2024-04-12T05:29:22.935 INFO:root:teuthology version: 0.0.1.dev267+g6c63784 2024-04-12T05:29:22.936 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi --owner scheduled_yuriw@teuthology --verbose --description orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --archive /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724 -- /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/orig.config.yaml 2024-04-12T05:29:22.991 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T05:29:23.080 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724 branch: reef-release description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: null first_in_suite: false job_id: '7652724' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: reef-release ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef-release sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2064 sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: reef-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 targets: smithi008.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBHXrxyScZM767rrL0sGHsSXpHS0uRqNFIrwqtCyUM/tVsNzRVrGtWwPsCPEeSSUdpDpLpexQRk+YL4Hd9p5PA= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLpm4x954BAugiC+FwA2zC+KhkrhLe0fOwwblJYsUgGIoXjcgpgaKzLpYnftQsmtTjghg2Ik4qXx70rbQDMt3Q= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 6c637841c215537a4502385240412f1966e0faab timestamp: 2024-04-11_20:35:13 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-12T05:29:23.080 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa; will attempt to use it 2024-04-12T05:29:23.081 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks 2024-04-12T05:29:23.081 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-12T05:29:23.083 INFO:teuthology.task.internal:Checking packages... 2024-04-12T05:29:23.109 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd540ebaca6b131a1dd560e7f69e024b133bbaa42' 2024-04-12T05:29:23.109 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-12T05:29:23.109 INFO:teuthology.packaging:ref: None 2024-04-12T05:29:23.109 INFO:teuthology.packaging:tag: None 2024-04-12T05:29:23.109 INFO:teuthology.packaging:branch: reef-release 2024-04-12T05:29:23.109 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:29:23.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef-release 2024-04-12T05:29:23.259 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1187.gd540ebac 2024-04-12T05:29:23.260 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-12T05:29:23.267 INFO:teuthology.task.internal:no buildpackages task found 2024-04-12T05:29:23.267 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-12T05:29:23.360 INFO:teuthology.task.internal:Saving configuration 2024-04-12T05:29:23.382 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-12T05:29:23.389 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T05:29:23.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 05:24:21.381449', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBHXrxyScZM767rrL0sGHsSXpHS0uRqNFIrwqtCyUM/tVsNzRVrGtWwPsCPEeSSUdpDpLpexQRk+YL4Hd9p5PA='} 2024-04-12T05:29:23.431 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 05:24:21.379489', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLpm4x954BAugiC+FwA2zC+KhkrhLe0fOwwblJYsUgGIoXjcgpgaKzLpYnftQsmtTjghg2Ik4qXx70rbQDMt3Q='} 2024-04-12T05:29:23.431 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-12T05:29:23.438 INFO:teuthology.task.internal:roles: ubuntu@smithi008.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-12T05:29:23.438 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-12T05:29:23.438 INFO:teuthology.run_tasks:Running task console_log... 2024-04-12T05:29:23.504 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ebf4e80d0>, signals=[15]) 2024-04-12T05:29:23.505 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-12T05:29:23.512 INFO:teuthology.task.internal:Opening connections... 2024-04-12T05:29:23.512 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:29:23.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:29:23.592 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-04-12T05:29:23.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:29:23.673 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-12T05:29:23.680 DEBUG:teuthology.orchestra.run.smithi008:> uname -m 2024-04-12T05:29:23.702 INFO:teuthology.orchestra.run.smithi008.stdout:x86_64 2024-04-12T05:29:23.703 DEBUG:teuthology.orchestra.run.smithi008:> cat /etc/os-release 2024-04-12T05:29:23.764 INFO:teuthology.orchestra.run.smithi008.stdout:NAME="CentOS Stream" 2024-04-12T05:29:23.764 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION="8" 2024-04-12T05:29:23.764 INFO:teuthology.orchestra.run.smithi008.stdout:ID="centos" 2024-04-12T05:29:23.764 INFO:teuthology.orchestra.run.smithi008.stdout:ID_LIKE="rhel fedora" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:VERSION_ID="8" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:PLATFORM_ID="platform:el8" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:ANSI_COLOR="0;31" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:HOME_URL="https://centos.org/" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-12T05:29:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T05:29:23.766 INFO:teuthology.lock.ops:Updating smithi008.front.sepia.ceph.com on lock server 2024-04-12T05:29:23.791 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-04-12T05:29:23.813 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-04-12T05:29:23.813 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-04-12T05:29:23.875 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-04-12T05:29:23.875 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="8" 2024-04-12T05:29:23.875 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-04-12T05:29:23.875 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="8" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el8" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 8" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-12T05:29:23.876 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-12T05:29:23.876 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-04-12T05:29:23.898 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-12T05:29:23.912 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-12T05:29:23.922 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-12T05:29:23.922 DEBUG:teuthology.orchestra.run.smithi008:> test '!' -e /home/ubuntu/cephtest 2024-04-12T05:29:23.925 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-04-12T05:29:23.944 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-12T05:29:23.952 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-12T05:29:23.952 DEBUG:teuthology.orchestra.run.smithi008:> test -z $(ls -A /var/lib/ceph) 2024-04-12T05:29:23.986 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-04-12T05:29:24.012 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-12T05:29:24.068 INFO:teuthology.run_tasks:Running task kernel... 2024-04-12T05:29:24.084 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-12T05:29:24.084 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-12T05:29:24.084 DEBUG:teuthology.orchestra.run.smithi008:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T05:29:24.084 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-12T05:29:24.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:29:24.104 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-04-12T05:29:24.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:29:24.106 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2024-04-12T05:29:24.166 INFO:teuthology.orchestra.run.smithi194.stdout:4.18.0-489.el8.x86_64 2024-04-12T05:29:24.166 INFO:teuthology.task.kernel:Running kernel on smithi194: 4.18.0-489.el8.x86_64 2024-04-12T05:29:24.166 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum install -y kernel 2024-04-12T05:29:24.167 INFO:teuthology.orchestra.run.smithi008.stdout:4.18.0-489.el8.x86_64 2024-04-12T05:29:24.167 INFO:teuthology.task.kernel:Running kernel on smithi008: 4.18.0-489.el8.x86_64 2024-04-12T05:29:24.168 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum install -y kernel 2024-04-12T05:29:35.846 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - AppStream 3.1 MB/s | 28 MB 00:09 2024-04-12T05:29:43.232 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - BaseOS 5.0 MB/s | 10 MB 00:02 2024-04-12T05:29:45.332 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - Extras 46 kB/s | 18 kB 00:00 2024-04-12T05:29:45.708 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - Extras common packages 30 kB/s | 7.7 kB 00:00 2024-04-12T05:29:45.951 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2024-04-12T05:29:47.788 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 16 MB 00:01 2024-04-12T05:29:51.744 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 444 kB/s | 24 kB 00:00 2024-04-12T05:29:54.966 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-12T05:29:54.966 INFO:teuthology.orchestra.run.smithi008.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-12T05:29:55.081 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-12T05:29:55.082 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout:Install 3 Packages 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 90 M 2024-04-12T05:29:55.083 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 96 M 2024-04-12T05:29:55.084 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:29:57.246 INFO:teuthology.orchestra.run.smithi008.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 4.9 MB/s | 10 MB 00:02 2024-04-12T05:29:57.946 INFO:teuthology.orchestra.run.smithi008.stdout:(2/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 13 MB/s | 36 MB 00:02 2024-04-12T05:30:00.254 INFO:teuthology.orchestra.run.smithi008.stdout:(3/3): kernel-core-4.18.0-552.el8.x86_64.rpm 8.5 MB/s | 44 MB 00:05 2024-04-12T05:30:00.255 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:30:00.255 INFO:teuthology.orchestra.run.smithi008.stdout:Total 17 MB/s | 90 MB 00:05 2024-04-12T05:30:00.972 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:30:01.435 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:30:01.435 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:30:02.480 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - AppStream 797 kB/s | 28 MB 00:35 2024-04-12T05:30:02.751 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:30:02.766 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:30:05.678 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:30:06.211 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:30:09.924 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:30:10.286 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:30:13.688 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - BaseOS 1.7 MB/s | 10 MB 00:06 2024-04-12T05:30:13.969 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:30:14.282 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:30:15.829 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - Extras 33 kB/s | 18 kB 00:00 2024-04-12T05:30:18.330 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - Extras common packages 3.2 kB/s | 7.7 kB 00:02 2024-04-12T05:30:18.598 INFO:teuthology.orchestra.run.smithi194.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2024-04-12T05:30:20.284 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-04-12T05:30:24.024 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 627 kB/s | 24 kB 00:00 2024-04-12T05:30:26.954 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2024-04-12T05:30:26.954 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2024-04-12T05:30:27.068 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout: kernel x86_64 4.18.0-552.el8 baseos 10 M 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout: kernel-core x86_64 4.18.0-552.el8 baseos 44 M 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout: kernel-modules x86_64 4.18.0-552.el8 baseos 36 M 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:30:27.069 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout:Install 3 Packages 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 90 M 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 96 M 2024-04-12T05:30:27.070 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:30:28.950 INFO:teuthology.orchestra.run.smithi194.stdout:(1/3): kernel-4.18.0-552.el8.x86_64.rpm 6.1 MB/s | 10 MB 00:01 2024-04-12T05:30:32.757 INFO:teuthology.orchestra.run.smithi194.stdout:(2/3): kernel-core-4.18.0-552.el8.x86_64.rpm 8.0 MB/s | 44 MB 00:05 2024-04-12T05:30:35.241 INFO:teuthology.orchestra.run.smithi194.stdout:(3/3): kernel-modules-4.18.0-552.el8.x86_64.rpm 4.6 MB/s | 36 MB 00:07 2024-04-12T05:30:35.241 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:30:35.241 INFO:teuthology.orchestra.run.smithi194.stdout:Total 11 MB/s | 90 MB 00:08 2024-04-12T05:30:35.922 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:30:36.334 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:30:36.334 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:30:37.580 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:30:37.598 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:30:37.672 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:30:37.695 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:30:38.059 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:30:38.060 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:30:38.060 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:30:39.069 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:30:39.069 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:30:39.069 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:30:39.069 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-12T05:30:39.069 INFO:teuthology.orchestra.run.smithi008.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-12T05:30:39.070 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:30:39.070 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:30:39.211 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel | sort -rV | head -n 1 2024-04-12T05:30:39.435 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-12T05:30:39.435 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-12T05:30:39.435 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-12T05:30:39.436 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-12T05:30:39.436 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-12T05:30:39.436 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-04-12T05:30:39.436 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi008.front.sepia.ceph.com, path=None, version=distro) 2024-04-12T05:30:39.436 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q kernel | sort -rV | head -n 1 2024-04-12T05:30:39.657 INFO:teuthology.orchestra.run.smithi008.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-12T05:30:39.657 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-12T05:30:39.657 DEBUG:teuthology.orchestra.run.smithi008:> sudo rpm -qi grub2-tools 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Name : grub2-tools 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Epoch : 1 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Version : 2.02 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Release : 129.el8 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Architecture: x86_64 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Group : System Environment/Base 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:Size : 9265586 2024-04-12T05:30:39.692 INFO:teuthology.orchestra.run.smithi008.stdout:License : GPLv3+ 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Relocations : (not relocatable) 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Packager : CentOS BuildSystem 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Vendor : CentOS 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Summary : Support tools for GRUB. 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:Description : 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T05:30:39.693 INFO:teuthology.orchestra.run.smithi008.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T05:30:39.694 INFO:teuthology.orchestra.run.smithi008.stdout:hardware devices. 2024-04-12T05:30:39.694 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:30:39.694 INFO:teuthology.orchestra.run.smithi008.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T05:30:39.695 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T05:30:39.695 INFO:teuthology.task.kernel:Updating grub on smithi008 to boot 4.18.0-552.el8.x86_64 2024-04-12T05:30:39.695 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T05:30:40.422 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:30:40.943 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:30:41.014 INFO:teuthology.orchestra.run.smithi008.stderr:Generating grub configuration file ... 2024-04-12T05:30:42.736 INFO:teuthology.orchestra.run.smithi008.stderr:done 2024-04-12T05:30:42.738 DEBUG:teuthology.orchestra.run.smithi008:> mktemp 2024-04-12T05:30:42.772 INFO:teuthology.orchestra.run.smithi008.stdout:/tmp/tmp.e2X2XrFlvB 2024-04-12T05:30:42.772 DEBUG:teuthology.orchestra.run.smithi008:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.e2X2XrFlvB 2024-04-12T05:30:42.844 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0666 /tmp/tmp.e2X2XrFlvB 2024-04-12T05:30:43.004 DEBUG:teuthology.orchestra.remote:smithi008:/tmp/tmp.e2X2XrFlvB is 6KB 2024-04-12T05:30:43.054 DEBUG:teuthology.orchestra.run.smithi008:> rm -fr /tmp/tmp.e2X2XrFlvB 2024-04-12T05:30:43.071 DEBUG:teuthology.orchestra.run.smithi008:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T05:30:43.147 INFO:teuthology.orchestra.run.smithi008.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-12T05:30:43.147 INFO:teuthology.orchestra.run.smithi008.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-12T05:30:43.147 INFO:teuthology.orchestra.run.smithi008.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-12T05:30:43.147 INFO:teuthology.orchestra.run.smithi008.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-12T05:30:43.148 DEBUG:teuthology.orchestra.run.smithi008:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-12T05:30:43.305 DEBUG:teuthology.orchestra.run.smithi008:> sudo shutdown -r now 2024-04-12T05:30:44.506 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:30:44.868 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:30:48.299 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:30:48.592 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : kernel-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:31:10.553 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-core-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:31:10.578 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:31:10.931 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:31:10.931 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : kernel-4.18.0-552.el8.x86_64 1/3 2024-04-12T05:31:10.931 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : kernel-core-4.18.0-552.el8.x86_64 2/3 2024-04-12T05:31:11.634 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : kernel-modules-4.18.0-552.el8.x86_64 3/3 2024-04-12T05:31:11.634 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:31:11.635 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:31:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: kernel-4.18.0-552.el8.x86_64 kernel-core-4.18.0-552.el8.x86_64 2024-04-12T05:31:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: kernel-modules-4.18.0-552.el8.x86_64 2024-04-12T05:31:11.635 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:31:11.635 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:31:11.943 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q kernel | sort -rV | head -n 1 2024-04-12T05:31:12.158 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-12T05:31:12.159 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-12T05:31:12.159 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-552.el8.x86_64 2024-04-12T05:31:12.159 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-12T05:31:12.159 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-12T05:31:12.159 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-04-12T05:31:12.159 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi194.front.sepia.ceph.com, path=None, version=distro) 2024-04-12T05:31:12.159 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q kernel | sort -rV | head -n 1 2024-04-12T05:31:12.371 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-4.18.0-552.el8.x86_64 2024-04-12T05:31:12.372 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-552.el8.x86_64 2024-04-12T05:31:12.372 DEBUG:teuthology.orchestra.run.smithi194:> sudo rpm -qi grub2-tools 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Name : grub2-tools 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Epoch : 1 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Version : 2.02 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Release : 129.el8 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Architecture: x86_64 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Group : System Environment/Base 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Size : 9265586 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:License : GPLv3+ 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2024-04-12T05:31:12.405 INFO:teuthology.orchestra.run.smithi194.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Build Host : kbuilder.bsys.centos.org 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Relocations : (not relocatable) 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Packager : CentOS BuildSystem 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Vendor : CentOS 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Summary : Support tools for GRUB. 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:Description : 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout:hardware devices. 2024-04-12T05:31:12.406 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:31:12.407 INFO:teuthology.orchestra.run.smithi194.stdout:This subpackage provides tools for support of all platforms. 2024-04-12T05:31:12.407 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-12T05:31:12.408 INFO:teuthology.task.kernel:Updating grub on smithi194 to boot 4.18.0-552.el8.x86_64 2024-04-12T05:31:12.408 DEBUG:teuthology.orchestra.run.smithi194:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-12T05:31:13.308 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-04-12T05:31:13.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:31:13.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:31:13.686 INFO:teuthology.orchestra.run.smithi194.stderr:Generating grub configuration file ... 2024-04-12T05:31:15.871 INFO:teuthology.orchestra.run.smithi194.stderr:done 2024-04-12T05:31:15.873 DEBUG:teuthology.orchestra.run.smithi194:> mktemp 2024-04-12T05:31:15.914 INFO:teuthology.orchestra.run.smithi194.stdout:/tmp/tmp.PEqsFjhC8H 2024-04-12T05:31:15.915 DEBUG:teuthology.orchestra.run.smithi194:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PEqsFjhC8H 2024-04-12T05:31:15.989 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0666 /tmp/tmp.PEqsFjhC8H 2024-04-12T05:31:16.133 DEBUG:teuthology.orchestra.remote:smithi194:/tmp/tmp.PEqsFjhC8H is 6KB 2024-04-12T05:31:16.181 DEBUG:teuthology.orchestra.run.smithi194:> rm -fr /tmp/tmp.PEqsFjhC8H 2024-04-12T05:31:16.199 DEBUG:teuthology.orchestra.run.smithi194:> sudo /bin/ls /boot/loader/entries || true 2024-04-12T05:31:16.280 INFO:teuthology.orchestra.run.smithi194.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2024-04-12T05:31:16.281 INFO:teuthology.orchestra.run.smithi194.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2024-04-12T05:31:16.281 INFO:teuthology.orchestra.run.smithi194.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2024-04-12T05:31:16.281 INFO:teuthology.orchestra.run.smithi194.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64.conf 2024-04-12T05:31:16.282 DEBUG:teuthology.orchestra.run.smithi194:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-552.el8.x86_64 2024-04-12T05:31:16.438 DEBUG:teuthology.orchestra.run.smithi194:> sudo shutdown -r now 2024-04-12T05:31:31.747 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-12T05:31:40.756 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:31:40.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:31:43.841 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.8 2024-04-12T05:31:46.442 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-04-12T05:31:46.442 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:31:46.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:31:55.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:31:55.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:32:04.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-12T05:32:13.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:32:13.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:32:16.830 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.194 2024-04-12T05:32:27.942 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-04-12T05:32:28.412 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:32:28.412 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-552.el8.x86_64"... 2024-04-12T05:32:28.413 DEBUG:teuthology.orchestra.run.smithi008:> uname -r 2024-04-12T05:32:28.471 INFO:teuthology.orchestra.run.smithi008.stdout:4.18.0-552.el8.x86_64 2024-04-12T05:32:28.471 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-12T05:32:28.471 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T05:32:28.471 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T05:32:28.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:32:28.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:32:29.472 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-12T05:32:29.472 DEBUG:teuthology.orchestra.run.smithi008:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-12T05:32:29.540 INFO:teuthology.orchestra.run.smithi008.stdout:ttyS1 2024-04-12T05:32:29.552 DEBUG:teuthology.parallel:result is None 2024-04-12T05:33:01.218 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-04-12T05:33:01.485 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:33:01.486 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-552.el8.x86_64"... 2024-04-12T05:33:01.486 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-04-12T05:33:01.503 INFO:teuthology.orchestra.run.smithi194.stdout:4.18.0-552.el8.x86_64 2024-04-12T05:33:01.503 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-552.el8.x86_64 vs 4.18.0-552.el8.x86_64 2024-04-12T05:33:01.503 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-12T05:33:01.503 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-12T05:33:02.504 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-12T05:33:02.505 DEBUG:teuthology.orchestra.run.smithi194:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-12T05:33:02.588 INFO:teuthology.orchestra.run.smithi194.stdout:ttyS1 2024-04-12T05:33:02.604 DEBUG:teuthology.parallel:result is None 2024-04-12T05:33:02.604 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-12T05:33:02.612 INFO:teuthology.task.internal:Creating test directory... 2024-04-12T05:33:02.612 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T05:33:02.615 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-12T05:33:02.635 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-12T05:33:02.643 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-12T05:33:02.651 INFO:teuthology.task.internal:Creating archive directory... 2024-04-12T05:33:02.652 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T05:33:02.678 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-12T05:33:02.715 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-12T05:33:02.725 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-12T05:33:02.725 DEBUG:teuthology.orchestra.run.smithi008:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T05:33:02.753 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-12T05:33:02.790 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T05:33:02.797 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T05:33:02.801 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T05:33:02.810 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-12T05:33:02.811 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-12T05:33:02.819 INFO:teuthology.task.internal:Configuring sudo... 2024-04-12T05:33:02.819 DEBUG:teuthology.orchestra.run.smithi008:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T05:33:02.822 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-12T05:33:02.873 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-12T05:33:02.882 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-12T05:33:02.882 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T05:33:02.893 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-12T05:33:02.932 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T05:33:02.999 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T05:33:03.052 DEBUG:teuthology.orchestra.run.smithi008:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T05:33:03.139 DEBUG:teuthology.orchestra.run.smithi008:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T05:33:03.183 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:33:03.183 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T05:33:03.253 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T05:33:03.301 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-12T05:33:03.356 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T05:33:03.441 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-12T05:33:03.486 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:33:03.486 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-12T05:33:03.553 DEBUG:teuthology.orchestra.run.smithi008:> sudo service rsyslog restart 2024-04-12T05:33:03.556 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-04-12T05:33:03.629 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T05:33:03.657 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T05:33:04.129 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-12T05:33:04.137 INFO:teuthology.task.internal:Starting timer... 2024-04-12T05:33:04.137 INFO:teuthology.run_tasks:Running task pcp... 2024-04-12T05:33:04.156 INFO:teuthology.run_tasks:Running task selinux... 2024-04-12T05:33:04.168 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-12T05:33:04.168 DEBUG:teuthology.orchestra.run.smithi008:> sudo service auditd rotate 2024-04-12T05:33:04.238 INFO:teuthology.orchestra.run.smithi008.stdout:Rotating logs: 2024-04-12T05:33:04.240 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-04-12T05:33:04.323 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-04-12T05:33:04.325 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-12T05:33:04.325 DEBUG:teuthology.orchestra.run.smithi008:> /usr/sbin/getenforce 2024-04-12T05:33:04.386 INFO:teuthology.orchestra.run.smithi008.stdout:Permissive 2024-04-12T05:33:04.387 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-04-12T05:33:04.410 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-04-12T05:33:04.411 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi008.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-04-12T05:33:04.411 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-12T05:33:04.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:33:04.440 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-12T05:33:04.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:33:04.478 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-12T05:33:04.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo /usr/sbin/setenforce permissive 2024-04-12T05:33:04.507 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-04-12T05:33:04.544 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-12T05:33:04.555 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-12T05:33:04.560 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-12T05:33:04.672 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-12T05:33:04.688 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-12T05:33:04.689 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi008.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-12T05:39:40.729 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi008.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-04-12T05:39:40.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:39:40.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi008.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:39:40.809 DEBUG:teuthology.orchestra.run.smithi008:> true 2024-04-12T05:39:40.880 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi008.front.sepia.ceph.com' 2024-04-12T05:39:40.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:39:40.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-12T05:39:40.954 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-04-12T05:39:41.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-04-12T05:39:41.026 INFO:teuthology.run_tasks:Running task clock... 2024-04-12T05:39:41.038 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-12T05:39:41.038 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T05:39:41.038 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T05:39:41.041 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-12T05:39:41.041 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T05:39:41.072 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T05:39:41.086 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T05:39:41.099 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-12T05:39:41.113 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-12T05:39:41.152 INFO:teuthology.orchestra.run.smithi008.stderr:sudo: ntpd: command not found 2024-04-12T05:39:41.164 INFO:teuthology.orchestra.run.smithi008.stdout:506 Cannot talk to daemon 2024-04-12T05:39:41.178 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T05:39:41.186 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-04-12T05:39:41.192 INFO:teuthology.orchestra.run.smithi008.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T05:39:41.196 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-04-12T05:39:41.210 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-12T05:39:41.223 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-12T05:39:41.258 INFO:teuthology.orchestra.run.smithi008.stderr:bash: ntpq: command not found 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.262 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.292 INFO:teuthology.orchestra.run.smithi194.stderr:bash: ntpq: command not found 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.296 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T05:39:41.297 INFO:teuthology.run_tasks:Running task pexec... 2024-04-12T05:39:41.308 INFO:teuthology.task.pexec:Executing custom commands... 2024-04-12T05:39:41.308 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:39:41.309 DEBUG:teuthology.orchestra.run.smithi008:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-04-12T05:39:41.309 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi194.front.sepia.ceph.com 2024-04-12T05:39:41.309 DEBUG:teuthology.orchestra.run.smithi194:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-04-12T05:39:42.090 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:04 ago on Fri 12 Apr 2024 05:37:37 AM UTC. 2024-04-12T05:39:42.095 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:56 ago on Fri 12 Apr 2024 05:37:45 AM UTC. 2024-04-12T05:39:44.218 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:39:44.218 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:44.218 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:39:44.218 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout:Resetting modules: 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:44.219 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:39:44.387 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:39:44.388 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout:Resetting modules: 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:39:44.389 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:44.390 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:39:44.470 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:39:44.647 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:39:45.296 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:08 ago on Fri 12 Apr 2024 05:37:37 AM UTC. 2024-04-12T05:39:45.506 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:00 ago on Fri 12 Apr 2024 05:37:45 AM UTC. 2024-04-12T05:39:47.443 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:39:47.480 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:47.480 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:39:47.480 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout:Installing group/module packages: 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-04-12T05:39:47.481 INFO:teuthology.orchestra.run.smithi194.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-04-12T05:39:47.482 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Installing module profiles: 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools/common 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Enabling module streams: 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: container-tools rhel8 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:39:47.483 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:39:47.484 INFO:teuthology.orchestra.run.smithi194.stdout:Install 13 Packages 2024-04-12T05:39:47.484 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:39:47.484 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 27 M 2024-04-12T05:39:47.485 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 91 M 2024-04-12T05:39:47.485 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:39:47.800 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout:Installing group/module packages: 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout: aardvark-dns x86_64 2:1.10.0-1.module_el8+871+552be958 appstream 1.0 M 2024-04-12T05:39:47.841 INFO:teuthology.orchestra.run.smithi008.stdout: buildah x86_64 2:1.33.6-2.module_el8+976+11360731 appstream 9.6 M 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: cockpit-podman noarch 84.1-1.module_el8+977+8ab54275 appstream 689 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: crun x86_64 1.14.3-2.module_el8+968+fbb249c7 appstream 258 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: netavark x86_64 2:1.10.3-1.module_el8+951+32019cde appstream 4.1 M 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: python3-podman noarch 4.9.0-1.module_el8+847+7863d4e6 appstream 180 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: skopeo x86_64 2:1.14.3-0.1.module_el8+885+7da147f3 appstream 8.8 M 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: toolbox x86_64 0.0.99.5-2.module_el8+971+3d3df00d appstream 2.5 M 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: udica noarch 0.2.6-20.module_el8+804+f131391c appstream 53 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 appstream 25 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyxdg noarch 0.25-16.el8 appstream 94 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: yajl x86_64 2.1.0-12.el8 appstream 41 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout:Installing weak dependencies: 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout: criu-libs x86_64 3.15-3.module_el8.7.0+1216+b022c01d appstream 38 k 2024-04-12T05:39:47.842 INFO:teuthology.orchestra.run.smithi008.stdout:Installing module profiles: 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools/common 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling module streams: 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout: container-tools rhel8 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout:Install 13 Packages 2024-04-12T05:39:47.843 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:39:47.844 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 27 M 2024-04-12T05:39:47.844 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 91 M 2024-04-12T05:39:47.844 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:39:49.007 INFO:teuthology.orchestra.run.smithi194.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 515 kB/s | 689 kB 00:01 2024-04-12T05:39:49.327 INFO:teuthology.orchestra.run.smithi008.stdout:(1/13): cockpit-podman-84.1-1.module_el8+977+8a 534 kB/s | 689 kB 00:01 2024-04-12T05:39:49.332 INFO:teuthology.orchestra.run.smithi194.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 117 kB/s | 38 kB 00:00 2024-04-12T05:39:49.532 INFO:teuthology.orchestra.run.smithi194.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 553 kB/s | 1.0 MB 00:01 2024-04-12T05:39:49.661 INFO:teuthology.orchestra.run.smithi008.stdout:(2/13): criu-libs-3.15-3.module_el8.7.0+1216+b0 114 kB/s | 38 kB 00:00 2024-04-12T05:39:49.753 INFO:teuthology.orchestra.run.smithi008.stdout:(3/13): aardvark-dns-1.10.0-1.module_el8+871+55 601 kB/s | 1.0 MB 00:01 2024-04-12T05:39:50.008 INFO:teuthology.orchestra.run.smithi194.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 382 kB/s | 258 kB 00:00 2024-04-12T05:39:50.320 INFO:teuthology.orchestra.run.smithi008.stdout:(4/13): crun-1.14.3-2.module_el8+968+fbb249c7.x 391 kB/s | 258 kB 00:00 2024-04-12T05:39:50.592 INFO:teuthology.orchestra.run.smithi194.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 309 kB/s | 180 kB 00:00 2024-04-12T05:39:50.886 INFO:teuthology.orchestra.run.smithi194.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 85 kB/s | 25 kB 00:00 2024-04-12T05:39:50.904 INFO:teuthology.orchestra.run.smithi008.stdout:(5/13): python3-podman-4.9.0-1.module_el8+847+7 309 kB/s | 180 kB 00:00 2024-04-12T05:39:51.101 INFO:teuthology.orchestra.run.smithi194.stdout:(7/13): buildah-1.33.6-2.module_el8+976+1136073 2.8 MB/s | 9.6 MB 00:03 2024-04-12T05:39:51.188 INFO:teuthology.orchestra.run.smithi008.stdout:(6/13): python3-pytoml-0.1.14-5.git7dea353.el8. 87 kB/s | 25 kB 00:00 2024-04-12T05:39:51.348 INFO:teuthology.orchestra.run.smithi194.stdout:(8/13): python3-pyxdg-0.25-16.el8.noarch.rpm 208 kB/s | 94 kB 00:00 2024-04-12T05:39:51.629 INFO:teuthology.orchestra.run.smithi008.stdout:(7/13): python3-pyxdg-0.25-16.el8.noarch.rpm 213 kB/s | 94 kB 00:00 2024-04-12T05:39:55.450 INFO:teuthology.orchestra.run.smithi194.stdout:(9/13): toolbox-0.0.99.5-2.module_el8+971+3d3df 629 kB/s | 2.5 MB 00:04 2024-04-12T05:39:55.577 INFO:teuthology.orchestra.run.smithi194.stdout:(10/13): netavark-1.10.3-1.module_el8+951+32019 695 kB/s | 4.1 MB 00:06 2024-04-12T05:39:55.884 INFO:teuthology.orchestra.run.smithi194.stdout:(11/13): udica-0.2.6-20.module_el8+804+f131391c 121 kB/s | 53 kB 00:00 2024-04-12T05:39:55.901 INFO:teuthology.orchestra.run.smithi194.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 127 kB/s | 41 kB 00:00 2024-04-12T05:39:56.137 INFO:teuthology.orchestra.run.smithi008.stdout:(8/13): netavark-1.10.3-1.module_el8+951+32019c 658 kB/s | 4.1 MB 00:06 2024-04-12T05:39:58.637 INFO:teuthology.orchestra.run.smithi008.stdout:(9/13): skopeo-1.14.3-0.1.module_el8+885+7da147 1.3 MB/s | 8.8 MB 00:07 2024-04-12T05:39:59.029 INFO:teuthology.orchestra.run.smithi008.stdout:(10/13): udica-0.2.6-20.module_el8+804+f131391c 134 kB/s | 53 kB 00:00 2024-04-12T05:39:59.354 INFO:teuthology.orchestra.run.smithi008.stdout:(11/13): yajl-2.1.0-12.el8.x86_64.rpm 126 kB/s | 41 kB 00:00 2024-04-12T05:40:00.230 INFO:teuthology.orchestra.run.smithi008.stdout:(12/13): toolbox-0.0.99.5-2.module_el8+971+3d3d 632 kB/s | 2.5 MB 00:04 2024-04-12T05:40:01.597 INFO:teuthology.orchestra.run.smithi008.stdout:(13/13): buildah-1.33.6-2.module_el8+976+113607 728 kB/s | 9.6 MB 00:13 2024-04-12T05:40:01.598 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:40:01.598 INFO:teuthology.orchestra.run.smithi008.stdout:Total 2.0 MB/s | 27 MB 00:13 2024-04-12T05:40:01.879 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:40:01.902 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:40:01.902 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:40:01.968 INFO:teuthology.orchestra.run.smithi194.stdout:(13/13): skopeo-1.14.3-0.1.module_el8+885+7da14 830 kB/s | 8.8 MB 00:10 2024-04-12T05:40:01.969 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:40:01.969 INFO:teuthology.orchestra.run.smithi194.stdout:Total 1.9 MB/s | 27 MB 00:14 2024-04-12T05:40:02.185 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:40:02.188 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:40:02.244 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:40:02.265 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:40:02.266 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:40:02.526 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:40:02.530 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:40:02.882 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:40:03.247 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:40:03.876 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-04-12T05:40:04.219 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-04-12T05:40:04.232 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-04-12T05:40:04.409 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2/13 2024-04-12T05:40:04.410 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-04-12T05:40:04.621 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/13 2024-04-12T05:40:04.653 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-04-12T05:40:04.835 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 4/13 2024-04-12T05:40:04.892 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-04-12T05:40:05.090 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 5/13 2024-04-12T05:40:05.485 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-04-12T05:40:05.664 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 6/13 2024-04-12T05:40:06.548 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-04-12T05:40:06.644 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 7/13 2024-04-12T05:40:06.769 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-04-12T05:40:06.942 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 8/13 2024-04-12T05:40:07.034 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-04-12T05:40:07.178 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 9/13 2024-04-12T05:40:07.514 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-04-12T05:40:07.628 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 10/13 2024-04-12T05:40:07.710 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-12T05:40:07.866 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-12T05:40:07.941 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-12T05:40:08.101 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-12T05:40:08.158 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-12T05:40:08.304 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-04-12T05:40:08.611 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-12T05:40:08.612 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 13/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x8 1/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cockpit-podman-84.1-1.module_el8+977+8ab54275.noar 3/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_ 4/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 5/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 6/13 2024-04-12T05:40:08.743 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-podman-4.9.0-1.module_el8+847+7863d4e6.noa 7/13 2024-04-12T05:40:08.744 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 8/13 2024-04-12T05:40:08.744 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 9/13 2024-04-12T05:40:08.744 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 10/13 2024-04-12T05:40:08.744 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 11/13 2024-04-12T05:40:08.744 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : udica-0.2.6-20.module_el8+804+f131391c.noarch 12/13 2024-04-12T05:40:09.104 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-04-12T05:40:09.105 INFO:teuthology.orchestra.run.smithi008.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-04-12T05:40:09.106 INFO:teuthology.orchestra.run.smithi008.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-04-12T05:40:09.106 INFO:teuthology.orchestra.run.smithi008.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-04-12T05:40:09.106 INFO:teuthology.orchestra.run.smithi008.stdout: yajl-2.1.0-12.el8.x86_64 2024-04-12T05:40:09.106 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:40:09.106 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:40:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: aardvark-dns-2:1.10.0-1.module_el8+871+552be958.x86_64 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: buildah-2:1.33.6-2.module_el8+976+11360731.x86_64 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: cockpit-podman-84.1-1.module_el8+977+8ab54275.noarch 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: criu-libs-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: crun-1.14.3-2.module_el8+968+fbb249c7.x86_64 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: netavark-2:1.10.3-1.module_el8+951+32019cde.x86_64 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: python3-podman-4.9.0-1.module_el8+847+7863d4e6.noarch 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-04-12T05:40:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: skopeo-2:1.14.3-0.1.module_el8+885+7da147f3.x86_64 2024-04-12T05:40:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: toolbox-0.0.99.5-2.module_el8+971+3d3df00d.x86_64 2024-04-12T05:40:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: udica-0.2.6-20.module_el8+804+f131391c.noarch 2024-04-12T05:40:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: yajl-2.1.0-12.el8.x86_64 2024-04-12T05:40:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:40:09.278 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:40:09.399 DEBUG:teuthology.parallel:result is None 2024-04-12T05:40:09.574 DEBUG:teuthology.parallel:result is None 2024-04-12T05:40:09.575 INFO:teuthology.run_tasks:Running task install... 2024-04-12T05:40:09.585 DEBUG:teuthology.task.install:project ceph 2024-04-12T05:40:09.585 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42'}} 2024-04-12T05:40:09.585 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42'} 2024-04-12T05:40:09.585 INFO:teuthology.task.install:Using flavor: default 2024-04-12T05:40:09.594 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-12T05:40:09.595 INFO:teuthology.task.install:extra packages: [] 2024-04-12T05:40:09.595 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-12T05:40:09.596 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:40:09.596 INFO:teuthology.packaging:ref: None 2024-04-12T05:40:09.596 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:40:09.596 INFO:teuthology.packaging:branch: None 2024-04-12T05:40:09.596 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:40:09.804 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph-ci v18.2.1^{} -> None 2024-04-12T05:40:09.804 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://github.com/ceph/ceph-ci; will also look in https://github.com/ceph/ceph 2024-04-12T05:40:10.084 DEBUG:teuthology.repo_utils:git ls-remote https://github.com/ceph/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-12T05:40:10.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-12T05:40:10.087 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-04-12T05:40:10.088 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:40:10.088 INFO:teuthology.packaging:ref: None 2024-04-12T05:40:10.088 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:40:10.088 INFO:teuthology.packaging:branch: None 2024-04-12T05:40:10.088 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:40:10.088 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://github.com/ceph/ceph-ci; will also look in https://github.com/ceph/ceph 2024-04-12T05:40:10.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-12T05:40:10.235 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/ 2024-04-12T05:40:10.236 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-12T05:40:10.276 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T05:40:10.276 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:40:10.276 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T05:40:10.306 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-12T05:40:10.307 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:40:10.307 INFO:teuthology.packaging:ref: None 2024-04-12T05:40:10.307 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:40:10.307 INFO:teuthology.packaging:branch: None 2024-04-12T05:40:10.307 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:40:10.307 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T05:40:10.315 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/ 2024-04-12T05:40:10.315 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-04-12T05:40:10.347 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-12T05:40:10.348 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:40:10.348 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-12T05:40:10.377 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T05:40:10.382 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-12T05:40:10.382 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:40:10.382 INFO:teuthology.packaging:ref: None 2024-04-12T05:40:10.383 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:40:10.383 INFO:teuthology.packaging:branch: None 2024-04-12T05:40:10.383 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:40:10.383 DEBUG:teuthology.orchestra.run.smithi008:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-12T05:40:10.457 DEBUG:teuthology.orchestra.run.smithi008:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-12T05:40:10.458 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T05:40:10.526 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-04-12T05:40:10.528 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-04-12T05:40:10.539 DEBUG:teuthology.orchestra.run.smithi008:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-12T05:40:10.611 INFO:teuthology.orchestra.run.smithi008.stdout:check_obsoletes = 1 2024-04-12T05:40:10.612 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum clean all 2024-04-12T05:40:10.790 INFO:teuthology.orchestra.run.smithi194.stdout:76 files removed 2024-04-12T05:40:10.817 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw 2024-04-12T05:40:10.901 INFO:teuthology.orchestra.run.smithi008.stdout:76 files removed 2024-04-12T05:40:10.929 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-radosgw 2024-04-12T05:40:11.342 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 686 kB/s | 79 kB 00:00 2024-04-12T05:40:11.451 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 134 kB/s | 9.4 kB 00:00 2024-04-12T05:40:11.455 INFO:teuthology.orchestra.run.smithi008.stdout:ceph packages for x86_64 793 kB/s | 79 kB 00:00 2024-04-12T05:40:11.585 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 17 kB/s | 1.8 kB 00:00 2024-04-12T05:40:12.774 INFO:teuthology.orchestra.run.smithi008.stdout:ceph noarch packages 7.4 kB/s | 9.4 kB 00:01 2024-04-12T05:40:12.883 INFO:teuthology.orchestra.run.smithi008.stdout:ceph source packages 23 kB/s | 1.8 kB 00:00 2024-04-12T05:40:44.591 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - AppStream 866 kB/s | 28 MB 00:32 2024-04-12T05:40:46.156 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - AppStream 859 kB/s | 28 MB 00:33 2024-04-12T05:40:51.224 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - BaseOS 6.2 MB/s | 10 MB 00:01 2024-04-12T05:40:52.906 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - BaseOS 7.0 MB/s | 10 MB 00:01 2024-04-12T05:40:53.208 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - Extras 44 kB/s | 18 kB 00:00 2024-04-12T05:40:53.683 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 7.7 kB 00:00 2024-04-12T05:40:54.998 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - Extras 45 kB/s | 18 kB 00:00 2024-04-12T05:40:55.077 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 8 - PowerTools 3.8 MB/s | 5.0 MB 00:01 2024-04-12T05:40:55.324 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - Extras common packages 37 kB/s | 7.7 kB 00:00 2024-04-12T05:40:56.211 INFO:teuthology.orchestra.run.smithi194.stdout:Copr repo for python3-asyncssh owned by ceph 19 kB/s | 3.5 kB 00:00 2024-04-12T05:40:56.692 INFO:teuthology.orchestra.run.smithi008.stdout:CentOS Stream 8 - PowerTools 3.9 MB/s | 5.0 MB 00:01 2024-04-12T05:40:57.860 INFO:teuthology.orchestra.run.smithi008.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-04-12T05:40:58.123 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2024-04-12T05:40:59.988 INFO:teuthology.orchestra.run.smithi008.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 16 MB 00:02 2024-04-12T05:41:01.937 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 471 kB/s | 24 kB 00:00 2024-04-12T05:41:04.002 INFO:teuthology.orchestra.run.smithi008.stdout:lab-extras 549 kB/s | 24 kB 00:00 2024-04-12T05:41:05.522 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:41:05.524 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:41:05.524 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:05.524 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:41:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:18.2.1-0.el8 ceph 13 M 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:18.2.1-0.el8 ceph 3.7 M 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:18.2.1-0.el8 ceph 3.6 M 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:18.2.1-0.el8 ceph 5.4 M 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:18.2.1-0.el8 ceph 22 M 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:18.2.1-0.el8 ceph 24 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:18.2.1-0.el8 ceph 774 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:18.2.1-0.el8 ceph 474 k 2024-04-12T05:41:05.525 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:18.2.1-0.el8 ceph 6.5 M 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el8 ceph 45 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:18.2.1-0.el8 ceph 123 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:18.2.1-0.el8 ceph 197 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:18.2.1-0.el8 ceph 371 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:18.2.1-0.el8 ceph 365 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:18.2.1-0.el8 ceph 107 k 2024-04-12T05:41:05.526 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout:Install 21 Packages 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 2 Packages 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:05.527 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 59 M 2024-04-12T05:41:05.528 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:41:06.079 INFO:teuthology.orchestra.run.smithi194.stdout:(1/23): ceph-base-18.2.1-0.el8.x86_64.rpm 46 MB/s | 5.4 MB 00:00 2024-04-12T05:41:06.104 INFO:teuthology.orchestra.run.smithi194.stdout:(2/23): ceph-selinux-18.2.1-0.el8.x86_64.rpm 975 kB/s | 24 kB 00:00 2024-04-12T05:41:06.146 INFO:teuthology.orchestra.run.smithi194.stdout:(3/23): libcephfs2-18.2.1-0.el8.x86_64.rpm 18 MB/s | 774 kB 00:00 2024-04-12T05:41:06.180 INFO:teuthology.orchestra.run.smithi194.stdout:(4/23): libradosstriper1-18.2.1-0.el8.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-12T05:41:06.314 INFO:teuthology.orchestra.run.smithi194.stdout:(5/23): librgw2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 6.5 MB 00:00 2024-04-12T05:41:06.339 INFO:teuthology.orchestra.run.smithi194.stdout:(6/23): python3-ceph-argparse-18.2.1-0.el8.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-12T05:41:06.364 INFO:teuthology.orchestra.run.smithi194.stdout:(7/23): python3-ceph-common-18.2.1-0.el8.x86_64 4.8 MB/s | 123 kB 00:00 2024-04-12T05:41:06.389 INFO:teuthology.orchestra.run.smithi194.stdout:(8/23): python3-cephfs-18.2.1-0.el8.x86_64.rpm 7.7 MB/s | 197 kB 00:00 2024-04-12T05:41:06.415 INFO:teuthology.orchestra.run.smithi194.stdout:(9/23): python3-rados-18.2.1-0.el8.x86_64.rpm 15 MB/s | 371 kB 00:00 2024-04-12T05:41:06.440 INFO:teuthology.orchestra.run.smithi194.stdout:(10/23): python3-rbd-18.2.1-0.el8.x86_64.rpm 14 MB/s | 365 kB 00:00 2024-04-12T05:41:06.490 INFO:teuthology.orchestra.run.smithi194.stdout:(11/23): python3-rgw-18.2.1-0.el8.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-04-12T05:41:06.641 INFO:teuthology.orchestra.run.smithi194.stdout:(12/23): ceph-radosgw-18.2.1-0.el8.x86_64.rpm 19 MB/s | 13 MB 00:00 2024-04-12T05:41:06.900 INFO:teuthology.orchestra.run.smithi194.stdout:(13/23): ceph-common-18.2.1-0.el8.x86_64.rpm 23 MB/s | 22 MB 00:00 2024-04-12T05:41:07.283 INFO:teuthology.orchestra.run.smithi194.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 115 kB/s | 44 kB 00:00 2024-04-12T05:41:07.483 INFO:teuthology.orchestra.run.smithi194.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 308 kB/s | 259 kB 00:00 2024-04-12T05:41:07.584 INFO:teuthology.orchestra.run.smithi194.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 3.0 MB/s | 306 kB 00:00 2024-04-12T05:41:07.659 INFO:teuthology.orchestra.run.smithi194.stdout:(17/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 125 kB/s | 47 kB 00:00 2024-04-12T05:41:07.684 INFO:teuthology.orchestra.run.smithi194.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 586 kB/s | 59 kB 00:00 2024-04-12T05:41:07.743 INFO:teuthology.orchestra.run.smithi194.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 897 kB/s | 75 kB 00:00 2024-04-12T05:41:07.785 INFO:teuthology.orchestra.run.smithi194.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2024-04-12T05:41:07.885 INFO:teuthology.orchestra.run.smithi194.stdout:(21/23): librbd1-18.2.1-0.el8.x86_64.rpm 36 MB/s | 3.6 MB 00:00 2024-04-12T05:41:07.919 INFO:teuthology.orchestra.run.smithi194.stdout:(22/23): librdkafka-1.6.1-1.el8.x86_64.rpm 474 kB/s | 677 kB 00:01 2024-04-12T05:41:07.968 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:41:07.970 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:07.970 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw x86_64 2:18.2.1-0.el8 ceph 13 M 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrading: 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: librados2 x86_64 2:18.2.1-0.el8 ceph 3.7 M 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: librbd1 x86_64 2:18.2.1-0.el8 ceph 3.6 M 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base x86_64 2:18.2.1-0.el8 ceph 5.4 M 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common x86_64 2:18.2.1-0.el8 ceph 22 M 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux x86_64 2:18.2.1-0.el8 ceph 24 k 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2 x86_64 2:18.2.1-0.el8 ceph 774 k 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-12T05:41:07.971 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq x86_64 0.9.0-5.el8 baseos 47 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1 x86_64 2:18.2.1-0.el8 ceph 474 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka x86_64 1.6.1-1.el8 appstream 677 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2 x86_64 2:18.2.1-0.el8 ceph 6.5 M 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el8 ceph 45 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common x86_64 2:18.2.1-0.el8 ceph 123 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs x86_64 2:18.2.1-0.el8 ceph 197 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados x86_64 2:18.2.1-0.el8 ceph 371 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd x86_64 2:18.2.1-0.el8 ceph 365 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw x86_64 2:18.2.1-0.el8 ceph 107 k 2024-04-12T05:41:07.972 INFO:teuthology.orchestra.run.smithi008.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout:Install 21 Packages 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout:Upgrade 2 Packages 2024-04-12T05:41:07.973 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:07.974 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 59 M 2024-04-12T05:41:07.974 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:41:07.977 INFO:teuthology.orchestra.run.smithi194.stdout:(23/23): librados2-18.2.1-0.el8.x86_64.rpm 16 MB/s | 3.7 MB 00:00 2024-04-12T05:41:07.979 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:41:07.980 INFO:teuthology.orchestra.run.smithi194.stdout:Total 24 MB/s | 59 MB 00:02 2024-04-12T05:41:08.023 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:41:08.069 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:41:08.069 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:41:08.443 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:41:08.446 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:41:08.554 INFO:teuthology.orchestra.run.smithi008.stdout:(1/23): ceph-base-18.2.1-0.el8.x86_64.rpm 32 MB/s | 5.4 MB 00:00 2024-04-12T05:41:08.579 INFO:teuthology.orchestra.run.smithi008.stdout:(2/23): ceph-selinux-18.2.1-0.el8.x86_64.rpm 978 kB/s | 24 kB 00:00 2024-04-12T05:41:08.621 INFO:teuthology.orchestra.run.smithi008.stdout:(3/23): libcephfs2-18.2.1-0.el8.x86_64.rpm 18 MB/s | 774 kB 00:00 2024-04-12T05:41:08.663 INFO:teuthology.orchestra.run.smithi008.stdout:(4/23): libradosstriper1-18.2.1-0.el8.x86_64.rp 11 MB/s | 474 kB 00:00 2024-04-12T05:41:08.796 INFO:teuthology.orchestra.run.smithi008.stdout:(5/23): librgw2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 6.5 MB 00:00 2024-04-12T05:41:08.821 INFO:teuthology.orchestra.run.smithi008.stdout:(6/23): python3-ceph-argparse-18.2.1-0.el8.x86_ 1.7 MB/s | 45 kB 00:00 2024-04-12T05:41:08.847 INFO:teuthology.orchestra.run.smithi008.stdout:(7/23): python3-ceph-common-18.2.1-0.el8.x86_64 4.8 MB/s | 123 kB 00:00 2024-04-12T05:41:08.873 INFO:teuthology.orchestra.run.smithi008.stdout:(8/23): python3-cephfs-18.2.1-0.el8.x86_64.rpm 7.7 MB/s | 197 kB 00:00 2024-04-12T05:41:08.906 INFO:teuthology.orchestra.run.smithi008.stdout:(9/23): python3-rados-18.2.1-0.el8.x86_64.rpm 11 MB/s | 371 kB 00:00 2024-04-12T05:41:08.939 INFO:teuthology.orchestra.run.smithi008.stdout:(10/23): python3-rbd-18.2.1-0.el8.x86_64.rpm 11 MB/s | 365 kB 00:00 2024-04-12T05:41:09.105 INFO:teuthology.orchestra.run.smithi008.stdout:(11/23): ceph-radosgw-18.2.1-0.el8.x86_64.rpm 18 MB/s | 13 MB 00:00 2024-04-12T05:41:09.140 INFO:teuthology.orchestra.run.smithi008.stdout:(12/23): python3-rgw-18.2.1-0.el8.x86_64.rpm 532 kB/s | 107 kB 00:00 2024-04-12T05:41:09.391 INFO:teuthology.orchestra.run.smithi008.stdout:(13/23): ceph-common-18.2.1-0.el8.x86_64.rpm 22 MB/s | 22 MB 00:01 2024-04-12T05:41:09.669 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:41:09.741 INFO:teuthology.orchestra.run.smithi008.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 125 kB/s | 44 kB 00:00 2024-04-12T05:41:09.875 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-12T05:41:09.974 INFO:teuthology.orchestra.run.smithi008.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 311 kB/s | 259 kB 00:00 2024-04-12T05:41:10.089 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-12T05:41:10.091 INFO:teuthology.orchestra.run.smithi008.stdout:(16/23): librabbitmq-0.9.0-5.el8.x86_64.rpm 134 kB/s | 47 kB 00:00 2024-04-12T05:41:10.200 INFO:teuthology.orchestra.run.smithi008.stdout:(17/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2024-04-12T05:41:10.259 INFO:teuthology.orchestra.run.smithi008.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 352 kB/s | 59 kB 00:00 2024-04-12T05:41:10.284 INFO:teuthology.orchestra.run.smithi008.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 899 kB/s | 75 kB 00:00 2024-04-12T05:41:10.294 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-12T05:41:10.359 INFO:teuthology.orchestra.run.smithi008.stdout:(20/23): librados2-18.2.1-0.el8.x86_64.rpm 49 MB/s | 3.7 MB 00:00 2024-04-12T05:41:10.434 INFO:teuthology.orchestra.run.smithi008.stdout:(21/23): librbd1-18.2.1-0.el8.x86_64.rpm 48 MB/s | 3.6 MB 00:00 2024-04-12T05:41:10.585 INFO:teuthology.orchestra.run.smithi008.stdout:(22/23): thrift-0.13.0-2.el8.x86_64.rpm 5.3 MB/s | 1.7 MB 00:00 2024-04-12T05:41:10.618 INFO:teuthology.orchestra.run.smithi008.stdout:(23/23): librdkafka-1.6.1-1.el8.x86_64.rpm 447 kB/s | 677 kB 00:01 2024-04-12T05:41:10.621 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:41:10.622 INFO:teuthology.orchestra.run.smithi008.stdout:Total 22 MB/s | 59 MB 00:02 2024-04-12T05:41:10.668 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:41:10.704 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-12T05:41:10.718 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:41:10.718 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:41:10.877 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:10.985 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:11.086 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:41:11.089 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:41:11.241 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:18.2.1-0.el8.x86_64 5/25 2024-04-12T05:41:11.469 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:11.858 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:12.052 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:12.157 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:12.282 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:41:12.353 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-12T05:41:12.456 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2024-04-12T05:41:12.508 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-12T05:41:12.697 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2024-04-12T05:41:12.705 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 10/25 2024-04-12T05:41:12.892 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-12T05:41:12.946 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:18.2.1-0.el8.x86_64 11/25 2024-04-12T05:41:13.186 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-12T05:41:13.335 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:13.336 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2024-04-12T05:41:13.399 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:13.533 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:13.630 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:13.636 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-12T05:41:13.912 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rados-2:18.2.1-0.el8.x86_64 5/25 2024-04-12T05:41:14.092 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:14.426 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-12T05:41:14.499 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:14.602 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-12T05:41:14.666 INFO:teuthology.orchestra.run.smithi008.stdout: Upgrading : librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:14.673 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-12T05:41:14.733 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:14.876 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:18.2.1-0.el8.x86_64 17/25 2024-04-12T05:41:14.963 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librabbitmq-0.9.0-5.el8.x86_64 8/25 2024-04-12T05:41:15.094 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-12T05:41:15.140 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librdkafka-1.6.1-1.el8.x86_64 9/25 2024-04-12T05:41:15.304 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 10/25 2024-04-12T05:41:15.336 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:18.2.1-0.el8.x86_64 19/25 2024-04-12T05:41:15.544 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cephfs-2:18.2.1-0.el8.x86_64 11/25 2024-04-12T05:41:15.753 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-12T05:41:15.916 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:16.022 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:16.229 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2024-04-12T05:41:17.046 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2024-04-12T05:41:17.224 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-12T05:41:17.304 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librgw2-2:18.2.1-0.el8.x86_64 16/25 2024-04-12T05:41:17.492 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rgw-2:18.2.1-0.el8.x86_64 17/25 2024-04-12T05:41:17.687 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-12T05:41:17.887 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:17.949 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-ceph-common-2:18.2.1-0.el8.x86_64 19/25 2024-04-12T05:41:18.185 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:18.797 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:19.311 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-12T05:41:19.414 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-12T05:41:19.611 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:20.653 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:20.883 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:21.515 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el8.x86_64 20/25 2024-04-12T05:41:21.804 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-12T05:41:21.919 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el8.x86_64 21/25 2024-04-12T05:41:22.132 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:39.038 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:39.038 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-04-12T05:41:39.038 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-04-12T05:41:39.038 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /run 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /dev 2024-04-12T05:41:39.039 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:40.480 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-12T05:41:40.501 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-12T05:41:40.501 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-12T05:41:40.734 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-12T05:41:40.734 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:18.2.1-0.el8.x86_64 1/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:18.2.1-0.el8.x86_64 2/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el8.x86_64 3/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:41.478 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:18.2.1-0.el8.x86_64 5/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 8/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el8.x86_64 9/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:18.2.1-0.el8.x86_64 10/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:18.2.1-0.el8.x86_64 11/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-12T05:41:41.479 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-12T05:41:41.480 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:18.2.1-0.el8.x86_64 24/25 2024-04-12T05:41:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:42.377 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:42.377 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:18.2.1-0.el8.x86_64 librbd1-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.378 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.379 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:42.380 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:41:42.734 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-test 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /sys 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /proc 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /run 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /mnt 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /var/tmp 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /home 2024-04-12T05:41:42.810 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /tmp 2024-04-12T05:41:42.811 INFO:teuthology.orchestra.run.smithi008.stdout:skipping the directory /dev 2024-04-12T05:41:42.811 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:43.463 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:42 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:41:44.293 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-12T05:41:44.315 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el8.x86_64 23/25 2024-04-12T05:41:44.315 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-12T05:41:44.507 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-12T05:41:44.507 INFO:teuthology.orchestra.run.smithi008.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:45.075 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:41:45.076 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:18.2.1-0.el8 ceph 47 M 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:18.2.1-0.el8 ceph 141 k 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:45.077 INFO:teuthology.orchestra.run.smithi194.stdout:Install 6 Packages 2024-04-12T05:41:45.078 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:45.078 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 48 M 2024-04-12T05:41:45.078 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 212 M 2024-04-12T05:41:45.078 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-base-2:18.2.1-0.el8.x86_64 1/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-common-2:18.2.1-0.el8.x86_64 2/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el8.x86_64 3/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-selinux-2:18.2.1-0.el8.x86_64 4/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs2-2:18.2.1-0.el8.x86_64 5/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libradosstriper1-2:18.2.1-0.el8.x86_64 6/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librgw2-2:18.2.1-0.el8.x86_64 7/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el8.x86_64 8/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el8.x86_64 9/25 2024-04-12T05:41:45.205 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cephfs-2:18.2.1-0.el8.x86_64 10/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rados-2:18.2.1-0.el8.x86_64 11/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rbd-2:18.2.1-0.el8.x86_64 12/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rgw-2:18.2.1-0.el8.x86_64 13/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librdkafka-1.6.1-1.el8.x86_64 14/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librabbitmq-0.9.0-5.el8.x86_64 17/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-2:18.2.1-0.el8.x86_64 22/25 2024-04-12T05:41:45.206 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-12T05:41:45.207 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-2:18.2.1-0.el8.x86_64 24/25 2024-04-12T05:41:45.247 INFO:teuthology.orchestra.run.smithi194.stdout:(1/6): libcephsqlite-18.2.1-0.el8.x86_64.rpm 1.2 MB/s | 141 kB 00:00 2024-04-12T05:41:45.780 INFO:teuthology.orchestra.run.smithi194.stdout:(2/6): ceph-test-18.2.1-0.el8.x86_64.rpm 73 MB/s | 47 MB 00:00 2024-04-12T05:41:45.814 INFO:teuthology.orchestra.run.smithi194.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 332 kB/s | 188 kB 00:00 2024-04-12T05:41:45.864 INFO:teuthology.orchestra.run.smithi194.stdout:(4/6): jq-1.6-8.el8.x86_64.rpm 279 kB/s | 203 kB 00:00 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout:Upgraded: 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: librados2-2:18.2.1-0.el8.x86_64 librbd1-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-base-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-common-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-radosgw-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-selinux-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs2-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.079 INFO:teuthology.orchestra.run.smithi008.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: librabbitmq-0.9.0-5.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: libradosstriper1-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: librdkafka-1.6.1-1.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: librgw2-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-argparse-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-ceph-common-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cephfs-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rados-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rbd-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rgw-2:18.2.1-0.el8.x86_64 2024-04-12T05:41:46.080 INFO:teuthology.orchestra.run.smithi008.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-12T05:41:46.081 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:46.081 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:41:46.223 INFO:teuthology.orchestra.run.smithi194.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 173 kB/s | 71 kB 00:00 2024-04-12T05:41:46.364 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-test 2024-04-12T05:41:46.523 INFO:teuthology.orchestra.run.smithi194.stdout:(6/6): socat-1.7.4.1-1.el8.x86_64.rpm 435 kB/s | 323 kB 00:00 2024-04-12T05:41:46.525 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:41:46.525 INFO:teuthology.orchestra.run.smithi194.stdout:Total 33 MB/s | 48 MB 00:01 2024-04-12T05:41:46.568 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:41:46.589 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:41:46.589 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:41:46.887 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:41:46.890 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:41:47.126 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:00:44 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:41:47.382 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:41:47.596 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-12T05:41:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-12T05:41:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-12T05:41:48.071 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-12T05:41:48.242 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-12T05:41:48.421 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-12T05:41:48.886 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test x86_64 2:18.2.1-0.el8 ceph 47 M 2024-04-12T05:41:48.887 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: jq x86_64 1.6-8.el8 appstream 203 k 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite x86_64 2:18.2.1-0.el8 ceph 141 k 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: oniguruma x86_64 6.8.2-3.el8 appstream 188 k 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout:Install 6 Packages 2024-04-12T05:41:48.888 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:48.889 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 48 M 2024-04-12T05:41:48.889 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 212 M 2024-04-12T05:41:48.889 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:41:49.018 INFO:teuthology.orchestra.run.smithi008.stdout:(1/6): libcephsqlite-18.2.1-0.el8.x86_64.rpm 2.1 MB/s | 141 kB 00:00 2024-04-12T05:41:49.683 INFO:teuthology.orchestra.run.smithi008.stdout:(2/6): ceph-test-18.2.1-0.el8.x86_64.rpm 64 MB/s | 47 MB 00:00 2024-04-12T05:41:49.708 INFO:teuthology.orchestra.run.smithi008.stdout:(3/6): oniguruma-6.8.2-3.el8.x86_64.rpm 272 kB/s | 188 kB 00:00 2024-04-12T05:41:49.725 INFO:teuthology.orchestra.run.smithi008.stdout:(4/6): jq-1.6-8.el8.x86_64.rpm 262 kB/s | 203 kB 00:00 2024-04-12T05:41:50.134 INFO:teuthology.orchestra.run.smithi008.stdout:(5/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 166 kB/s | 71 kB 00:00 2024-04-12T05:41:50.417 INFO:teuthology.orchestra.run.smithi008.stdout:(6/6): socat-1.7.4.1-1.el8.x86_64.rpm 440 kB/s | 323 kB 00:00 2024-04-12T05:41:50.419 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:41:50.419 INFO:teuthology.orchestra.run.smithi008.stdout:Total 31 MB/s | 48 MB 00:01 2024-04-12T05:41:50.464 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:41:50.486 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:41:50.487 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:41:50.793 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:41:50.796 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:41:51.278 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:41:51.501 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2024-04-12T05:41:51.707 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2024-04-12T05:41:51.878 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-12T05:41:51.947 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: oniguruma-6.8.2-3.el8.x86_64 3/6 2024-04-12T05:41:52.116 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : jq-1.6-8.el8.x86_64 4/6 2024-04-12T05:41:52.269 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-12T05:41:52.926 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-12T05:41:53.177 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:18.2.1-0.el8.x86_64 1/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:18.2.1-0.el8.x86_64 2/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-12T05:41:53.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:18.2.1-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:18.2.1-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:53.912 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:41:54.154 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph 2024-04-12T05:41:54.887 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:53 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:41:56.515 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:41:56.518 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:56.518 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:41:56.518 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:18.2.1-0.el8 ceph 6.3 k 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:18.2.1-0.el8 ceph 2.3 M 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:18.2.1-0.el8 ceph 1.6 M 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el8 ceph-noarch 242 k 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:18.2.1-0.el8 ceph 4.6 M 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:18.2.1-0.el8 ceph 18 M 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-12T05:41:56.519 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-12T05:41:56.520 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-12T05:41:56.521 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:56.522 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:41:56.522 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:41:56.522 INFO:teuthology.orchestra.run.smithi194.stdout:Install 32 Packages 2024-04-12T05:41:56.522 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:41:56.522 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 30 M 2024-04-12T05:41:56.523 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 119 M 2024-04-12T05:41:56.523 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:41:56.898 INFO:teuthology.orchestra.run.smithi194.stdout:(1/32): ceph-18.2.1-0.el8.x86_64.rpm 146 kB/s | 6.3 kB 00:00 2024-04-12T05:41:56.972 INFO:teuthology.orchestra.run.smithi194.stdout:(2/32): ceph-mgr-18.2.1-0.el8.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-12T05:41:57.024 INFO:teuthology.orchestra.run.smithi194.stdout:(3/32): ceph-mds-18.2.1-0.el8.x86_64.rpm 14 MB/s | 2.3 MB 00:00 2024-04-12T05:41:57.056 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el8.x86_64 5/6 2024-04-12T05:41:57.083 INFO:teuthology.orchestra.run.smithi194.stdout:(4/32): ceph-mgr-modules-core-18.2.1-0.el8.noar 4.1 MB/s | 242 kB 00:00 2024-04-12T05:41:57.141 INFO:teuthology.orchestra.run.smithi194.stdout:(5/32): ceph-mon-18.2.1-0.el8.x86_64.rpm 19 MB/s | 4.6 MB 00:00 2024-04-12T05:41:57.318 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-12T05:41:57.358 INFO:teuthology.orchestra.run.smithi194.stdout:(6/32): ceph-osd-18.2.1-0.el8.x86_64.rpm 46 MB/s | 18 MB 00:00 2024-04-12T05:41:57.606 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el8.x86_64 6/6 2024-04-12T05:41:57.607 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-test-2:18.2.1-0.el8.x86_64 1/6 2024-04-12T05:41:57.607 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephsqlite-2:18.2.1-0.el8.x86_64 2/6 2024-04-12T05:41:57.607 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : jq-1.6-8.el8.x86_64 3/6 2024-04-12T05:41:57.607 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : oniguruma-6.8.2-3.el8.x86_64 4/6 2024-04-12T05:41:57.607 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2024-04-12T05:41:57.617 INFO:teuthology.orchestra.run.smithi194.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 294 kB/s | 157 kB 00:00 2024-04-12T05:41:57.676 INFO:teuthology.orchestra.run.smithi194.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 73 kB/s | 39 kB 00:00 2024-04-12T05:41:57.809 INFO:teuthology.orchestra.run.smithi194.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 230 kB/s | 103 kB 00:00 2024-04-12T05:41:58.026 INFO:teuthology.orchestra.run.smithi194.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 139 kB/s | 57 kB 00:00 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-test-2:18.2.1-0.el8.x86_64 jq-1.6-8.el8.x86_64 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: libcephsqlite-2:18.2.1-0.el8.x86_64 oniguruma-6.8.2-3.el8.x86_64 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: socat-1.7.4.1-1.el8.x86_64 xmlstarlet-1.6.1-20.el8.x86_64 2024-04-12T05:41:58.049 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:41:58.050 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:41:58.326 INFO:teuthology.orchestra.run.smithi194.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 481 kB/s | 249 kB 00:00 2024-04-12T05:41:58.420 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph 2024-04-12T05:41:58.605 INFO:teuthology.orchestra.run.smithi194.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 492 kB/s | 457 kB 00:00 2024-04-12T05:41:58.643 INFO:teuthology.orchestra.run.smithi194.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 1.0 MB/s | 641 kB 00:00 2024-04-12T05:41:58.760 INFO:teuthology.orchestra.run.smithi194.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 251 kB/s | 109 kB 00:00 2024-04-12T05:41:58.919 INFO:teuthology.orchestra.run.smithi194.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 139 kB/s | 44 kB 00:00 2024-04-12T05:41:59.103 INFO:teuthology.orchestra.run.smithi194.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 404 kB/s | 185 kB 00:00 2024-04-12T05:41:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:00:56 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:41:59.203 INFO:teuthology.orchestra.run.smithi194.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-04-12T05:41:59.228 INFO:teuthology.orchestra.run.smithi194.stdout:(18/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 369 kB/s | 173 kB 00:00 2024-04-12T05:41:59.254 INFO:teuthology.orchestra.run.smithi194.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.1 MB/s | 384 kB 00:00 2024-04-12T05:41:59.329 INFO:teuthology.orchestra.run.smithi194.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 140 kB/s | 18 kB 00:00 2024-04-12T05:41:59.354 INFO:teuthology.orchestra.run.smithi194.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 388 kB/s | 49 kB 00:00 2024-04-12T05:41:59.379 INFO:teuthology.orchestra.run.smithi194.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 468 kB/s | 59 kB 00:00 2024-04-12T05:41:59.446 INFO:teuthology.orchestra.run.smithi194.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2024-04-12T05:41:59.481 INFO:teuthology.orchestra.run.smithi194.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 187 kB/s | 19 kB 00:00 2024-04-12T05:41:59.505 INFO:teuthology.orchestra.run.smithi194.stdout:(25/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2024-04-12T05:41:59.564 INFO:teuthology.orchestra.run.smithi194.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 204 kB/s | 24 kB 00:00 2024-04-12T05:41:59.589 INFO:teuthology.orchestra.run.smithi194.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 268 kB/s | 29 kB 00:00 2024-04-12T05:41:59.614 INFO:teuthology.orchestra.run.smithi194.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 253 kB/s | 27 kB 00:00 2024-04-12T05:41:59.690 INFO:teuthology.orchestra.run.smithi194.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.0 MB/s | 255 kB 00:00 2024-04-12T05:41:59.715 INFO:teuthology.orchestra.run.smithi194.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 852 kB/s | 85 kB 00:00 2024-04-12T05:41:59.790 INFO:teuthology.orchestra.run.smithi194.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.2 MB/s | 251 kB 00:00 2024-04-12T05:41:59.815 INFO:teuthology.orchestra.run.smithi194.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 183 kB/s | 23 kB 00:00 2024-04-12T05:41:59.817 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:41:59.818 INFO:teuthology.orchestra.run.smithi194.stdout:Total 9.1 MB/s | 30 MB 00:03 2024-04-12T05:41:59.898 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:41:59.935 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:41:59.935 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:42:00.255 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:42:00.258 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:42:00.930 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:00.933 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:42:00.933 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:00.933 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:42:00.933 INFO:teuthology.orchestra.run.smithi008.stdout: ceph x86_64 2:18.2.1-0.el8 ceph 6.3 k 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds x86_64 2:18.2.1-0.el8 ceph 2.3 M 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr x86_64 2:18.2.1-0.el8 ceph 1.6 M 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el8 ceph-noarch 242 k 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon x86_64 2:18.2.1-0.el8 ceph 4.6 M 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd x86_64 2:18.2.1-0.el8 ceph 18 M 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-12T05:42:00.934 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography x86_64 3.2.1-7.el8 baseos 641 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-12T05:42:00.935 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-12T05:42:00.936 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout:Install 32 Packages 2024-04-12T05:42:00.937 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:00.938 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 30 M 2024-04-12T05:42:00.938 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 119 M 2024-04-12T05:42:00.938 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:42:01.170 INFO:teuthology.orchestra.run.smithi008.stdout:(1/32): ceph-18.2.1-0.el8.x86_64.rpm 114 kB/s | 6.3 kB 00:00 2024-04-12T05:42:01.211 INFO:teuthology.orchestra.run.smithi008.stdout:(2/32): ceph-mgr-18.2.1-0.el8.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-12T05:42:01.270 INFO:teuthology.orchestra.run.smithi008.stdout:(3/32): ceph-mds-18.2.1-0.el8.x86_64.rpm 15 MB/s | 2.3 MB 00:00 2024-04-12T05:42:01.298 INFO:teuthology.orchestra.run.smithi008.stdout:(4/32): ceph-mgr-modules-core-18.2.1-0.el8.noar 9.5 MB/s | 242 kB 00:00 2024-04-12T05:42:01.371 INFO:teuthology.orchestra.run.smithi008.stdout:(5/32): ceph-mon-18.2.1-0.el8.x86_64.rpm 23 MB/s | 4.6 MB 00:00 2024-04-12T05:42:01.444 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:42:01.635 INFO:teuthology.orchestra.run.smithi008.stdout:(6/32): ceph-osd-18.2.1-0.el8.x86_64.rpm 42 MB/s | 18 MB 00:00 2024-04-12T05:42:01.723 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-12T05:42:01.838 INFO:teuthology.orchestra.run.smithi008.stdout:(7/32): python3-mako-1.0.6-14.el8.noarch.rpm 289 kB/s | 157 kB 00:00 2024-04-12T05:42:01.905 INFO:teuthology.orchestra.run.smithi008.stdout:(8/32): python3-markupsafe-0.23-19.el8.x86_64.r 73 kB/s | 39 kB 00:00 2024-04-12T05:42:01.968 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-12T05:42:02.089 INFO:teuthology.orchestra.run.smithi008.stdout:(9/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 228 kB/s | 103 kB 00:00 2024-04-12T05:42:02.218 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-12T05:42:02.247 INFO:teuthology.orchestra.run.smithi008.stdout:(10/32): python3-pytz-2017.2-11.el8.noarch.rpm 139 kB/s | 57 kB 00:00 2024-04-12T05:42:02.508 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-12T05:42:02.589 INFO:teuthology.orchestra.run.smithi008.stdout:(11/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 497 kB/s | 249 kB 00:00 2024-04-12T05:42:02.739 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-12T05:42:02.840 INFO:teuthology.orchestra.run.smithi008.stdout:(12/32): python3-werkzeug-0.12.2-4.el8.noarch.r 490 kB/s | 457 kB 00:00 2024-04-12T05:42:02.865 INFO:teuthology.orchestra.run.smithi008.stdout:(13/32): python3-cryptography-3.2.1-7.el8.x86_6 1.0 MB/s | 641 kB 00:00 2024-04-12T05:42:02.961 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-12T05:42:03.040 INFO:teuthology.orchestra.run.smithi008.stdout:(14/32): python3-pycparser-2.14-14.el8.noarch.r 242 kB/s | 109 kB 00:00 2024-04-12T05:42:03.155 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-12T05:42:03.190 INFO:teuthology.orchestra.run.smithi008.stdout:(15/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 124 kB/s | 44 kB 00:00 2024-04-12T05:42:03.290 INFO:teuthology.orchestra.run.smithi008.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 435 kB/s | 185 kB 00:00 2024-04-12T05:42:03.366 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-12T05:42:03.466 INFO:teuthology.orchestra.run.smithi008.stdout:(17/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 406 kB/s | 173 kB 00:00 2024-04-12T05:42:03.491 INFO:teuthology.orchestra.run.smithi008.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 55 kB/s | 11 kB 00:00 2024-04-12T05:42:03.584 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-12T05:42:03.716 INFO:teuthology.orchestra.run.smithi008.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 731 kB/s | 384 kB 00:00 2024-04-12T05:42:03.741 INFO:teuthology.orchestra.run.smithi008.stdout:(20/32): python3-jaraco-functools-2.0-4.el8.noa 64 kB/s | 18 kB 00:00 2024-04-12T05:42:03.820 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-12T05:42:03.842 INFO:teuthology.orchestra.run.smithi008.stdout:(21/32): python3-logutils-0.3.5-11.el8.noarch.r 139 kB/s | 49 kB 00:00 2024-04-12T05:42:04.066 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-12T05:42:04.067 INFO:teuthology.orchestra.run.smithi008.stdout:(22/32): python3-more-itertools-7.2.0-3.el8.noa 167 kB/s | 59 kB 00:00 2024-04-12T05:42:04.109 INFO:teuthology.orchestra.run.smithi008.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 58 kB/s | 16 kB 00:00 2024-04-12T05:42:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 538 kB/s | 283 kB 00:00 2024-04-12T05:42:04.323 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-12T05:42:04.335 INFO:teuthology.orchestra.run.smithi008.stdout:(25/32): python3-simplegeneric-0.8.1-17.el8.noa 70 kB/s | 19 kB 00:00 2024-04-12T05:42:04.377 INFO:teuthology.orchestra.run.smithi008.stdout:(26/32): python3-singledispatch-3.4.0.3-18.el8. 89 kB/s | 24 kB 00:00 2024-04-12T05:42:04.535 INFO:teuthology.orchestra.run.smithi008.stdout:(27/32): python3-tempora-1.14.1-5.el8.noarch.rp 109 kB/s | 29 kB 00:00 2024-04-12T05:42:04.603 INFO:teuthology.orchestra.run.smithi008.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 103 kB/s | 27 kB 00:00 2024-04-12T05:42:04.741 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-12T05:42:04.878 INFO:teuthology.orchestra.run.smithi008.stdout:(29/32): python3-waitress-1.4.3-1.el8.noarch.rp 510 kB/s | 255 kB 00:00 2024-04-12T05:42:04.961 INFO:teuthology.orchestra.run.smithi008.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 238 kB/s | 85 kB 00:00 2024-04-12T05:42:05.029 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-12T05:42:05.036 INFO:teuthology.orchestra.run.smithi008.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 501 kB/s | 251 kB 00:00 2024-04-12T05:42:05.153 INFO:teuthology.orchestra.run.smithi008.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 83 kB/s | 23 kB 00:00 2024-04-12T05:42:05.157 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:05.157 INFO:teuthology.orchestra.run.smithi008.stdout:Total 7.1 MB/s | 30 MB 00:04 2024-04-12T05:42:05.241 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:42:05.251 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-12T05:42:05.282 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:42:05.282 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:42:05.524 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-12T05:42:05.611 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:42:05.614 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:42:05.761 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-12T05:42:06.105 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-12T05:42:06.390 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-12T05:42:06.562 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-12T05:42:06.773 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:42:06.798 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-12T05:42:07.035 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-12T05:42:07.065 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-12T05:42:07.268 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-12T05:42:07.322 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-12T05:42:07.529 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-12T05:42:07.548 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2024-04-12T05:42:07.800 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 4/32 2024-04-12T05:42:07.840 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-12T05:42:08.023 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2024-04-12T05:42:08.166 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-12T05:42:08.222 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2024-04-12T05:42:08.430 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2024-04-12T05:42:08.570 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 27/32 2024-04-12T05:42:08.649 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2024-04-12T05:42:08.842 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-12T05:42:08.861 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2024-04-12T05:42:09.073 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2024-04-12T05:42:09.335 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2024-04-12T05:42:09.586 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2024-04-12T05:42:10.027 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 13/32 2024-04-12T05:42:10.347 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cryptography-3.2.1-7.el8.x86_64 14/32 2024-04-12T05:42:10.422 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-12T05:42:10.568 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2024-04-12T05:42:10.650 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-12T05:42:10.833 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2024-04-12T05:42:11.070 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-12T05:42:11.108 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-12T05:42:11.334 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-12T05:42:11.432 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2024-04-12T05:42:11.549 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-12T05:42:11.659 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2024-04-12T05:42:11.742 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-12T05:42:11.759 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-12T05:42:11.839 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 20/32 2024-04-12T05:42:11.908 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-12T05:42:12.056 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-12T05:42:12.303 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:18.2.1-0.el8.x86_64 1/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:18.2.1-0.el8.x86_64 2/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:18.2.1-0.el8.x86_64 3/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:18.2.1-0.el8.x86_64 4/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:18.2.1-0.el8.x86_64 5/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 6/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-12T05:42:12.315 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-12T05:42:12.316 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-12T05:42:12.317 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-12T05:42:12.317 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-12T05:42:12.317 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-12T05:42:12.317 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-12T05:42:12.318 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-12T05:42:12.318 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-12T05:42:12.318 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-12T05:42:12.318 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-12T05:42:12.565 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-12T05:42:12.830 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2024-04-12T05:42:13.150 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-12T05:42:13.150 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:13.150 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:42:13.150 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:13.150 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-12T05:42:13.151 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-2.6-1.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-12T05:42:13.152 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-12T05:42:13.153 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-12T05:42:13.153 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-12T05:42:13.153 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:13.153 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:13.166 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 25/32 2024-04-12T05:42:13.456 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-base 2024-04-12T05:42:13.482 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-12T05:42:13.886 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 27/32 2024-04-12T05:42:14.106 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-12T05:42:14.239 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:13 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:15.789 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el8.x86_64 28/32 2024-04-12T05:42:15.822 INFO:teuthology.orchestra.run.smithi194.stdout:Package ceph-base-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:42:15.871 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:15.872 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:42:15.872 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:15.964 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-12T05:42:15.979 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install cephadm 2024-04-12T05:42:16.453 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el8.x86_64 29/32 2024-04-12T05:42:16.667 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-12T05:42:16.709 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:15 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:16.900 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el8.x86_64 30/32 2024-04-12T05:42:17.109 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-12T05:42:17.128 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el8.x86_64 31/32 2024-04-12T05:42:17.300 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-12T05:42:17.719 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-2:18.2.1-0.el8.x86_64 32/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-2:18.2.1-0.el8.x86_64 1/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mds-2:18.2.1-0.el8.x86_64 2/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-2:18.2.1-0.el8.x86_64 3/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mon-2:18.2.1-0.el8.x86_64 4/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-osd-2:18.2.1-0.el8.x86_64 5/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 6/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 7/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 10/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 12/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cryptography-3.2.1-7.el8.x86_64 13/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2024-04-12T05:42:17.720 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2024-04-12T05:42:17.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 29/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2024-04-12T05:42:17.722 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2024-04-12T05:42:18.341 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:18.2.1-0.el8 ceph-noarch 171 k 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:42:18.342 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:18.343 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:42:18.343 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:18.343 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 171 k 2024-04-12T05:42:18.343 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 680 k 2024-04-12T05:42:18.343 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:42:18.410 INFO:teuthology.orchestra.run.smithi194.stdout:cephadm-18.2.1-0.el8.noarch.rpm 2.5 MB/s | 171 kB 00:00 2024-04-12T05:42:18.410 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:18.410 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.4 MB/s | 171 kB 00:00 2024-04-12T05:42:18.411 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:42:18.418 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:42:18.418 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:42:18.420 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:42:18.424 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:42:18.471 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:42:19.006 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2024-04-12T05:42:19.006 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:19.006 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:42:19.006 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mds-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-modules-core-2:18.2.1-0.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mon-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-osd-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cryptography-3.2.1-7.el8.x86_64 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-12T05:42:19.007 INFO:teuthology.orchestra.run.smithi008.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-portend-2.6-1.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-12T05:42:19.008 INFO:teuthology.orchestra.run.smithi008.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-12T05:42:19.009 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:19.009 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:19.043 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:19.253 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:19.278 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-base 2024-04-12T05:42:19.482 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:19.904 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:19.904 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:19.904 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:42:19.905 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:18.2.1-0.el8.noarch 2024-04-12T05:42:19.905 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:19.905 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:20.046 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:17 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:21.741 INFO:teuthology.orchestra.run.smithi008.stdout:Package ceph-base-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:42:21.793 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:21.794 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:42:21.794 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:21.857 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-immutable-object-cache 2024-04-12T05:42:21.892 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install cephadm 2024-04-12T05:42:22.600 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:21 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:22.658 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:19 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:24.235 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repo Size 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el8 ceph 165 k 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:42:24.236 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:24.237 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 165 k 2024-04-12T05:42:24.237 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 503 k 2024-04-12T05:42:24.237 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:42:24.291 INFO:teuthology.orchestra.run.smithi194.stdout:ceph-immutable-object-cache-18.2.1-0.el8.x86_64 3.0 MB/s | 165 kB 00:00 2024-04-12T05:42:24.292 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:24.292 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.9 MB/s | 165 kB 00:00 2024-04-12T05:42:24.292 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:42:24.303 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:42:24.303 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:42:24.350 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:42:24.353 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:42:24.399 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:24.400 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:24.400 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:42:24.400 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:24.400 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm noarch 2:18.2.1-0.el8 ceph-noarch 171 k 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 171 k 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 680 k 2024-04-12T05:42:24.401 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:42:24.453 INFO:teuthology.orchestra.run.smithi008.stdout:cephadm-18.2.1-0.el8.noarch.rpm 3.2 MB/s | 171 kB 00:00 2024-04-12T05:42:24.454 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:24.454 INFO:teuthology.orchestra.run.smithi008.stdout:Total 3.1 MB/s | 171 kB 00:00 2024-04-12T05:42:24.454 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:42:24.459 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:42:24.462 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:42:24.462 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:42:24.465 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:42:24.468 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:42:24.513 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:42:24.632 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:25.057 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:25.065 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:25.242 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:25.484 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:25.496 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:25.733 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-mgr 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : cephadm-2:18.2.1-0.el8.noarch 1/1 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout: cephadm-2:18.2.1-0.el8.noarch 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:25.887 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:26.125 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-immutable-object-cache 2024-04-12T05:42:26.479 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:25 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:26.884 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:23 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:28.060 INFO:teuthology.orchestra.run.smithi194.stdout:Package ceph-mgr-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:42:28.109 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:28.110 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:42:28.111 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:28.214 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-mgr-dashboard 2024-04-12T05:42:28.616 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:28.616 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:28.616 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repo Size 2024-04-12T05:42:28.616 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el8 ceph 165 k 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:42:28.617 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:28.618 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 165 k 2024-04-12T05:42:28.618 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 503 k 2024-04-12T05:42:28.618 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:42:28.675 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-immutable-object-cache-18.2.1-0.el8.x86_64 2.9 MB/s | 165 kB 00:00 2024-04-12T05:42:28.675 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:28.675 INFO:teuthology.orchestra.run.smithi008.stdout:Total 2.8 MB/s | 165 kB 00:00 2024-04-12T05:42:28.676 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:42:28.688 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:42:28.688 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:42:28.738 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:42:28.741 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:42:28.854 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:42:28.936 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:27 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:30.240 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:30.577 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el8 ceph-noarch 1.8 M 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el8 ceph-noarch 23 k 2024-04-12T05:42:30.578 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el8 ceph-noarch 14 k 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout:Install 6 Packages 2024-04-12T05:42:30.579 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:30.580 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 2.1 M 2024-04-12T05:42:30.580 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 8.5 M 2024-04-12T05:42:30.580 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:42:30.684 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:30.877 INFO:teuthology.orchestra.run.smithi194.stdout:(1/6): ceph-grafana-dashboards-18.2.1-0.el8.noa 402 kB/s | 23 kB 00:00 2024-04-12T05:42:30.902 INFO:teuthology.orchestra.run.smithi194.stdout:(2/6): ceph-prometheus-alerts-18.2.1-0.el8.noar 176 kB/s | 14 kB 00:00 2024-04-12T05:42:30.953 INFO:teuthology.orchestra.run.smithi194.stdout:(3/6): ceph-mgr-dashboard-18.2.1-0.el8.noarch.r 13 MB/s | 1.8 MB 00:00 2024-04-12T05:42:31.069 INFO:teuthology.orchestra.run.smithi194.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 201 kB/s | 34 kB 00:00 2024-04-12T05:42:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:42:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:31.086 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:42:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-immutable-object-cache-2:18.2.1-0.el8.x86_64 2024-04-12T05:42:31.086 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:31.087 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:31.161 INFO:teuthology.orchestra.run.smithi194.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 943 kB/s | 196 kB 00:00 2024-04-12T05:42:31.278 INFO:teuthology.orchestra.run.smithi194.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 106 kB/s | 43 kB 00:00 2024-04-12T05:42:31.281 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:31.281 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.9 MB/s | 2.1 MB 00:00 2024-04-12T05:42:31.292 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:42:31.302 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:42:31.302 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:42:31.312 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr 2024-04-12T05:42:31.387 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:42:31.390 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:42:31.694 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:42:31.937 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-12T05:42:32.084 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:29 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:32.207 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-12T05:42:32.463 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-12T05:42:32.595 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 4/6 2024-04-12T05:42:33.070 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 5/6 2024-04-12T05:42:33.337 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-12T05:42:33.623 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-12T05:42:33.624 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 1/6 2024-04-12T05:42:33.624 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2/6 2024-04-12T05:42:33.624 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 3/6 2024-04-12T05:42:33.624 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-12T05:42:33.624 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-12T05:42:33.776 INFO:teuthology.orchestra.run.smithi008.stdout:Package ceph-mgr-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:42:33.828 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:33.829 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:42:33.830 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:33.928 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-dashboard 2024-04-12T05:42:34.048 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:34.049 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:42:34.181 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-12T05:42:34.682 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:31 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:34.912 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:01:33 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:42:36.428 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el8 ceph-noarch 1.8 M 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el8 ceph-noarch 23 k 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el8 ceph-noarch 14 k 2024-04-12T05:42:36.429 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout:Install 6 Packages 2024-04-12T05:42:36.430 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 2.1 M 2024-04-12T05:42:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 8.5 M 2024-04-12T05:42:36.431 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:42:36.549 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:42:36.550 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-12T05:42:36.550 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:42:36.550 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el8 ceph-noarch 7.4 M 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:36.551 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:42:36.552 INFO:teuthology.orchestra.run.smithi194.stdout:============================================================================================ 2024-04-12T05:42:36.552 INFO:teuthology.orchestra.run.smithi194.stdout:Install 8 Packages 2024-04-12T05:42:36.552 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:42:36.552 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 36 M 2024-04-12T05:42:36.552 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 210 M 2024-04-12T05:42:36.553 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:42:36.729 INFO:teuthology.orchestra.run.smithi008.stdout:(1/6): ceph-grafana-dashboards-18.2.1-0.el8.noa 312 kB/s | 23 kB 00:00 2024-04-12T05:42:36.753 INFO:teuthology.orchestra.run.smithi008.stdout:(2/6): ceph-prometheus-alerts-18.2.1-0.el8.noar 147 kB/s | 14 kB 00:00 2024-04-12T05:42:36.804 INFO:teuthology.orchestra.run.smithi008.stdout:(3/6): ceph-mgr-dashboard-18.2.1-0.el8.noarch.r 12 MB/s | 1.8 MB 00:00 2024-04-12T05:42:36.963 INFO:teuthology.orchestra.run.smithi194.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.1-0.e 47 MB/s | 7.4 MB 00:00 2024-04-12T05:42:37.039 INFO:teuthology.orchestra.run.smithi008.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 118 kB/s | 34 kB 00:00 2024-04-12T05:42:37.063 INFO:teuthology.orchestra.run.smithi008.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 760 kB/s | 196 kB 00:00 2024-04-12T05:42:37.096 INFO:teuthology.orchestra.run.smithi008.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 116 kB/s | 43 kB 00:00 2024-04-12T05:42:37.099 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:42:37.099 INFO:teuthology.orchestra.run.smithi008.stdout:Total 3.1 MB/s | 2.1 MB 00:00 2024-04-12T05:42:37.110 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:42:37.120 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:42:37.120 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:42:37.218 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:42:37.221 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:42:37.482 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:42:37.706 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2024-04-12T05:42:37.966 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2024-04-12T05:42:38.192 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2024-04-12T05:42:38.321 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 4/6 2024-04-12T05:42:38.700 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 5/6 2024-04-12T05:42:38.910 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 6/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 1/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 3/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2024-04-12T05:42:39.197 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2024-04-12T05:42:39.610 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2024-04-12T05:42:39.610 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-grafana-dashboards-2:18.2.1-0.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-dashboard-2:18.2.1-0.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-prometheus-alerts-2:18.2.1-0.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:39.611 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:42:39.728 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-12T05:42:40.489 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:01:37 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:42:42.242 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el8 ceph-noarch 7.4 M 2024-04-12T05:42:42.243 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran x86_64 8.5.0-21.el8 baseos 645 k 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath x86_64 8.5.0-21.el8 baseos 172 k 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy x86_64 1.0.0-21.module_el8+762+77bd8591 appstream 14 M 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout:============================================================================================ 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout:Install 8 Packages 2024-04-12T05:42:42.244 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:42:42.245 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 36 M 2024-04-12T05:42:42.245 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 210 M 2024-04-12T05:42:42.245 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:42:42.593 INFO:teuthology.orchestra.run.smithi194.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 679 kB/s | 3.7 MB 00:05 2024-04-12T05:42:42.629 INFO:teuthology.orchestra.run.smithi008.stdout:(1/8): ceph-mgr-diskprediction-local-18.2.1-0.e 52 MB/s | 7.4 MB 00:00 2024-04-12T05:42:43.595 INFO:teuthology.orchestra.run.smithi194.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 225 kB/s | 225 kB 00:01 2024-04-12T05:42:43.962 INFO:teuthology.orchestra.run.smithi194.stdout:(4/8): openblas-0.3.15-4.el8.x86_64.rpm 676 kB/s | 4.7 MB 00:07 2024-04-12T05:42:44.546 INFO:teuthology.orchestra.run.smithi194.stdout:(5/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.1 MB/s | 645 kB 00:00 2024-04-12T05:42:44.646 INFO:teuthology.orchestra.run.smithi194.stdout:(6/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 634 kB/s | 4.9 MB 00:07 2024-04-12T05:42:44.996 INFO:teuthology.orchestra.run.smithi194.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 383 kB/s | 172 kB 00:00 2024-04-12T05:42:48.047 INFO:teuthology.orchestra.run.smithi008.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 705 kB/s | 3.7 MB 00:05 2024-04-12T05:42:48.697 INFO:teuthology.orchestra.run.smithi008.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 346 kB/s | 225 kB 00:00 2024-04-12T05:42:49.531 INFO:teuthology.orchestra.run.smithi008.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 706 kB/s | 4.9 MB 00:07 2024-04-12T05:42:49.815 INFO:teuthology.orchestra.run.smithi008.stdout:(5/8): openblas-0.3.15-4.el8.x86_64.rpm 660 kB/s | 4.7 MB 00:07 2024-04-12T05:42:50.142 INFO:teuthology.orchestra.run.smithi008.stdout:(6/8): libgfortran-8.5.0-21.el8.x86_64.rpm 1.0 MB/s | 645 kB 00:00 2024-04-12T05:42:50.240 INFO:teuthology.orchestra.run.smithi008.stdout:(7/8): libquadmath-8.5.0-21.el8.x86_64.rpm 406 kB/s | 172 kB 00:00 2024-04-12T05:43:04.744 INFO:teuthology.orchestra.run.smithi194.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 686 kB/s | 14 MB 00:21 2024-04-12T05:43:04.745 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:04.746 INFO:teuthology.orchestra.run.smithi194.stdout:Total 1.3 MB/s | 36 MB 00:28 2024-04-12T05:43:04.976 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:04.994 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:04.994 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:05.393 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:05.396 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:05.983 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:06.141 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-12T05:43:06.284 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-12T05:43:06.442 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-12T05:43:07.043 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-12T05:43:07.199 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:07.827 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:07.999 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-12T05:43:08.689 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-12T05:43:09.044 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-12T05:43:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-12T05:43:11.075 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-12T05:43:11.239 INFO:teuthology.orchestra.run.smithi008.stdout:(8/8): python3-scipy-1.0.0-21.module_el8+762+77 644 kB/s | 14 MB 00:22 2024-04-12T05:43:11.240 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:11.240 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.2 MB/s | 36 MB 00:28 2024-04-12T05:43:11.482 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:11.501 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:11.502 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:11.900 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:11.903 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:12.324 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-12T05:43:12.512 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:12.557 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-12T05:43:12.676 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-12T05:43:12.819 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libquadmath-8.5.0-21.el8.x86_64 1/8 2024-04-12T05:43:12.907 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 1/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-12T05:43:12.908 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-12T05:43:12.968 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-12T05:43:13.347 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-12T05:43:13.348 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-12T05:43:13.348 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-12T05:43:13.348 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:13.348 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:13.492 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-mgr-rook 2024-04-12T05:43:13.597 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libgfortran-8.5.0-21.el8.x86_64 2/8 2024-04-12T05:43:13.784 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:14.220 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:13 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:14.433 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:14.559 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-12T05:43:15.363 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2024-04-12T05:43:15.728 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-12T05:43:15.848 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el8 ceph-noarch 50 k 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-12T05:43:15.850 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout:Install 13 Packages 2024-04-12T05:43:15.851 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:15.852 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 1.9 M 2024-04-12T05:43:15.852 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 25 M 2024-04-12T05:43:15.852 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:43:15.976 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-12T05:43:16.198 INFO:teuthology.orchestra.run.smithi194.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el8.noarch.rpm 1.0 MB/s | 50 kB 00:00 2024-04-12T05:43:16.457 INFO:teuthology.orchestra.run.smithi194.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 65 kB/s | 20 kB 00:00 2024-04-12T05:43:16.482 INFO:teuthology.orchestra.run.smithi194.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 80 kB/s | 27 kB 00:00 2024-04-12T05:43:16.766 INFO:teuthology.orchestra.run.smithi194.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 223 kB/s | 126 kB 00:00 2024-04-12T05:43:16.916 INFO:teuthology.orchestra.run.smithi194.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 357 kB/s | 155 kB 00:00 2024-04-12T05:43:17.017 INFO:teuthology.orchestra.run.smithi194.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 197 kB/s | 110 kB 00:00 2024-04-12T05:43:17.117 INFO:teuthology.orchestra.run.smithi194.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 122 kB/s | 43 kB 00:00 2024-04-12T05:43:17.342 INFO:teuthology.orchestra.run.smithi194.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 77 kB/s | 33 kB 00:00 2024-04-12T05:43:17.392 INFO:teuthology.orchestra.run.smithi194.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 44 kB/s | 16 kB 00:00 2024-04-12T05:43:17.626 INFO:teuthology.orchestra.run.smithi194.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 161 kB/s | 81 kB 00:00 2024-04-12T05:43:17.660 INFO:teuthology.orchestra.run.smithi194.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 240 kB/s | 64 kB 00:00 2024-04-12T05:43:17.793 INFO:teuthology.orchestra.run.smithi194.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 367 kB/s | 61 kB 00:00 2024-04-12T05:43:17.893 INFO:teuthology.orchestra.run.smithi194.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.1 MB/s | 1.2 MB 00:00 2024-04-12T05:43:17.897 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:17.897 INFO:teuthology.orchestra.run.smithi194.stdout:Total 968 kB/s | 1.9 MB 00:02 2024-04-12T05:43:17.953 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:17.969 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:17.969 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:18.031 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-12T05:43:18.058 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:18.061 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:18.658 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:18.908 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-12T05:43:19.090 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-12T05:43:19.309 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-12T05:43:19.364 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 7/8 2024-04-12T05:43:19.563 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-12T05:43:19.675 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-12T05:43:19.805 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 8/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 1/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_6 6/8 2024-04-12T05:43:20.009 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libgfortran-8.5.0-21.el8.x86_64 7/8 2024-04-12T05:43:20.054 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-12T05:43:20.362 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-12T05:43:20.466 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libquadmath-8.5.0-21.el8.x86_64 8/8 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el8.noarch 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: libgfortran-8.5.0-21.el8.x86_64 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: libquadmath-8.5.0-21.el8.x86_64 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-0.3.15-4.el8.x86_64 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: openblas-threads-0.3.15-4.el8.x86_64 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-12T05:43:20.467 INFO:teuthology.orchestra.run.smithi008.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-12T05:43:20.468 INFO:teuthology.orchestra.run.smithi008.stdout: python3-scipy-1.0.0-21.module_el8+762+77bd8591.x86_64 2024-04-12T05:43:20.468 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:20.468 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:20.594 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-rook 2024-04-12T05:43:20.620 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-12T05:43:21.352 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:18 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:21.625 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-12T05:43:21.920 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-12T05:43:22.154 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-12T05:43:22.384 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-12T05:43:22.613 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el8.noarch 1/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-12T05:43:22.948 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-12T05:43:22.949 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-12T05:43:23.086 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el8 ceph-noarch 50 k 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-12T05:43:23.088 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:23.089 INFO:teuthology.orchestra.run.smithi008.stdout:Install 13 Packages 2024-04-12T05:43:23.090 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:23.090 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 1.9 M 2024-04-12T05:43:23.090 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 25 M 2024-04-12T05:43:23.090 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:18.2.1-0.el8.noarch 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-12T05:43:23.419 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:23.420 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:23.531 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-mgr-cephadm 2024-04-12T05:43:23.579 INFO:teuthology.orchestra.run.smithi008.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el8.noarch.rpm 791 kB/s | 50 kB 00:00 2024-04-12T05:43:23.796 INFO:teuthology.orchestra.run.smithi008.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 71 kB/s | 20 kB 00:00 2024-04-12T05:43:23.821 INFO:teuthology.orchestra.run.smithi008.stdout:(3/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 87 kB/s | 27 kB 00:00 2024-04-12T05:43:24.089 INFO:teuthology.orchestra.run.smithi008.stdout:(4/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 248 kB/s | 126 kB 00:00 2024-04-12T05:43:24.239 INFO:teuthology.orchestra.run.smithi008.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 372 kB/s | 155 kB 00:00 2024-04-12T05:43:24.262 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:23 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:24.289 INFO:teuthology.orchestra.run.smithi008.stdout:(6/13): python3-pyasn1-modules-0.3.7-6.el8.noar 223 kB/s | 110 kB 00:00 2024-04-12T05:43:24.339 INFO:teuthology.orchestra.run.smithi008.stdout:(7/13): python3-cachetools-3.1.1-4.el8.noarch.r 329 kB/s | 33 kB 00:00 2024-04-12T05:43:24.381 INFO:teuthology.orchestra.run.smithi008.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 178 kB/s | 16 kB 00:00 2024-04-12T05:43:24.423 INFO:teuthology.orchestra.run.smithi008.stdout:(9/13): python3-google-auth-1.1.1-10.el8.noarch 972 kB/s | 81 kB 00:00 2024-04-12T05:43:24.448 INFO:teuthology.orchestra.run.smithi008.stdout:(10/13): python3-requests-oauthlib-1.0.0-1.el8. 118 kB/s | 43 kB 00:00 2024-04-12T05:43:24.474 INFO:teuthology.orchestra.run.smithi008.stdout:(11/13): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2024-04-12T05:43:24.582 INFO:teuthology.orchestra.run.smithi008.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 5.8 MB/s | 1.2 MB 00:00 2024-04-12T05:43:24.607 INFO:teuthology.orchestra.run.smithi008.stdout:(13/13): python3-websocket-client-0.56.0-5.el8. 386 kB/s | 61 kB 00:00 2024-04-12T05:43:24.608 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:24.609 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2024-04-12T05:43:24.668 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:24.685 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:24.686 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:24.780 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:24.783 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:25.320 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:25.554 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-12T05:43:25.743 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2024-04-12T05:43:25.912 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el8 ceph-noarch 132 k 2024-04-12T05:43:25.913 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout:Install 5 Packages 2024-04-12T05:43:25.914 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:25.915 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 6.0 M 2024-04-12T05:43:25.915 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 26 M 2024-04-12T05:43:25.915 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:43:26.004 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-rsa-4.9-2.el8.noarch 3/13 2024-04-12T05:43:26.249 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2024-04-12T05:43:26.283 INFO:teuthology.orchestra.run.smithi194.stdout:(1/5): ceph-mgr-cephadm-18.2.1-0.el8.noarch.rpm 2.5 MB/s | 132 kB 00:00 2024-04-12T05:43:26.524 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2024-04-12T05:43:26.767 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2024-04-12T05:43:26.892 INFO:teuthology.orchestra.run.smithi194.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 782 kB/s | 476 kB 00:00 2024-04-12T05:43:26.992 INFO:teuthology.orchestra.run.smithi194.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 602 kB/s | 60 kB 00:00 2024-04-12T05:43:27.096 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2024-04-12T05:43:27.350 INFO:teuthology.orchestra.run.smithi194.stdout:(4/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 517 kB/s | 578 kB 00:01 2024-04-12T05:43:27.366 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2024-04-12T05:43:28.462 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2024-04-12T05:43:28.773 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2024-04-12T05:43:28.982 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2024-04-12T05:43:29.196 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2024-04-12T05:43:29.407 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-12T05:43:29.720 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el8.noarch 13/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el8.noarch 1/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2024-04-12T05:43:29.721 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 12/13 2024-04-12T05:43:30.184 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2024-04-12T05:43:30.184 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:30.184 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-rook-2:18.2.1-0.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-12T05:43:30.185 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:30.186 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:30.305 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-mgr-cephadm 2024-04-12T05:43:31.067 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:28 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:32.816 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:32.817 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:32.817 INFO:teuthology.orchestra.run.smithi008.stdout: Package Arch Version Repository Size 2024-04-12T05:43:32.817 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el8 ceph-noarch 132 k 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: python3-jinja2 noarch 2.10.1-4.el8 appstream 578 k 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout:Install 5 Packages 2024-04-12T05:43:32.818 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:32.819 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 6.0 M 2024-04-12T05:43:32.819 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 26 M 2024-04-12T05:43:32.819 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:43:33.000 INFO:teuthology.orchestra.run.smithi008.stdout:(1/5): ceph-mgr-cephadm-18.2.1-0.el8.noarch.rpm 2.9 MB/s | 132 kB 00:00 2024-04-12T05:43:33.025 INFO:teuthology.orchestra.run.smithi194.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 722 kB/s | 4.8 MB 00:06 2024-04-12T05:43:33.026 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:33.026 INFO:teuthology.orchestra.run.smithi194.stdout:Total 864 kB/s | 6.0 MB 00:07 2024-04-12T05:43:33.078 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:33.090 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:33.091 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:33.200 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:33.202 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:33.259 INFO:teuthology.orchestra.run.smithi008.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2024-04-12T05:43:33.317 INFO:teuthology.orchestra.run.smithi008.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 1.0 MB/s | 60 kB 00:00 2024-04-12T05:43:33.467 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:33.808 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-12T05:43:34.059 INFO:teuthology.orchestra.run.smithi008.stdout:(4/5): python3-jinja2-2.10.1-4.el8.noarch.rpm 524 kB/s | 578 kB 00:01 2024-04-12T05:43:34.847 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-12T05:43:35.265 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-12T05:43:35.586 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-12T05:43:35.786 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-12T05:43:36.074 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-12T05:43:36.074 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 1/5 2024-04-12T05:43:36.074 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-12T05:43:36.074 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-12T05:43:36.074 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch python3-asyncssh-2.7.0-2.el8.noarch 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.5.1-7.el8.noarch python3-jinja2-2.10.1-4.el8.noarch 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:36.475 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:36.605 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-fuse 2024-04-12T05:43:37.340 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:36 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:38.972 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:18.2.1-0.el8 ceph 851 k 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout:Install 2 Packages 2024-04-12T05:43:38.973 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:38.974 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 934 k 2024-04-12T05:43:38.974 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 3.0 M 2024-04-12T05:43:38.974 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:43:39.215 INFO:teuthology.orchestra.run.smithi194.stdout:(1/2): ceph-fuse-18.2.1-0.el8.x86_64.rpm 17 MB/s | 851 kB 00:00 2024-04-12T05:43:39.524 INFO:teuthology.orchestra.run.smithi194.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 233 kB/s | 83 kB 00:00 2024-04-12T05:43:39.524 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:39.525 INFO:teuthology.orchestra.run.smithi194.stdout:Total 1.7 MB/s | 934 kB 00:00 2024-04-12T05:43:39.535 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:39.557 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:39.557 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:39.645 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:39.648 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:39.792 INFO:teuthology.orchestra.run.smithi008.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 717 kB/s | 4.8 MB 00:06 2024-04-12T05:43:39.793 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:39.794 INFO:teuthology.orchestra.run.smithi008.stdout:Total 881 kB/s | 6.0 MB 00:06 2024-04-12T05:43:39.850 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:39.862 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:39.862 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:39.872 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:39.973 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:39.976 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:40.175 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-12T05:43:40.258 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:40.577 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2024-04-12T05:43:41.453 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-12T05:43:41.573 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-12T05:43:41.942 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2024-04-12T05:43:42.240 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : python3-jinja2-2.10.1-4.el8.noarch 4/5 2024-04-12T05:43:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-12T05:43:42.404 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:18.2.1-0.el8.x86_64 1/2 2024-04-12T05:43:42.405 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-12T05:43:42.712 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 5/5 2024-04-12T05:43:42.712 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el8.noarch 1/5 2024-04-12T05:43:42.712 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2024-04-12T05:43:42.712 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-jinja2-2.10.1-4.el8.noarch 3/5 2024-04-12T05:43:42.712 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-12T05:43:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-12T05:43:42.989 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:42.990 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:18.2.1-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-12T05:43:42.990 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:42.990 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-mgr-cephadm-2:18.2.1-0.el8.noarch python3-asyncssh-2.7.0-2.el8.noarch 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: python3-babel-2.5.1-7.el8.noarch python3-jinja2-2.10.1-4.el8.noarch 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:43.121 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:43.228 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install librados-devel 2024-04-12T05:43:43.636 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install ceph-fuse 2024-04-12T05:43:43.975 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:42 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:44.402 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:41 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:45.612 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:18.2.1-0.el8 ceph 139 k 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:43:45.613 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:45.614 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 139 k 2024-04-12T05:43:45.614 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 513 k 2024-04-12T05:43:45.614 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:43:45.673 INFO:teuthology.orchestra.run.smithi194.stdout:librados-devel-18.2.1-0.el8.x86_64.rpm 2.3 MB/s | 139 kB 00:00 2024-04-12T05:43:45.673 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:45.674 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.3 MB/s | 139 kB 00:00 2024-04-12T05:43:45.674 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:45.679 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:45.679 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:45.729 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:45.732 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:45.844 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:46.109 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:46.141 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:46.141 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse x86_64 2:18.2.1-0.el8 ceph 851 k 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:Installing dependencies: 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout: fuse x86_64 2.9.7-19.el8 baseos 83 k 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout:Install 2 Packages 2024-04-12T05:43:46.142 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:46.143 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 934 k 2024-04-12T05:43:46.143 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 3.0 M 2024-04-12T05:43:46.143 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:43:46.250 INFO:teuthology.orchestra.run.smithi008.stdout:(1/2): ceph-fuse-18.2.1-0.el8.x86_64.rpm 16 MB/s | 851 kB 00:00 2024-04-12T05:43:46.393 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:46.542 INFO:teuthology.orchestra.run.smithi008.stdout:(2/2): fuse-2.9.7-19.el8.x86_64.rpm 243 kB/s | 83 kB 00:00 2024-04-12T05:43:46.543 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:46.543 INFO:teuthology.orchestra.run.smithi008.stdout:Total 2.3 MB/s | 934 kB 00:00 2024-04-12T05:43:46.554 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:46.577 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:46.577 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:46.670 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:46.673 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:18.2.1-0.el8.x86_64 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:46.765 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:46.854 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:46.983 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install libcephfs2 2024-04-12T05:43:47.117 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : fuse-2.9.7-19.el8.x86_64 1/2 2024-04-12T05:43:47.347 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-12T05:43:47.727 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:46 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:47.760 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el8.x86_64 2/2 2024-04-12T05:43:47.760 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : ceph-fuse-2:18.2.1-0.el8.x86_64 1/2 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : fuse-2.9.7-19.el8.x86_64 2/2 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout: ceph-fuse-2:18.2.1-0.el8.x86_64 fuse-2.9.7-19.el8.x86_64 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:48.153 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:48.356 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librados-devel 2024-04-12T05:43:49.121 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:46 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:49.322 INFO:teuthology.orchestra.run.smithi194.stdout:Package libcephfs2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:43:49.369 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:49.370 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:43:49.370 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:49.473 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install libcephfs-devel 2024-04-12T05:43:50.199 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:49 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:50.905 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel x86_64 2:18.2.1-0.el8 ceph 139 k 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:43:50.906 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:50.907 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 139 k 2024-04-12T05:43:50.907 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 513 k 2024-04-12T05:43:50.907 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:43:50.950 INFO:teuthology.orchestra.run.smithi008.stdout:librados-devel-18.2.1-0.el8.x86_64.rpm 3.1 MB/s | 139 kB 00:00 2024-04-12T05:43:50.951 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:50.951 INFO:teuthology.orchestra.run.smithi008.stdout:Total 3.0 MB/s | 139 kB 00:00 2024-04-12T05:43:50.951 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:50.957 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:50.957 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:51.012 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:51.015 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:51.134 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:51.379 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:51.682 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:51.819 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:51.819 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:51.819 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:18.2.1-0.el8 ceph 31 k 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:43:51.820 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:51.821 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 31 k 2024-04-12T05:43:51.821 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 139 k 2024-04-12T05:43:51.821 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:43:51.884 INFO:teuthology.orchestra.run.smithi194.stdout:libcephfs-devel-18.2.1-0.el8.x86_64.rpm 494 kB/s | 31 kB 00:00 2024-04-12T05:43:52.259 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : librados-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:52.259 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:52.260 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:52.260 INFO:teuthology.orchestra.run.smithi008.stdout: librados-devel-2:18.2.1-0.el8.x86_64 2024-04-12T05:43:52.260 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:52.260 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Total 484 kB/s | 31 kB 00:00 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:43:52.262 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:43:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:43:52.263 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:52.278 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install libcephfs2 2024-04-12T05:43:52.461 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:52.809 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:52.810 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:52.810 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:43:52.810 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:18.2.1-0.el8.x86_64 2024-04-12T05:43:52.810 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:43:52.810 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:52.931 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install librados2 2024-04-12T05:43:53.042 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:50 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:53.676 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:52 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:54.823 INFO:teuthology.orchestra.run.smithi008.stdout:Package libcephfs2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:43:54.877 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:54.878 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:43:54.878 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:54.971 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install libcephfs-devel 2024-04-12T05:43:55.254 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:43:55.302 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:55.303 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:43:55.303 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:55.406 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install librbd1 2024-04-12T05:43:55.734 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:52 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:43:56.139 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:55 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:57.486 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:43:57.487 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:57.487 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:43:57.487 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:57.487 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel x86_64 2:18.2.1-0.el8 ceph 31 k 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 31 k 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 139 k 2024-04-12T05:43:57.488 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:43:57.551 INFO:teuthology.orchestra.run.smithi008.stdout:libcephfs-devel-18.2.1-0.el8.x86_64.rpm 487 kB/s | 31 kB 00:00 2024-04-12T05:43:57.552 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:43:57.552 INFO:teuthology.orchestra.run.smithi008.stdout:Total 477 kB/s | 31 kB 00:00 2024-04-12T05:43:57.553 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:43:57.556 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:43:57.556 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:43:57.589 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:43:57.593 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:43:57.654 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:43:57.724 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:43:57.772 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:43:57.773 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:43:57.773 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:43:57.837 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:57.873 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install python3-rados 2024-04-12T05:43:58.112 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout: libcephfs-devel-2:18.2.1-0.el8.x86_64 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:43:58.485 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:43:58.599 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:02:57 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:43:58.602 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librados2 2024-04-12T05:43:59.367 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:56 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:00.187 INFO:teuthology.orchestra.run.smithi194.stdout:Package python3-rados-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:00.237 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:00.238 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:44:00.238 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:00.360 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install python3-rgw 2024-04-12T05:44:01.052 INFO:teuthology.orchestra.run.smithi008.stdout:Package librados2-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:01.088 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:00 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:01.104 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:01.105 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:01.105 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:01.206 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install librbd1 2024-04-12T05:44:01.972 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:02:58 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:02.686 INFO:teuthology.orchestra.run.smithi194.stdout:Package python3-rgw-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:02.733 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:02.735 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:44:02.735 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:02.837 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install python3-cephfs 2024-04-12T05:44:03.570 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:02 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:03.665 INFO:teuthology.orchestra.run.smithi008.stdout:Package librbd1-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:03.719 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:03.720 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:03.720 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:03.825 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rados 2024-04-12T05:44:04.589 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:01 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:05.157 INFO:teuthology.orchestra.run.smithi194.stdout:Package python3-cephfs-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:05.205 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:05.206 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:44:05.206 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:05.310 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install python3-rbd 2024-04-12T05:44:06.037 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:05 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:06.285 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rados-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:06.337 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:06.338 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:06.338 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:06.437 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rgw 2024-04-12T05:44:07.198 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:04 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:07.621 INFO:teuthology.orchestra.run.smithi194.stdout:Package python3-rbd-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:07.669 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:07.670 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-04-12T05:44:07.671 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:07.804 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install rbd-fuse 2024-04-12T05:44:08.532 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:07 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:08.895 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rgw-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:08.948 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:08.949 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:08.949 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:09.050 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-cephfs 2024-04-12T05:44:09.814 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:06 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:18.2.1-0.el8 ceph 94 k 2024-04-12T05:44:10.165 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 94 k 2024-04-12T05:44:10.166 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 276 k 2024-04-12T05:44:10.167 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:44:10.214 INFO:teuthology.orchestra.run.smithi194.stdout:rbd-fuse-18.2.1-0.el8.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-12T05:44:10.215 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:10.215 INFO:teuthology.orchestra.run.smithi194.stdout:Total 1.9 MB/s | 94 kB 00:00 2024-04-12T05:44:10.215 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:44:10.225 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:44:10.225 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:44:10.274 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:44:10.277 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:44:10.393 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:44:10.596 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:10.890 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:11.286 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:11.287 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:11.287 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:44:11.287 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:11.287 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:11.287 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:11.479 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install rbd-mirror 2024-04-12T05:44:11.502 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-cephfs-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:11.554 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:11.556 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:11.556 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:11.654 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install python3-rbd 2024-04-12T05:44:12.225 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:11 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:12.416 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:09 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:13.856 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:18.2.1-0.el8 ceph 3.4 M 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:44:13.857 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:13.858 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:44:13.858 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:13.858 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 3.4 M 2024-04-12T05:44:13.858 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 16 M 2024-04-12T05:44:13.858 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:44:13.943 INFO:teuthology.orchestra.run.smithi194.stdout:rbd-mirror-18.2.1-0.el8.x86_64.rpm 41 MB/s | 3.4 MB 00:00 2024-04-12T05:44:13.943 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:13.944 INFO:teuthology.orchestra.run.smithi194.stdout:Total 40 MB/s | 3.4 MB 00:00 2024-04-12T05:44:13.944 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:44:13.956 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:44:13.956 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:44:14.019 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:44:14.022 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:44:14.114 INFO:teuthology.orchestra.run.smithi008.stdout:Package python3-rbd-2:18.2.1-0.el8.x86_64 is already installed. 2024-04-12T05:44:14.168 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:14.169 INFO:teuthology.orchestra.run.smithi008.stdout:Nothing to do. 2024-04-12T05:44:14.169 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:14.264 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-fuse 2024-04-12T05:44:14.479 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:44:14.822 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:15.029 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:12 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:15.240 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:15.623 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:15.851 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install rbd-nbd 2024-04-12T05:44:16.591 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:03:15 ago on Fri 12 Apr 2024 05:41:01 AM UTC. 2024-04-12T05:44:16.767 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:16.767 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse x86_64 2:18.2.1-0.el8 ceph 94 k 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:44:16.768 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:16.769 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 94 k 2024-04-12T05:44:16.769 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 276 k 2024-04-12T05:44:16.769 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:44:16.819 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-fuse-18.2.1-0.el8.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-12T05:44:16.819 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:16.819 INFO:teuthology.orchestra.run.smithi008.stdout:Total 1.8 MB/s | 94 kB 00:00 2024-04-12T05:44:16.820 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:44:16.831 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:44:16.831 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:44:16.883 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:44:16.886 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:44:17.003 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:44:17.225 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:17.526 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-fuse-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-fuse-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:17.941 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:18.160 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-mirror 2024-04-12T05:44:18.209 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:18.2.1-0.el8 ceph 181 k 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-04-12T05:44:18.210 INFO:teuthology.orchestra.run.smithi194.stdout:Install 1 Package 2024-04-12T05:44:18.211 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:18.211 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 181 k 2024-04-12T05:44:18.211 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 568 k 2024-04-12T05:44:18.211 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-04-12T05:44:18.275 INFO:teuthology.orchestra.run.smithi194.stdout:rbd-nbd-18.2.1-0.el8.x86_64.rpm 2.8 MB/s | 181 kB 00:00 2024-04-12T05:44:18.275 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:18.275 INFO:teuthology.orchestra.run.smithi194.stdout:Total 2.8 MB/s | 181 kB 00:00 2024-04-12T05:44:18.275 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-04-12T05:44:18.287 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-04-12T05:44:18.287 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-04-12T05:44:18.337 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-04-12T05:44:18.341 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-04-12T05:44:18.459 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-04-12T05:44:18.678 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:18.932 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:15 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:18.996 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:44:19.409 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-04-12T05:44:19.663 DEBUG:teuthology.parallel:result is None 2024-04-12T05:44:20.676 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror x86_64 2:18.2.1-0.el8 ceph 3.4 M 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:44:20.677 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:20.678 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 3.4 M 2024-04-12T05:44:20.678 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 16 M 2024-04-12T05:44:20.678 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:44:20.763 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-mirror-18.2.1-0.el8.x86_64.rpm 41 MB/s | 3.4 MB 00:00 2024-04-12T05:44:20.763 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:20.764 INFO:teuthology.orchestra.run.smithi008.stdout:Total 40 MB/s | 3.4 MB 00:00 2024-04-12T05:44:20.764 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:44:20.777 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:44:20.778 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:44:20.843 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:44:20.846 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:44:21.307 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:44:21.500 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:21.957 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-mirror-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:22.373 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:22.373 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:44:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-mirror-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:22.374 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:22.374 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:22.594 DEBUG:teuthology.orchestra.run.smithi008:> sudo yum -y install rbd-nbd 2024-04-12T05:44:23.362 INFO:teuthology.orchestra.run.smithi008.stdout:Last metadata expiration check: 0:03:20 ago on Fri 12 Apr 2024 05:41:03 AM UTC. 2024-04-12T05:44:25.119 INFO:teuthology.orchestra.run.smithi008.stdout:Dependencies resolved. 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout: Package Architecture Version Repository Size 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout:Installing: 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd x86_64 2:18.2.1-0.el8 ceph 181 k 2024-04-12T05:44:25.120 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction Summary 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:================================================================================ 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:Install 1 Package 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:Total download size: 181 k 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:Installed size: 568 k 2024-04-12T05:44:25.121 INFO:teuthology.orchestra.run.smithi008.stdout:Downloading Packages: 2024-04-12T05:44:25.166 INFO:teuthology.orchestra.run.smithi008.stdout:rbd-nbd-18.2.1-0.el8.x86_64.rpm 3.9 MB/s | 181 kB 00:00 2024-04-12T05:44:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:Total 3.8 MB/s | 181 kB 00:00 2024-04-12T05:44:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction check 2024-04-12T05:44:25.179 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction check succeeded. 2024-04-12T05:44:25.179 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction test 2024-04-12T05:44:25.232 INFO:teuthology.orchestra.run.smithi008.stdout:Transaction test succeeded. 2024-04-12T05:44:25.236 INFO:teuthology.orchestra.run.smithi008.stdout:Running transaction 2024-04-12T05:44:25.357 INFO:teuthology.orchestra.run.smithi008.stdout: Preparing : 1/1 2024-04-12T05:44:25.549 INFO:teuthology.orchestra.run.smithi008.stdout: Installing : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:25.856 INFO:teuthology.orchestra.run.smithi008.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:26.255 INFO:teuthology.orchestra.run.smithi008.stdout: Verifying : rbd-nbd-2:18.2.1-0.el8.x86_64 1/1 2024-04-12T05:44:26.255 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:26.255 INFO:teuthology.orchestra.run.smithi008.stdout:Installed: 2024-04-12T05:44:26.255 INFO:teuthology.orchestra.run.smithi008.stdout: rbd-nbd-2:18.2.1-0.el8.x86_64 2024-04-12T05:44:26.255 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:26.256 INFO:teuthology.orchestra.run.smithi008.stdout:Complete! 2024-04-12T05:44:26.516 DEBUG:teuthology.parallel:result is None 2024-04-12T05:44:26.517 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:44:26.517 INFO:teuthology.packaging:ref: None 2024-04-12T05:44:26.517 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:44:26.517 INFO:teuthology.packaging:branch: None 2024-04-12T05:44:26.517 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:44:26.517 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://github.com/ceph/ceph-ci; will also look in https://github.com/ceph/ceph 2024-04-12T05:44:26.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-12T05:44:26.666 DEBUG:teuthology.orchestra.run.smithi008:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T05:44:26.689 INFO:teuthology.orchestra.run.smithi008.stdout:18.2.1-0.el8 2024-04-12T05:44:26.689 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el8 2024-04-12T05:44:26.690 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-12T05:44:26.691 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-12T05:44:26.691 INFO:teuthology.packaging:ref: None 2024-04-12T05:44:26.691 INFO:teuthology.packaging:tag: v18.2.1 2024-04-12T05:44:26.691 INFO:teuthology.packaging:branch: None 2024-04-12T05:44:26.692 INFO:teuthology.packaging:sha1: d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:44:26.692 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://github.com/ceph/ceph-ci; will also look in https://github.com/ceph/ceph 2024-04-12T05:44:26.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-04-12T05:44:26.832 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-12T05:44:26.860 INFO:teuthology.orchestra.run.smithi194.stdout:18.2.1-0.el8 2024-04-12T05:44:26.860 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el8 2024-04-12T05:44:26.860 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-04-12T05:44:26.862 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-12T05:44:26.862 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:44:26.863 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T05:44:26.891 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:44:26.891 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-12T05:44:26.932 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-12T05:44:26.932 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:44:26.932 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T05:44:26.959 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T05:44:27.029 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:44:27.030 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-04-12T05:44:27.060 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-12T05:44:27.128 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-12T05:44:27.129 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:44:27.129 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T05:44:27.156 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T05:44:27.224 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:44:27.224 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-12T05:44:27.253 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-12T05:44:27.322 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-12T05:44:27.322 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:44:27.322 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T05:44:27.350 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T05:44:27.417 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:44:27.418 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-04-12T05:44:27.447 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-12T05:44:27.517 INFO:teuthology.run_tasks:Running task print... 2024-04-12T05:44:27.527 INFO:teuthology.task.print:**** done install task... 2024-04-12T05:44:27.527 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-12T05:44:27.626 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42'} 2024-04-12T05:44:27.626 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-04-12T05:44:27.626 INFO:tasks.cephadm:Cluster fsid is b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:44:27.627 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-12T05:44:27.627 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-12T05:44:27.627 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi008': '172.21.15.8', 'mon.smithi194': '172.21.15.194'} 2024-04-12T05:44:27.627 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-12T05:44:27.627 DEBUG:teuthology.orchestra.run.smithi008:> sudo hostname $(hostname -s) 2024-04-12T05:44:27.656 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-04-12T05:44:27.685 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-12T05:44:27.686 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:44:27.909 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/54853/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1187-gd540ebac', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1187.gd540ebac'}, 'url': 'https://3.chacra.ceph.com/r/ceph/reef/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-04-09 00:37:59.513625', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/reef/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd540ebaca6b131a1dd560e7f69e024b133bbaa42', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/78385/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.2-1187-gd540ebac', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.2-1187.gd540ebac'}, 'url': 'https://4.chacra.ceph.com/r/ceph/reef-release/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-04-09 00:18:04.650798', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'reef-release', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/reef-release/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-12T05:44:28.132 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 d540ebaca6b131a1dd560e7f69e024b133bbaa42 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:44:28.133 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/x86_64/flavors/default/cephadm 2024-04-12T05:44:28.134 DEBUG:teuthology.orchestra.run.smithi008:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.533 INFO:teuthology.orchestra.run.smithi008.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Apr 12 05:44 /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.533 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/d540ebaca6b131a1dd560e7f69e024b133bbaa42/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.928 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-rw-r--. 1 ubuntu ubuntu 703022 Apr 12 05:44 /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.929 DEBUG:teuthology.orchestra.run.smithi008:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.952 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-12T05:44:28.997 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-04-12T05:44:28.997 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-12T05:44:29.000 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-04-12T05:44:30.236 INFO:teuthology.orchestra.run.smithi008.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-12T05:44:30.371 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-12T05:44:48.750 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: "repo_digests": [ 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout: ] 2024-04-12T05:44:48.751 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:44:49.637 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-04-12T05:44:49.638 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-12T05:44:49.658 DEBUG:teuthology.orchestra.run.smithi008:> sudo mkdir -p /etc/ceph 2024-04-12T05:44:49.695 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-04-12T05:44:49.725 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 777 /etc/ceph 2024-04-12T05:44:49.769 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-04-12T05:44:49.795 INFO:tasks.cephadm:Writing seed config... 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-12T05:44:49.796 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-12T05:44:49.797 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-12T05:44:49.798 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-12T05:44:49.799 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-12T05:44:49.800 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-12T05:44:49.800 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-12T05:44:49.800 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-12T05:44:49.800 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:44:49.800 DEBUG:teuthology.orchestra.run.smithi008:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-12T05:44:49.830 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b9cc93fe-f88f-11ee-bc8c-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-12T05:44:49.830 DEBUG:teuthology.orchestra.run.smithi008:mon.smithi008> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service 2024-04-12T05:44:49.873 INFO:tasks.cephadm:Bootstrapping... 2024-04-12T05:44:49.873 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.8 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-12T05:44:49.902 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:-- Logs begin at Fri 2024-04-12 05:32:04 UTC. -- 2024-04-12T05:44:50.046 INFO:teuthology.orchestra.run.smithi008.stdout:-------------------------------------------------------------------------------- 2024-04-12T05:44:50.046 INFO:teuthology.orchestra.run.smithi008.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', 'b9cc93fe-f88f-11ee-bc8c-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.8', '--skip-admin-label'] 2024-04-12T05:44:50.083 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T05:44:50.083 INFO:teuthology.orchestra.run.smithi008.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-12T05:44:50.084 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying podman|docker is present... 2024-04-12T05:44:50.114 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T05:44:50.114 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying lvm2 is present... 2024-04-12T05:44:50.114 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying time synchronization is in place... 2024-04-12T05:44:50.121 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-12T05:44:50.121 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-12T05:44:50.127 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-12T05:44:50.127 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:44:50.133 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2024-04-12T05:44:50.139 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2024-04-12T05:44:50.139 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2024-04-12T05:44:50.139 INFO:teuthology.orchestra.run.smithi008.stdout:Repeating the final host check... 2024-04-12T05:44:50.169 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-12T05:44:50.169 INFO:teuthology.orchestra.run.smithi008.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-12T05:44:50.169 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl is present 2024-04-12T05:44:50.169 INFO:teuthology.orchestra.run.smithi008.stdout:lvcreate is present 2024-04-12T05:44:50.174 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-12T05:44:50.174 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-12T05:44:50.180 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-12T05:44:50.180 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:44:50.186 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout enabled 2024-04-12T05:44:50.192 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout active 2024-04-12T05:44:50.192 INFO:teuthology.orchestra.run.smithi008.stdout:Unit chronyd.service is enabled and running 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Host looks OK 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Cluster fsid: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Acquiring lock 139880440752616 on /run/cephadm/b9cc93fe-f88f-11ee-bc8c-c7b262605968.lock 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Lock 139880440752616 acquired on /run/cephadm/b9cc93fe-f88f-11ee-bc8c-c7b262605968.lock 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 3300 ... 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying IP 172.21.15.8 port 6789 ... 2024-04-12T05:44:50.193 INFO:teuthology.orchestra.run.smithi008.stdout:Base mon IP(s) is [172.21.15.8:3300, 172.21.15.8:6789], mon addrv is [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2024-04-12T05:44:50.197 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.8 metric 100 2024-04-12T05:44:50.197 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.8 metric 100 2024-04-12T05:44:50.200 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-12T05:44:50.200 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-04-12T05:44:50.200 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-04-12T05:44:50.202 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-12T05:44:50.202 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:Mon IP `172.21.15.8` is in CIDR network `172.21.0.0/20` 2024-04-12T05:44:50.203 INFO:teuthology.orchestra.run.smithi008.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-12T05:44:50.204 INFO:teuthology.orchestra.run.smithi008.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-12T05:44:50.205 INFO:teuthology.orchestra.run.smithi008.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-04-12T05:44:51.038 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Getting image source signatures 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-04-12T05:44:51.039 INFO:teuthology.orchestra.run.smithi008.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-12T05:44:52.198 INFO:teuthology.orchestra.run.smithi008.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-12T05:44:52.199 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-04-12T05:44:52.199 INFO:teuthology.orchestra.run.smithi008.stdout:Extracting ceph user uid/gid from container image... 2024-04-12T05:44:53.313 INFO:teuthology.orchestra.run.smithi008.stdout:stat: stdout 167 167 2024-04-12T05:44:53.314 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial keys... 2024-04-12T05:44:54.443 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQBVyhhmSip3LxAAzqYb9GYu5MCnqCiabRb2WQ== 2024-04-12T05:44:55.555 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQBWyhhmjb8ENBAAp9tLlPB9hBKdM9WSFxQliA== 2024-04-12T05:44:56.612 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph-authtool: stdout AQBXyhhmJ9RZOhAA3IdKWMrlizbizsctSdR/bg== 2024-04-12T05:44:56.613 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial monmap... 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:monmaptool for smithi008 [v2:172.21.15.8:3300,v1:172.21.15.8:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-12T05:44:57.672 INFO:teuthology.orchestra.run.smithi008.stdout:setting min_mon_release = pacific 2024-04-12T05:44:57.673 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: set fsid to b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:44:57.673 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-12T05:44:57.673 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:44:57.673 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mon... 2024-04-12T05:44:58.794 INFO:teuthology.orchestra.run.smithi008.stdout:create mon.smithi008 on 2024-04-12T05:44:58.931 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-04-12T05:44:59.025 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-12T05:44:59.151 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target → /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target. 2024-04-12T05:44:59.151 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target → /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target. 2024-04-12T05:44:59.338 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008 2024-04-12T05:44:59.338 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service: Unit ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service not loaded. 2024-04-12T05:44:59.432 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target.wants/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service → /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@.service. 2024-04-12T05:45:00.038 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T05:45:00.038 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-04-12T05:45:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T05:45:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:45:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-04-12T05:45:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2024-04-12T05:45:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon to start... 2024-04-12T05:45:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mon... 2024-04-12T05:45:00.277 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:00 smithi008 bash[39203]: 4c9fb5740bb0863f951f411e87b52e7c0bf48b6355c4da3233c1ec14537f50b5 2024-04-12T05:45:00.277 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:00 smithi008 systemd[1]: Started Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:45:01.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout cluster: 2024-04-12T05:45:01.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout id: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:01.287 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout services: 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi008 (age 0.636175s) 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout data: 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout pgs: 2024-04-12T05:45:01.288 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.578+0000 7f6c10720700 1 Processor -- start 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.578+0000 7f6c10720700 1 -- start start 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c10720700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c10720700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08106d20 con 0x7f6c08106330 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c0e4bc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c0e4bc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48656/0 (socket says 172.21.15.8:48656) 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c0e4bc700 1 -- 172.21.15.8:0/644004236 learned_addr learned my addr 172.21.15.8:0/644004236 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.579+0000 7f6c0e4bc700 1 -- 172.21.15.8:0/644004236 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08107530 con 0x7f6c08106330 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c0e4bc700 1 --2- 172.21.15.8:0/644004236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80098a0 tx=0x7f6bf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0534f2cd9a6922 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c0d4ba700 1 -- 172.21.15.8:0/644004236 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bf8004180 con 0x7f6c08106330 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c0d4ba700 1 -- 172.21.15.8:0/644004236 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf80042e0 con 0x7f6c08106330 2024-04-12T05:45:01.289 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c0d4ba700 1 -- 172.21.15.8:0/644004236 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bf80036a0 con 0x7f6c08106330 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 -- 172.21.15.8:0/644004236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 msgr2=0x7f6c08106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 --2- 172.21.15.8:0/644004236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6bf80098a0 tx=0x7f6bf8009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 -- 172.21.15.8:0/644004236 shutdown_connections 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 --2- 172.21.15.8:0/644004236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08106750 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 -- 172.21.15.8:0/644004236 >> 172.21.15.8:0/644004236 conn(0x7f6c081018d0 msgr2=0x7f6c08103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 -- 172.21.15.8:0/644004236 shutdown_connections 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.580+0000 7f6c10720700 1 -- 172.21.15.8:0/644004236 wait complete. 2024-04-12T05:45:01.290 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c10720700 1 Processor -- start 2024-04-12T05:45:01.290 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: mon.smithi008 is new leader, mons smithi008 in quorum (ranks 0) 2024-04-12T05:45:01.290 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: monmap e1: 1 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:45:01.291 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: fsmap 2024-04-12T05:45:01.291 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: osdmap e1: 0 total, 0 up, 0 in 2024-04-12T05:45:01.291 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: mgrmap e1: no daemons active 2024-04-12T05:45:01.291 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:01 smithi008 ceph-mon[39240]: from='client.? 172.21.15.8:0/3707706915' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c10720700 1 -- start start 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c10720700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c10720700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08199da0 con 0x7f6c08106330 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c0e4bc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c0e4bc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48668/0 (socket says 172.21.15.8:48668) 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c0e4bc700 1 -- 172.21.15.8:0/3707706915 learned_addr learned my addr 172.21.15.8:0/3707706915 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.581+0000 7f6c0e4bc700 1 -- 172.21.15.8:0/3707706915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8009550 con 0x7f6c08106330 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6c0e4bc700 1 --2- 172.21.15.8:0/3707706915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8004000 tx=0x7f6bf8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bf8003ce0 con 0x7f6c08106330 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8020070 con 0x7f6c08106330 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c08199fa0 con 0x7f6c08106330 2024-04-12T05:45:01.294 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6bf8011630 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0819a3c0 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6bf8022030 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.582+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6bf8025d00 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.583+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c08193110 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.585+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f6bf8025480 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.657+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6c0802cdf0 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.657+0000 7f6bff7fe700 1 -- 172.21.15.8:0/3707706915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f6bf8011bd0 con 0x7f6c08106330 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 msgr2=0x7f6c08199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 --2- 172.21.15.8:0/3707706915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8004000 tx=0x7f6bf8004750 comp rx=0 tx=0).stop 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 shutdown_connections 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 --2- 172.21.15.8:0/3707706915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c08106330 0x7f6c08199860 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 >> 172.21.15.8:0/3707706915 conn(0x7f6c081018d0 msgr2=0x7f6c0818dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 shutdown_connections 2024-04-12T05:45:01.295 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:00.658+0000 7f6c10720700 1 -- 172.21.15.8:0/3707706915 wait complete. 2024-04-12T05:45:01.296 INFO:teuthology.orchestra.run.smithi008.stdout:mon is available 2024-04-12T05:45:01.296 INFO:teuthology.orchestra.run.smithi008.stdout:Assimilating anything we can from ceph.conf... 2024-04-12T05:45:02.262 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:02 smithi008 ceph-mon[39240]: from='client.? 172.21.15.8:0/4078933044' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-12T05:45:02.262 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:02 smithi008 ceph-mon[39240]: from='client.? 172.21.15.8:0/4078933044' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-12T05:45:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.8:3300,v1:172.21.15.8:6789] 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-12T05:45:02.547 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f250c6700 1 Processor -- start 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f250c6700 1 -- start start 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f250c6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f250c6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20106d00 con 0x7f8f20106310 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f1ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f1ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48674/0 (socket says 172.21.15.8:48674) 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.826+0000 7f8f1ed9d700 1 -- 172.21.15.8:0/3013134285 learned_addr learned my addr 172.21.15.8:0/3013134285 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:02.548 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.827+0000 7f8f1ed9d700 1 -- 172.21.15.8:0/3013134285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20107510 con 0x7f8f20106310 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.827+0000 7f8f1ed9d700 1 --2- 172.21.15.8:0/3013134285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009e80 tx=0x7f8f1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fb582f2ec82368 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.827+0000 7f8f1dd9b700 1 -- 172.21.15.8:0/3013134285 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f1800e070 con 0x7f8f20106310 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.827+0000 7f8f1dd9b700 1 -- 172.21.15.8:0/3013134285 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18003a20 con 0x7f8f20106310 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.827+0000 7f8f1dd9b700 1 -- 172.21.15.8:0/3013134285 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f18003bd0 con 0x7f8f20106310 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 -- 172.21.15.8:0/3013134285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 msgr2=0x7f8f20106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 --2- 172.21.15.8:0/3013134285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009e80 tx=0x7f8f1800b330 comp rx=0 tx=0).stop 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 -- 172.21.15.8:0/3013134285 shutdown_connections 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 --2- 172.21.15.8:0/3013134285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f20106730 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 -- 172.21.15.8:0/3013134285 >> 172.21.15.8:0/3013134285 conn(0x7f8f201018d0 msgr2=0x7f8f20103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 -- 172.21.15.8:0/3013134285 shutdown_connections 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 -- 172.21.15.8:0/3013134285 wait complete. 2024-04-12T05:45:02.549 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.828+0000 7f8f250c6700 1 Processor -- start 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f250c6700 1 -- start start 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f250c6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f250c6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f20199de0 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f1ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f1ed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48690/0 (socket says 172.21.15.8:48690) 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f1ed9d700 1 -- 172.21.15.8:0/4078933044 learned_addr learned my addr 172.21.15.8:0/4078933044 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f1ed9d700 1 -- 172.21.15.8:0/4078933044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18009b30 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.829+0000 7f8f1ed9d700 1 --2- 172.21.15.8:0/4078933044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f18004000 tx=0x7f8f18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f18004240 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f180043e0 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f20199fe0 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f180046e0 con 0x7f8f20106310 2024-04-12T05:45:02.550 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2019a400 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8f18019050 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.830+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8f180257e0 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.831+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f20193150 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.833+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f8f1802a060 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.902+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8f200623c0 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 7 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f8f18012d10 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f17fff700 1 -- 172.21.15.8:0/4078933044 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f8f18020030 con 0x7f8f20106310 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 msgr2=0x7f8f201998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f250c6700 1 --2- 172.21.15.8:0/4078933044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f18004000 tx=0x7f8f18004ad0 comp rx=0 tx=0).stop 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 shutdown_connections 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f250c6700 1 --2- 172.21.15.8:0/4078933044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8f20106310 0x7f8f201998a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.906+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 >> 172.21.15.8:0/4078933044 conn(0x7f8f201018d0 msgr2=0x7f8f2018dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.907+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 shutdown_connections 2024-04-12T05:45:02.551 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:01.907+0000 7f8f250c6700 1 -- 172.21.15.8:0/4078933044 wait complete. 2024-04-12T05:45:02.552 INFO:teuthology.orchestra.run.smithi008.stdout:Generating new minimal ceph.conf... 2024-04-12T05:45:03.585 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:03 smithi008 ceph-mon[39240]: from='client.? 172.21.15.8:0/2883674679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:45:03.810 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a983f700 1 Processor -- start 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a983f700 1 -- start start 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a983f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a983f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4105000 con 0x7f34a4104610 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48704/0 (socket says 172.21.15.8:48704) 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.117+0000 7f34a2ffd700 1 -- 172.21.15.8:0/2964756450 learned_addr learned my addr 172.21.15.8:0/2964756450 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a2ffd700 1 -- 172.21.15.8:0/2964756450 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a41075a0 con 0x7f34a4104610 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a2ffd700 1 --2- 172.21.15.8:0/2964756450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34980098a0 tx=0x7f3498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5ffb41c554899c server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a1ffb700 1 -- 172.21.15.8:0/2964756450 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3498004180 con 0x7f34a4104610 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a1ffb700 1 -- 172.21.15.8:0/2964756450 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f34980042e0 con 0x7f34a4104610 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a1ffb700 1 -- 172.21.15.8:0/2964756450 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34980119d0 con 0x7f34a4104610 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a983f700 1 -- 172.21.15.8:0/2964756450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 msgr2=0x7f34a4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:03.811 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a983f700 1 --2- 172.21.15.8:0/2964756450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f34980098a0 tx=0x7f3498009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a983f700 1 -- 172.21.15.8:0/2964756450 shutdown_connections 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a983f700 1 --2- 172.21.15.8:0/2964756450 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4104a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.118+0000 7f34a983f700 1 -- 172.21.15.8:0/2964756450 >> 172.21.15.8:0/2964756450 conn(0x7f34a40fff40 msgr2=0x7f34a4102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.119+0000 7f34a983f700 1 -- 172.21.15.8:0/2964756450 shutdown_connections 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.119+0000 7f34a983f700 1 -- 172.21.15.8:0/2964756450 wait complete. 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.119+0000 7f34a983f700 1 Processor -- start 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a983f700 1 -- start start 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a983f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a983f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a419a510 con 0x7f34a4104610 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48706/0 (socket says 172.21.15.8:48706) 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a2ffd700 1 -- 172.21.15.8:0/2883674679 learned_addr learned my addr 172.21.15.8:0/2883674679 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.120+0000 7f34a2ffd700 1 -- 172.21.15.8:0/2883674679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3498009550 con 0x7f34a4104610 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a2ffd700 1 --2- 172.21.15.8:0/2883674679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3498006af0 tx=0x7f3498011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:03.812 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3498024430 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f34980249e0 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f349801a970 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a419a710 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a419ab30 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f349802d3f0 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.121+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3498019750 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.122+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a40623c0 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.124+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3498024b50 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.193+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f34a410a790 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a883d700 1 -- 172.21.15.8:0/2883674679 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f3498024cf0 con 0x7f34a4104610 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 msgr2=0x7f34a4199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a983f700 1 --2- 172.21.15.8:0/2883674679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3498006af0 tx=0x7f3498011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 shutdown_connections 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a983f700 1 --2- 172.21.15.8:0/2883674679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f34a4104610 0x7f34a4199fd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:03.813 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.194+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 >> 172.21.15.8:0/2883674679 conn(0x7f34a40fff40 msgr2=0x7f34a4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:03.814 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.195+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 shutdown_connections 2024-04-12T05:45:03.814 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:03.195+0000 7f34a983f700 1 -- 172.21.15.8:0/2883674679 wait complete. 2024-04-12T05:45:03.814 INFO:teuthology.orchestra.run.smithi008.stdout:Restarting the monitor... 2024-04-12T05:45:03.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:03 smithi008 systemd[1]: Stopping Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T05:45:04.339 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[39217]: 2024-04-12T05:45:04.062+0000 7f7ebf601700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi008 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T05:45:04.339 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[39217]: 2024-04-12T05:45:04.062+0000 7f7ebf601700 -1 mon.smithi008@0(leader) e1 *** Got Signal Terminated *** 2024-04-12T05:45:04.649 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 bash[39566]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008 2024-04-12T05:45:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service: Succeeded. 2024-04-12T05:45:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 systemd[1]: Stopped Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:45:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:04 smithi008 systemd[1]: Starting Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T05:45:05.430 INFO:teuthology.orchestra.run.smithi008.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 7 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: pidfile_write: ignore empty --pid-file 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: load: jerasure load: lrc 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: RocksDB version: 7.9.2 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Git sha 0 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Compile date 2023-12-11 22:07:34 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: DB SUMMARY 2024-04-12T05:45:05.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: DB Session ID: GTUXUEKCMHW1H106VFON 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: CURRENT file: CURRENT 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi008/store.db dir, Total Num: 1, files: 000008.sst 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi008/store.db: 000009.log size: 96908 ; 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.error_if_exists: 0 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.create_if_missing: 0 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.paranoid_checks: 1 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.env: 0x5611878ce720 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.fs: PosixFileSystem 2024-04-12T05:45:05.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.info_log: 0x56118a4bb2e0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.statistics: (nil) 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.use_fsync: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_log_file_size: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_fallocate: 1 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.use_direct_reads: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T05:45:05.652 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T05:45:05.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.db_log_dir: 2024-04-12T05:45:05.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.wal_dir: 2024-04-12T05:45:05.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T05:45:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.write_buffer_manager: 0x56118972c140 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.unordered_write: 0 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T05:45:05.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.row_cache: None 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.wal_filter: None 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.two_write_queues: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.wal_compression: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.atomic_flush: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.log_readahead_size: 0 2024-04-12T05:45:05.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_background_jobs: 2 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_background_compactions: -1 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_subcompactions: 1 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T05:45:05.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_open_files: -1 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_background_flushes: -1 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Compression algorithms supported: 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kZSTD supported: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kXpressCompression supported: 0 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kZlibCompression supported: 1 2024-04-12T05:45:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kSnappyCompression supported: 1 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kLZ4Compression supported: 1 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: kBZip2Compression supported: 0 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi008/store.db/MANIFEST-000010 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.merge_operator: 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_filter: None 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56118a4bb3e0) 2024-04-12T05:45:05.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: cache_index_and_filter_blocks: 1 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: pin_top_level_index_and_filter: 1 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_type: 0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: data_block_index_type: 0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_shortening: 1 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: checksum: 4 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: no_block_cache: 0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache: 0x5611897cd1f0 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_name: BinnedLRUCache 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_options: 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: capacity : 536870912 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: num_shard_bits : 4 2024-04-12T05:45:05.660 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: strict_capacity_limit : 0 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: high_pri_pool_ratio: 0.000 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_compressed: (nil) 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: persistent_cache: (nil) 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_size: 4096 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_size_deviation: 10 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_restart_interval: 16 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_block_restart_interval: 1 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: metadata_block_size: 4096 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: partition_filters: 0 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: use_delta_encoding: 1 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: filter_policy: bloomfilter 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: whole_key_filtering: 1 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: verify_compression: 0 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: read_amp_bytes_per_bit: 0 2024-04-12T05:45:05.661 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: format_version: 5 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: enable_index_compression: 1 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_align: 0 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: max_auto_readahead_size: 262144 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: prepopulate_block_cache: 0 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: initial_auto_readahead_size: 8192 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: num_file_reads_for_auto_readahead: 2 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression: NoCompression 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.num_levels: 7 2024-04-12T05:45:05.662 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T05:45:05.663 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T05:45:05.664 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.arena_block_size: 1048576 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T05:45:05.665 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.table_properties_collectors: 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.inplace_update_support: 0 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T05:45:05.666 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.bloom_locality: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.max_successive_merges: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.ttl: 2592000 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enable_blob_files: false 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.min_blob_size: 0 2024-04-12T05:45:05.667 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.blob_file_starting_level: 0 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi008/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 68c12ccb-cf4a-4751-83b9-27e5c20b0020 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712900705393865, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712900705394887, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92355, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 308, "table_properties": {"data_size": 90372, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13225, "raw_average_key_size": 49, "raw_value_size": 83111, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 267, "num_filter_entries": 267, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712900705, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "68c12ccb-cf4a-4751-83b9-27e5c20b0020", "db_session_id": "GTUXUEKCMHW1H106VFON", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712900705394976, "job": 1, "event": "recovery_finished"} 2024-04-12T05:45:05.668 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi008/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56118986a000 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: DB pointer 0x561189856000 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** DB Stats ** 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T05:45:05.669 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** Compaction Stats [default] ** 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: L0 2/0 92.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Sum 2/0 92.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** Compaction Stats [default] ** 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T05:45:05.670 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative compaction: 0.00 GB write, 27.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval compaction: 0.00 GB write, 27.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Block cache BinnedLRUCache@0x5611897cd1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-12T05:45:05.671 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: starting mon.smithi008 rank 0 at public addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] at bind addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi008 fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???) e1 preinit fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).mds e1 new map 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).mds e1 print_map 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: e1 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: legacy client fscid: -1 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: No filesystems configured 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T05:45:05.672 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-12T05:45:05.673 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 ceph-mon[39722]: mon.smithi008@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-12T05:45:05.673 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 bash[39685]: 281a8a1d6515157aacf43cc7f57bc4ba63213c300619cffda8ecae3c25bbd8ea 2024-04-12T05:45:05.673 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:05 smithi008 systemd[1]: Started Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: mon.smithi008 is new leader, mons smithi008 in quorum (ranks 0) 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: monmap e1: 1 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: fsmap 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: osdmap e1: 0 total, 0 up, 0 in 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: mgrmap e1: no daemons active 2024-04-12T05:45:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:06 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2317955756' entity='client.admin' 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.908+0000 7fe0c5b9b700 1 Processor -- start 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.908+0000 7fe0c5b9b700 1 -- start start 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0c5b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0c5b9b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0106110 con 0x7fe0c0105720 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43906/0 (socket says 172.21.15.8:43906) 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0bf7fe700 1 -- 172.21.15.8:0/3937893470 learned_addr learned my addr 172.21.15.8:0/3937893470 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:09.328 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.909+0000 7fe0bf7fe700 1 -- 172.21.15.8:0/3937893470 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0106440 con 0x7fe0c0105720 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0bf7fe700 1 --2- 172.21.15.8:0/3937893470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00098a0 tx=0x7fe0b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b2362d433f02a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0be7fc700 1 -- 172.21.15.8:0/3937893470 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0b0004180 con 0x7fe0c0105720 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0be7fc700 1 -- 172.21.15.8:0/3937893470 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fe0b00042e0 con 0x7fe0c0105720 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0be7fc700 1 -- 172.21.15.8:0/3937893470 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0b00119d0 con 0x7fe0c0105720 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/3937893470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 msgr2=0x7fe0c0105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 --2- 172.21.15.8:0/3937893470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00098a0 tx=0x7fe0b0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/3937893470 shutdown_connections 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 --2- 172.21.15.8:0/3937893470 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c0105b40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/3937893470 >> 172.21.15.8:0/3937893470 conn(0x7fe0c00fff40 msgr2=0x7fe0c0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/3937893470 shutdown_connections 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.910+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/3937893470 wait complete. 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0c5b9b700 1 Processor -- start 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0c5b9b700 1 -- start start 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0c5b9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:09.329 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0c5b9b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c019c710 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.911+0000 7fe0bf7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43908/0 (socket says 172.21.15.8:43908) 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bf7fe700 1 -- 172.21.15.8:0/2317955756 learned_addr learned my addr 172.21.15.8:0/2317955756 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bf7fe700 1 -- 172.21.15.8:0/2317955756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0009550 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bf7fe700 1 --2- 172.21.15.8:0/2317955756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0011980 tx=0x7fe0b0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0b0003b40 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c019c910 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fe0b001a880 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0b0019910 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.912+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c019cd30 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.913+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe0b002d440 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.913+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe0b00248b0 con 0x7fe0c0105720 2024-04-12T05:45:09.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.913+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c00623c0 con 0x7fe0c0105720 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.915+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fe0b001adf0 con 0x7fe0c0105720 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.984+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe0c0109640 con 0x7fe0c0105720 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.986+0000 7fe0bcff9700 1 -- 172.21.15.8:0/2317955756 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe0b001a9f0 con 0x7fe0c0105720 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 msgr2=0x7fe0c019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 --2- 172.21.15.8:0/2317955756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0011980 tx=0x7fe0b0011a60 comp rx=0 tx=0).stop 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 shutdown_connections 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 --2- 172.21.15.8:0/2317955756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c0105720 0x7fe0c019c1d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 >> 172.21.15.8:0/2317955756 conn(0x7fe0c00fff40 msgr2=0x7fe0c0104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 shutdown_connections 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:05.987+0000 7fe0c5b9b700 1 -- 172.21.15.8:0/2317955756 wait complete. 2024-04-12T05:45:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-12T05:45:09.354 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-12T05:45:09.354 INFO:teuthology.orchestra.run.smithi008.stdout:Creating mgr... 2024-04-12T05:45:09.355 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-12T05:45:09.356 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-12T05:45:09.356 INFO:teuthology.orchestra.run.smithi008.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-12T05:45:09.590 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mgr.smithi008.ixmnlp 2024-04-12T05:45:09.590 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mgr.smithi008.ixmnlp.service: Unit ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mgr.smithi008.ixmnlp.service not loaded. 2024-04-12T05:45:09.692 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968.target.wants/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mgr.smithi008.ixmnlp.service → /etc/systemd/system/ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@.service. 2024-04-12T05:45:10.317 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T05:45:10.317 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-04-12T05:45:10.326 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T05:45:10.326 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:45:10.326 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-04-12T05:45:10.326 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to enable service . firewalld.service is not available 2024-04-12T05:45:10.335 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T05:45:10.335 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr to start... 2024-04-12T05:45:10.344 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr... 2024-04-12T05:45:11.629 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:11.629 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "b9cc93fe-f88f-11ee-bc8c-c7b262605968", 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "smithi008" 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:11.630 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T05:45:11.631 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T05:45:11.635 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T05:45:11.636 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:11.637 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T05:45:00.023490+0000", 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.897+0000 7f37892ff700 1 Processor -- start 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.897+0000 7f37892ff700 1 -- start start 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f37892ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:11.638 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f37892ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784105000 con 0x7f3784104610 2024-04-12T05:45:11.639 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:11 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2784697145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43936/0 (socket says 172.21.15.8:43936) 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f3782ffd700 1 -- 172.21.15.8:0/1612586451 learned_addr learned my addr 172.21.15.8:0/1612586451 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f3782ffd700 1 -- 172.21.15.8:0/1612586451 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37841075a0 con 0x7f3784104610 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.898+0000 7f3782ffd700 1 --2- 172.21.15.8:0/1612586451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f377400f960 tx=0x7f377400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de20bf3af755ec64 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f3781ffb700 1 -- 172.21.15.8:0/1612586451 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3774009b30 con 0x7f3784104610 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f3781ffb700 1 -- 172.21.15.8:0/1612586451 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3774004480 con 0x7f3784104610 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f3781ffb700 1 -- 172.21.15.8:0/1612586451 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3774003cc0 con 0x7f3784104610 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 -- 172.21.15.8:0/1612586451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 msgr2=0x7f3784104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:11.643 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 --2- 172.21.15.8:0/1612586451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f377400f960 tx=0x7f377400fc70 comp rx=0 tx=0).stop 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 -- 172.21.15.8:0/1612586451 shutdown_connections 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 --2- 172.21.15.8:0/1612586451 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784104a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 -- 172.21.15.8:0/1612586451 >> 172.21.15.8:0/1612586451 conn(0x7f37840fff40 msgr2=0x7f3784102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 -- 172.21.15.8:0/1612586451 shutdown_connections 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.899+0000 7f37892ff700 1 -- 172.21.15.8:0/1612586451 wait complete. 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.900+0000 7f37892ff700 1 Processor -- start 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.900+0000 7f37892ff700 1 -- start start 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.900+0000 7f37892ff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.900+0000 7f37892ff700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784198290 con 0x7f3784104610 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f3782ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43944/0 (socket says 172.21.15.8:43944) 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f3782ffd700 1 -- 172.21.15.8:0/2784697145 learned_addr learned my addr 172.21.15.8:0/2784697145 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:11.644 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f3782ffd700 1 -- 172.21.15.8:0/2784697145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377400f610 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f3782ffd700 1 --2- 172.21.15.8:0/2784697145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3774003750 tx=0x7f3774004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3774017660 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3774017c10 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3784198490 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3774022b20 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3784198850 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f377402b6f0 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.901+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3774021660 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.903+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3784191c90 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.905+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f3774030060 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.979+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3784108320 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.980+0000 7f376bfff700 1 -- 172.21.15.8:0/2784697145 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f377401c070 con 0x7f3784104610 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.980+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 msgr2=0x7f3784197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:11.645 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.980+0000 7f37892ff700 1 --2- 172.21.15.8:0/2784697145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3774003750 tx=0x7f3774004250 comp rx=0 tx=0).stop 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.981+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 shutdown_connections 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.981+0000 7f37892ff700 1 --2- 172.21.15.8:0/2784697145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3784104610 0x7f3784197d50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.981+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 >> 172.21.15.8:0/2784697145 conn(0x7f37840fff40 msgr2=0x7f37841017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.981+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 shutdown_connections 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:10.981+0000 7f37892ff700 1 -- 172.21.15.8:0/2784697145 wait complete. 2024-04-12T05:45:11.646 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (1/15)... 2024-04-12T05:45:14.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:14 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1304534504' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T05:45:17.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: Activating manager daemon smithi008.ixmnlp 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: mgrmap e2: smithi008.ixmnlp(active, starting, since 0.00301168s) 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:17.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:17 smithi008 ceph-mon[39722]: from='mgr.14100 172.21.15.8:0/3148209192' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:19 smithi008 ceph-mon[39722]: mgrmap e3: smithi008.ixmnlp(active, since 1.00732s) 2024-04-12T05:45:19.750 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:19.752 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "b9cc93fe-f88f-11ee-bc8c-c7b262605968", 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "smithi008" 2024-04-12T05:45:19.753 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T05:45:19.754 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T05:45:19.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T05:45:19.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T05:45:00.023490+0000", 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e76e23700 1 Processor -- start 2024-04-12T05:45:19.765 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e76e23700 1 -- start start 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e76e23700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e76e23700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70074530 con 0x7f8e70106da0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e74bbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e74bbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36740/0 (socket says 172.21.15.8:36740) 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e74bbf700 1 -- 172.21.15.8:0/1242578196 learned_addr learned my addr 172.21.15.8:0/1242578196 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.179+0000 7f8e74bbf700 1 -- 172.21.15.8:0/1242578196 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701096d0 con 0x7f8e70106da0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e74bbf700 1 --2- 172.21.15.8:0/1242578196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e640098a0 tx=0x7f8e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db88b3686978e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e6f7fe700 1 -- 172.21.15.8:0/1242578196 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e64004180 con 0x7f8e70106da0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e6f7fe700 1 -- 172.21.15.8:0/1242578196 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f8e640042e0 con 0x7f8e70106da0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e6f7fe700 1 -- 172.21.15.8:0/1242578196 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e640119d0 con 0x7f8e70106da0 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e76e23700 1 -- 172.21.15.8:0/1242578196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 msgr2=0x7f8e70109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:19.766 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e76e23700 1 --2- 172.21.15.8:0/1242578196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8e640098a0 tx=0x7f8e64009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e76e23700 1 -- 172.21.15.8:0/1242578196 shutdown_connections 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e76e23700 1 --2- 172.21.15.8:0/1242578196 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70109190 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.180+0000 7f8e76e23700 1 -- 172.21.15.8:0/1242578196 >> 172.21.15.8:0/1242578196 conn(0x7f8e701007f0 msgr2=0x7f8e70102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 -- 172.21.15.8:0/1242578196 shutdown_connections 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 -- 172.21.15.8:0/1242578196 wait complete. 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 Processor -- start 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 -- start start 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e76e23700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70106c40 con 0x7f8e70106da0 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e74bbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e74bbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36746/0 (socket says 172.21.15.8:36746) 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.181+0000 7f8e74bbf700 1 -- 172.21.15.8:0/1304534504 learned_addr learned my addr 172.21.15.8:0/1304534504 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e74bbf700 1 -- 172.21.15.8:0/1304534504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e64009550 con 0x7f8e70106da0 2024-04-12T05:45:19.767 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e74bbf700 1 --2- 172.21.15.8:0/1304534504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009130 tx=0x7f8e64011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e64003a00 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f8e6401a880 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e70104d50 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e70105170 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8e640197b0 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8e6402d440 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.182+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e64024820 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.183+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e54005320 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.186+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f8e6401ad60 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.259+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8e54005cc0 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.259+0000 7f8e6dffb700 1 -- 172.21.15.8:0/1304534504 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f8e6401a9f0 con 0x7f8e70106da0 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 msgr2=0x7f8e70106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 --2- 172.21.15.8:0/1304534504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009130 tx=0x7f8e64011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:19.768 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 shutdown_connections 2024-04-12T05:45:19.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 --2- 172.21.15.8:0/1304534504 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e70106da0 0x7f8e70106700 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:19.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 >> 172.21.15.8:0/1304534504 conn(0x7f8e701007f0 msgr2=0x7f8e70101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:19.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 shutdown_connections 2024-04-12T05:45:19.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:14.260+0000 7f8e76e23700 1 -- 172.21.15.8:0/1304534504 wait complete. 2024-04-12T05:45:19.769 INFO:teuthology.orchestra.run.smithi008.stdout:mgr not available, waiting (2/15)... 2024-04-12T05:45:21.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:21 smithi008 ceph-mon[39722]: mgrmap e4: smithi008.ixmnlp(active, since 3s) 2024-04-12T05:45:24.891 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3021762344' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-12T05:45:25.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:25.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:45:25.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsid": "b9cc93fe-f88f-11ee-bc8c-c7b262605968", 2024-04-12T05:45:25.156 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "health": { 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 0 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "smithi008" 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "quorum_age": 19, 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-12T05:45:25.157 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-12T05:45:25.158 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-12T05:45:25.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "restful" 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ], 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:25.168 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "modified": "2024-04-12T05:45:00.023490+0000", 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout }, 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f08371d3700 1 Processor -- start 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f08371d3700 1 -- start start 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f08371d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f08371d3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0830105c20 con 0x7f0830105230 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f0834f6f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f0834f6f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49006/0 (socket says 172.21.15.8:49006) 2024-04-12T05:45:25.169 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.265+0000 7f0834f6f700 1 -- 172.21.15.8:0/1483495039 learned_addr learned my addr 172.21.15.8:0/1483495039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.266+0000 7f0834f6f700 1 -- 172.21.15.8:0/1483495039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830106430 con 0x7f0830105230 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.266+0000 7f0834f6f700 1 --2- 172.21.15.8:0/1483495039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08240098a0 tx=0x7f0824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0dc66be8876d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.266+0000 7f082f7fe700 1 -- 172.21.15.8:0/1483495039 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0824004180 con 0x7f0830105230 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.266+0000 7f082f7fe700 1 -- 172.21.15.8:0/1483495039 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f08240042e0 con 0x7f0830105230 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.266+0000 7f082f7fe700 1 -- 172.21.15.8:0/1483495039 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08240119d0 con 0x7f0830105230 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- 172.21.15.8:0/1483495039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 msgr2=0x7f0830105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 --2- 172.21.15.8:0/1483495039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08240098a0 tx=0x7f0824009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- 172.21.15.8:0/1483495039 shutdown_connections 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 --2- 172.21.15.8:0/1483495039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f0830105650 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- 172.21.15.8:0/1483495039 >> 172.21.15.8:0/1483495039 conn(0x7f08301007f0 msgr2=0x7f0830102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- 172.21.15.8:0/1483495039 shutdown_connections 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- 172.21.15.8:0/1483495039 wait complete. 2024-04-12T05:45:25.170 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 Processor -- start 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- start start 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.267+0000 7f08371d3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083019ab90 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f0834f6f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f0834f6f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49018/0 (socket says 172.21.15.8:49018) 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f0834f6f700 1 -- 172.21.15.8:0/3021762344 learned_addr learned my addr 172.21.15.8:0/3021762344 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f0834f6f700 1 -- 172.21.15.8:0/3021762344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824009550 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f0834f6f700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0824009130 tx=0x7f0824011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0824003900 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f0824003a60 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083019f1a0 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08240246e0 con 0x7f0830105230 2024-04-12T05:45:25.171 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.268+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083019f500 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.269+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f082402d3f0 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.269+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08300623c0 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.269+0000 7f082dffb700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 0x7f082003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.269+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f082404c120 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.270+0000 7f082ffff700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 0x7f082003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.270+0000 7f082ffff700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 0x7f082003a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f081c006fd0 tx=0x7f081c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.275+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0824010380 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.553+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0830109660 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.553+0000 7f082dffb700 1 -- 172.21.15.8:0/3021762344 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f0824010380 con 0x7f0830105230 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 msgr2=0x7f082003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 0x7f082003a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f081c006fd0 tx=0x7f081c006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 msgr2=0x7f08300741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:25.172 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0824009130 tx=0x7f0824011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 shutdown_connections 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f0820038320 0x7f082003a7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 --2- 172.21.15.8:0/3021762344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0830105230 0x7f08300741b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.555+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 >> 172.21.15.8:0/3021762344 conn(0x7f08301007f0 msgr2=0x7f0830101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.556+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 shutdown_connections 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:24.556+0000 7f08371d3700 1 -- 172.21.15.8:0/3021762344 wait complete. 2024-04-12T05:45:25.173 INFO:teuthology.orchestra.run.smithi008.stdout:mgr is available 2024-04-12T05:45:26.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:26 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3527279432' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [global] 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout fsid = b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-12T05:45:26.721 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout [osd] 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.728+0000 7efca0b87700 1 Processor -- start 2024-04-12T05:45:26.722 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.728+0000 7efca0b87700 1 -- start start 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.728+0000 7efca0b87700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.728+0000 7efca0b87700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c106110 con 0x7efc9c105720 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49020/0 (socket says 172.21.15.8:49020) 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9a59c700 1 -- 172.21.15.8:0/1342816508 learned_addr learned my addr 172.21.15.8:0/1342816508 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9a59c700 1 -- 172.21.15.8:0/1342816508 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c106440 con 0x7efc9c105720 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9a59c700 1 --2- 172.21.15.8:0/1342816508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0098a0 tx=0x7efc8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74dd87392471939 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9959a700 1 -- 172.21.15.8:0/1342816508 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c004180 con 0x7efc9c105720 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9959a700 1 -- 172.21.15.8:0/1342816508 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7efc8c0042e0 con 0x7efc9c105720 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.729+0000 7efc9959a700 1 -- 172.21.15.8:0/1342816508 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c0119d0 con 0x7efc9c105720 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 -- 172.21.15.8:0/1342816508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 msgr2=0x7efc9c105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 --2- 172.21.15.8:0/1342816508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0098a0 tx=0x7efc8c009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 -- 172.21.15.8:0/1342816508 shutdown_connections 2024-04-12T05:45:26.723 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 --2- 172.21.15.8:0/1342816508 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c105b40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 -- 172.21.15.8:0/1342816508 >> 172.21.15.8:0/1342816508 conn(0x7efc9c0fff40 msgr2=0x7efc9c102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 -- 172.21.15.8:0/1342816508 shutdown_connections 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.730+0000 7efca0b87700 1 -- 172.21.15.8:0/1342816508 wait complete. 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efca0b87700 1 Processor -- start 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efca0b87700 1 -- start start 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efca0b87700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efca0b87700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c19a4d0 con 0x7efc9c105720 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efc9a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efc9a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49034/0 (socket says 172.21.15.8:49034) 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efc9a59c700 1 -- 172.21.15.8:0/3527279432 learned_addr learned my addr 172.21.15.8:0/3527279432 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efc9a59c700 1 -- 172.21.15.8:0/3527279432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c009550 con 0x7efc9c105720 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.731+0000 7efc9a59c700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006af0 tx=0x7efc8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c024430 con 0x7efc9c105720 2024-04-12T05:45:26.724 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9c19a6d0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7efc8c0249e0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9c19aaf0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc8c01a8a0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.732+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7efc8c01aa00 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.733+0000 7efc8b7fe700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 0x7efc8403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.733+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9c0623c0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.733+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efc8c04c1f0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.733+0000 7efc99d9b700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 0x7efc8403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.733+0000 7efc99d9b700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 0x7efc8403a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc90006fd0 tx=0x7efc90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.738+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efc8c01acb0 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.932+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7efc9c19d950 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.933+0000 7efc8b7fe700 1 -- 172.21.15.8:0/3527279432 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7efc8c019e40 con 0x7efc9c105720 2024-04-12T05:45:26.725 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.935+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 msgr2=0x7efc8403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.935+0000 7efca0b87700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 0x7efc8403a790 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc90006fd0 tx=0x7efc90006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 msgr2=0x7efc9c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc8c006af0 tx=0x7efc8c011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 shutdown_connections 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7efc840382d0 0x7efc8403a790 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 --2- 172.21.15.8:0/3527279432 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efc9c105720 0x7efc9c199f90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 >> 172.21.15.8:0/3527279432 conn(0x7efc9c0fff40 msgr2=0x7efc9c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 shutdown_connections 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:25.936+0000 7efca0b87700 1 -- 172.21.15.8:0/3527279432 wait complete. 2024-04-12T05:45:26.726 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling cephadm module... 2024-04-12T05:45:27.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:27 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1863476038' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c870e1700 1 Processor -- start 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c870e1700 1 -- start start 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c870e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c870e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c80106110 con 0x7f8c80105720 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c84e7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:28.330 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c84e7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49048/0 (socket says 172.21.15.8:49048) 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.262+0000 7f8c84e7d700 1 -- 172.21.15.8:0/1789245934 learned_addr learned my addr 172.21.15.8:0/1789245934 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c84e7d700 1 -- 172.21.15.8:0/1789245934 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80106440 con 0x7f8c80105720 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c84e7d700 1 --2- 172.21.15.8:0/1789245934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400bb40 tx=0x7f8c7400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60c9db4f49208b server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c7f7fe700 1 -- 172.21.15.8:0/1789245934 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c7400dc00 con 0x7f8c80105720 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c7f7fe700 1 -- 172.21.15.8:0/1789245934 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f8c74004480 con 0x7f8c80105720 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c7f7fe700 1 -- 172.21.15.8:0/1789245934 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c74003e60 con 0x7f8c80105720 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c870e1700 1 -- 172.21.15.8:0/1789245934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 msgr2=0x7f8c80105b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.263+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1789245934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400bb40 tx=0x7f8c7400be50 comp rx=0 tx=0).stop 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 -- 172.21.15.8:0/1789245934 shutdown_connections 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1789245934 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80105b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 -- 172.21.15.8:0/1789245934 >> 172.21.15.8:0/1789245934 conn(0x7f8c800fff40 msgr2=0x7f8c80102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 -- 172.21.15.8:0/1789245934 shutdown_connections 2024-04-12T05:45:28.331 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 -- 172.21.15.8:0/1789245934 wait complete. 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 Processor -- start 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.264+0000 7f8c870e1700 1 -- start start 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c870e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c870e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801982f0 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c84e7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c84e7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49064/0 (socket says 172.21.15.8:49064) 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c84e7d700 1 -- 172.21.15.8:0/1863476038 learned_addr learned my addr 172.21.15.8:0/1863476038 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c84e7d700 1 -- 172.21.15.8:0/1863476038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7400b7f0 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c84e7d700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c740104d0 tx=0x7f8c74003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c7401e070 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f8c740109a0 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c801984f0 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.265+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8c74022a30 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.266+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c80198910 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.266+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f8c7402c440 con 0x7f8c80105720 2024-04-12T05:45:28.332 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.266+0000 7f8c7dffb700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 0x7f8c7003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.266+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8c7404e760 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.267+0000 7f8c7ffff700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 0x7f8c7003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.267+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c80191ad0 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.267+0000 7f8c7ffff700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 0x7f8c7003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006fd0 tx=0x7f8c6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.271+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f8c74027030 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.524+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8c801071f0 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.577+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f8c7402c740 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.578+0000 7f8c7dffb700 1 -- 172.21.15.8:0/1863476038 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8c7402a950 con 0x7f8c80105720 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 msgr2=0x7f8c7003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 0x7f8c7003a7e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006fd0 tx=0x7f8c6c006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 msgr2=0x7f8c80197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c740104d0 tx=0x7f8c74003fb0 comp rx=0 tx=0).stop 2024-04-12T05:45:28.333 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 shutdown_connections 2024-04-12T05:45:28.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f8c70038320 0x7f8c7003a7e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:28.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 --2- 172.21.15.8:0/1863476038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c80105720 0x7f8c80197db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:28.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 >> 172.21.15.8:0/1863476038 conn(0x7f8c800fff40 msgr2=0x7f8c8018ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:28.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 shutdown_connections 2024-04-12T05:45:28.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:27.582+0000 7f8c870e1700 1 -- 172.21.15.8:0/1863476038 wait complete. 2024-04-12T05:45:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:28 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1863476038' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-12T05:45:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:28 smithi008 ceph-mon[39722]: mgrmap e5: smithi008.ixmnlp(active, since 10s) 2024-04-12T05:45:29.863 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:45:29.863 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-12T05:45:29.863 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T05:45:29.863 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "smithi008.ixmnlp", 2024-04-12T05:45:29.863 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f917a2f9700 1 Processor -- start 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f917a2f9700 1 -- start start 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f917a2f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f917a2f9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9174106d20 con 0x7f9174106330 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49096/0 (socket says 172.21.15.8:49096) 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f9173fff700 1 -- 172.21.15.8:0/80235709 learned_addr learned my addr 172.21.15.8:0/80235709 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.900+0000 7f9173fff700 1 -- 172.21.15.8:0/80235709 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174107530 con 0x7f9174106330 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f9173fff700 1 --2- 172.21.15.8:0/80235709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56d4419bb7172bf server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f9172ffd700 1 -- 172.21.15.8:0/80235709 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9164004180 con 0x7f9174106330 2024-04-12T05:45:29.864 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f9172ffd700 1 -- 172.21.15.8:0/80235709 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f91640042e0 con 0x7f9174106330 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f9172ffd700 1 -- 172.21.15.8:0/80235709 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f91640119d0 con 0x7f9174106330 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 -- 172.21.15.8:0/80235709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 msgr2=0x7f9174106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 --2- 172.21.15.8:0/80235709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f91640098a0 tx=0x7f9164009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 -- 172.21.15.8:0/80235709 shutdown_connections 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 --2- 172.21.15.8:0/80235709 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174106750 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 -- 172.21.15.8:0/80235709 >> 172.21.15.8:0/80235709 conn(0x7f91741018d0 msgr2=0x7f9174103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 -- 172.21.15.8:0/80235709 shutdown_connections 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.901+0000 7f917a2f9700 1 -- 172.21.15.8:0/80235709 wait complete. 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.902+0000 7f917a2f9700 1 Processor -- start 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.902+0000 7f917a2f9700 1 -- start start 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f917a2f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f917a2f9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917419a510 con 0x7f9174106330 2024-04-12T05:45:29.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f9173fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49108/0 (socket says 172.21.15.8:49108) 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f9173fff700 1 -- 172.21.15.8:0/2088353375 learned_addr learned my addr 172.21.15.8:0/2088353375 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f9173fff700 1 -- 172.21.15.8:0/2088353375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9164009550 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f9173fff700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9164006af0 tx=0x7f9164011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.903+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9164024430 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.904+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f917419a710 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.904+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f91640249e0 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.904+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f916401a970 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.904+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f917419ab30 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.904+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f916401ac30 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.905+0000 7f91717fa700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 0x7f915c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.905+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f916404c3b0 con 0x7f9174106330 2024-04-12T05:45:29.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.905+0000 7f91737fe700 1 -- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 msgr2=0x7f915c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.905+0000 7f91737fe700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 0x7f915c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.905+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91740623c0 con 0x7f9174106330 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:28.910+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f91640193c0 con 0x7f9174106330 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.106+0000 7f91737fe700 1 -- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 msgr2=0x7f915c03a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.106+0000 7f91737fe700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 0x7f915c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.184+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f917410a730 con 0x7f9174106330 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.184+0000 7f91717fa700 1 -- 172.21.15.8:0/2088353375 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f9164029210 con 0x7f9174106330 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 msgr2=0x7f915c03a7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 0x7f915c03a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 msgr2=0x7f9174199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9164006af0 tx=0x7f9164011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 shutdown_connections 2024-04-12T05:45:29.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7f915c038320 0x7f915c03a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:29.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 --2- 172.21.15.8:0/2088353375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9174106330 0x7f9174199fd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:29.868 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:29 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2088353375' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-12T05:45:29.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 >> 172.21.15.8:0/2088353375 conn(0x7f91741018d0 msgr2=0x7f9174102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:29.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 shutdown_connections 2024-04-12T05:45:29.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:29.187+0000 7f917a2f9700 1 -- 172.21.15.8:0/2088353375 wait complete. 2024-04-12T05:45:29.869 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2024-04-12T05:45:29.869 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 5... 2024-04-12T05:45:34.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: Active manager daemon smithi008.ixmnlp restarted 2024-04-12T05:45:34.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: Activating manager daemon smithi008.ixmnlp 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: osdmap e2: 0 total, 0 up, 0 in 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: mgrmap e6: smithi008.ixmnlp(active, starting, since 0.00303698s) 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:34 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: Found migration_current of "None". Setting to last migration. 2024-04-12T05:45:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:45:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:45:35.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:35.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:35.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:35 smithi008 ceph-mon[39722]: mgrmap e7: smithi008.ixmnlp(active, since 1.00487s) 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde1bebd700 1 Processor -- start 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde1bebd700 1 -- start start 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde1bebd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde1bebd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde14106d20 con 0x7fde14106330 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde19c59700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:36.222 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde19c59700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49116/0 (socket says 172.21.15.8:49116) 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde19c59700 1 -- 172.21.15.8:0/3768268378 learned_addr learned my addr 172.21.15.8:0/3768268378 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.453+0000 7fde19c59700 1 -- 172.21.15.8:0/3768268378 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14107530 con 0x7fde14106330 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.454+0000 7fde19c59700 1 --2- 172.21.15.8:0/3768268378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde100098a0 tx=0x7fde10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51270dadf14b5823 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.454+0000 7fde18c57700 1 -- 172.21.15.8:0/3768268378 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde10004180 con 0x7fde14106330 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.454+0000 7fde18c57700 1 -- 172.21.15.8:0/3768268378 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fde100042e0 con 0x7fde14106330 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.454+0000 7fde18c57700 1 -- 172.21.15.8:0/3768268378 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde100119d0 con 0x7fde14106330 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- 172.21.15.8:0/3768268378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 msgr2=0x7fde14106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 --2- 172.21.15.8:0/3768268378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde100098a0 tx=0x7fde10009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- 172.21.15.8:0/3768268378 shutdown_connections 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 --2- 172.21.15.8:0/3768268378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14106750 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- 172.21.15.8:0/3768268378 >> 172.21.15.8:0/3768268378 conn(0x7fde141018d0 msgr2=0x7fde14103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- 172.21.15.8:0/3768268378 shutdown_connections 2024-04-12T05:45:36.223 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- 172.21.15.8:0/3768268378 wait complete. 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 Processor -- start 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.455+0000 7fde1bebd700 1 -- start start 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde1bebd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde1bebd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1419a510 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde19c59700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde19c59700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49128/0 (socket says 172.21.15.8:49128) 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde19c59700 1 -- 172.21.15.8:0/1688695466 learned_addr learned my addr 172.21.15.8:0/1688695466 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde19c59700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde10009550 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde19c59700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fde10006af0 tx=0x7fde10011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde10003b40 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1419a710 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fde1001a880 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde10019890 con 0x7fde14106330 2024-04-12T05:45:36.224 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.456+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1419ab30 con 0x7fde14106330 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.457+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fde10019b10 con 0x7fde14106330 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.458+0000 7fde0affd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.458+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde10003b40 con 0x7fde000382e0 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.458+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fde1004c6d0 con 0x7fde14106330 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.458+0000 7fde19458700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.458+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.659+0000 7fde19458700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:30.659+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:31.060+0000 7fde19458700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:31.060+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:31.861+0000 7fde19458700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:31.861+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:33.462+0000 7fde19458700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3166946531 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:33.462+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-12T05:45:36.225 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:34.569+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7fde10016d60 con 0x7fde14106330 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:34.569+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 msgr2=0x7fde0003a7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:34.569+0000 7fde0affd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.570+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7fde1001a9f0 con 0x7fde14106330 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.570+0000 7fde0affd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 0x7fde0003dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.570+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fde10003b40 con 0x7fde0003baf0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.570+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 0x7fde0003dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.571+0000 7fde19458700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 0x7fde0003dee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde04003a10 tx=0x7fde04009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.572+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fde10003b40 con 0x7fde0003baf0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fde1419b310 con 0x7fde0003baf0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde0affd700 1 -- 172.21.15.8:0/1688695466 <== mgr.14118 v2:172.21.15.8:6800/1010894527 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fde1419b310 con 0x7fde0003baf0 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 msgr2=0x7fde0003dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 0x7fde0003dee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fde04003a10 tx=0x7fde04009290 comp rx=0 tx=0).stop 2024-04-12T05:45:36.226 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 msgr2=0x7fde14199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fde10006af0 tx=0x7fde10011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 shutdown_connections 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fde0003baf0 0x7fde0003dee0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde14106330 0x7fde14199fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 --2- 172.21.15.8:0/1688695466 >> [v2:172.21.15.8:6800/3166946531,v1:172.21.15.8:6801/3166946531] conn(0x7fde000382e0 0x7fde0003a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 >> 172.21.15.8:0/1688695466 conn(0x7fde141018d0 msgr2=0x7fde14102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 shutdown_connections 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:35.578+0000 7fde1bebd700 1 -- 172.21.15.8:0/1688695466 wait complete. 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 5 is available 2024-04-12T05:45:36.227 INFO:teuthology.orchestra.run.smithi008.stdout:Setting orchestrator backend to cephadm... 2024-04-12T05:45:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:36 smithi008 ceph-mon[39722]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-12T05:45:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:36 smithi008 ceph-mon[39722]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-12T05:45:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:36 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:37.567 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.773+0000 7f6a81584700 1 Processor -- start 2024-04-12T05:45:37.567 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.773+0000 7f6a81584700 1 -- start start 2024-04-12T05:45:37.567 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.773+0000 7f6a81584700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:37.567 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.773+0000 7f6a81584700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c079d60 con 0x7f6a7c104c10 2024-04-12T05:45:37.567 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a7affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a7affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46382/0 (socket says 172.21.15.8:46382) 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a7affd700 1 -- 172.21.15.8:0/2656451332 learned_addr learned my addr 172.21.15.8:0/2656451332 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a7affd700 1 -- 172.21.15.8:0/2656451332 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c105570 con 0x7f6a7c104c10 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a7affd700 1 --2- 172.21.15.8:0/2656451332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0098a0 tx=0x7f6a6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf877ce9b219b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a79ffb700 1 -- 172.21.15.8:0/2656451332 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a6c004180 con 0x7f6a7c104c10 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a79ffb700 1 -- 172.21.15.8:0/2656451332 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f6a6c0042e0 con 0x7f6a7c104c10 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.774+0000 7f6a79ffb700 1 -- 172.21.15.8:0/2656451332 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a6c0119d0 con 0x7f6a7c104c10 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 -- 172.21.15.8:0/2656451332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 msgr2=0x7f6a7c105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 --2- 172.21.15.8:0/2656451332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0098a0 tx=0x7f6a6c009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 -- 172.21.15.8:0/2656451332 shutdown_connections 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 --2- 172.21.15.8:0/2656451332 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c105030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 -- 172.21.15.8:0/2656451332 >> 172.21.15.8:0/2656451332 conn(0x7f6a7c1007d0 msgr2=0x7f6a7c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:37.568 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 -- 172.21.15.8:0/2656451332 shutdown_connections 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 -- 172.21.15.8:0/2656451332 wait complete. 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.775+0000 7f6a81584700 1 Processor -- start 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a81584700 1 -- start start 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a81584700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a81584700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c19c6f0 con 0x7f6a7c104c10 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a7affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a7affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46398/0 (socket says 172.21.15.8:46398) 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a7affd700 1 -- 172.21.15.8:0/595293960 learned_addr learned my addr 172.21.15.8:0/595293960 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a7affd700 1 -- 172.21.15.8:0/595293960 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009550 con 0x7f6a7c104c10 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a7affd700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c006af0 tx=0x7f6a6c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.776+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a6c024430 con 0x7f6a7c104c10 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f6a6c0249e0 con 0x7f6a7c104c10 2024-04-12T05:45:37.569 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a6c01aa00 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c19c8f0 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c19cd10 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f6a6c01ac70 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.777+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c195a40 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.778+0000 7f6a73fff700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 0x7f6a6803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.778+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6a6c04c450 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.778+0000 7f6a7a7fc700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 0x7f6a6803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.778+0000 7f6a7a7fc700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 0x7f6a6803a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a64006fd0 tx=0x7f6a64006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.783+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6a6c0189a0 con 0x7f6a7c104c10 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:36.996+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a7c0611d0 con 0x7f6a68038370 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.002+0000 7f6a73fff700 1 -- 172.21.15.8:0/595293960 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6a7c0611d0 con 0x7f6a68038370 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.004+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 msgr2=0x7f6a6803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:37.570 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.004+0000 7f6a81584700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 0x7f6a6803a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6a64006fd0 tx=0x7f6a64006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.004+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 msgr2=0x7f6a7c19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.004+0000 7f6a81584700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c006af0 tx=0x7f6a6c011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 shutdown_connections 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f6a68038370 0x7f6a6803a830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 --2- 172.21.15.8:0/595293960 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6a7c104c10 0x7f6a7c19c1b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 >> 172.21.15.8:0/595293960 conn(0x7f6a7c1007d0 msgr2=0x7f6a7c1907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 shutdown_connections 2024-04-12T05:45:37.571 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:37.005+0000 7f6a81584700 1 -- 172.21.15.8:0/595293960 wait complete. 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: [12/Apr/2024:05:45:35] ENGINE Bus STARTING 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: [12/Apr/2024:05:45:36] ENGINE Serving on https://172.21.15.8:7150 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: [12/Apr/2024:05:45:36] ENGINE Serving on http://172.21.15.8:8765 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: [12/Apr/2024:05:45:36] ENGINE Bus STARTED 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: mgrmap e8: smithi008.ixmnlp(active, since 2s) 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:37.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:37 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:38 smithi008 ceph-mon[39722]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c2cd1700 1 Processor -- start 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c2cd1700 1 -- start start 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c2cd1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c2cd1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc105c20 con 0x7f25bc105230 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c0a6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c0a6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46408/0 (socket says 172.21.15.8:46408) 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c0a6d700 1 -- 172.21.15.8:0/1300412272 learned_addr learned my addr 172.21.15.8:0/1300412272 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c0a6d700 1 -- 172.21.15.8:0/1300412272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc106430 con 0x7f25bc105230 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.142+0000 7f25c0a6d700 1 --2- 172.21.15.8:0/1300412272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25b00098a0 tx=0x7f25b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad033f3f2c5043de server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:39.008 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25bb7fe700 1 -- 172.21.15.8:0/1300412272 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25b0004180 con 0x7f25bc105230 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25bb7fe700 1 -- 172.21.15.8:0/1300412272 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f25b00042e0 con 0x7f25bc105230 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25bb7fe700 1 -- 172.21.15.8:0/1300412272 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25b00119d0 con 0x7f25bc105230 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 -- 172.21.15.8:0/1300412272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 msgr2=0x7f25bc105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/1300412272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f25b00098a0 tx=0x7f25b0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 -- 172.21.15.8:0/1300412272 shutdown_connections 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/1300412272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc105650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 -- 172.21.15.8:0/1300412272 >> 172.21.15.8:0/1300412272 conn(0x7f25bc1007f0 msgr2=0x7f25bc102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 -- 172.21.15.8:0/1300412272 shutdown_connections 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.143+0000 7f25c2cd1700 1 -- 172.21.15.8:0/1300412272 wait complete. 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c2cd1700 1 Processor -- start 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c2cd1700 1 -- start start 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c2cd1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c2cd1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc190940 con 0x7f25bc105230 2024-04-12T05:45:39.009 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c0a6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c0a6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46418/0 (socket says 172.21.15.8:46418) 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c0a6d700 1 -- 172.21.15.8:0/461455506 learned_addr learned my addr 172.21.15.8:0/461455506 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.144+0000 7f25c0a6d700 1 -- 172.21.15.8:0/461455506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009550 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25c0a6d700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009130 tx=0x7f25b0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25b0003a00 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f25b001a880 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc18ea50 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25b0019760 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.145+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc18ee70 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.146+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f25b0019a20 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.146+0000 7f25b9ffb700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 0x7f25ac03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.146+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc0623c0 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.146+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f25b004c700 con 0x7f25bc105230 2024-04-12T05:45:39.010 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.148+0000 7f25bbfff700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 0x7f25ac03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.148+0000 7f25bbfff700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 0x7f25ac03a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006fd0 tx=0x7f25a8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.151+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f25b002c3a0 con 0x7f25bc105230 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.350+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f25bc1029e0 con 0x7f25ac0382e0 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.351+0000 7f25b9ffb700 1 -- 172.21.15.8:0/461455506 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f25bc1029e0 con 0x7f25ac0382e0 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.353+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 msgr2=0x7f25ac03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.353+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 0x7f25ac03a7a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006fd0 tx=0x7f25a8006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.353+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 msgr2=0x7f25bc190400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.353+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009130 tx=0x7f25b0011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.353+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 shutdown_connections 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.354+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f25ac0382e0 0x7f25ac03a7a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.354+0000 7f25c2cd1700 1 --2- 172.21.15.8:0/461455506 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25bc105230 0x7f25bc190400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.354+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 >> 172.21.15.8:0/461455506 conn(0x7f25bc1007f0 msgr2=0x7f25bc073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.354+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 shutdown_connections 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:38.354+0000 7f25c2cd1700 1 -- 172.21.15.8:0/461455506 wait complete. 2024-04-12T05:45:39.011 INFO:teuthology.orchestra.run.smithi008.stdout:Generating ssh key... 2024-04-12T05:45:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:39 smithi008 ceph-mon[39722]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:41.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.761+0000 7f22cd0ce700 1 Processor -- start 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.761+0000 7f22cd0ce700 1 -- start start 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.761+0000 7f22cd0ce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22cd0ce700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8106d00 con 0x7f22c8106310 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46428/0 (socket says 172.21.15.8:46428) 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c6d9d700 1 -- 172.21.15.8:0/3589960959 learned_addr learned my addr 172.21.15.8:0/3589960959 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c6d9d700 1 -- 172.21.15.8:0/3589960959 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8107510 con 0x7f22c8106310 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c6d9d700 1 --2- 172.21.15.8:0/3589960959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098a0 tx=0x7f22b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba8800fb83d92a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c5d9b700 1 -- 172.21.15.8:0/3589960959 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22b8004180 con 0x7f22c8106310 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c5d9b700 1 -- 172.21.15.8:0/3589960959 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f22b80042e0 con 0x7f22c8106310 2024-04-12T05:45:41.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.762+0000 7f22c5d9b700 1 -- 172.21.15.8:0/3589960959 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22b80119d0 con 0x7f22c8106310 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 -- 172.21.15.8:0/3589960959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 msgr2=0x7f22c8106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/3589960959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098a0 tx=0x7f22b8009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 -- 172.21.15.8:0/3589960959 shutdown_connections 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/3589960959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c8106730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 -- 172.21.15.8:0/3589960959 >> 172.21.15.8:0/3589960959 conn(0x7f22c81018d0 msgr2=0x7f22c8103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 -- 172.21.15.8:0/3589960959 shutdown_connections 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.763+0000 7f22cd0ce700 1 -- 172.21.15.8:0/3589960959 wait complete. 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22cd0ce700 1 Processor -- start 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22cd0ce700 1 -- start start 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22cd0ce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22cd0ce700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819a550 con 0x7f22c8106310 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46436/0 (socket says 172.21.15.8:46436) 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.764+0000 7f22c6d9d700 1 -- 172.21.15.8:0/1725606659 learned_addr learned my addr 172.21.15.8:0/1725606659 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:41.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22c6d9d700 1 -- 172.21.15.8:0/1725606659 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8009550 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22c6d9d700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006af0 tx=0x7f22b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22b8024430 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c819a750 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f22b80249e0 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22b801a970 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.765+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c819ab70 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.766+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f22b801ac30 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.766+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c80623c0 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.766+0000 7f22bffff700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 0x7f22b403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.766+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f22b804c3f0 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.766+0000 7f22c659c700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 0x7f22b403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.767+0000 7f22c659c700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 0x7f22b403a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006fd0 tx=0x7f22b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.771+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f22b80193c0 con 0x7f22c8106310 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:39.969+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f22c8103bd0 con 0x7f22b4038320 2024-04-12T05:45:41.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.536+0000 7f22bffff700 1 -- 172.21.15.8:0/1725606659 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22c8103bd0 con 0x7f22b4038320 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 msgr2=0x7f22b403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 0x7f22b403a7e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006fd0 tx=0x7f22b0006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 msgr2=0x7f22c819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22b8006af0 tx=0x7f22b8011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 shutdown_connections 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f22b4038320 0x7f22b403a7e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 --2- 172.21.15.8:0/1725606659 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22c8106310 0x7f22c819a010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 >> 172.21.15.8:0/1725606659 conn(0x7f22c81018d0 msgr2=0x7f22c8102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 shutdown_connections 2024-04-12T05:45:41.185 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:40.538+0000 7f22cd0ce700 1 -- 172.21.15.8:0/1725606659 wait complete. 2024-04-12T05:45:41.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:41 smithi008 ceph-mon[39722]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:41.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:41 smithi008 ceph-mon[39722]: Generating ssh key... 2024-04-12T05:45:41.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:41 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:41.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:41 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAi/AVRPGp614QppzArhmxjIl/6lnCQMfg6xlznI4WRGm7AafuZqG4qsSxR4B7i6UOSTQ0JaekQ0gOJM4mHozI5wwFkqTexfCPngq9Y31VL5ANzU6txfuyiFn7OVQre+ObSx8W6P3lzfWRpmi08ksmLS7vEtpm9VnpX0YujOKbB7fIgfRHMTqQpygkhQYsKmCqmMsavGpIUlOsABgsL8u9hUxJjyg5+pktIjtvlxgfPCxWq+7Ru4ylxdbh6aEMZm/5E0JOOs6IWJph+I04VzcmAzGfU93qlgpWJxA4eVxIiiu0VpBdF1qR1mPV8mPfA/C0NfFIfcoTkNb3j3yfBtvG3ZjGyHM/EKKuJucnCiyaugTgpxhIG7nRPqXiN12PbigIgLtIBny55EnMgxnnJdlHDHhr7v1CLV3ww0zd0AvHJHPpdZDZ5I013U39TtQ0l/WreCl5Knea+Lad+WUvw8g9b6tLEwwVDa7QXR2FnlJX4YNSQN0ihL+psa4F8mgHDwE= ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.704+0000 7f3b8c955700 1 Processor -- start 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8c955700 1 -- start start 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8c955700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8c955700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b880744f0 con 0x7f3b88106de0 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46450/0 (socket says 172.21.15.8:46450) 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8659c700 1 -- 172.21.15.8:0/1518680621 learned_addr learned my addr 172.21.15.8:0/1518680621 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:42.513 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.705+0000 7f3b8659c700 1 -- 172.21.15.8:0/1518680621 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b88109710 con 0x7f3b88106de0 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8659c700 1 --2- 172.21.15.8:0/1518680621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098a0 tx=0x7f3b78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fb471f90095e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8559a700 1 -- 172.21.15.8:0/1518680621 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b78004180 con 0x7f3b88106de0 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8559a700 1 -- 172.21.15.8:0/1518680621 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3b780042e0 con 0x7f3b88106de0 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8559a700 1 -- 172.21.15.8:0/1518680621 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b780119d0 con 0x7f3b88106de0 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 -- 172.21.15.8:0/1518680621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 msgr2=0x7f3b881091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 --2- 172.21.15.8:0/1518680621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3b780098a0 tx=0x7f3b78009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 -- 172.21.15.8:0/1518680621 shutdown_connections 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 --2- 172.21.15.8:0/1518680621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b881091d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 -- 172.21.15.8:0/1518680621 >> 172.21.15.8:0/1518680621 conn(0x7f3b881007f0 msgr2=0x7f3b88102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.706+0000 7f3b8c955700 1 -- 172.21.15.8:0/1518680621 shutdown_connections 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8c955700 1 -- 172.21.15.8:0/1518680621 wait complete. 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8c955700 1 Processor -- start 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8c955700 1 -- start start 2024-04-12T05:45:42.514 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8c955700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8c955700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b88106c80 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.707+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46456/0 (socket says 172.21.15.8:46456) 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8659c700 1 -- 172.21.15.8:0/2976422909 learned_addr learned my addr 172.21.15.8:0/2976422909 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8659c700 1 -- 172.21.15.8:0/2976422909 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b78009550 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8659c700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009130 tx=0x7f3b78011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b78003a00 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f3b7801a880 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b780197b0 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b88104d90 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.708+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b881051b0 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.709+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f3b78019a20 con 0x7f3b88106de0 2024-04-12T05:45:42.515 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.709+0000 7f3b7f7fe700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 0x7f3b7403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.709+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b7804c780 con 0x7f3b88106de0 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.709+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8810a400 con 0x7f3b88106de0 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.709+0000 7f3b85d9b700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 0x7f3b7403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.710+0000 7f3b85d9b700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 0x7f3b7403a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006fd0 tx=0x7f3b70006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.714+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3b7802c3a0 con 0x7f3b88106de0 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.911+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b88102b00 con 0x7f3b74038320 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.912+0000 7f3b7f7fe700 1 -- 172.21.15.8:0/2976422909 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3b88102b00 con 0x7f3b74038320 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 msgr2=0x7f3b7403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 0x7f3b7403a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006fd0 tx=0x7f3b70006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 msgr2=0x7f3b88106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3b78009130 tx=0x7f3b78011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 shutdown_connections 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f3b74038320 0x7f3b7403a7e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 --2- 172.21.15.8:0/2976422909 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b88106de0 0x7f3b88106740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:42.516 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 >> 172.21.15.8:0/2976422909 conn(0x7f3b881007f0 msgr2=0x7f3b88101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:42.517 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 shutdown_connections 2024-04-12T05:45:42.517 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:41.914+0000 7f3b8c955700 1 -- 172.21.15.8:0/2976422909 wait complete. 2024-04-12T05:45:42.517 INFO:teuthology.orchestra.run.smithi008.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-12T05:45:42.517 INFO:teuthology.orchestra.run.smithi008.stdout:Adding key to root@localhost authorized_keys... 2024-04-12T05:45:42.517 INFO:teuthology.orchestra.run.smithi008.stdout:Adding host smithi008... 2024-04-12T05:45:42.781 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:42 smithi008 ceph-mon[39722]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:43.851 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:43 smithi008 ceph-mon[39722]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:45.049 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:44 smithi008 ceph-mon[39722]: Deploying cephadm binary to smithi008 2024-04-12T05:45:46.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Added host 'smithi008' with addr '172.21.15.8' 2024-04-12T05:45:46.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.087+0000 7fc4da25c700 1 Processor -- start 2024-04-12T05:45:46.163 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.087+0000 7fc4da25c700 1 -- start start 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.087+0000 7fc4da25c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.087+0000 7fc4da25c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4106d20 con 0x7fc4d4106330 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46466/0 (socket says 172.21.15.8:46466) 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d37fe700 1 -- 172.21.15.8:0/2807214098 learned_addr learned my addr 172.21.15.8:0/2807214098 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d37fe700 1 -- 172.21.15.8:0/2807214098 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4107530 con 0x7fc4d4106330 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d37fe700 1 --2- 172.21.15.8:0/2807214098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ae05a2e532713f server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d27fc700 1 -- 172.21.15.8:0/2807214098 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4bc004180 con 0x7fc4d4106330 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d27fc700 1 -- 172.21.15.8:0/2807214098 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fc4bc0042e0 con 0x7fc4d4106330 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.088+0000 7fc4d27fc700 1 -- 172.21.15.8:0/2807214098 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4bc0119d0 con 0x7fc4d4106330 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 -- 172.21.15.8:0/2807214098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 msgr2=0x7fc4d4106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 --2- 172.21.15.8:0/2807214098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0098a0 tx=0x7fc4bc009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 -- 172.21.15.8:0/2807214098 shutdown_connections 2024-04-12T05:45:46.164 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 --2- 172.21.15.8:0/2807214098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d4106750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 -- 172.21.15.8:0/2807214098 >> 172.21.15.8:0/2807214098 conn(0x7fc4d41018d0 msgr2=0x7fc4d4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 -- 172.21.15.8:0/2807214098 shutdown_connections 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 -- 172.21.15.8:0/2807214098 wait complete. 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.089+0000 7fc4da25c700 1 Processor -- start 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4da25c700 1 -- start start 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4da25c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4da25c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d419c760 con 0x7fc4d4106330 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46478/0 (socket says 172.21.15.8:46478) 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4d37fe700 1 -- 172.21.15.8:0/931312891 learned_addr learned my addr 172.21.15.8:0/931312891 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4d37fe700 1 -- 172.21.15.8:0/931312891 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc009550 con 0x7fc4d4106330 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.090+0000 7fc4d37fe700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006af0 tx=0x7fc4bc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4bc003b40 con 0x7fc4d4106330 2024-04-12T05:45:46.165 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7fc4bc01a880 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d419c960 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc4bc019890 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d419cd80 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.091+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc4bc019ac0 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.092+0000 7fc4d0ff9700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 0x7fc4c003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.092+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc4bc04c670 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.092+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d40623c0 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.092+0000 7fc4d2ffd700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 0x7fc4c003a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.092+0000 7fc4d2ffd700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 0x7fc4c003a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4006fd0 tx=0x7fc4c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.097+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc4bc02c3a0 con 0x7fc4d4106330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:43.303+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi008", "addr": "172.21.15.8", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4d4103ba0 con 0x7fc4c0038330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.500+0000 7fc4d0ff9700 1 -- 172.21.15.8:0/931312891 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc4d4103ba0 con 0x7fc4c0038330 2024-04-12T05:45:46.166 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 msgr2=0x7fc4c003a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 0x7fc4c003a7f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4006fd0 tx=0x7fc4c4006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 msgr2=0x7fc4d419c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc006af0 tx=0x7fc4bc011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 shutdown_connections 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc4c0038330 0x7fc4c003a7f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 --2- 172.21.15.8:0/931312891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4d4106330 0x7fc4d419c220 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.502+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 >> 172.21.15.8:0/931312891 conn(0x7fc4d41018d0 msgr2=0x7fc4d4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.503+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 shutdown_connections 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:45.503+0000 7fc4da25c700 1 -- 172.21.15.8:0/931312891 wait complete. 2024-04-12T05:45:46.167 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying mon service with default placement... 2024-04-12T05:45:46.800 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:46 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:46.801 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:46 smithi008 ceph-mon[39722]: Added host smithi008 2024-04-12T05:45:46.801 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:46 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:45:47.865 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92e523a700 1 Processor -- start 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92e523a700 1 -- start start 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92e523a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92e523a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0079da0 con 0x7f92e0106de0 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43082/0 (socket says 172.21.15.8:43082) 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92ded9d700 1 -- 172.21.15.8:0/4131930248 learned_addr learned my addr 172.21.15.8:0/4131930248 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92ded9d700 1 -- 172.21.15.8:0/4131930248 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0109710 con 0x7f92e0106de0 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.695+0000 7f92ded9d700 1 --2- 172.21.15.8:0/4131930248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f92d00098a0 tx=0x7f92d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b8759253b602d server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92ddd9b700 1 -- 172.21.15.8:0/4131930248 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92d0004180 con 0x7f92e0106de0 2024-04-12T05:45:47.866 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92ddd9b700 1 -- 172.21.15.8:0/4131930248 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f92d00042e0 con 0x7f92e0106de0 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92ddd9b700 1 -- 172.21.15.8:0/4131930248 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92d00119d0 con 0x7f92e0106de0 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 -- 172.21.15.8:0/4131930248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 msgr2=0x7f92e01091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 --2- 172.21.15.8:0/4131930248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f92d00098a0 tx=0x7f92d0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 -- 172.21.15.8:0/4131930248 shutdown_connections 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 --2- 172.21.15.8:0/4131930248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e01091d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 -- 172.21.15.8:0/4131930248 >> 172.21.15.8:0/4131930248 conn(0x7f92e01007d0 msgr2=0x7f92e0102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 -- 172.21.15.8:0/4131930248 shutdown_connections 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.696+0000 7f92e523a700 1 -- 172.21.15.8:0/4131930248 wait complete. 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92e523a700 1 Processor -- start 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92e523a700 1 -- start start 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92e523a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92e523a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e019a520 con 0x7f92e0106de0 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:47.867 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.697+0000 7f92ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43094/0 (socket says 172.21.15.8:43094) 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.698+0000 7f92ded9d700 1 -- 172.21.15.8:0/2277680386 learned_addr learned my addr 172.21.15.8:0/2277680386 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.698+0000 7f92ded9d700 1 -- 172.21.15.8:0/2277680386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d0009550 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92ded9d700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92d0006af0 tx=0x7f92d0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92d0024430 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e019a720 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7f92d00249e0 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f92d001aa00 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.701+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e019ab40 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.702+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f92d001acc0 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.702+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e00623c0 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.702+0000 7f92c7fff700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 0x7f92c803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.702+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f92d004c420 con 0x7f92e0106de0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.702+0000 7f92de59c700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 0x7f92c803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.703+0000 7f92de59c700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 0x7f92c803a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:47.868 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.707+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f92d002de30 con 0x7f92e0106de0 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.917+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e0102ad0 con 0x7f92c8038320 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.921+0000 7f92c7fff700 1 -- 172.21.15.8:0/2277680386 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f92e0102ad0 con 0x7f92c8038320 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 msgr2=0x7f92c803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 0x7f92c803a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 msgr2=0x7f92e0199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f92d0006af0 tx=0x7f92d0011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 shutdown_connections 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f92c8038320 0x7f92c803a7e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 --2- 172.21.15.8:0/2277680386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f92e0106de0 0x7f92e0199fe0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.923+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 >> 172.21.15.8:0/2277680386 conn(0x7f92e01007d0 msgr2=0x7f92e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.924+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 shutdown_connections 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:46.924+0000 7f92e523a700 1 -- 172.21.15.8:0/2277680386 wait complete. 2024-04-12T05:45:47.869 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying mgr service with default placement... 2024-04-12T05:45:48.171 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:47 smithi008 ceph-mon[39722]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:48.171 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:47 smithi008 ceph-mon[39722]: Saving service mon spec with placement count:5 2024-04-12T05:45:48.171 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:47 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.472+0000 7ff1566f5700 1 Processor -- start 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.472+0000 7ff1566f5700 1 -- start start 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.472+0000 7ff1566f5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.472+0000 7ff1566f5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff150106d20 con 0x7ff150106330 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43096/0 (socket says 172.21.15.8:43096) 2024-04-12T05:45:49.375 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14ffff700 1 -- 172.21.15.8:0/938637412 learned_addr learned my addr 172.21.15.8:0/938637412 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14ffff700 1 -- 172.21.15.8:0/938637412 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150107530 con 0x7ff150106330 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14ffff700 1 --2- 172.21.15.8:0/938637412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5232572d852ad690 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14effd700 1 -- 172.21.15.8:0/938637412 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff140004180 con 0x7ff150106330 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14effd700 1 -- 172.21.15.8:0/938637412 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7ff1400042e0 con 0x7ff150106330 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.473+0000 7ff14effd700 1 -- 172.21.15.8:0/938637412 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1400119d0 con 0x7ff150106330 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 -- 172.21.15.8:0/938637412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 msgr2=0x7ff150106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 --2- 172.21.15.8:0/938637412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 -- 172.21.15.8:0/938637412 shutdown_connections 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 --2- 172.21.15.8:0/938637412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150106750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 -- 172.21.15.8:0/938637412 >> 172.21.15.8:0/938637412 conn(0x7ff1501018d0 msgr2=0x7ff150103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 -- 172.21.15.8:0/938637412 shutdown_connections 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 -- 172.21.15.8:0/938637412 wait complete. 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.474+0000 7ff1566f5700 1 Processor -- start 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff1566f5700 1 -- start start 2024-04-12T05:45:49.376 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff1566f5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff1566f5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15019a510 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff14ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43098/0 (socket says 172.21.15.8:43098) 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff14ffff700 1 -- 172.21.15.8:0/683166621 learned_addr learned my addr 172.21.15.8:0/683166621 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff14ffff700 1 -- 172.21.15.8:0/683166621 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009550 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.475+0000 7ff14ffff700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff140006af0 tx=0x7ff140011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff140024430 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1002+0+0 (secure 0 0 0) 0x7ff1400249e0 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15019a710 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff14001a970 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15019ab30 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.476+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff14001ac30 con 0x7ff150106330 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.477+0000 7ff14d7fa700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 0x7ff13803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:49.377 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.477+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1500623c0 con 0x7ff150106330 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.477+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff14004c530 con 0x7ff150106330 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.477+0000 7ff14f7fe700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 0x7ff13803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.477+0000 7ff14f7fe700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 0x7ff13803a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff144006fd0 tx=0x7ff144006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.482+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff14002d3f0 con 0x7ff150106330 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.692+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7ff150103ba0 con 0x7ff138038320 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.697+0000 7ff14d7fa700 1 -- 172.21.15.8:0/683166621 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff150103ba0 con 0x7ff138038320 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.698+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 msgr2=0x7ff13803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.698+0000 7ff1566f5700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 0x7ff13803a7e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff144006fd0 tx=0x7ff144006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.698+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 msgr2=0x7ff150199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.698+0000 7ff1566f5700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff140006af0 tx=0x7ff140011a30 comp rx=0 tx=0).stop 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 shutdown_connections 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff138038320 0x7ff13803a7e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:49.378 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 --2- 172.21.15.8:0/683166621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff150106330 0x7ff150199fd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:49.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 >> 172.21.15.8:0/683166621 conn(0x7ff1501018d0 msgr2=0x7ff150102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:49.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 shutdown_connections 2024-04-12T05:45:49.379 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:48.699+0000 7ff1566f5700 1 -- 172.21.15.8:0/683166621 wait complete. 2024-04-12T05:45:49.379 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying crash service with default placement... 2024-04-12T05:45:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:49 smithi008 ceph-mon[39722]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:49 smithi008 ceph-mon[39722]: Saving service mgr spec with placement count:2 2024-04-12T05:45:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:49 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:49 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:49 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:50.908 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-12T05:45:50.908 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.941+0000 7f5e70052700 1 Processor -- start 2024-04-12T05:45:50.908 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.941+0000 7f5e70052700 1 -- start start 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e70052700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e70052700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68105c20 con 0x7f5e68105230 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e6ddee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e6ddee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43112/0 (socket says 172.21.15.8:43112) 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e6ddee700 1 -- 172.21.15.8:0/3542798233 learned_addr learned my addr 172.21.15.8:0/3542798233 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e6ddee700 1 -- 172.21.15.8:0/3542798233 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e68106430 con 0x7f5e68105230 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.942+0000 7f5e6ddee700 1 --2- 172.21.15.8:0/3542798233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098a0 tx=0x7f5e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c808af6a44495050 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e6cdec700 1 -- 172.21.15.8:0/3542798233 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e64004180 con 0x7f5e68105230 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e6cdec700 1 -- 172.21.15.8:0/3542798233 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5e640042e0 con 0x7f5e68105230 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e6cdec700 1 -- 172.21.15.8:0/3542798233 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e64011a30 con 0x7f5e68105230 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e70052700 1 -- 172.21.15.8:0/3542798233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 msgr2=0x7f5e68105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e70052700 1 --2- 172.21.15.8:0/3542798233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e640098a0 tx=0x7f5e64009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:50.909 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e70052700 1 -- 172.21.15.8:0/3542798233 shutdown_connections 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e70052700 1 --2- 172.21.15.8:0/3542798233 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e68105650 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.943+0000 7f5e70052700 1 -- 172.21.15.8:0/3542798233 >> 172.21.15.8:0/3542798233 conn(0x7f5e681007d0 msgr2=0x7f5e68102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 -- 172.21.15.8:0/3542798233 shutdown_connections 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 -- 172.21.15.8:0/3542798233 wait complete. 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 Processor -- start 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 -- start start 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.944+0000 7f5e70052700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68190900 con 0x7f5e68105230 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e6ddee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e6ddee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43118/0 (socket says 172.21.15.8:43118) 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e6ddee700 1 -- 172.21.15.8:0/3783664705 learned_addr learned my addr 172.21.15.8:0/3783664705 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e6ddee700 1 -- 172.21.15.8:0/3783664705 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e64009550 con 0x7f5e68105230 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e6ddee700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c00 tx=0x7f5e64011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:50.910 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e64003bc0 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6818ea10 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5e640248d0 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.945+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5e6401a890 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.946+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6818ee30 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.946+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e680623c0 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.947+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f5e6401ab50 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.947+0000 7f5e5effd700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 0x7f5e5403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.947+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5e6404c580 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.947+0000 7f5e6d5ed700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 0x7f5e5403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.947+0000 7f5e6d5ed700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 0x7f5e5403a7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006fd0 tx=0x7f5e58006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:49.954+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5e6401ae00 con 0x7f5e68105230 2024-04-12T05:45:50.911 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.169+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e68102b20 con 0x7f5e54038320 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.173+0000 7f5e5effd700 1 -- 172.21.15.8:0/3783664705 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5e68102b20 con 0x7f5e54038320 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 msgr2=0x7f5e5403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 0x7f5e5403a7e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5e58006fd0 tx=0x7f5e58006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 msgr2=0x7f5e681903c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e64000c00 tx=0x7f5e64011ab0 comp rx=0 tx=0).stop 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 shutdown_connections 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f5e54038320 0x7f5e5403a7e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 --2- 172.21.15.8:0/3783664705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5e68105230 0x7f5e681903c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:50.912 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 >> 172.21.15.8:0/3783664705 conn(0x7f5e681007d0 msgr2=0x7f5e68101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:50.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 shutdown_connections 2024-04-12T05:45:50.913 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:50.175+0000 7f5e70052700 1 -- 172.21.15.8:0/3783664705 wait complete. 2024-04-12T05:45:50.913 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying ceph-exporter service with default placement... 2024-04-12T05:45:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:51 smithi008 ceph-mon[39722]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:51 smithi008 ceph-mon[39722]: Saving service crash spec with placement * 2024-04-12T05:45:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:51 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:52.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-12T05:45:52.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.509+0000 7fa5cf6ea700 1 Processor -- start 2024-04-12T05:45:52.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cf6ea700 1 -- start start 2024-04-12T05:45:52.463 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cf6ea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cf6ea700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8105c20 con 0x7fa5c8105230 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cd486700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cd486700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43134/0 (socket says 172.21.15.8:43134) 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cd486700 1 -- 172.21.15.8:0/4170996902 learned_addr learned my addr 172.21.15.8:0/4170996902 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.510+0000 7fa5cd486700 1 -- 172.21.15.8:0/4170996902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8106430 con 0x7fa5c8105230 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cd486700 1 --2- 172.21.15.8:0/4170996902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80098a0 tx=0x7fa5b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71253fa844595c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5bffff700 1 -- 172.21.15.8:0/4170996902 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b8004180 con 0x7fa5c8105230 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5bffff700 1 -- 172.21.15.8:0/4170996902 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa5b80042e0 con 0x7fa5c8105230 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5bffff700 1 -- 172.21.15.8:0/4170996902 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b8011a30 con 0x7fa5c8105230 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/4170996902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 msgr2=0x7fa5c8105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/4170996902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80098a0 tx=0x7fa5b8009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/4170996902 shutdown_connections 2024-04-12T05:45:52.464 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/4170996902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c8105650 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/4170996902 >> 172.21.15.8:0/4170996902 conn(0x7fa5c81007f0 msgr2=0x7fa5c8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/4170996902 shutdown_connections 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.511+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/4170996902 wait complete. 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cf6ea700 1 Processor -- start 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cf6ea700 1 -- start start 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cf6ea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cf6ea700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819a5d0 con 0x7fa5c8105230 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cd486700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:52.465 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cd486700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43136/0 (socket says 172.21.15.8:43136) 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cd486700 1 -- 172.21.15.8:0/1548245905 learned_addr learned my addr 172.21.15.8:0/1548245905 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.512+0000 7fa5cd486700 1 -- 172.21.15.8:0/1548245905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8009550 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5cd486700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006af0 tx=0x7fa5b8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b8003a80 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa5b8003be0 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa5b8019770 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c819a7d0 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.513+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c819abf0 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa5b80199e0 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c80623c0 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5be7fc700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 0x7fa5b403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa5b804c720 con 0x7fa5c8105230 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5ccc85700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 0x7fa5b403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:52.466 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.514+0000 7fa5ccc85700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 0x7fa5b403a7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.519+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa5b802b750 con 0x7fa5c8105230 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.730+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c8102a70 con 0x7fa5b4038330 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.734+0000 7fa5be7fc700 1 -- 172.21.15.8:0/1548245905 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa5c8102a70 con 0x7fa5b4038330 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 msgr2=0x7fa5b403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 0x7fa5b403a7f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4006fd0 tx=0x7fa5c4006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 msgr2=0x7fa5c819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006af0 tx=0x7fa5b8011b20 comp rx=0 tx=0).stop 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 shutdown_connections 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa5b4038330 0x7fa5b403a7f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 --2- 172.21.15.8:0/1548245905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c8105230 0x7fa5c819a090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 >> 172.21.15.8:0/1548245905 conn(0x7fa5c81007f0 msgr2=0x7fa5c8073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 shutdown_connections 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:51.736+0000 7fa5cf6ea700 1 -- 172.21.15.8:0/1548245905 wait complete. 2024-04-12T05:45:52.467 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying prometheus service with default placement... 2024-04-12T05:45:53.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:52 smithi008 ceph-mon[39722]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:53.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:52 smithi008 ceph-mon[39722]: Saving service ceph-exporter spec with placement * 2024-04-12T05:45:53.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:52 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:53.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:52 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:54.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-12T05:45:54.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.100+0000 7f10ebff8700 1 Processor -- start 2024-04-12T05:45:54.101 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.100+0000 7f10ebff8700 1 -- start start 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10ebff8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10ebff8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40744b0 con 0x7f10e4104c10 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e9d94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e9d94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43138/0 (socket says 172.21.15.8:43138) 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e9d94700 1 -- 172.21.15.8:0/3551006516 learned_addr learned my addr 172.21.15.8:0/3551006516 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e9d94700 1 -- 172.21.15.8:0/3551006516 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4105570 con 0x7f10e4104c10 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e9d94700 1 --2- 172.21.15.8:0/3551006516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10e00098a0 tx=0x7f10e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f4d25096680be server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.101+0000 7f10e8d92700 1 -- 172.21.15.8:0/3551006516 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e0004180 con 0x7f10e4104c10 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10e8d92700 1 -- 172.21.15.8:0/3551006516 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10e00042e0 con 0x7f10e4104c10 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10e8d92700 1 -- 172.21.15.8:0/3551006516 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e0011a30 con 0x7f10e4104c10 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 -- 172.21.15.8:0/3551006516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 msgr2=0x7f10e4105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 --2- 172.21.15.8:0/3551006516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10e00098a0 tx=0x7f10e0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:54.102 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 -- 172.21.15.8:0/3551006516 shutdown_connections 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 --2- 172.21.15.8:0/3551006516 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e4105030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 -- 172.21.15.8:0/3551006516 >> 172.21.15.8:0/3551006516 conn(0x7f10e41007f0 msgr2=0x7f10e4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 -- 172.21.15.8:0/3551006516 shutdown_connections 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.102+0000 7f10ebff8700 1 -- 172.21.15.8:0/3551006516 wait complete. 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10ebff8700 1 Processor -- start 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10ebff8700 1 -- start start 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10ebff8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10ebff8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e419c790 con 0x7f10e4104c10 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10e9d94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10e9d94700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43154/0 (socket says 172.21.15.8:43154) 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.103+0000 7f10e9d94700 1 -- 172.21.15.8:0/2946404359 learned_addr learned my addr 172.21.15.8:0/2946404359 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10e9d94700 1 -- 172.21.15.8:0/2946404359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0009550 con 0x7f10e4104c10 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10e9d94700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006af0 tx=0x7f10e0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:54.103 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e0003a80 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10e0003be0 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e0019770 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e419c990 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.104+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e419cdb0 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.105+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f10e0019a30 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.105+0000 7f10daffd700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 0x7f10d003a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.105+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f10e004c5d0 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.105+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e4195b20 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.106+0000 7f10e9593700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 0x7f10d003a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.106+0000 7f10e9593700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 0x7f10d003a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.114+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f10e002c3a0 con 0x7f10e4104c10 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.335+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f10e40611d0 con 0x7f10d00382e0 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.339+0000 7f10daffd700 1 -- 172.21.15.8:0/2946404359 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f10e40611d0 con 0x7f10d00382e0 2024-04-12T05:45:54.104 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 msgr2=0x7f10d003a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:54.105 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 0x7f10d003a7a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10d4006fd0 tx=0x7f10d4006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:54.105 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 msgr2=0x7f10e419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:54.105 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:53 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:54.105 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:53 smithi008 ceph-mon[39722]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:54.105 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:53 smithi008 ceph-mon[39722]: Saving service prometheus spec with placement count:1 2024-04-12T05:45:54.105 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:53 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:54.106 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f10e0006af0 tx=0x7f10e0011b20 comp rx=0 tx=0).stop 2024-04-12T05:45:54.106 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 shutdown_connections 2024-04-12T05:45:54.106 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f10d00382e0 0x7f10d003a7a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:54.106 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 --2- 172.21.15.8:0/2946404359 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4104c10 0x7f10e419c250 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:54.107 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 >> 172.21.15.8:0/2946404359 conn(0x7f10e41007f0 msgr2=0x7f10e41907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:54.107 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 shutdown_connections 2024-04-12T05:45:54.107 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:53.341+0000 7f10ebff8700 1 -- 172.21.15.8:0/2946404359 wait complete. 2024-04-12T05:45:54.107 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying grafana service with default placement... 2024-04-12T05:45:55.854 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:55 smithi008 ceph-mon[39722]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:45:55.854 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:55 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:56.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-12T05:45:56.180 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.282+0000 7fd590728700 1 Processor -- start 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.282+0000 7fd590728700 1 -- start start 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.283+0000 7fd590728700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.283+0000 7fd590728700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd588079da0 con 0x7fd588106de0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.283+0000 7fd58e4c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.283+0000 7fd58e4c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34818/0 (socket says 172.21.15.8:34818) 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.283+0000 7fd58e4c4700 1 -- 172.21.15.8:0/2451529657 learned_addr learned my addr 172.21.15.8:0/2451529657 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd58e4c4700 1 -- 172.21.15.8:0/2451529657 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd588109710 con 0x7fd588106de0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd58e4c4700 1 --2- 172.21.15.8:0/2451529657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098a0 tx=0x7fd584009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e05e4d311e02d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd58d4c2700 1 -- 172.21.15.8:0/2451529657 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd584004180 con 0x7fd588106de0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd58d4c2700 1 -- 172.21.15.8:0/2451529657 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd5840042e0 con 0x7fd588106de0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd58d4c2700 1 -- 172.21.15.8:0/2451529657 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd584011a30 con 0x7fd588106de0 2024-04-12T05:45:56.181 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd590728700 1 -- 172.21.15.8:0/2451529657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 msgr2=0x7fd5881091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.284+0000 7fd590728700 1 --2- 172.21.15.8:0/2451529657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5840098a0 tx=0x7fd584009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- 172.21.15.8:0/2451529657 shutdown_connections 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 --2- 172.21.15.8:0/2451529657 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd5881091d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- 172.21.15.8:0/2451529657 >> 172.21.15.8:0/2451529657 conn(0x7fd5881007d0 msgr2=0x7fd588102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- 172.21.15.8:0/2451529657 shutdown_connections 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- 172.21.15.8:0/2451529657 wait complete. 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 Processor -- start 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- start start 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.285+0000 7fd590728700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58819c760 con 0x7fd588106de0 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd58e4c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd58e4c4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34826/0 (socket says 172.21.15.8:34826) 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd58e4c4700 1 -- 172.21.15.8:0/174694213 learned_addr learned my addr 172.21.15.8:0/174694213 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:56.182 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd58e4c4700 1 -- 172.21.15.8:0/174694213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584009550 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd58e4c4700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd584011a00 tx=0x7fd584011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd584003bc0 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd5840248c0 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58819c960 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd58401a850 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.286+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58819cd80 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.287+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fd584024a30 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.287+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5880623c0 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.287+0000 7fd57f7fe700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 0x7fd57403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.287+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd58404c660 con 0x7fd588106de0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.288+0000 7fd58dcc3700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 0x7fd57403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.288+0000 7fd58dcc3700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 0x7fd57403a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:56.183 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.292+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd584024ce0 con 0x7fd588106de0 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.505+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fd588102ae0 con 0x7fd574038370 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.509+0000 7fd57f7fe700 1 -- 172.21.15.8:0/174694213 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fd588102ae0 con 0x7fd574038370 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 msgr2=0x7fd57403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 0x7fd57403a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 msgr2=0x7fd58819c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd584011a00 tx=0x7fd584011ae0 comp rx=0 tx=0).stop 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 shutdown_connections 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fd574038370 0x7fd57403a830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 --2- 172.21.15.8:0/174694213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd588106de0 0x7fd58819c220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 >> 172.21.15.8:0/174694213 conn(0x7fd5881007d0 msgr2=0x7fd588101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 shutdown_connections 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:55.511+0000 7fd590728700 1 -- 172.21.15.8:0/174694213 wait complete. 2024-04-12T05:45:56.184 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying node-exporter service with default placement... 2024-04-12T05:45:56.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:56 smithi008 ceph-mon[39722]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:56.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:56 smithi008 ceph-mon[39722]: Saving service grafana spec with placement count:1 2024-04-12T05:45:57.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-12T05:45:57.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b52d66700 1 Processor -- start 2024-04-12T05:45:57.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b52d66700 1 -- start start 2024-04-12T05:45:57.538 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b52d66700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b52d66700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c079da0 con 0x7f9b4c106de0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b50b02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b50b02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34834/0 (socket says 172.21.15.8:34834) 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.701+0000 7f9b50b02700 1 -- 172.21.15.8:0/4133200210 learned_addr learned my addr 172.21.15.8:0/4133200210 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b50b02700 1 -- 172.21.15.8:0/4133200210 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c109710 con 0x7f9b4c106de0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b50b02700 1 --2- 172.21.15.8:0/4133200210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b400098a0 tx=0x7f9b40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa00a0f296b56ac server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b4b7fe700 1 -- 172.21.15.8:0/4133200210 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b40004180 con 0x7f9b4c106de0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b4b7fe700 1 -- 172.21.15.8:0/4133200210 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b400042e0 con 0x7f9b4c106de0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b4b7fe700 1 -- 172.21.15.8:0/4133200210 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b40011a30 con 0x7f9b4c106de0 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b52d66700 1 -- 172.21.15.8:0/4133200210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 msgr2=0x7f9b4c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b52d66700 1 --2- 172.21.15.8:0/4133200210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b400098a0 tx=0x7f9b40009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b52d66700 1 -- 172.21.15.8:0/4133200210 shutdown_connections 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b52d66700 1 --2- 172.21.15.8:0/4133200210 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c1091d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:57.539 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.702+0000 7f9b52d66700 1 -- 172.21.15.8:0/4133200210 >> 172.21.15.8:0/4133200210 conn(0x7f9b4c1007d0 msgr2=0x7f9b4c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.703+0000 7f9b52d66700 1 -- 172.21.15.8:0/4133200210 shutdown_connections 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.703+0000 7f9b52d66700 1 -- 172.21.15.8:0/4133200210 wait complete. 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.703+0000 7f9b52d66700 1 Processor -- start 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.703+0000 7f9b52d66700 1 -- start start 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b52d66700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b52d66700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4c19a560 con 0x7f9b4c106de0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b50b02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b50b02700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34844/0 (socket says 172.21.15.8:34844) 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b50b02700 1 -- 172.21.15.8:0/3353487108 learned_addr learned my addr 172.21.15.8:0/3353487108 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b50b02700 1 -- 172.21.15.8:0/3353487108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b40009550 con 0x7f9b4c106de0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b50b02700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006af0 tx=0x7f9b40011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b40003bc0 con 0x7f9b4c106de0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c19a760 con 0x7f9b4c106de0 2024-04-12T05:45:57.540 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9b4001a8c0 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.704+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b40019860 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.705+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4c19ab80 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.705+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f9b4001aa30 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.706+0000 7f9b49ffb700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 0x7f9b3c03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.706+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4c0623c0 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.706+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9b4004c710 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.706+0000 7f9b4bfff700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 0x7f9b3c03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.706+0000 7f9b4bfff700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 0x7f9b3c03a7a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b38006fd0 tx=0x7f9b38006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.710+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9b4001a430 con 0x7f9b4c106de0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.920+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b4c19d5c0 con 0x7f9b3c0382e0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.924+0000 7f9b49ffb700 1 -- 172.21.15.8:0/3353487108 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f9b4c19d5c0 con 0x7f9b3c0382e0 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 msgr2=0x7f9b3c03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 0x7f9b3c03a7a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9b38006fd0 tx=0x7f9b38006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:57.541 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 msgr2=0x7f9b4c19a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006af0 tx=0x7f9b40011ab0 comp rx=0 tx=0).stop 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 shutdown_connections 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f9b3c0382e0 0x7f9b3c03a7a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 --2- 172.21.15.8:0/3353487108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9b4c106de0 0x7f9b4c19a020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 >> 172.21.15.8:0/3353487108 conn(0x7f9b4c1007d0 msgr2=0x7f9b4c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 shutdown_connections 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:56.926+0000 7f9b52d66700 1 -- 172.21.15.8:0/3353487108 wait complete. 2024-04-12T05:45:57.542 INFO:teuthology.orchestra.run.smithi008.stdout:Deploying alertmanager service with default placement... 2024-04-12T05:45:57.805 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:57 smithi008 ceph-mon[39722]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:45:57.805 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:57 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:58 smithi008 ceph-mon[39722]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:58 smithi008 ceph-mon[39722]: Saving service node-exporter spec with placement * 2024-04-12T05:45:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:58 smithi008 ceph-mon[39722]: from='mgr.14118 172.21.15.8:0/3198895465' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:45:58.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-12T05:45:58.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.078+0000 7fa72ec08700 1 Processor -- start 2024-04-12T05:45:58.967 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72ec08700 1 -- start start 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72ec08700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72ec08700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728105000 con 0x7fa728104610 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72c9a4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72c9a4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34850/0 (socket says 172.21.15.8:34850) 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72c9a4700 1 -- 172.21.15.8:0/339032766 learned_addr learned my addr 172.21.15.8:0/339032766 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72c9a4700 1 -- 172.21.15.8:0/339032766 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7281075a0 con 0x7fa728104610 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa72c9a4700 1 --2- 172.21.15.8:0/339032766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0098a0 tx=0x7fa71c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49087d9ef4bdfe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa7277fe700 1 -- 172.21.15.8:0/339032766 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71c004180 con 0x7fa728104610 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.079+0000 7fa7277fe700 1 -- 172.21.15.8:0/339032766 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa71c0042e0 con 0x7fa728104610 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa7277fe700 1 -- 172.21.15.8:0/339032766 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71c011a30 con 0x7fa728104610 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 -- 172.21.15.8:0/339032766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 msgr2=0x7fa728104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 --2- 172.21.15.8:0/339032766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0098a0 tx=0x7fa71c009bb0 comp rx=0 tx=0).stop 2024-04-12T05:45:58.968 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 -- 172.21.15.8:0/339032766 shutdown_connections 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 --2- 172.21.15.8:0/339032766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728104a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 -- 172.21.15.8:0/339032766 >> 172.21.15.8:0/339032766 conn(0x7fa7280fff40 msgr2=0x7fa728102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 -- 172.21.15.8:0/339032766 shutdown_connections 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.080+0000 7fa72ec08700 1 -- 172.21.15.8:0/339032766 wait complete. 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72ec08700 1 Processor -- start 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72ec08700 1 -- start start 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72ec08700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72ec08700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72819a530 con 0x7fa728104610 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72c9a4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72c9a4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34862/0 (socket says 172.21.15.8:34862) 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72c9a4700 1 -- 172.21.15.8:0/2315713195 learned_addr learned my addr 172.21.15.8:0/2315713195 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72c9a4700 1 -- 172.21.15.8:0/2315713195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009550 con 0x7fa728104610 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.081+0000 7fa72c9a4700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006af0 tx=0x7fa71c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.082+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71c003bc0 con 0x7fa728104610 2024-04-12T05:45:58.969 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.082+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa71c01a8c0 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.082+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72819a730 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.082+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa71c019820 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.082+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72819ab50 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa71c01aa30 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7280623c0 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa725ffb700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 0x7fa71803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa71c04c710 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa727fff700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 0x7fa71803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.083+0000 7fa727fff700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 0x7fa71803a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.088+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa71c01a430 con 0x7fa728104610 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.295+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fa728102210 con 0x7fa718038330 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.299+0000 7fa725ffb700 1 -- 172.21.15.8:0/2315713195 <== mgr.14118 v2:172.21.15.8:6800/1010894527 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fa728102210 con 0x7fa718038330 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.301+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 msgr2=0x7fa71803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:58.970 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.301+0000 7fa72ec08700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 0x7fa71803a7f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa714006fd0 tx=0x7fa714006e40 comp rx=0 tx=0).stop 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.301+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 msgr2=0x7fa728199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.301+0000 7fa72ec08700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa71c006af0 tx=0x7fa71c011ab0 comp rx=0 tx=0).stop 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 shutdown_connections 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fa718038330 0x7fa71803a7f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 --2- 172.21.15.8:0/2315713195 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa728104610 0x7fa728199ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 >> 172.21.15.8:0/2315713195 conn(0x7fa7280fff40 msgr2=0x7fa7281015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 shutdown_connections 2024-04-12T05:45:58.971 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:58.302+0000 7fa72ec08700 1 -- 172.21.15.8:0/2315713195 wait complete. 2024-04-12T05:45:59.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:59 smithi008 ceph-mon[39722]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:45:59.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:59 smithi008 ceph-mon[39722]: Saving service alertmanager spec with placement count:1 2024-04-12T05:45:59.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:45:59 smithi008 ceph-mon[39722]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:00.301 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf3075700 1 Processor -- start 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf3075700 1 -- start start 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf3075700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf3075700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec105c20 con 0x7ffbec105230 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf0e11700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf0e11700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34868/0 (socket says 172.21.15.8:34868) 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.509+0000 7ffbf0e11700 1 -- 172.21.15.8:0/4074612329 learned_addr learned my addr 172.21.15.8:0/4074612329 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.510+0000 7ffbf0e11700 1 -- 172.21.15.8:0/4074612329 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec106430 con 0x7ffbec105230 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.510+0000 7ffbf0e11700 1 --2- 172.21.15.8:0/4074612329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00098a0 tx=0x7ffbe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35237f600671ab8e server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.510+0000 7ffbeb7fe700 1 -- 172.21.15.8:0/4074612329 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe0004180 con 0x7ffbec105230 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.510+0000 7ffbeb7fe700 1 -- 172.21.15.8:0/4074612329 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffbe00042e0 con 0x7ffbec105230 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.510+0000 7ffbeb7fe700 1 -- 172.21.15.8:0/4074612329 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe0011a30 con 0x7ffbec105230 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- 172.21.15.8:0/4074612329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 msgr2=0x7ffbec105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:00.302 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 --2- 172.21.15.8:0/4074612329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00098a0 tx=0x7ffbe0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- 172.21.15.8:0/4074612329 shutdown_connections 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 --2- 172.21.15.8:0/4074612329 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec105650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- 172.21.15.8:0/4074612329 >> 172.21.15.8:0/4074612329 conn(0x7ffbec1007f0 msgr2=0x7ffbec102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- 172.21.15.8:0/4074612329 shutdown_connections 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- 172.21.15.8:0/4074612329 wait complete. 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 Processor -- start 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- start start 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.511+0000 7ffbf3075700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec19cdd0 con 0x7ffbec105230 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf0e11700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf0e11700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34870/0 (socket says 172.21.15.8:34870) 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf0e11700 1 -- 172.21.15.8:0/729757679 learned_addr learned my addr 172.21.15.8:0/729757679 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf0e11700 1 -- 172.21.15.8:0/729757679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe0009550 con 0x7ffbec105230 2024-04-12T05:46:00.303 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf0e11700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0000c00 tx=0x7ffbe0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe0003b40 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffbe00248d0 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec19cf10 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbe001a820 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.512+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec19d2d0 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.513+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ffbe001aae0 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.513+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec193010 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.513+0000 7ffbe9ffb700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 0x7ffbdc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.514+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffbe004c4e0 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.514+0000 7ffbebfff700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 0x7ffbdc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.514+0000 7ffbebfff700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 0x7ffbdc03a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006fd0 tx=0x7ffbd8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.518+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffbe001ad90 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.714+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ffbec02d090 con 0x7ffbec105230 2024-04-12T05:46:00.304 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.717+0000 7ffbe9ffb700 1 -- 172.21.15.8:0/729757679 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ffbe002c350 con 0x7ffbec105230 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 msgr2=0x7ffbdc03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 0x7ffbdc03a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8006fd0 tx=0x7ffbd8006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 msgr2=0x7ffbec0741b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0000c00 tx=0x7ffbe0011ab0 comp rx=0 tx=0).stop 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 shutdown_connections 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ffbdc038320 0x7ffbdc03a7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 --2- 172.21.15.8:0/729757679 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffbec105230 0x7ffbec0741b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 >> 172.21.15.8:0/729757679 conn(0x7ffbec1007f0 msgr2=0x7ffbec102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 shutdown_connections 2024-04-12T05:46:00.305 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:45:59.720+0000 7ffbf3075700 1 -- 172.21.15.8:0/729757679 wait complete. 2024-04-12T05:46:01.073 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/729757679' entity='client.admin' 2024-04-12T05:46:01.074 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:00 smithi008 ceph-mon[39722]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:01.692 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.842+0000 7ff140dfc700 1 Processor -- start 2024-04-12T05:46:01.692 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff140dfc700 1 -- start start 2024-04-12T05:46:01.692 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff140dfc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:01.692 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff140dfc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c079de0 con 0x7ff13c106da0 2024-04-12T05:46:01.692 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34878/0 (socket says 172.21.15.8:34878) 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff13a59c700 1 -- 172.21.15.8:0/3575564371 learned_addr learned my addr 172.21.15.8:0/3575564371 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff13a59c700 1 -- 172.21.15.8:0/3575564371 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c1096d0 con 0x7ff13c106da0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.843+0000 7ff13a59c700 1 --2- 172.21.15.8:0/3575564371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098a0 tx=0x7ff130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a51fe64d109f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff13959a700 1 -- 172.21.15.8:0/3575564371 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff130004180 con 0x7ff13c106da0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff13959a700 1 -- 172.21.15.8:0/3575564371 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff1300042e0 con 0x7ff13c106da0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff13959a700 1 -- 172.21.15.8:0/3575564371 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff130011a30 con 0x7ff13c106da0 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 -- 172.21.15.8:0/3575564371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 msgr2=0x7ff13c109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 --2- 172.21.15.8:0/3575564371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098a0 tx=0x7ff130009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 -- 172.21.15.8:0/3575564371 shutdown_connections 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 --2- 172.21.15.8:0/3575564371 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c109190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 -- 172.21.15.8:0/3575564371 >> 172.21.15.8:0/3575564371 conn(0x7ff13c1007d0 msgr2=0x7ff13c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 -- 172.21.15.8:0/3575564371 shutdown_connections 2024-04-12T05:46:01.693 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.844+0000 7ff140dfc700 1 -- 172.21.15.8:0/3575564371 wait complete. 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff140dfc700 1 Processor -- start 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff140dfc700 1 -- start start 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff140dfc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff140dfc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c19a550 con 0x7ff13c106da0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34886/0 (socket says 172.21.15.8:34886) 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.845+0000 7ff13a59c700 1 -- 172.21.15.8:0/2768748670 learned_addr learned my addr 172.21.15.8:0/2768748670 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff13a59c700 1 -- 172.21.15.8:0/2768748670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff130009550 con 0x7ff13c106da0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff13a59c700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff130011a00 tx=0x7ff130011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff130003bc0 con 0x7ff13c106da0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c19a750 con 0x7ff13c106da0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff1300248c0 con 0x7ff13c106da0 2024-04-12T05:46:01.694 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff13001a850 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.846+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c19ab70 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.847+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff13001ab20 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.847+0000 7ff12b7fe700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 0x7ff12403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.847+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c0623c0 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.847+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff13004c620 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.847+0000 7ff139d9b700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 0x7ff12403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.848+0000 7ff139d9b700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 0x7ff12403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:00.852+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff13001add0 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.049+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff13c19d5b0 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.052+0000 7ff12b7fe700 1 -- 172.21.15.8:0/2768748670 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff1300172b0 con 0x7ff13c106da0 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 msgr2=0x7ff12403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 0x7ff12403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff12c006fd0 tx=0x7ff12c006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 msgr2=0x7ff13c19a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:01.695 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff130011a00 tx=0x7ff130011ae0 comp rx=0 tx=0).stop 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 shutdown_connections 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7ff124038370 0x7ff12403a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 --2- 172.21.15.8:0/2768748670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff13c106da0 0x7ff13c19a010 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 >> 172.21.15.8:0/2768748670 conn(0x7ff13c1007d0 msgr2=0x7ff13c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.054+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 shutdown_connections 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:01.055+0000 7ff140dfc700 1 -- 172.21.15.8:0/2768748670 wait complete. 2024-04-12T05:46:01.696 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling the dashboard module... 2024-04-12T05:46:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:02 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2768748670' entity='client.admin' 2024-04-12T05:46:03.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:03 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1000553949' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-12T05:46:03.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:03 smithi008 ceph-mon[39722]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:03.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.289+0000 7f1ad4b6c700 1 Processor -- start 2024-04-12T05:46:03.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.289+0000 7f1ad4b6c700 1 -- start start 2024-04-12T05:46:03.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ad4b6c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:03.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ad4b6c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad0106d20 con 0x7f1ad0106330 2024-04-12T05:46:03.849 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ace59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ace59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34900/0 (socket says 172.21.15.8:34900) 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ace59c700 1 -- 172.21.15.8:0/1058141668 learned_addr learned my addr 172.21.15.8:0/1058141668 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ace59c700 1 -- 172.21.15.8:0/1058141668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad0107530 con 0x7f1ad0106330 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1ace59c700 1 --2- 172.21.15.8:0/1058141668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00098a0 tx=0x7f1ac0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4991bd80d81adab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1acd59a700 1 -- 172.21.15.8:0/1058141668 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ac0004180 con 0x7f1ad0106330 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1acd59a700 1 -- 172.21.15.8:0/1058141668 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1ac00042e0 con 0x7f1ad0106330 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.290+0000 7f1acd59a700 1 -- 172.21.15.8:0/1058141668 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ac0011a30 con 0x7f1ad0106330 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1058141668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 msgr2=0x7f1ad0106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1058141668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00098a0 tx=0x7f1ac0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1058141668 shutdown_connections 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1058141668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad0106750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1058141668 >> 172.21.15.8:0/1058141668 conn(0x7f1ad01018d0 msgr2=0x7f1ad0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1058141668 shutdown_connections 2024-04-12T05:46:03.850 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.291+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1058141668 wait complete. 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ad4b6c700 1 Processor -- start 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ad4b6c700 1 -- start start 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ad4b6c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ad4b6c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad019a5a0 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ace59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ace59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34914/0 (socket says 172.21.15.8:34914) 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.292+0000 7f1ace59c700 1 -- 172.21.15.8:0/1000553949 learned_addr learned my addr 172.21.15.8:0/1000553949 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ace59c700 1 -- 172.21.15.8:0/1000553949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0009550 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ace59c700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ac0003bc0 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1ac00248d0 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad019a7a0 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1ac001a890 con 0x7f1ad0106330 2024-04-12T05:46:03.851 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.293+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad019abc0 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.294+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad00623c0 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.294+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1ac0024a40 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.294+0000 7f1ac77fe700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 0x7f1abc03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.295+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1ac004ca70 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.295+0000 7f1acdd9b700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 0x7f1abc03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.295+0000 7f1acdd9b700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 0x7f1abc03a7e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.299+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1ac0018710 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:02.556+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f1ad010a730 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.056+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f1ac004b8a0 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.057+0000 7f1ac77fe700 1 -- 172.21.15.8:0/1000553949 <== mon.0 v2:172.21.15.8:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f1ac000be20 con 0x7f1ad0106330 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 msgr2=0x7f1abc03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 0x7f1abc03a7e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8006fd0 tx=0x7f1ab8006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 msgr2=0x7f1ad019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:03.852 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0000c00 tx=0x7f1ac0011ab0 comp rx=0 tx=0).stop 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 shutdown_connections 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f1abc038320 0x7f1abc03a7e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 --2- 172.21.15.8:0/1000553949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1ad0106330 0x7f1ad019a060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 >> 172.21.15.8:0/1000553949 conn(0x7f1ad01018d0 msgr2=0x7f1ad0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.060+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 shutdown_connections 2024-04-12T05:46:03.853 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:03.061+0000 7f1ad4b6c700 1 -- 172.21.15.8:0/1000553949 wait complete. 2024-04-12T05:46:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:04 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1000553949' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-12T05:46:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:04 smithi008 ceph-mon[39722]: mgrmap e9: smithi008.ixmnlp(active, since 28s) 2024-04-12T05:46:05.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "active_name": "smithi008.ixmnlp", 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.402+0000 7f7ffe79f700 1 Processor -- start 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.402+0000 7f7ffe79f700 1 -- start start 2024-04-12T05:46:05.340 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.402+0000 7f7ffe79f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.402+0000 7f7ffe79f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8074530 con 0x7f7ff8106da0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38186/0 (socket says 172.21.15.8:38186) 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff7fff700 1 -- 172.21.15.8:0/3293161725 learned_addr learned my addr 172.21.15.8:0/3293161725 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff7fff700 1 -- 172.21.15.8:0/3293161725 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff81096d0 con 0x7f7ff8106da0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff7fff700 1 --2- 172.21.15.8:0/3293161725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51c82f171423546 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff6ffd700 1 -- 172.21.15.8:0/3293161725 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe8004180 con 0x7f7ff8106da0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff6ffd700 1 -- 172.21.15.8:0/3293161725 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7fe80042e0 con 0x7f7ff8106da0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.403+0000 7f7ff6ffd700 1 -- 172.21.15.8:0/3293161725 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe8011a30 con 0x7f7ff8106da0 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 -- 172.21.15.8:0/3293161725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 msgr2=0x7f7ff8109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/3293161725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80098a0 tx=0x7f7fe8009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.341 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 -- 172.21.15.8:0/3293161725 shutdown_connections 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/3293161725 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff8109190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 -- 172.21.15.8:0/3293161725 >> 172.21.15.8:0/3293161725 conn(0x7f7ff81007f0 msgr2=0x7f7ff8102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 -- 172.21.15.8:0/3293161725 shutdown_connections 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.404+0000 7f7ffe79f700 1 -- 172.21.15.8:0/3293161725 wait complete. 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ffe79f700 1 Processor -- start 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ffe79f700 1 -- start start 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ffe79f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ffe79f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff819c780 con 0x7f7ff8106da0 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ff7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38198/0 (socket says 172.21.15.8:38198) 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ff7fff700 1 -- 172.21.15.8:0/1461366700 learned_addr learned my addr 172.21.15.8:0/1461366700 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ff7fff700 1 -- 172.21.15.8:0/1461366700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8009550 con 0x7f7ff8106da0 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.405+0000 7f7ff7fff700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8011a00 tx=0x7f7fe8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:05.342 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.406+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe8003bd0 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.406+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff819c980 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.406+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7fe80248d0 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.406+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe801a7e0 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.406+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff819cda0 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff80623c0 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f7fe8024a40 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ff57fa700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 0x7f7fe003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7fe804c490 con 0x7f7ff8106da0 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ff77fe700 1 -- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 msgr2=0x7f7fe003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:05.343 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.407+0000 7f7ff77fe700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 0x7f7fe003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:46:05.343 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:05 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1461366700' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-12T05:46:05.346 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.412+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7fe8018db0 con 0x7f7ff8106da0 2024-04-12T05:46:05.346 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.608+0000 7f7ff77fe700 1 -- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 msgr2=0x7f7fe003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:05.346 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.608+0000 7f7ff77fe700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 0x7f7fe003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T05:46:05.346 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.672+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7ff819fe90 con 0x7f7ff8106da0 2024-04-12T05:46:05.346 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.673+0000 7f7ff57fa700 1 -- 172.21.15.8:0/1461366700 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f7fe8018350 con 0x7f7ff8106da0 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 msgr2=0x7f7fe003a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 0x7f7fe003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 msgr2=0x7f7ff819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8011a00 tx=0x7f7fe8011ae0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 shutdown_connections 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7f7fe0038370 0x7f7fe003a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 --2- 172.21.15.8:0/1461366700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ff8106da0 0x7f7ff819c240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 >> 172.21.15.8:0/1461366700 conn(0x7f7ff81007f0 msgr2=0x7f7ff8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 shutdown_connections 2024-04-12T05:46:05.347 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:04.675+0000 7f7ffe79f700 1 -- 172.21.15.8:0/1461366700 wait complete. 2024-04-12T05:46:05.348 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for the mgr to restart... 2024-04-12T05:46:05.348 INFO:teuthology.orchestra.run.smithi008.stdout:Waiting for mgr epoch 9... 2024-04-12T05:46:10.144 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: Active manager daemon smithi008.ixmnlp restarted 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: Activating manager daemon smithi008.ixmnlp 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: osdmap e3: 0 total, 0 up, 0 in 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: mgrmap e10: smithi008.ixmnlp(active, starting, since 0.00325737s) 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:46:10.145 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:46:10.146 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:46:10.146 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:10 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:46:11.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:11 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:46:11.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:11 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:46:11.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:11 smithi008 ceph-mon[39722]: mgrmap e11: smithi008.ixmnlp(active, since 1.0059s) 2024-04-12T05:46:11.759 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout { 2024-04-12T05:46:11.759 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout } 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.863+0000 7fc706082700 1 Processor -- start 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.863+0000 7fc706082700 1 -- start start 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.863+0000 7fc706082700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.863+0000 7fc706082700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000744f0 con 0x7fc700106de0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38202/0 (socket says 172.21.15.8:38202) 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6ff7fe700 1 -- 172.21.15.8:0/3637369585 learned_addr learned my addr 172.21.15.8:0/3637369585 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6ff7fe700 1 -- 172.21.15.8:0/3637369585 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc700109710 con 0x7fc700106de0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6ff7fe700 1 --2- 172.21.15.8:0/3637369585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00098a0 tx=0x7fc6f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eb4da114c089e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6fe7fc700 1 -- 172.21.15.8:0/3637369585 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6f0004180 con 0x7fc700106de0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6fe7fc700 1 -- 172.21.15.8:0/3637369585 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc6f00042e0 con 0x7fc700106de0 2024-04-12T05:46:11.760 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.864+0000 7fc6fe7fc700 1 -- 172.21.15.8:0/3637369585 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6f0011a30 con 0x7fc700106de0 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 -- 172.21.15.8:0/3637369585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 msgr2=0x7fc7001091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 --2- 172.21.15.8:0/3637369585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00098a0 tx=0x7fc6f0009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 -- 172.21.15.8:0/3637369585 shutdown_connections 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 --2- 172.21.15.8:0/3637369585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc7001091d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 -- 172.21.15.8:0/3637369585 >> 172.21.15.8:0/3637369585 conn(0x7fc7001007f0 msgr2=0x7fc700102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 -- 172.21.15.8:0/3637369585 shutdown_connections 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 -- 172.21.15.8:0/3637369585 wait complete. 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.865+0000 7fc706082700 1 Processor -- start 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc706082700 1 -- start start 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc706082700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc706082700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70019a580 con 0x7fc700106de0 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc6ff7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38214/0 (socket says 172.21.15.8:38214) 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc6ff7fe700 1 -- 172.21.15.8:0/4138420131 learned_addr learned my addr 172.21.15.8:0/4138420131 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:11.761 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc6ff7fe700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0009550 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.866+0000 7fc6ff7fe700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006af0 tx=0x7fc6f0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.867+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6f0003a80 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.867+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc6f0003be0 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.867+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70019a780 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.867+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6f0019770 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.867+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70019aba0 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fc6f0019a30 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6fcff9700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e803af00 con 0x7fc6e8038330 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6f004c710 con 0x7fc700106de0 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6feffd700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:05.868+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:46:11.762 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:06.069+0000 7fc6feffd700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:06.069+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:06.470+0000 7fc6feffd700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:06.470+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:07.271+0000 7fc6feffd700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:07.271+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:08.872+0000 7fc6feffd700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/1010894527 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:08.872+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:10.033+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7fc6f002dce0 con 0x7fc700106de0 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:10.033+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 msgr2=0x7fc6e803a7f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:10.033+0000 7fc6fcff9700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.034+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fc6f004e0e0 con 0x7fc700106de0 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.035+0000 7fc6fcff9700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 0x7fc6e803e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.035+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6e803af00 con 0x7fc6e803bdf0 2024-04-12T05:46:11.763 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.035+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 0x7fc6e803e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.035+0000 7fc6feffd700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 0x7fc6e803e1e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003a10 tx=0x7fc6f4009270 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.036+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fc6e803af00 con 0x7fc6e803bdf0 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.042+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc70019b380 con 0x7fc6e803bdf0 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc6fcff9700 1 -- 172.21.15.8:0/4138420131 <== mgr.14162 v2:172.21.15.8:6800/3827652772 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc70019b380 con 0x7fc6e803bdf0 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 msgr2=0x7fc6e803e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 0x7fc6e803e1e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4003a10 tx=0x7fc6f4009270 comp rx=0 tx=0).stop 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 msgr2=0x7fc70019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006af0 tx=0x7fc6f0011b20 comp rx=0 tx=0).stop 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 shutdown_connections 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc6e803bdf0 0x7fc6e803e1e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc700106de0 0x7fc70019a040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 --2- 172.21.15.8:0/4138420131 >> [v2:172.21.15.8:6800/1010894527,v1:172.21.15.8:6801/1010894527] conn(0x7fc6e8038330 0x7fc6e803a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 >> 172.21.15.8:0/4138420131 conn(0x7fc7001007f0 msgr2=0x7fc700101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 shutdown_connections 2024-04-12T05:46:11.764 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:11.043+0000 7fc706082700 1 -- 172.21.15.8:0/4138420131 wait complete. 2024-04-12T05:46:11.765 INFO:teuthology.orchestra.run.smithi008.stdout:mgr epoch 9 is available 2024-04-12T05:46:11.765 INFO:teuthology.orchestra.run.smithi008.stdout:Generating a dashboard self-signed certificate... 2024-04-12T05:46:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: [12/Apr/2024:05:46:10] ENGINE Bus STARTING 2024-04-12T05:46:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: [12/Apr/2024:05:46:10] ENGINE Serving on http://172.21.15.8:8765 2024-04-12T05:46:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: [12/Apr/2024:05:46:10] ENGINE Serving on https://172.21.15.8:7150 2024-04-12T05:46:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: [12/Apr/2024:05:46:10] ENGINE Bus STARTED 2024-04-12T05:46:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-12T05:46:12.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-12T05:46:12.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:12 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:13.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:13 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:13.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:13 smithi008 ceph-mon[39722]: mgrmap e12: smithi008.ixmnlp(active, since 2s) 2024-04-12T05:46:13.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:13 smithi008 ceph-mon[39722]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:46:13.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:13 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:13.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:13 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.351+0000 7fdeacb1a700 1 Processor -- start 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.351+0000 7fdeacb1a700 1 -- start start 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.351+0000 7fdeacb1a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdeacb1a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8105000 con 0x7fdea8104610 2024-04-12T05:46:13.769 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38294/0 (socket says 172.21.15.8:38294) 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea659c700 1 -- 172.21.15.8:0/1677978462 learned_addr learned my addr 172.21.15.8:0/1677978462 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea659c700 1 -- 172.21.15.8:0/1677978462 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea81075a0 con 0x7fdea8104610 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea659c700 1 --2- 172.21.15.8:0/1677978462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c016c17a2b98b0ca server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea559a700 1 -- 172.21.15.8:0/1677978462 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde98004180 con 0x7fdea8104610 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea559a700 1 -- 172.21.15.8:0/1677978462 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fde980042e0 con 0x7fdea8104610 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.352+0000 7fdea559a700 1 -- 172.21.15.8:0/1677978462 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde98011a30 con 0x7fdea8104610 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.353+0000 7fdeacb1a700 1 -- 172.21.15.8:0/1677978462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 msgr2=0x7fdea8104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.353+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/1677978462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fde980098a0 tx=0x7fde98009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.353+0000 7fdeacb1a700 1 -- 172.21.15.8:0/1677978462 shutdown_connections 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.353+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/1677978462 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8104a30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.353+0000 7fdeacb1a700 1 -- 172.21.15.8:0/1677978462 >> 172.21.15.8:0/1677978462 conn(0x7fdea80fff40 msgr2=0x7fdea8102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:13.770 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 -- 172.21.15.8:0/1677978462 shutdown_connections 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 -- 172.21.15.8:0/1677978462 wait complete. 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 Processor -- start 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 -- start start 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.354+0000 7fdeacb1a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8107440 con 0x7fdea8104610 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdea659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdea659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38304/0 (socket says 172.21.15.8:38304) 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdea659c700 1 -- 172.21.15.8:0/223720072 learned_addr learned my addr 172.21.15.8:0/223720072 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdea659c700 1 -- 172.21.15.8:0/223720072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98009550 con 0x7fdea8104610 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdea659c700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde98003bc0 con 0x7fdea8104610 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fde980248d0 con 0x7fdea8104610 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8105550 con 0x7fdea8104610 2024-04-12T05:46:13.771 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde9801a890 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.355+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea8105970 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.356+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7fde9801ab50 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.356+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea80623c0 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.357+0000 7fde977fe700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 0x7fde9003a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.357+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fde9804c5c0 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.357+0000 7fdea5d9b700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 0x7fde9003a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.357+0000 7fdea5d9b700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 0x7fde9003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006fd0 tx=0x7fde9c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.362+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fde98022340 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.399+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fde9802dc90 con 0x7fdea8104610 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.571+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fdea8102210 con 0x7fde90038250 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.641+0000 7fde977fe700 1 -- 172.21.15.8:0/223720072 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fdea8102210 con 0x7fde90038250 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 msgr2=0x7fde9003a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 0x7fde9003a710 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde9c006fd0 tx=0x7fde9c006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:13.772 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 msgr2=0x7fdea8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98011ab0 comp rx=0 tx=0).stop 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 shutdown_connections 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fde90038250 0x7fde9003a710 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 --2- 172.21.15.8:0/223720072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdea8104610 0x7fdea8106f00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 >> 172.21.15.8:0/223720072 conn(0x7fdea80fff40 msgr2=0x7fdea81015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 shutdown_connections 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:12.643+0000 7fdeacb1a700 1 -- 172.21.15.8:0/223720072 wait complete. 2024-04-12T05:46:13.773 INFO:teuthology.orchestra.run.smithi008.stdout:Creating initial admin user... 2024-04-12T05:46:15.505 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FfpVd7XeJPJcj2PsjfY5K.UsLo0JK0QP3/Wha1kFhGM47sqEBH0NK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712900774, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26f696700 1 Processor -- start 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26f696700 1 -- start start 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26f696700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26f696700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc268079d60 con 0x7fc268104c10 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26d432700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26d432700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46998/0 (socket says 172.21.15.8:46998) 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.361+0000 7fc26d432700 1 -- 172.21.15.8:0/2171589273 learned_addr learned my addr 172.21.15.8:0/2171589273 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc26d432700 1 -- 172.21.15.8:0/2171589273 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc268105570 con 0x7fc268104c10 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc26d432700 1 --2- 172.21.15.8:0/2171589273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d04d8afbe49c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc25ffff700 1 -- 172.21.15.8:0/2171589273 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc264004180 con 0x7fc268104c10 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc25ffff700 1 -- 172.21.15.8:0/2171589273 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc2640042e0 con 0x7fc268104c10 2024-04-12T05:46:15.506 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc25ffff700 1 -- 172.21.15.8:0/2171589273 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc264011a30 con 0x7fc268104c10 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc26f696700 1 -- 172.21.15.8:0/2171589273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 msgr2=0x7fc268105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc26f696700 1 --2- 172.21.15.8:0/2171589273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc2640098a0 tx=0x7fc264009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.362+0000 7fc26f696700 1 -- 172.21.15.8:0/2171589273 shutdown_connections 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 --2- 172.21.15.8:0/2171589273 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc268105030 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 -- 172.21.15.8:0/2171589273 >> 172.21.15.8:0/2171589273 conn(0x7fc2681007d0 msgr2=0x7fc268102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 -- 172.21.15.8:0/2171589273 shutdown_connections 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 -- 172.21.15.8:0/2171589273 wait complete. 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 Processor -- start 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 -- start start 2024-04-12T05:46:15.507 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26f696700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26819c780 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26d432700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26d432700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47010/0 (socket says 172.21.15.8:47010) 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.363+0000 7fc26d432700 1 -- 172.21.15.8:0/4222732693 learned_addr learned my addr 172.21.15.8:0/4222732693 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc26d432700 1 -- 172.21.15.8:0/4222732693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264009550 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc26d432700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc264011a00 tx=0x7fc264011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc264003bc0 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc2640248c0 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26819c980 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc26401a850 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.364+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26819cda0 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fc26401aac0 con 0x7fc268104c10 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc25e7fc700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 0x7fc25403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:15.508 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc26404c740 con 0x7fc268104c10 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc26cc31700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 0x7fc25403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc268195af0 con 0x7fc268104c10 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.365+0000 7fc26cc31700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 0x7fc25403a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.370+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc26401adc0 con 0x7fc268104c10 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.584+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fc2680621e0 con 0x7fc2540383c0 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.830+0000 7fc25e7fc700 1 -- 172.21.15.8:0/4222732693 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fc2680621e0 con 0x7fc2540383c0 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 msgr2=0x7fc25403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 0x7fc25403a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc258006fd0 tx=0x7fc258006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 msgr2=0x7fc26819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc264011a00 tx=0x7fc264011ae0 comp rx=0 tx=0).stop 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 shutdown_connections 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fc2540383c0 0x7fc25403a880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 --2- 172.21.15.8:0/4222732693 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc268104c10 0x7fc26819c240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 >> 172.21.15.8:0/4222732693 conn(0x7fc2681007d0 msgr2=0x7fc2681907d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:15.509 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 shutdown_connections 2024-04-12T05:46:15.510 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:14.832+0000 7fc26f696700 1 -- 172.21.15.8:0/4222732693 wait complete. 2024-04-12T05:46:15.510 INFO:teuthology.orchestra.run.smithi008.stdout:Fetching dashboard port number... 2024-04-12T05:46:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:15 smithi008 ceph-mon[39722]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:46:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:15 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:16 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1943603939' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stdout 8443 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.080+0000 7fcb5682c700 1 Processor -- start 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.080+0000 7fcb5682c700 1 -- start start 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.080+0000 7fcb5682c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.080+0000 7fcb5682c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb50079d60 con 0x7fcb50106e20 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.080+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:16.955 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47018/0 (socket says 172.21.15.8:47018) 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4ffff700 1 -- 172.21.15.8:0/3656906104 learned_addr learned my addr 172.21.15.8:0/3656906104 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4ffff700 1 -- 172.21.15.8:0/3656906104 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50109750 con 0x7fcb50106e20 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4ffff700 1 --2- 172.21.15.8:0/3656906104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb400098a0 tx=0x7fcb40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567e332f14ad37e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4effd700 1 -- 172.21.15.8:0/3656906104 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb40004180 con 0x7fcb50106e20 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4effd700 1 -- 172.21.15.8:0/3656906104 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcb400042e0 con 0x7fcb50106e20 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.081+0000 7fcb4effd700 1 -- 172.21.15.8:0/3656906104 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb40011a30 con 0x7fcb50106e20 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 -- 172.21.15.8:0/3656906104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 msgr2=0x7fcb50109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 --2- 172.21.15.8:0/3656906104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb400098a0 tx=0x7fcb40009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 -- 172.21.15.8:0/3656906104 shutdown_connections 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 --2- 172.21.15.8:0/3656906104 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50109210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 -- 172.21.15.8:0/3656906104 >> 172.21.15.8:0/3656906104 conn(0x7fcb501007d0 msgr2=0x7fcb50102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 -- 172.21.15.8:0/3656906104 shutdown_connections 2024-04-12T05:46:16.956 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 -- 172.21.15.8:0/3656906104 wait complete. 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.082+0000 7fcb5682c700 1 Processor -- start 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb5682c700 1 -- start start 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb5682c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb5682c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5019a530 con 0x7fcb50106e20 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb4ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47034/0 (socket says 172.21.15.8:47034) 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb4ffff700 1 -- 172.21.15.8:0/1943603939 learned_addr learned my addr 172.21.15.8:0/1943603939 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.083+0000 7fcb4ffff700 1 -- 172.21.15.8:0/1943603939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009550 con 0x7fcb50106e20 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb4ffff700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb40011a00 tx=0x7fcb40011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb40003bc0 con 0x7fcb50106e20 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcb400248c0 con 0x7fcb50106e20 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5019a730 con 0x7fcb50106e20 2024-04-12T05:46:16.957 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb4001a850 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5019ab50 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.084+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7fcb4001aac0 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.085+0000 7fcb4d7fa700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 0x7fcb3c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.085+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcb4004c7b0 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.085+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb500623c0 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.085+0000 7fcb4f7fe700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 0x7fcb3c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.085+0000 7fcb4f7fe700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 0x7fcb3c03a880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.090+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcb4001adc0 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.287+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fcb5019d900 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.288+0000 7fcb4d7fa700 1 -- 172.21.15.8:0/1943603939 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fcb4002c350 con 0x7fcb50106e20 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 msgr2=0x7fcb3c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:16.958 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 0x7fcb3c03a880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcb38006fd0 tx=0x7fcb38006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 msgr2=0x7fcb50199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcb40011a00 tx=0x7fcb40011ae0 comp rx=0 tx=0).stop 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 shutdown_connections 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcb3c0383c0 0x7fcb3c03a880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 --2- 172.21.15.8:0/1943603939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb50106e20 0x7fcb50199ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 >> 172.21.15.8:0/1943603939 conn(0x7fcb501007d0 msgr2=0x7fcb50101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 shutdown_connections 2024-04-12T05:46:16.959 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:16.290+0000 7fcb5682c700 1 -- 172.21.15.8:0/1943603939 wait complete. 2024-04-12T05:46:16.960 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-12T05:46:16.961 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout disabled 2024-04-12T05:46:16.970 INFO:teuthology.orchestra.run.smithi008.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-12T05:46:16.970 INFO:teuthology.orchestra.run.smithi008.stdout:systemctl: stdout inactive 2024-04-12T05:46:16.970 INFO:teuthology.orchestra.run.smithi008.stdout:firewalld.service is not enabled 2024-04-12T05:46:16.970 INFO:teuthology.orchestra.run.smithi008.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout:Ceph Dashboard is now available at: 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout: URL: https://smithi008.front.sepia.ceph.com:8443/ 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout: User: admin 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout: Password: 13bqbhz9hx 2024-04-12T05:46:16.972 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:16.973 INFO:teuthology.orchestra.run.smithi008.stdout:Saving cluster configuration to /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config directory 2024-04-12T05:46:16.973 INFO:teuthology.orchestra.run.smithi008.stdout:Enabling autotune for osd_memory_target 2024-04-12T05:46:18.068 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:17 smithi008 ceph-mon[39722]: mgrmap e13: smithi008.ixmnlp(active, since 6s) 2024-04-12T05:46:18.334 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.526+0000 7f65d8d72700 1 Processor -- start 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d8d72700 1 -- start start 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d8d72700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d8d72700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4079d60 con 0x7f65d4106e20 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47046/0 (socket says 172.21.15.8:47046) 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d259c700 1 -- 172.21.15.8:0/205693779 learned_addr learned my addr 172.21.15.8:0/205693779 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d259c700 1 -- 172.21.15.8:0/205693779 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4109750 con 0x7f65d4106e20 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.527+0000 7f65d259c700 1 --2- 172.21.15.8:0/205693779 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65c80098a0 tx=0x7f65c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a4b7400399bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d159a700 1 -- 172.21.15.8:0/205693779 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c8004180 con 0x7f65d4106e20 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d159a700 1 -- 172.21.15.8:0/205693779 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f65c80042e0 con 0x7f65d4106e20 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d159a700 1 -- 172.21.15.8:0/205693779 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c8011a30 con 0x7f65d4106e20 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 -- 172.21.15.8:0/205693779 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 msgr2=0x7f65d4109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:18.335 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 --2- 172.21.15.8:0/205693779 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f65c80098a0 tx=0x7f65c8009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 -- 172.21.15.8:0/205693779 shutdown_connections 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 --2- 172.21.15.8:0/205693779 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d4109210 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 -- 172.21.15.8:0/205693779 >> 172.21.15.8:0/205693779 conn(0x7f65d41007d0 msgr2=0x7f65d4102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 -- 172.21.15.8:0/205693779 shutdown_connections 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.528+0000 7f65d8d72700 1 -- 172.21.15.8:0/205693779 wait complete. 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d8d72700 1 Processor -- start 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d8d72700 1 -- start start 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d8d72700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d8d72700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d419c740 con 0x7f65d4106e20 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47056/0 (socket says 172.21.15.8:47056) 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.529+0000 7f65d259c700 1 -- 172.21.15.8:0/2001364455 learned_addr learned my addr 172.21.15.8:0/2001364455 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65d259c700 1 -- 172.21.15.8:0/2001364455 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009550 con 0x7f65d4106e20 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65d259c700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65c8011a00 tx=0x7f65c8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:18.336 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c8003bc0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f65c80248c0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d419c940 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f65c801a850 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.530+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d419cd60 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.531+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f65c801aac0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.531+0000 7f65c37fe700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 0x7f65bc03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.531+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f65c804c7f0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.531+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d40623c0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.531+0000 7f65d1d9b700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 0x7f65bc03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.532+0000 7f65d1d9b700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 0x7f65bc03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.536+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f65c801adc0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.733+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f65d419f7a0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.734+0000 7f65c37fe700 1 -- 172.21.15.8:0/2001364455 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f65c80172b0 con 0x7f65d4106e20 2024-04-12T05:46:18.337 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 msgr2=0x7f65bc03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 0x7f65bc03a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 msgr2=0x7f65d419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f65c8011a00 tx=0x7f65c8011ae0 comp rx=0 tx=0).stop 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 shutdown_connections 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f65bc038460 0x7f65bc03a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 --2- 172.21.15.8:0/2001364455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4106e20 0x7f65d419c200 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 >> 172.21.15.8:0/2001364455 conn(0x7f65d41007d0 msgr2=0x7f65d4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 shutdown_connections 2024-04-12T05:46:18.338 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:17.736+0000 7f65d8d72700 1 -- 172.21.15.8:0/2001364455 wait complete. 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f24195f1700 1 Processor -- start 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f24195f1700 1 -- start start 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f24195f1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f24195f1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414104fd0 con 0x7f24141045e0 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47064/0 (socket says 172.21.15.8:47064) 2024-04-12T05:46:19.784 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f2412ffd700 1 -- 172.21.15.8:0/2645541262 learned_addr learned my addr 172.21.15.8:0/2645541262 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.891+0000 7f2412ffd700 1 -- 172.21.15.8:0/2645541262 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414105110 con 0x7f24141045e0 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f2412ffd700 1 --2- 172.21.15.8:0/2645541262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0098a0 tx=0x7f23fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c217387bfce764 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f2411ffb700 1 -- 172.21.15.8:0/2645541262 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc004180 con 0x7f24141045e0 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f2411ffb700 1 -- 172.21.15.8:0/2645541262 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f23fc0042e0 con 0x7f24141045e0 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f2411ffb700 1 -- 172.21.15.8:0/2645541262 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc011a30 con 0x7f24141045e0 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 -- 172.21.15.8:0/2645541262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 msgr2=0x7f2414104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 --2- 172.21.15.8:0/2645541262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f23fc0098a0 tx=0x7f23fc009bb0 comp rx=0 tx=0).stop 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 -- 172.21.15.8:0/2645541262 shutdown_connections 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 --2- 172.21.15.8:0/2645541262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414104a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 -- 172.21.15.8:0/2645541262 >> 172.21.15.8:0/2645541262 conn(0x7f24140fff40 msgr2=0x7f2414102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 -- 172.21.15.8:0/2645541262 shutdown_connections 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.892+0000 7f24195f1700 1 -- 172.21.15.8:0/2645541262 wait complete. 2024-04-12T05:46:19.785 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f24195f1700 1 Processor -- start 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f24195f1700 1 -- start start 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f24195f1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f24195f1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414107500 con 0x7f24141045e0 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f2412ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47072/0 (socket says 172.21.15.8:47072) 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.893+0000 7f2412ffd700 1 -- 172.21.15.8:0/2978449973 learned_addr learned my addr 172.21.15.8:0/2978449973 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f2412ffd700 1 -- 172.21.15.8:0/2978449973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc009550 con 0x7f24141045e0 2024-04-12T05:46:19.786 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f2412ffd700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23fc006af0 tx=0x7f23fc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc003bc0 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f23fc01a8c0 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2414105610 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f23fc019820 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.894+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2414105a30 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.895+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f23fc019a50 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.895+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24140623c0 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.895+0000 7f240bfff700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 0x7f240003a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.895+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f23fc04ca90 con 0x7f24141045e0 2024-04-12T05:46:19.787 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.895+0000 7f24127fc700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 0x7f240003a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.896+0000 7f24127fc700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 0x7f240003a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2404006fd0 tx=0x7f2404006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:18.900+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f23fc018760 con 0x7f24141045e0 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.206+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2414108270 con 0x7f24141045e0 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.208+0000 7f240bfff700 1 -- 172.21.15.8:0/2978449973 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f23fc00b3e0 con 0x7f24141045e0 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 msgr2=0x7f240003a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 0x7f240003a8e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2404006fd0 tx=0x7f2404006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 msgr2=0x7f2414106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23fc006af0 tx=0x7f23fc011ab0 comp rx=0 tx=0).stop 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 shutdown_connections 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2400038420 0x7f240003a8e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:19.788 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 --2- 172.21.15.8:0/2978449973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24141045e0 0x7f2414106fc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 >> 172.21.15.8:0/2978449973 conn(0x7f24140fff40 msgr2=0x7f2414101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 shutdown_connections 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr 2024-04-12T05:46:19.210+0000 7f24195f1700 1 -- 172.21.15.8:0/2978449973 wait complete. 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout:Or, if you are only running a single cluster on this host: 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-12T05:46:19.789 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: ceph telemetry on 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout:For more information see: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:19.790 INFO:teuthology.orchestra.run.smithi008.stdout:Bootstrap complete. 2024-04-12T05:46:19.818 INFO:tasks.cephadm:Fetching config... 2024-04-12T05:46:19.818 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:46:19.819 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-12T05:46:19.877 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-12T05:46:19.878 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:46:19.878 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-12T05:46:19.936 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-12T05:46:19.937 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:46:19.937 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/keyring of=/dev/stdout 2024-04-12T05:46:20.005 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-12T05:46:20.005 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:46:20.005 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-12T05:46:20.063 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-12T05:46:20.063 DEBUG:teuthology.orchestra.run.smithi008:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAi/AVRPGp614QppzArhmxjIl/6lnCQMfg6xlznI4WRGm7AafuZqG4qsSxR4B7i6UOSTQ0JaekQ0gOJM4mHozI5wwFkqTexfCPngq9Y31VL5ANzU6txfuyiFn7OVQre+ObSx8W6P3lzfWRpmi08ksmLS7vEtpm9VnpX0YujOKbB7fIgfRHMTqQpygkhQYsKmCqmMsavGpIUlOsABgsL8u9hUxJjyg5+pktIjtvlxgfPCxWq+7Ru4ylxdbh6aEMZm/5E0JOOs6IWJph+I04VzcmAzGfU93qlgpWJxA4eVxIiiu0VpBdF1qR1mPV8mPfA/C0NfFIfcoTkNb3j3yfBtvG3ZjGyHM/EKKuJucnCiyaugTgpxhIG7nRPqXiN12PbigIgLtIBny55EnMgxnnJdlHDHhr7v1CLV3ww0zd0AvHJHPpdZDZ5I013U39TtQ0l/WreCl5Knea+Lad+WUvw8g9b6tLEwwVDa7QXR2FnlJX4YNSQN0ihL+psa4F8mgHDwE= ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-12T05:46:20.144 INFO:teuthology.orchestra.run.smithi008.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAi/AVRPGp614QppzArhmxjIl/6lnCQMfg6xlznI4WRGm7AafuZqG4qsSxR4B7i6UOSTQ0JaekQ0gOJM4mHozI5wwFkqTexfCPngq9Y31VL5ANzU6txfuyiFn7OVQre+ObSx8W6P3lzfWRpmi08ksmLS7vEtpm9VnpX0YujOKbB7fIgfRHMTqQpygkhQYsKmCqmMsavGpIUlOsABgsL8u9hUxJjyg5+pktIjtvlxgfPCxWq+7Ru4ylxdbh6aEMZm/5E0JOOs6IWJph+I04VzcmAzGfU93qlgpWJxA4eVxIiiu0VpBdF1qR1mPV8mPfA/C0NfFIfcoTkNb3j3yfBtvG3ZjGyHM/EKKuJucnCiyaugTgpxhIG7nRPqXiN12PbigIgLtIBny55EnMgxnnJdlHDHhr7v1CLV3ww0zd0AvHJHPpdZDZ5I013U39TtQ0l/WreCl5Knea+Lad+WUvw8g9b6tLEwwVDa7QXR2FnlJX4YNSQN0ihL+psa4F8mgHDwE= ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:46:20.156 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAi/AVRPGp614QppzArhmxjIl/6lnCQMfg6xlznI4WRGm7AafuZqG4qsSxR4B7i6UOSTQ0JaekQ0gOJM4mHozI5wwFkqTexfCPngq9Y31VL5ANzU6txfuyiFn7OVQre+ObSx8W6P3lzfWRpmi08ksmLS7vEtpm9VnpX0YujOKbB7fIgfRHMTqQpygkhQYsKmCqmMsavGpIUlOsABgsL8u9hUxJjyg5+pktIjtvlxgfPCxWq+7Ru4ylxdbh6aEMZm/5E0JOOs6IWJph+I04VzcmAzGfU93qlgpWJxA4eVxIiiu0VpBdF1qR1mPV8mPfA/C0NfFIfcoTkNb3j3yfBtvG3ZjGyHM/EKKuJucnCiyaugTgpxhIG7nRPqXiN12PbigIgLtIBny55EnMgxnnJdlHDHhr7v1CLV3ww0zd0AvHJHPpdZDZ5I013U39TtQ0l/WreCl5Knea+Lad+WUvw8g9b6tLEwwVDa7QXR2FnlJX4YNSQN0ihL+psa4F8mgHDwE= ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-12T05:46:20.201 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAi/AVRPGp614QppzArhmxjIl/6lnCQMfg6xlznI4WRGm7AafuZqG4qsSxR4B7i6UOSTQ0JaekQ0gOJM4mHozI5wwFkqTexfCPngq9Y31VL5ANzU6txfuyiFn7OVQre+ObSx8W6P3lzfWRpmi08ksmLS7vEtpm9VnpX0YujOKbB7fIgfRHMTqQpygkhQYsKmCqmMsavGpIUlOsABgsL8u9hUxJjyg5+pktIjtvlxgfPCxWq+7Ru4ylxdbh6aEMZm/5E0JOOs6IWJph+I04VzcmAzGfU93qlgpWJxA4eVxIiiu0VpBdF1qR1mPV8mPfA/C0NfFIfcoTkNb3j3yfBtvG3ZjGyHM/EKKuJucnCiyaugTgpxhIG7nRPqXiN12PbigIgLtIBny55EnMgxnnJdlHDHhr7v1CLV3ww0zd0AvHJHPpdZDZ5I013U39TtQ0l/WreCl5Knea+Lad+WUvw8g9b6tLEwwVDa7QXR2FnlJX4YNSQN0ihL+psa4F8mgHDwE= ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:46:20.213 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-12T05:46:20.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:20 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2978449973' entity='client.admin' 2024-04-12T05:46:20.483 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/312207815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 msgr2=0x7f9bec0ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/312207815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0ffba0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009910 tx=0x7f9be8009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/312207815 shutdown_connections 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/312207815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0ffba0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/312207815 >> 172.21.15.8:0/312207815 conn(0x7f9bec0fad20 msgr2=0x7f9bec0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/312207815 shutdown_connections 2024-04-12T05:46:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.087+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/312207815 wait complete. 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf3dd2700 1 Processor -- start 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf3dd2700 1 -- start start 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf3dd2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf3dd2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0fd200 con 0x7f9bec0ff780 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf1b6e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf1b6e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47092/0 (socket says 172.21.15.8:47092) 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf1b6e700 1 -- 172.21.15.8:0/667443285 learned_addr learned my addr 172.21.15.8:0/667443285 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.088+0000 7f9bf1b6e700 1 -- 172.21.15.8:0/667443285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bec0ff780 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9bf1b6e700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004f70 tx=0x7f9be8005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9be801d070 con 0x7f9bec0ff780 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9be800bdf0 con 0x7f9bec0ff780 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec0fb310 con 0x7f9bec0ff780 2024-04-12T05:46:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9be8022e40 con 0x7f9bec0ff780 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.089+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec0fb7b0 con 0x7f9bec0ff780 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f9be8022700 con 0x7f9bec0ff780 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9be2ffd700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 0x7f9bd803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9be804c410 con 0x7f9bec0ff780 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9bf136d700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 0x7f9bd803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec0623c0 con 0x7f9bec0ff780 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.090+0000 7f9bf136d700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 0x7f9bd803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.095+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9be8027070 con 0x7f9bec0ff780 2024-04-12T05:46:23.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.291+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f9bec18ec50 con 0x7f9bec0ff780 2024-04-12T05:46:23.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.295+0000 7f9be2ffd700 1 -- 172.21.15.8:0/667443285 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9bec18ec50 con 0x7f9bec0ff780 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 msgr2=0x7f9bd803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 0x7f9bd803a880 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 msgr2=0x7f9bec0fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9be8004f70 tx=0x7f9be8005ee0 comp rx=0 tx=0).stop 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 shutdown_connections 2024-04-12T05:46:23.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f9bd80383c0 0x7f9bd803a880 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 --2- 172.21.15.8:0/667443285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9bec0ff780 0x7f9bec0fccc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 >> 172.21.15.8:0/667443285 conn(0x7f9bec0fad20 msgr2=0x7f9bec068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 shutdown_connections 2024-04-12T05:46:23.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:23.298+0000 7f9bf3dd2700 1 -- 172.21.15.8:0/667443285 wait complete. 2024-04-12T05:46:24.230 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-12T05:46:24.231 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-12T05:46:24.594 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/667443285' entity='client.admin' 2024-04-12T05:46:24.645 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi008", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:46:25.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:25 smithi008 ceph-mon[39722]: Deploying daemon ceph-exporter.smithi008 on smithi008 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3601019654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 msgr2=0x7f4d440fea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3601019654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d440fea30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009910 tx=0x7f4d38009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3601019654 shutdown_connections 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3601019654 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d440fea30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3601019654 >> 172.21.15.8:0/3601019654 conn(0x7f4d440fa050 msgr2=0x7f4d440fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3601019654 shutdown_connections 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.438+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3601019654 wait complete. 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d4ae80700 1 Processor -- start 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d4ae80700 1 -- start start 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d4ae80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:27.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d4ae80700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44195df0 con 0x7f4d440fe610 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d48c1c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d48c1c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36792/0 (socket says 172.21.15.8:36792) 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d48c1c700 1 -- 172.21.15.8:0/3488589562 learned_addr learned my addr 172.21.15.8:0/3488589562 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d48c1c700 1 -- 172.21.15.8:0/3488589562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380095f0 con 0x7f4d440fe610 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.439+0000 7f4d48c1c700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d38004f40 tx=0x7f4d380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.440+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d3801d070 con 0x7f4d440fe610 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.440+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4d3800bdf0 con 0x7f4d440fe610 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.440+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d44195ff0 con 0x7f4d440fe610 2024-04-12T05:46:27.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.440+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4d38022e00 con 0x7f4d440fe610 2024-04-12T05:46:27.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.440+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d44196490 con 0x7f4d440fe610 2024-04-12T05:46:27.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.441+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f4d3800f4b0 con 0x7f4d440fe610 2024-04-12T05:46:27.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.441+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4418f380 con 0x7f4d440fe610 2024-04-12T05:46:27.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.441+0000 7f4d41ffb700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 0x7f4d3403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:27.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.441+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4d3804c410 con 0x7f4d440fe610 2024-04-12T05:46:27.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.441+0000 7f4d43fff700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 0x7f4d3403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:27.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.442+0000 7f4d43fff700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 0x7f4d3403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:27.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.446+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4d38020920 con 0x7f4d440fe610 2024-04-12T05:46:27.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.651+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d44061260 con 0x7f4d340383c0 2024-04-12T05:46:27.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.655+0000 7f4d41ffb700 1 -- 172.21.15.8:0/3488589562 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4d44061260 con 0x7f4d340383c0 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 msgr2=0x7f4d3403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 0x7f4d3403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 msgr2=0x7f4d441958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4d38004f40 tx=0x7f4d380057b0 comp rx=0 tx=0).stop 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 shutdown_connections 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4d340383c0 0x7f4d3403a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 --2- 172.21.15.8:0/3488589562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d440fe610 0x7f4d441958b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.657+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 >> 172.21.15.8:0/3488589562 conn(0x7f4d440fa050 msgr2=0x7f4d440fad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:27.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.658+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 shutdown_connections 2024-04-12T05:46:27.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:27.658+0000 7f4d4ae80700 1 -- 172.21.15.8:0/3488589562 wait complete. 2024-04-12T05:46:28.565 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-04-12T05:46:28.565 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:46:28.565 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-04-12T05:46:28.588 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:46:28.588 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:46:28.649 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-04-12T05:46:28.649 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch host add smithi194 2024-04-12T05:46:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-12T05:46:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:28 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:46:29.818 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:46:29.819 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:29 smithi008 ceph-mon[39722]: Deploying daemon crash.smithi008 on smithi008 2024-04-12T05:46:31.299 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:31 smithi008 ceph-mon[39722]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:31.299 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:31 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: Deploying daemon node-exporter.smithi008 on smithi008 2024-04-12T05:46:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:32 smithi008 ceph-mon[39722]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.188+0000 7f3ba9e81700 1 -- 172.21.15.8:0/1820952356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 msgr2=0x7f3ba40ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.188+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/1820952356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba40ffb40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400b0a0 tx=0x7f3b9400b3b0 comp rx=0 tx=0).stop 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 -- 172.21.15.8:0/1820952356 shutdown_connections 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/1820952356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba40ffb40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 -- 172.21.15.8:0/1820952356 >> 172.21.15.8:0/1820952356 conn(0x7f3ba40facc0 msgr2=0x7f3ba40fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 -- 172.21.15.8:0/1820952356 shutdown_connections 2024-04-12T05:46:32.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 -- 172.21.15.8:0/1820952356 wait complete. 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.189+0000 7f3ba9e81700 1 Processor -- start 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba9e81700 1 -- start start 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba9e81700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba9e81700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4194cc0 con 0x7f3ba40ff720 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36824/0 (socket says 172.21.15.8:36824) 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba37fe700 1 -- 172.21.15.8:0/838310796 learned_addr learned my addr 172.21.15.8:0/838310796 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:32.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.190+0000 7f3ba37fe700 1 -- 172.21.15.8:0/838310796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b94009d00 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba37fe700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b940062a0 tx=0x7f3b9400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b9400e040 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3b94012ad0 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b9401c860 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba4194ec0 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.191+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba4195360 con 0x7f3ba40ff720 2024-04-12T05:46:32.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.192+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba418e200 con 0x7f3ba40ff720 2024-04-12T05:46:32.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.193+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f3b94012c40 con 0x7f3ba40ff720 2024-04-12T05:46:32.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.193+0000 7f3ba0ff9700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 0x7f3b8c03a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:32.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.193+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b9404cfc0 con 0x7f3ba40ff720 2024-04-12T05:46:32.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.193+0000 7f3ba2ffd700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 0x7f3b8c03a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:32.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.194+0000 7f3ba2ffd700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 0x7f3b8c03a930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:32.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.197+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3b94024950 con 0x7f3ba40ff720 2024-04-12T05:46:32.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:32.414+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba40611d0 con 0x7f3b8c038470 2024-04-12T05:46:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:33 smithi008 ceph-mon[39722]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:46:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:33 smithi008 ceph-mon[39722]: Deploying cephadm binary to smithi194 2024-04-12T05:46:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:34 smithi008 ceph-mon[39722]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:34.668 INFO:teuthology.orchestra.run.smithi008.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-04-12T05:46:34.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.671+0000 7f3ba0ff9700 1 -- 172.21.15.8:0/838310796 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3ba40611d0 con 0x7f3b8c038470 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 msgr2=0x7f3b8c03a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 0x7f3b8c03a930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3b98006fd0 tx=0x7f3b98006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 msgr2=0x7f3ba4194780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3b940062a0 tx=0x7f3b9400bd40 comp rx=0 tx=0).stop 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 shutdown_connections 2024-04-12T05:46:34.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f3b8c038470 0x7f3b8c03a930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:34.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 --2- 172.21.15.8:0/838310796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ba40ff720 0x7f3ba4194780 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:34.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 >> 172.21.15.8:0/838310796 conn(0x7f3ba40facc0 msgr2=0x7f3ba40fb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:34.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 shutdown_connections 2024-04-12T05:46:34.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:34.674+0000 7f3ba9e81700 1 -- 172.21.15.8:0/838310796 wait complete. 2024-04-12T05:46:35.501 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch host ls --format=json 2024-04-12T05:46:35.752 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:46:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:35 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:35 smithi008 ceph-mon[39722]: Added host smithi194 2024-04-12T05:46:35.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:35 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: Deploying daemon alertmanager.smithi008 on smithi008 2024-04-12T05:46:37.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:36 smithi008 ceph-mon[39722]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.997+0000 7f980ae79700 1 -- 172.21.15.8:0/3289094796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 msgr2=0x7f9804100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.997+0000 7f980ae79700 1 --2- 172.21.15.8:0/3289094796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804100cc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009910 tx=0x7f97f8009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- 172.21.15.8:0/3289094796 shutdown_connections 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 --2- 172.21.15.8:0/3289094796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804100cc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- 172.21.15.8:0/3289094796 >> 172.21.15.8:0/3289094796 conn(0x7f98040fbe20 msgr2=0x7f98040fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- 172.21.15.8:0/3289094796 shutdown_connections 2024-04-12T05:46:38.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- 172.21.15.8:0/3289094796 wait complete. 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 Processor -- start 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- start start 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.998+0000 7f980ae79700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9804195e40 con 0x7f98041008a0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9808c15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9808c15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36488/0 (socket says 172.21.15.8:36488) 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9808c15700 1 -- 172.21.15.8:0/973584439 learned_addr learned my addr 172.21.15.8:0/973584439 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9808c15700 1 -- 172.21.15.8:0/973584439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80095f0 con 0x7f98041008a0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9808c15700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f801d070 con 0x7f98041008a0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f97f800bdf0 con 0x7f98041008a0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9804196040 con 0x7f98041008a0 2024-04-12T05:46:38.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f8022e50 con 0x7f98041008a0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:37.999+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98041964e0 con 0x7f98041008a0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.000+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980418f380 con 0x7f98041008a0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.001+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f97f8022700 con 0x7f98041008a0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.001+0000 7f9801ffb700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 0x7f97f403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.001+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f97f804c890 con 0x7f98041008a0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.001+0000 7f9803fff700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 0x7f97f403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.001+0000 7f9803fff700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 0x7f97f403a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97f0006fd0 tx=0x7f97f0006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:38.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.005+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97f800b960 con 0x7f98041008a0 2024-04-12T05:46:38.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.210+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98040611d0 con 0x7f97f4038370 2024-04-12T05:46:38.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.211+0000 7f9801ffb700 1 -- 172.21.15.8:0/973584439 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f98040611d0 con 0x7f97f4038370 2024-04-12T05:46:38.207 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:46:38.207 INFO:teuthology.orchestra.run.smithi008.stdout:[{"addr": "172.21.15.8", "hostname": "smithi008", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 msgr2=0x7f97f403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 0x7f97f403a830 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97f0006fd0 tx=0x7f97f0006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 msgr2=0x7f9804195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006010 tx=0x7f97f80057b0 comp rx=0 tx=0).stop 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 shutdown_connections 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f97f4038370 0x7f97f403a830 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 --2- 172.21.15.8:0/973584439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98041008a0 0x7f9804195900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 >> 172.21.15.8:0/973584439 conn(0x7f98040fbe20 msgr2=0x7f98040fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:38.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 shutdown_connections 2024-04-12T05:46:38.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:38.213+0000 7f980ae79700 1 -- 172.21.15.8:0/973584439 wait complete. 2024-04-12T05:46:39.339 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:39 smithi008 ceph-mon[39722]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:39.360 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-12T05:46:39.360 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd crush tunables default 2024-04-12T05:46:40.220 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:46:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:40 smithi008 ceph-mon[39722]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T05:46:41.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:41 smithi008 ceph-mon[39722]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:41.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:41 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:42.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.951+0000 7f413fe62700 1 -- 172.21.15.8:0/232517078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 msgr2=0x7f41380ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.951+0000 7f413fe62700 1 --2- 172.21.15.8:0/232517078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380ffba0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f412c009910 tx=0x7f412c009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.951+0000 7f413fe62700 1 -- 172.21.15.8:0/232517078 shutdown_connections 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.951+0000 7f413fe62700 1 --2- 172.21.15.8:0/232517078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380ffba0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.951+0000 7f413fe62700 1 -- 172.21.15.8:0/232517078 >> 172.21.15.8:0/232517078 conn(0x7f41380fad00 msgr2=0x7f41380fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 -- 172.21.15.8:0/232517078 shutdown_connections 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 -- 172.21.15.8:0/232517078 wait complete. 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 Processor -- start 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 -- start start 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.952+0000 7f413fe62700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380fd200 con 0x7f41380ff780 2024-04-12T05:46:42.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413dbfe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413dbfe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36492/0 (socket says 172.21.15.8:36492) 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413dbfe700 1 -- 172.21.15.8:0/216050707 learned_addr learned my addr 172.21.15.8:0/216050707 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413dbfe700 1 -- 172.21.15.8:0/216050707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0095f0 con 0x7f41380ff780 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413dbfe700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f70 tx=0x7f412c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f412c01d070 con 0x7f41380ff780 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f412c00bdf0 con 0x7f41380ff780 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380fb310 con 0x7f41380ff780 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f412c022e40 con 0x7f41380ff780 2024-04-12T05:46:42.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.953+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41380fb7b0 con 0x7f41380ff780 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.954+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f412c00f460 con 0x7f41380ff780 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.954+0000 7f412affd700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 0x7f412403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.954+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f412c04c470 con 0x7f41380ff780 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.954+0000 7f413d3fd700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 0x7f412403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.955+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41380623c0 con 0x7f41380ff780 2024-04-12T05:46:42.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.955+0000 7f413d3fd700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 0x7f412403a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:42.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:42.960+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f412c027020 con 0x7f41380ff780 2024-04-12T05:46:43.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:43.164+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f413818ec50 con 0x7f41380ff780 2024-04-12T05:46:43.354 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:43 smithi008 ceph-mon[39722]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.099+0000 7f412affd700 1 -- 172.21.15.8:0/216050707 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f412c034070 con 0x7f41380ff780 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 msgr2=0x7f412403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 0x7f412403a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4134006fd0 tx=0x7f4134006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 msgr2=0x7f41380fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f412c004f70 tx=0x7f412c005ee0 comp rx=0 tx=0).stop 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 shutdown_connections 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4124038410 0x7f412403a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 --2- 172.21.15.8:0/216050707 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41380ff780 0x7f41380fccc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.101+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 >> 172.21.15.8:0/216050707 conn(0x7f41380fad00 msgr2=0x7f4138069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.102+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 shutdown_connections 2024-04-12T05:46:44.098 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:46:44.102+0000 7f413fe62700 1 -- 172.21.15.8:0/216050707 wait complete. 2024-04-12T05:46:44.099 INFO:teuthology.orchestra.run.smithi008.stderr:adjusted tunables profile to default 2024-04-12T05:46:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:44 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/216050707' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-12T05:46:45.168 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:45 smithi008 ceph-mon[39722]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:45.168 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:45 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/216050707' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-12T05:46:45.168 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:45 smithi008 ceph-mon[39722]: osdmap e4: 0 total, 0 up, 0 in 2024-04-12T05:46:45.770 INFO:tasks.cephadm:Adding mon.smithi008 on smithi008 2024-04-12T05:46:45.770 INFO:tasks.cephadm:Adding mon.smithi194 on smithi194 2024-04-12T05:46:45.770 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch apply mon '2;smithi008:172.21.15.8=smithi008;smithi194:172.21.15.194=smithi194' 2024-04-12T05:46:46.022 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:46.085 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:46.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:46 smithi008 ceph-mon[39722]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:47 smithi008 ceph-mon[39722]: Deploying daemon grafana.smithi008 on smithi008 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.907+0000 7f54032ac700 1 -- 172.21.15.194:0/2315260613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 msgr2=0x7f53fc0fef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.907+0000 7f54032ac700 1 --2- 172.21.15.194:0/2315260613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc0fef70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009910 tx=0x7f53ec009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.907+0000 7f54032ac700 1 -- 172.21.15.194:0/2315260613 shutdown_connections 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.907+0000 7f54032ac700 1 --2- 172.21.15.194:0/2315260613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc0fef70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.907+0000 7f54032ac700 1 -- 172.21.15.194:0/2315260613 >> 172.21.15.194:0/2315260613 conn(0x7f53fc0fa490 msgr2=0x7f53fc0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 -- 172.21.15.194:0/2315260613 shutdown_connections 2024-04-12T05:46:47.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 -- 172.21.15.194:0/2315260613 wait complete. 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 Processor -- start 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 -- start start 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f54032ac700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc1019d0 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f5401048700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.908+0000 7f5401048700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:53938/0 (socket says 172.21.15.194:53938) 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f5401048700 1 -- 172.21.15.194:0/2495586203 learned_addr learned my addr 172.21.15.194:0/2495586203 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f5401048700 1 -- 172.21.15.194:0/2495586203 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0095f0 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f5401048700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ec01d070 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc0ffae0 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc0fff80 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f53ec00bdf0 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.909+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53ec022e00 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.910+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f53ec00b960 con 0x7f53fc0feb50 2024-04-12T05:46:47.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.910+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e0005320 con 0x7f53fc0feb50 2024-04-12T05:46:47.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.910+0000 7f53f27fc700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 0x7f53e803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:47.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.910+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f53ec04c130 con 0x7f53fc0feb50 2024-04-12T05:46:47.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.910+0000 7f5400847700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 0x7f53e803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:47.909 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.911+0000 7f5400847700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 0x7f53e803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:47.912 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:47.916+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f53ec0226e0 con 0x7f53fc0feb50 2024-04-12T05:46:48.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.115+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 --> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi008:172.21.15.8=smithi008;smithi194:172.21.15.194=smithi194", "target": ["mon-mgr", ""]}) v1 -- 0x7f53e0000ca0 con 0x7f53e80383c0 2024-04-12T05:46:48.117 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.121+0000 7f53f27fc700 1 -- 172.21.15.194:0/2495586203 <== mgr.14162 v2:172.21.15.8:6800/3827652772 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f53e0000ca0 con 0x7f53e80383c0 2024-04-12T05:46:48.118 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2024-04-12T05:46:48.119 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.122+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 msgr2=0x7f53e803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:48.119 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.122+0000 7f54032ac700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 0x7f53e803a880 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f53f8006fd0 tx=0x7f53f8006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:48.119 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 msgr2=0x7f53fc101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:48.119 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006010 tx=0x7f53ec0057b0 comp rx=0 tx=0).stop 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 shutdown_connections 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f53e80383c0 0x7f53e803a880 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 --2- 172.21.15.194:0/2495586203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53fc0feb50 0x7f53fc101490 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 >> 172.21.15.194:0/2495586203 conn(0x7f53fc0fa490 msgr2=0x7f53fc0fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 shutdown_connections 2024-04-12T05:46:48.120 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:48.123+0000 7f54032ac700 1 -- 172.21.15.194:0/2495586203 wait complete. 2024-04-12T05:46:48.789 DEBUG:teuthology.orchestra.run.smithi194:mon.smithi194> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi194.service 2024-04-12T05:46:48.792 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:46:48.792 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:46:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:-- Logs begin at Fri 2024-04-12 05:32:37 UTC. -- 2024-04-12T05:46:49.048 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:49.105 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:49 smithi008 ceph-mon[39722]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:49 smithi008 ceph-mon[39722]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi008:172.21.15.8=smithi008;smithi194:172.21.15.194=smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:46:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:49 smithi008 ceph-mon[39722]: Saving service mon spec with placement smithi008:172.21.15.8=smithi008;smithi194:172.21.15.194=smithi194;count:2 2024-04-12T05:46:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:49 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:50.934 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 -- 172.21.15.194:0/2777708749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 msgr2=0x7f72800ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2777708749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72800ffba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7274009910 tx=0x7f7274009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 -- 172.21.15.194:0/2777708749 shutdown_connections 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2777708749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72800ffba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 -- 172.21.15.194:0/2777708749 >> 172.21.15.194:0/2777708749 conn(0x7f72800fad00 msgr2=0x7f72800fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 -- 172.21.15.194:0/2777708749 shutdown_connections 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.935+0000 7f7286ed7700 1 -- 172.21.15.194:0/2777708749 wait complete. 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.936+0000 7f7286ed7700 1 Processor -- start 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.936+0000 7f7286ed7700 1 -- start start 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.936+0000 7f7286ed7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.936+0000 7f7286ed7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280194d00 con 0x7f72800ff780 2024-04-12T05:46:50.935 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7284c73700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7284c73700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:53956/0 (socket says 172.21.15.194:53956) 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7284c73700 1 -- 172.21.15.194:0/2592582075 learned_addr learned my addr 172.21.15.194:0/2592582075 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7284c73700 1 -- 172.21.15.194:0/2592582075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740095f0 con 0x7f72800ff780 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7284c73700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7274004f70 tx=0x7f7274005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f727401d070 con 0x7f72800ff780 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7280194f00 con 0x7f72800ff780 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f727400bdf0 con 0x7f72800ff780 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7274022e40 con 0x7f72800ff780 2024-04-12T05:46:50.936 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.937+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72801953a0 con 0x7f72800ff780 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.938+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f727400b960 con 0x7f72800ff780 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.938+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728018e260 con 0x7f72800ff780 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.938+0000 7f727dffb700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 0x7f727003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.938+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f727404c440 con 0x7f72800ff780 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.939+0000 7f727ffff700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 0x7f727003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:50.937 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.939+0000 7f727ffff700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 0x7f727003a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f726c006fd0 tx=0x7f726c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:50.940 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:50.944+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f727402bba0 con 0x7f72800ff780 2024-04-12T05:46:51.205 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.208+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f728002cc70 con 0x7f72800ff780 2024-04-12T05:46:51.205 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.208+0000 7f727dffb700 1 -- 172.21.15.194:0/2592582075 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f7274027070 con 0x7f72800ff780 2024-04-12T05:46:51.205 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:46:51.206 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:46:51.207 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 msgr2=0x7f727003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:51.207 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 0x7f727003a880 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f726c006fd0 tx=0x7f726c006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 msgr2=0x7f72801947c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7274004f70 tx=0x7f7274005ee0 comp rx=0 tx=0).stop 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 shutdown_connections 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f72700383c0 0x7f727003a880 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 --2- 172.21.15.194:0/2592582075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72800ff780 0x7f72801947c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 >> 172.21.15.194:0/2592582075 conn(0x7f72800fad00 msgr2=0x7f72800fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.210+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 shutdown_connections 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:51.211+0000 7f7286ed7700 1 -- 172.21.15.194:0/2592582075 wait complete. 2024-04-12T05:46:51.208 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:46:51.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:51 smithi008 ceph-mon[39722]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:51.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:51 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:46:52.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:52 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/2592582075' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:46:52.868 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:46:52.868 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:46:53.119 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:53.180 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:53 smithi008 ceph-mon[39722]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 -- 172.21.15.194:0/1923890025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 msgr2=0x7f4954100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 --2- 172.21.15.194:0/1923890025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f4954100cc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4944009910 tx=0x7f4944009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 -- 172.21.15.194:0/1923890025 shutdown_connections 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 --2- 172.21.15.194:0/1923890025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f4954100cc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 -- 172.21.15.194:0/1923890025 >> 172.21.15.194:0/1923890025 conn(0x7f49540fbe20 msgr2=0x7f49540fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 -- 172.21.15.194:0/1923890025 shutdown_connections 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.036+0000 7f495bab8700 1 -- 172.21.15.194:0/1923890025 wait complete. 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f495bab8700 1 Processor -- start 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f495bab8700 1 -- start start 2024-04-12T05:46:55.034 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f495bab8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f495bab8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954195e20 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f4959854700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f4959854700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:53978/0 (socket says 172.21.15.194:53978) 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.037+0000 7f4959854700 1 -- 172.21.15.194:0/1399906603 learned_addr learned my addr 172.21.15.194:0/1399906603 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f4959854700 1 -- 172.21.15.194:0/1399906603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49440095f0 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f4959854700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4944006010 tx=0x7f49440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f494401d070 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4954196020 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f494400bdf0 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4944022e50 con 0x7f49541008a0 2024-04-12T05:46:55.035 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.038+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49541964c0 con 0x7f49541008a0 2024-04-12T05:46:55.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.039+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f494400b960 con 0x7f49541008a0 2024-04-12T05:46:55.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.039+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495418f380 con 0x7f49541008a0 2024-04-12T05:46:55.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.039+0000 7f494affd700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 0x7f494003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:55.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.039+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f494404c390 con 0x7f49541008a0 2024-04-12T05:46:55.036 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.039+0000 7f4959053700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 0x7f494003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:55.037 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.040+0000 7f4959053700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 0x7f494003a830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:55.041 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.045+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4944022700 con 0x7f49541008a0 2024-04-12T05:46:55.300 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.304+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f495402cc70 con 0x7f49541008a0 2024-04-12T05:46:55.300 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.305+0000 7f494affd700 1 -- 172.21.15.194:0/1399906603 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f4944027020 con 0x7f49541008a0 2024-04-12T05:46:55.301 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:46:55.301 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 msgr2=0x7f494003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 0x7f494003a830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 msgr2=0x7f49541958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4944006010 tx=0x7f49440057b0 comp rx=0 tx=0).stop 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 shutdown_connections 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4940038370 0x7f494003a830 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 --2- 172.21.15.194:0/1399906603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49541008a0 0x7f49541958e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 >> 172.21.15.194:0/1399906603 conn(0x7f49540fbe20 msgr2=0x7f49540fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 shutdown_connections 2024-04-12T05:46:55.303 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:55.307+0000 7f495bab8700 1 -- 172.21.15.194:0/1399906603 wait complete. 2024-04-12T05:46:55.304 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:46:55.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:55 smithi008 ceph-mon[39722]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:56.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:56 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1399906603' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:46:56.992 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:46:56.993 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:46:57.250 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:57.307 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:46:57.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:57 smithi008 ceph-mon[39722]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:46:59.048 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 -- 172.21.15.194:0/1343923128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 msgr2=0x7fcf1c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1343923128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c101140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009910 tx=0x7fcf10009c20 comp rx=0 tx=0).stop 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 -- 172.21.15.194:0/1343923128 shutdown_connections 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1343923128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c101140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 -- 172.21.15.194:0/1343923128 >> 172.21.15.194:0/1343923128 conn(0x7fcf1c0fa490 msgr2=0x7fcf1c0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 -- 172.21.15.194:0/1343923128 shutdown_connections 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.042+0000 7fcf22b42700 1 -- 172.21.15.194:0/1343923128 wait complete. 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf22b42700 1 Processor -- start 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf22b42700 1 -- start start 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf22b42700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:59.049 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf22b42700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c194d10 con 0x7fcf1c100d20 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf208de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf208de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:58318/0 (socket says 172.21.15.194:58318) 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.043+0000 7fcf208de700 1 -- 172.21.15.194:0/1774118578 learned_addr learned my addr 172.21.15.194:0/1774118578 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf208de700 1 -- 172.21.15.194:0/1774118578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100095f0 con 0x7fcf1c100d20 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf208de700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006010 tx=0x7fcf100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf1001d070 con 0x7fcf1c100d20 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcf1000bdf0 con 0x7fcf1c100d20 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf10022e50 con 0x7fcf1c100d20 2024-04-12T05:46:59.050 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c194f10 con 0x7fcf1c100d20 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.044+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c1953b0 con 0x7fcf1c100d20 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.045+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7fcf10022700 con 0x7fcf1c100d20 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.045+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c18e270 con 0x7fcf1c100d20 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.045+0000 7fcf19ffb700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 0x7fcf0c03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.045+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf1004c410 con 0x7fcf1c100d20 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.045+0000 7fcf1bfff700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 0x7fcf0c03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.046+0000 7fcf1bfff700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 0x7fcf0c03a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcf08006fd0 tx=0x7fcf08006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:46:59.051 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.050+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcf1000b960 con 0x7fcf1c100d20 2024-04-12T05:46:59.307 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.311+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcf1c02cc70 con 0x7fcf1c100d20 2024-04-12T05:46:59.308 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.312+0000 7fcf19ffb700 1 -- 172.21.15.194:0/1774118578 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fcf10027020 con 0x7fcf1c100d20 2024-04-12T05:46:59.308 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:46:59.308 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:46:59.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 msgr2=0x7fcf0c03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:59.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 0x7fcf0c03a830 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcf08006fd0 tx=0x7fcf08006e40 comp rx=0 tx=0).stop 2024-04-12T05:46:59.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 msgr2=0x7fcf1c1947d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:46:59.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcf10006010 tx=0x7fcf100057b0 comp rx=0 tx=0).stop 2024-04-12T05:46:59.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 shutdown_connections 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fcf0c038370 0x7fcf0c03a830 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 --2- 172.21.15.194:0/1774118578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf1c100d20 0x7fcf1c1947d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 >> 172.21.15.194:0/1774118578 conn(0x7fcf1c0fa490 msgr2=0x7fcf1c0fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 shutdown_connections 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:46:59.314+0000 7fcf22b42700 1 -- 172.21.15.194:0/1774118578 wait complete. 2024-04-12T05:46:59.311 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:46:59.358 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:46:59 smithi008 ceph-mon[39722]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:00.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1774118578' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:00.951 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:00.951 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:01.201 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:01.260 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:01.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:01 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:02 smithi008 ceph-mon[39722]: Deploying daemon prometheus.smithi008 on smithi008 2024-04-12T05:47:03.244 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 -- 172.21.15.194:0/3349010987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 msgr2=0x7f228c100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:03.244 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 --2- 172.21.15.194:0/3349010987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c100cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2280009910 tx=0x7f2280009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:03.244 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 -- 172.21.15.194:0/3349010987 shutdown_connections 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 --2- 172.21.15.194:0/3349010987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c100cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 -- 172.21.15.194:0/3349010987 >> 172.21.15.194:0/3349010987 conn(0x7f228c0fbe20 msgr2=0x7f228c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 -- 172.21.15.194:0/3349010987 shutdown_connections 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.247+0000 7f22908dd700 1 -- 172.21.15.194:0/3349010987 wait complete. 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.248+0000 7f22908dd700 1 Processor -- start 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.248+0000 7f22908dd700 1 -- start start 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.248+0000 7f22908dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.248+0000 7f22908dd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c195e40 con 0x7f228c1008a0 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f228a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f228a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:58344/0 (socket says 172.21.15.194:58344) 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f228a59c700 1 -- 172.21.15.194:0/1765228677 learned_addr learned my addr 172.21.15.194:0/1765228677 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:03.245 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f228a59c700 1 -- 172.21.15.194:0/1765228677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22800095f0 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f228a59c700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f22800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f228001d070 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c196040 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f228000bdf0 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2280022e50 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.249+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c1964e0 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.250+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f2280022700 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.250+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c18f380 con 0x7f228c1008a0 2024-04-12T05:47:03.247 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.250+0000 7f227b7fe700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 0x7f227403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:03.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.250+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f228004c3d0 con 0x7f228c1008a0 2024-04-12T05:47:03.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.251+0000 7f2289d9b700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 0x7f227403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:03.248 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.251+0000 7f2289d9b700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 0x7f227403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:03.255 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.260+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f228000b960 con 0x7f228c1008a0 2024-04-12T05:47:03.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:03 smithi008 ceph-mon[39722]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:03.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.524+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f228c02cc70 con 0x7f228c1008a0 2024-04-12T05:47:03.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.525+0000 7f227b7fe700 1 -- 172.21.15.194:0/1765228677 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f2280027020 con 0x7f228c1008a0 2024-04-12T05:47:03.521 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:03.521 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:03.522 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 msgr2=0x7f227403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 0x7f227403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 msgr2=0x7f228c195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2280006010 tx=0x7f22800057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 shutdown_connections 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2274038370 0x7f227403a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 --2- 172.21.15.194:0/1765228677 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f228c1008a0 0x7f228c195900 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 >> 172.21.15.194:0/1765228677 conn(0x7f228c0fbe20 msgr2=0x7f228c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 shutdown_connections 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:03.527+0000 7f22908dd700 1 -- 172.21.15.194:0/1765228677 wait complete. 2024-04-12T05:47:03.523 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:04 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1765228677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:05.191 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:05.192 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:05.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:05 smithi008 ceph-mon[39722]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:05.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:05 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:05.441 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:05.498 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:06 smithi008 ceph-mon[39722]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:07.239 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 -- 172.21.15.194:0/833015981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 msgr2=0x7f4a680ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:07.239 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 --2- 172.21.15.194:0/833015981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a680ffba0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009910 tx=0x7f4a58009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 -- 172.21.15.194:0/833015981 shutdown_connections 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 --2- 172.21.15.194:0/833015981 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a680ffba0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 -- 172.21.15.194:0/833015981 >> 172.21.15.194:0/833015981 conn(0x7f4a680fad20 msgr2=0x7f4a680fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 -- 172.21.15.194:0/833015981 shutdown_connections 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.240+0000 7f4a6f710700 1 -- 172.21.15.194:0/833015981 wait complete. 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6f710700 1 Processor -- start 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6f710700 1 -- start start 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6f710700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6f710700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68195e20 con 0x7f4a680ff780 2024-04-12T05:47:07.240 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6d4ac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6d4ac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:39410/0 (socket says 172.21.15.194:39410) 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.241+0000 7f4a6d4ac700 1 -- 172.21.15.194:0/2064829914 learned_addr learned my addr 172.21.15.194:0/2064829914 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a6d4ac700 1 -- 172.21.15.194:0/2064829914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580095f0 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a6d4ac700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006010 tx=0x7f4a580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a5801d070 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a68196020 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4a5800bdf0 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a58022e50 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.242+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a681964c0 con 0x7f4a680ff780 2024-04-12T05:47:07.241 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.243+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f4a58022700 con 0x7f4a680ff780 2024-04-12T05:47:07.242 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.243+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6818f470 con 0x7f4a680ff780 2024-04-12T05:47:07.242 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.243+0000 7f4a5e7fc700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 0x7f4a5403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:07.242 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.243+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a5804c410 con 0x7f4a680ff780 2024-04-12T05:47:07.242 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.243+0000 7f4a6ccab700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 0x7f4a5403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:07.242 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.244+0000 7f4a6ccab700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 0x7f4a5403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a64006fd0 tx=0x7f4a64006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:07.244 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.249+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a5800b960 con 0x7f4a680ff780 2024-04-12T05:47:07.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.508+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4a6802cc70 con 0x7f4a680ff780 2024-04-12T05:47:07.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.509+0000 7f4a5e7fc700 1 -- 172.21.15.194:0/2064829914 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f4a58027020 con 0x7f4a680ff780 2024-04-12T05:47:07.505 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:07.505 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:07.507 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 msgr2=0x7f4a5403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:07.507 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 0x7f4a5403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a64006fd0 tx=0x7f4a64006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:07.507 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 msgr2=0x7f4a681958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:07.507 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4a58006010 tx=0x7f4a580057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:07.507 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 shutdown_connections 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f4a54038370 0x7f4a5403a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 --2- 172.21.15.194:0/2064829914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a680ff780 0x7f4a681958e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 >> 172.21.15.194:0/2064829914 conn(0x7f4a680fad20 msgr2=0x7f4a68103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 shutdown_connections 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:07.511+0000 7f4a6f710700 1 -- 172.21.15.194:0/2064829914 wait complete. 2024-04-12T05:47:07.508 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:07.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:07 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/2064829914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:08.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:08 smithi008 ceph-mon[39722]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:09.151 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:09.153 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:09.408 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:09.476 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 -- 172.21.15.194:0/3818835116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 msgr2=0x7f2098100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 --2- 172.21.15.194:0/3818835116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f2098100ca0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2088009910 tx=0x7f2088009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 -- 172.21.15.194:0/3818835116 shutdown_connections 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 --2- 172.21.15.194:0/3818835116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f2098100ca0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 -- 172.21.15.194:0/3818835116 >> 172.21.15.194:0/3818835116 conn(0x7f20980fbe00 msgr2=0x7f20980fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 -- 172.21.15.194:0/3818835116 shutdown_connections 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.254+0000 7f209fe59700 1 -- 172.21.15.194:0/3818835116 wait complete. 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.255+0000 7f209fe59700 1 Processor -- start 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.255+0000 7f209fe59700 1 -- start start 2024-04-12T05:47:11.263 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.255+0000 7f209fe59700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.255+0000 7f209fe59700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098195e00 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209dbf5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209dbf5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:39426/0 (socket says 172.21.15.194:39426) 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209dbf5700 1 -- 172.21.15.194:0/3653417638 learned_addr learned my addr 172.21.15.194:0/3653417638 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209dbf5700 1 -- 172.21.15.194:0/3653417638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880095f0 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209dbf5700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2088004f70 tx=0x7f2088005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f208801d070 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098196000 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f208800bdf0 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2088022e40 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.256+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20981964a0 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.257+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f2088022700 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.257+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209818f360 con 0x7f2098100880 2024-04-12T05:47:11.264 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.257+0000 7f208effd700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 0x7f208403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:11.265 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.257+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f208804c4d0 con 0x7f2098100880 2024-04-12T05:47:11.265 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.258+0000 7f209d3f4700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 0x7f208403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:11.265 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.258+0000 7f209d3f4700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 0x7f208403a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2094006fd0 tx=0x7f2094006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:11.265 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.263+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f208802ba00 con 0x7f2098100880 2024-04-12T05:47:11.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:11 smithi008 ceph-mon[39722]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:11.524 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.529+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f209802cc70 con 0x7f2098100880 2024-04-12T05:47:11.525 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.530+0000 7f208effd700 1 -- 172.21.15.194:0/3653417638 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f2088027030 con 0x7f2098100880 2024-04-12T05:47:11.526 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:11.526 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:11.527 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 msgr2=0x7f208403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 0x7f208403a8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2094006fd0 tx=0x7f2094006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 msgr2=0x7f20981958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2088004f70 tx=0x7f2088005ee0 comp rx=0 tx=0).stop 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 shutdown_connections 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2084038410 0x7f208403a8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 --2- 172.21.15.194:0/3653417638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2098100880 0x7f20981958c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 >> 172.21.15.194:0/3653417638 conn(0x7f20980fbe00 msgr2=0x7f20980fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 shutdown_connections 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:11.532+0000 7f209fe59700 1 -- 172.21.15.194:0/3653417638 wait complete. 2024-04-12T05:47:11.528 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:12.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:12 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/3653417638' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:13.150 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:13.151 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:13.394 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:13.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:13 smithi008 ceph-mon[39722]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:13.454 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:15.266 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/2455865433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 msgr2=0x7f2ab00ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/2455865433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab00ff330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009910 tx=0x7f2aa0009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/2455865433 shutdown_connections 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/2455865433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab00ff330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/2455865433 >> 172.21.15.194:0/2455865433 conn(0x7f2ab00fa490 msgr2=0x7f2ab00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/2455865433 shutdown_connections 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.259+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/2455865433 wait complete. 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab7dc7700 1 Processor -- start 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab7dc7700 1 -- start start 2024-04-12T05:47:15.267 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab7dc7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab7dc7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0197ff0 con 0x7f2ab00fef10 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab5b63700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab5b63700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:39446/0 (socket says 172.21.15.194:39446) 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab5b63700 1 -- 172.21.15.194:0/983442815 learned_addr learned my addr 172.21.15.194:0/983442815 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.260+0000 7f2ab5b63700 1 -- 172.21.15.194:0/983442815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00095f0 con 0x7f2ab00fef10 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2ab5b63700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0004f40 tx=0x7f2aa00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2aa001d070 con 0x7f2ab00fef10 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2aa000bdf0 con 0x7f2ab00fef10 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab01981f0 con 0x7f2ab00fef10 2024-04-12T05:47:15.268 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2aa0022e30 con 0x7f2ab00fef10 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.261+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0198690 con 0x7f2ab00fef10 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab0104b80 con 0x7f2ab00fef10 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7f2aa000b960 con 0x7f2ab00fef10 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2aa6ffd700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 0x7f2a9c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2aa004c390 con 0x7f2ab00fef10 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2ab5362700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 0x7f2a9c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.262+0000 7f2ab5362700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 0x7f2a9c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006fd0 tx=0x7f2aac006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:15.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.267+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2aa002a360 con 0x7f2ab00fef10 2024-04-12T05:47:15.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:15 smithi008 ceph-mon[39722]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:15.528 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.532+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2ab019b470 con 0x7f2ab00fef10 2024-04-12T05:47:15.529 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.533+0000 7f2aa6ffd700 1 -- 172.21.15.194:0/983442815 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f2aa0027030 con 0x7f2ab00fef10 2024-04-12T05:47:15.529 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:15.529 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 msgr2=0x7f2a9c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 0x7f2a9c03a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2aac006fd0 tx=0x7f2aac006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 msgr2=0x7f2ab0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0004f40 tx=0x7f2aa00057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 shutdown_connections 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f2a9c0383c0 0x7f2a9c03a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 --2- 172.21.15.194:0/983442815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ab00fef10 0x7f2ab0197ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 >> 172.21.15.194:0/983442815 conn(0x7f2ab00fa490 msgr2=0x7f2ab00fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 shutdown_connections 2024-04-12T05:47:15.531 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:15.535+0000 7f2ab7dc7700 1 -- 172.21.15.194:0/983442815 wait complete. 2024-04-12T05:47:15.532 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:16.644 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:16 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/983442815' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:16.644 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:16 smithi008 ceph-mon[39722]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:17.330 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:17.331 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:17.582 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:17.640 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:19.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:19 smithi008 ceph-mon[39722]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:19.475 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 -- 172.21.15.194:0/982644818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 msgr2=0x7fedcc100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:19.475 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 --2- 172.21.15.194:0/982644818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc100cc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009910 tx=0x7fedc0009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:19.475 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 -- 172.21.15.194:0/982644818 shutdown_connections 2024-04-12T05:47:19.475 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 --2- 172.21.15.194:0/982644818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc100cc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:19.475 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 -- 172.21.15.194:0/982644818 >> 172.21.15.194:0/982644818 conn(0x7fedcc0fbe20 msgr2=0x7fedcc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 -- 172.21.15.194:0/982644818 shutdown_connections 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 -- 172.21.15.194:0/982644818 wait complete. 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.475+0000 7fedd195e700 1 Processor -- start 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedd195e700 1 -- start start 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedd195e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedd195e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc195e40 con 0x7fedcc1008a0 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedcaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:39704/0 (socket says 172.21.15.194:39704) 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedcaffd700 1 -- 172.21.15.194:0/1567008673 learned_addr learned my addr 172.21.15.194:0/1567008673 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:19.476 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedcaffd700 1 -- 172.21.15.194:0/1567008673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00095f0 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.476+0000 7fedcaffd700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fedc0004f70 tx=0x7fedc0005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedc001d070 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fedc000bdf0 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc196040 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fedc0022e40 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc1964e0 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.477+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 13) v1 ==== 45338+0+0 (secure 0 0 0) 0x7fedc000f460 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.478+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc18f380 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.478+0000 7fedd095c700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 0x7fedb403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.478+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fedc004c4b0 con 0x7fedcc1008a0 2024-04-12T05:47:19.477 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.478+0000 7fedca7fc700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 0x7fedb403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:19.478 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.478+0000 7fedca7fc700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 0x7fedb403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:19.479 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.483+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fedc0017440 con 0x7fedcc1008a0 2024-04-12T05:47:19.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.747+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fedcc02cc70 con 0x7fedcc1008a0 2024-04-12T05:47:19.744 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.748+0000 7fedd095c700 1 -- 172.21.15.194:0/1567008673 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7fedc0027020 con 0x7fedcc1008a0 2024-04-12T05:47:19.745 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:19.745 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:19.746 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 msgr2=0x7fedb403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 0x7fedb403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fedbc006fd0 tx=0x7fedbc006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 msgr2=0x7fedcc195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fedc0004f70 tx=0x7fedc0005ee0 comp rx=0 tx=0).stop 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 shutdown_connections 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7fedb4038410 0x7fedb403a8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 --2- 172.21.15.194:0/1567008673 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedcc1008a0 0x7fedcc195900 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 >> 172.21.15.194:0/1567008673 conn(0x7fedcc0fbe20 msgr2=0x7fedcc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 shutdown_connections 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:19.750+0000 7fedd195e700 1 -- 172.21.15.194:0/1567008673 wait complete. 2024-04-12T05:47:19.747 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1567008673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:20 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:21.493 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:21.493 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:21.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:21 smithi008 ceph-mon[39722]: from='mgr.14162 172.21.15.8:0/2440004564' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-12T05:47:21.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:21 smithi008 ceph-mon[39722]: mgrmap e14: smithi008.ixmnlp(active, since 70s) 2024-04-12T05:47:21.739 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:21.808 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:23.510 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- 172.21.15.194:0/226679513 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 msgr2=0x7f09e40ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/226679513 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40ffba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009910 tx=0x7f09d8009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- 172.21.15.194:0/226679513 shutdown_connections 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/226679513 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40ffba0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- 172.21.15.194:0/226679513 >> 172.21.15.194:0/226679513 conn(0x7f09e40fad00 msgr2=0x7f09e40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- 172.21.15.194:0/226679513 shutdown_connections 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- 172.21.15.194:0/226679513 wait complete. 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 Processor -- start 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.500+0000 7f09e8c4b700 1 -- start start 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e8c4b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e8c4b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e40fd200 con 0x7f09e40ff780 2024-04-12T05:47:23.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:39726/0 (socket says 172.21.15.194:39726) 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e259c700 1 -- 172.21.15.194:0/3553935813 learned_addr learned my addr 172.21.15.194:0/3553935813 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e259c700 1 -- 172.21.15.194:0/3553935813 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80095f0 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e259c700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006010 tx=0x7f09d80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d801d070 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e40fb310 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f09d800bdf0 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f09d8022e50 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.501+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e40fb7b0 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.502+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e40623c0 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.502+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 14) v1 ==== 45352+0+0 (secure 0 0 0) 0x7f09d800b960 con 0x7f09e40ff780 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.503+0000 7f09d37fe700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 0x7f09cc03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:23.512 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.503+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09d804c370 con 0x7f09e40ff780 2024-04-12T05:47:23.513 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.503+0000 7f09e1d9b700 1 -- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 msgr2=0x7f09cc03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3827652772 2024-04-12T05:47:23.513 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.503+0000 7f09e1d9b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 0x7f09cc03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:47:23.513 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.508+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f09d802acf0 con 0x7f09e40ff780 2024-04-12T05:47:23.700 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.704+0000 7f09e1d9b700 1 -- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 msgr2=0x7f09cc03a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.8:6800/3827652772 2024-04-12T05:47:23.701 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.704+0000 7f09e1d9b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 0x7f09cc03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-12T05:47:23.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.775+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f09e418ec50 con 0x7f09e40ff780 2024-04-12T05:47:23.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.776+0000 7f09d37fe700 1 -- 172.21.15.194:0/3553935813 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f09d8027020 con 0x7f09e40ff780 2024-04-12T05:47:23.773 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:23.773 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:23.774 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 msgr2=0x7f09cc03a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 0x7f09cc03a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 msgr2=0x7f09e40fccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09d8006010 tx=0x7f09d80057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 shutdown_connections 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:6800/3827652772,v1:172.21.15.8:6801/3827652772] conn(0x7f09cc0383c0 0x7f09cc03a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 --2- 172.21.15.194:0/3553935813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09e40ff780 0x7f09e40fccc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 >> 172.21.15.194:0/3553935813 conn(0x7f09e40fad00 msgr2=0x7f09e4069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 shutdown_connections 2024-04-12T05:47:23.775 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:23.778+0000 7f09e8c4b700 1 -- 172.21.15.194:0/3553935813 wait complete. 2024-04-12T05:47:23.776 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:23.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:23 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/3553935813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:25.402 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:25.402 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:25.656 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:25.720 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 -- 172.21.15.194:0/3669315021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 msgr2=0x7f7728103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 --2- 172.21.15.194:0/3669315021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f7728103700 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f771c009910 tx=0x7f771c009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 -- 172.21.15.194:0/3669315021 shutdown_connections 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 --2- 172.21.15.194:0/3669315021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f7728103700 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 -- 172.21.15.194:0/3669315021 >> 172.21.15.194:0/3669315021 conn(0x7f77280fad20 msgr2=0x7f77280fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 -- 172.21.15.194:0/3669315021 shutdown_connections 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.302+0000 7f772ebfe700 1 -- 172.21.15.194:0/3669315021 wait complete. 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772ebfe700 1 Processor -- start 2024-04-12T05:47:27.337 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772ebfe700 1 -- start start 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772ebfe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772ebfe700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728195df0 con 0x7f7728101310 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772c99a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772c99a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:33552/0 (socket says 172.21.15.194:33552) 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772c99a700 1 -- 172.21.15.194:0/2981873089 learned_addr learned my addr 172.21.15.194:0/2981873089 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.303+0000 7f772c99a700 1 -- 172.21.15.194:0/2981873089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0095f0 con 0x7f7728101310 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f772c99a700 1 --2- 172.21.15.194:0/2981873089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f771c004f40 tx=0x7f771c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f771c01d070 con 0x7f7728101310 2024-04-12T05:47:27.338 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f771c00bdf0 con 0x7f7728101310 2024-04-12T05:47:27.341 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728195ff0 con 0x7f7728101310 2024-04-12T05:47:27.341 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f771c022e00 con 0x7f7728101310 2024-04-12T05:47:27.341 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.304+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728196490 con 0x7f7728101310 2024-04-12T05:47:27.341 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.305+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 15) v1 ==== 45077+0+0 (secure 0 0 0) 0x7f771c0226c0 con 0x7f7728101310 2024-04-12T05:47:27.341 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.305+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772818f360 con 0x7f7728101310 2024-04-12T05:47:27.342 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.305+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f771c04c190 con 0x7f7728101310 2024-04-12T05:47:27.342 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.311+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f771c02ba00 con 0x7f7728101310 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: Active manager daemon smithi008.ixmnlp restarted 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: Activating manager daemon smithi008.ixmnlp 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: mgrmap e15: smithi008.ixmnlp(active, starting, since 0.00305357s) 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:47:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:47:27.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:47:27.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:47:27.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:47:27.582 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.584+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f772802cc70 con 0x7f7728101310 2024-04-12T05:47:27.582 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.585+0000 7f7725ffb700 1 -- 172.21.15.194:0/2981873089 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f771c027020 con 0x7f7728101310 2024-04-12T05:47:27.582 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:27.583 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:27.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 msgr2=0x7f77281958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:27.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 --2- 172.21.15.194:0/2981873089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f771c004f40 tx=0x7f771c0057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:27.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 shutdown_connections 2024-04-12T05:47:27.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 --2- 172.21.15.194:0/2981873089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7728101310 0x7f77281958b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:27.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 >> 172.21.15.194:0/2981873089 conn(0x7f77280fad20 msgr2=0x7f77280fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:27.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 shutdown_connections 2024-04-12T05:47:27.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:27.587+0000 7f772ebfe700 1 -- 172.21.15.194:0/2981873089 wait complete. 2024-04-12T05:47:27.585 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/2981873089' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:28.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:28 smithi008 ceph-mon[39722]: mgrmap e16: smithi008.ixmnlp(active, since 1.00567s) 2024-04-12T05:47:29.364 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:29.364 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: [12/Apr/2024:05:47:27] ENGINE Bus STARTING 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: [12/Apr/2024:05:47:28] ENGINE Serving on https://172.21.15.8:7150 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: [12/Apr/2024:05:47:28] ENGINE Serving on http://172.21.15.8:8765 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: [12/Apr/2024:05:47:28] ENGINE Bus STARTED 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:29 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:30.189 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:30.727 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-12T05:47:31.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:31 smithi008 ceph-mon[39722]: mgrmap e17: smithi008.ixmnlp(active, since 2s) 2024-04-12T05:47:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:33 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:33 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:33 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:33 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:33.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:33 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi008", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:47:33.408 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 -- 172.21.15.194:0/2498261846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 msgr2=0x7ff74c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 --2- 172.21.15.194:0/2498261846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c0ff9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009910 tx=0x7ff73c009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 -- 172.21.15.194:0/2498261846 shutdown_connections 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 --2- 172.21.15.194:0/2498261846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c0ff9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 -- 172.21.15.194:0/2498261846 >> 172.21.15.194:0/2498261846 conn(0x7ff74c0fab20 msgr2=0x7ff74c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 -- 172.21.15.194:0/2498261846 shutdown_connections 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.410+0000 7ff75239d700 1 -- 172.21.15.194:0/2498261846 wait complete. 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75239d700 1 Processor -- start 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75239d700 1 -- start start 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75239d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75239d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c194b60 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75139b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:33.409 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75139b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:33574/0 (socket says 172.21.15.194:33574) 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75139b700 1 -- 172.21.15.194:0/3881732433 learned_addr learned my addr 172.21.15.194:0/3881732433 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.411+0000 7ff75139b700 1 -- 172.21.15.194:0/3881732433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0095f0 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff75139b700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006010 tx=0x7ff73c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff73c01d070 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c194d60 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff73c00bdf0 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff73c022e50 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.410 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.412+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c195200 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.413+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c18e080 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.413+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 45394+0+0 (secure 0 0 0) 0x7ff73c022700 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.413+0000 7ff7427fc700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 0x7ff73803a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.413+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ff73c04c570 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.413+0000 7ff750b9a700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 0x7ff73803a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:33.412 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.414+0000 7ff750b9a700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 0x7ff73803a920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:33.415 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.418+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff73c00b960 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.679 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.683+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff74c02cc70 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.680 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.683+0000 7ff7427fc700 1 -- 172.21.15.194:0/3881732433 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7ff73c027030 con 0x7ff74c0ff5a0 2024-04-12T05:47:33.680 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:33.681 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 msgr2=0x7ff73803a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 0x7ff73803a920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff748006fd0 tx=0x7ff748006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 msgr2=0x7ff74c194620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff73c006010 tx=0x7ff73c0057b0 comp rx=0 tx=0).stop 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 shutdown_connections 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff738038460 0x7ff73803a920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 --2- 172.21.15.194:0/3881732433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff74c0ff5a0 0x7ff74c194620 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 >> 172.21.15.194:0/3881732433 conn(0x7ff74c0fab20 msgr2=0x7ff74c0fb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:33.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 shutdown_connections 2024-04-12T05:47:33.683 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:33.685+0000 7ff75239d700 1 -- 172.21.15.194:0/3881732433 wait complete. 2024-04-12T05:47:33.683 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/3881732433' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:47:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:35.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:35.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:35.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:35.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:47:35.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-12T05:47:35.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:47:35.823 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:35.824 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:36 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:47:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:36 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:47:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:36 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:47:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:36 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:47:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:36 smithi008 ceph-mon[39722]: Deploying daemon ceph-exporter.smithi194 on smithi194 2024-04-12T05:47:36.681 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:38.713 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 -- 172.21.15.194:0/2569839327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 msgr2=0x7f332c0ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:38.713 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 --2- 172.21.15.194:0/2569839327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c0ff360 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3314009910 tx=0x7f3314009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:38.713 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 -- 172.21.15.194:0/2569839327 shutdown_connections 2024-04-12T05:47:38.713 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 --2- 172.21.15.194:0/2569839327 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c0ff360 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:38.713 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 -- 172.21.15.194:0/2569839327 >> 172.21.15.194:0/2569839327 conn(0x7f332c075df0 msgr2=0x7f332c076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 -- 172.21.15.194:0/2569839327 shutdown_connections 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.698+0000 7f3331000700 1 -- 172.21.15.194:0/2569839327 wait complete. 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f3331000700 1 Processor -- start 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f3331000700 1 -- start start 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f3331000700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f3331000700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c19bf60 con 0x7f332c0fef80 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f332a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f332a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:45960/0 (socket says 172.21.15.194:45960) 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f332a59c700 1 -- 172.21.15.194:0/3716165618 learned_addr learned my addr 172.21.15.194:0/3716165618 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.699+0000 7f332a59c700 1 -- 172.21.15.194:0/3716165618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140095f0 con 0x7f332c0fef80 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f332a59c700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3314004f40 tx=0x7f3314005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f331401d070 con 0x7f332c0fef80 2024-04-12T05:47:38.714 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c19c160 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3314005dc0 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c19c580 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.700+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3314022d30 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.701+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 45394+0+0 (secure 0 0 0) 0x7f331400b960 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.701+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c0623c0 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.701+0000 7f33237fe700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 0x7f331803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.701+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f331404c4c0 con 0x7f332c0fef80 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.701+0000 7f3329d9b700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 0x7f331803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.702+0000 7f3329d9b700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 0x7f331803a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f331c006fd0 tx=0x7f331c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:38.715 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.706+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f331400f460 con 0x7f332c0fef80 2024-04-12T05:47:38.973 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.976+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f332c196fb0 con 0x7f332c0fef80 2024-04-12T05:47:38.974 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.977+0000 7f33237fe700 1 -- 172.21.15.194:0/3716165618 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f3314027030 con 0x7f332c0fef80 2024-04-12T05:47:38.974 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:38.974 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 msgr2=0x7f331803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 0x7f331803a970 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f331c006fd0 tx=0x7f331c006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 msgr2=0x7f332c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3314004f40 tx=0x7f3314005020 comp rx=0 tx=0).stop 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 shutdown_connections 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f33180384b0 0x7f331803a970 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 --2- 172.21.15.194:0/3716165618 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c0fef80 0x7f332c19ba20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 >> 172.21.15.194:0/3716165618 conn(0x7f332c075df0 msgr2=0x7f332c104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.979+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 shutdown_connections 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:38.980+0000 7f3331000700 1 -- 172.21.15.194:0/3716165618 wait complete. 2024-04-12T05:47:38.979 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:39.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:39 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/3716165618' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:40.893 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:40.893 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:47:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:40 smithi008 ceph-mon[39722]: Deploying daemon crash.smithi194 on smithi194 2024-04-12T05:47:41.448 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:43.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:43.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:47:43.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:43.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:43.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:43.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:43.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 -- 172.21.15.194:0/4112954885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 msgr2=0x7f094c105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:43.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 --2- 172.21.15.194:0/4112954885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c105b60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0948009910 tx=0x7f0948009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:43.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 -- 172.21.15.194:0/4112954885 shutdown_connections 2024-04-12T05:47:43.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 --2- 172.21.15.194:0/4112954885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c105b60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:43.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 -- 172.21.15.194:0/4112954885 >> 172.21.15.194:0/4112954885 conn(0x7f094c075290 msgr2=0x7f094c0756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:43.683 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 -- 172.21.15.194:0/4112954885 shutdown_connections 2024-04-12T05:47:43.683 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.686+0000 7f09542c3700 1 -- 172.21.15.194:0/4112954885 wait complete. 2024-04-12T05:47:43.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.687+0000 7f09542c3700 1 Processor -- start 2024-04-12T05:47:43.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.687+0000 7f09542c3700 1 -- start start 2024-04-12T05:47:43.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.687+0000 7f09542c3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:43.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.687+0000 7f09542c3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c196ff0 con 0x7f094c105780 2024-04-12T05:47:43.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f095205f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f095205f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:46010/0 (socket says 172.21.15.194:46010) 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f095205f700 1 -- 172.21.15.194:0/19796062 learned_addr learned my addr 172.21.15.194:0/19796062 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f095205f700 1 -- 172.21.15.194:0/19796062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480095f0 con 0x7f094c105780 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f095205f700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0948004f40 tx=0x7f0948005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f094801d070 con 0x7f094c105780 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c197190 con 0x7f094c105780 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0948005dc0 con 0x7f094c105780 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0948022d70 con 0x7f094c105780 2024-04-12T05:47:43.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.688+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c1975b0 con 0x7f094c105780 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.689+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 45394+0+0 (secure 0 0 0) 0x7f094800b960 con 0x7f094c105780 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.689+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c1039c0 con 0x7f094c105780 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.689+0000 7f09437fe700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 0x7f093803a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.689+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f094804c640 con 0x7f094c105780 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.690+0000 7f095185e700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 0x7f093803a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:43.686 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.690+0000 7f095185e700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 0x7f093803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f093c006fd0 tx=0x7f093c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:43.690 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:43.695+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f094800f460 con 0x7f094c105780 2024-04-12T05:47:44.000 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.004+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f094c0623c0 con 0x7f094c105780 2024-04-12T05:47:44.001 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.005+0000 7f09437fe700 1 -- 172.21.15.194:0/19796062 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f0948027030 con 0x7f094c105780 2024-04-12T05:47:44.001 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:44.001 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:44.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 msgr2=0x7f093803a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:44.005 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 0x7f093803a970 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f093c006fd0 tx=0x7f093c006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 msgr2=0x7f094c196ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0948004f40 tx=0x7f0948005020 comp rx=0 tx=0).stop 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 shutdown_connections 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f09380384b0 0x7f093803a970 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 --2- 172.21.15.194:0/19796062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f094c105780 0x7f094c196ab0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:44.006 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.009+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 >> 172.21.15.194:0/19796062 conn(0x7f094c075290 msgr2=0x7f094c0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:44.008 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.012+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 shutdown_connections 2024-04-12T05:47:44.008 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:44.012+0000 7f09542c3700 1 -- 172.21.15.194:0/19796062 wait complete. 2024-04-12T05:47:44.012 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:44 smithi008 ceph-mon[39722]: Deploying daemon node-exporter.smithi194 on smithi194 2024-04-12T05:47:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:44 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/19796062' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:45.743 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:45.744 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:45.976 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:48 smithi008 ceph-mon[39722]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 -- 172.21.15.194:0/2024939115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 msgr2=0x7f46e8105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 --2- 172.21.15.194:0/2024939115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e8105b60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009910 tx=0x7f46dc009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 -- 172.21.15.194:0/2024939115 shutdown_connections 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 --2- 172.21.15.194:0/2024939115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e8105b60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 -- 172.21.15.194:0/2024939115 >> 172.21.15.194:0/2024939115 conn(0x7f46e8075290 msgr2=0x7f46e80756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 -- 172.21.15.194:0/2024939115 shutdown_connections 2024-04-12T05:47:48.845 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.712+0000 7f46f05dd700 1 -- 172.21.15.194:0/2024939115 wait complete. 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46f05dd700 1 Processor -- start 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46f05dd700 1 -- start start 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46f05dd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46f05dd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e80fe760 con 0x7f46e8105780 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46ee379700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46ee379700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:58094/0 (socket says 172.21.15.194:58094) 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46ee379700 1 -- 172.21.15.194:0/1940659069 learned_addr learned my addr 172.21.15.194:0/1940659069 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.713+0000 7f46ee379700 1 -- 172.21.15.194:0/1940659069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc0095f0 con 0x7f46e8105780 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46ee379700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46dc004f40 tx=0x7f46dc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46dc01d070 con 0x7f46e8105780 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e80fcd20 con 0x7f46e8105780 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f46dc009e60 con 0x7f46e8105780 2024-04-12T05:47:48.846 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46dc022ce0 con 0x7f46e8105780 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.714+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e80fd1a0 con 0x7f46e8105780 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.715+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e80623c0 con 0x7f46e8105780 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.715+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 45394+0+0 (secure 0 0 0) 0x7f46dc022470 con 0x7f46e8105780 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.715+0000 7f46db7fe700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 0x7f46d403a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.715+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f46dc04c5b0 con 0x7f46e8105780 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.715+0000 7f46edb78700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 0x7f46d403a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.716+0000 7f46edb78700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 0x7f46d403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006fd0 tx=0x7f46e4006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:48.847 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:48.720+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f46dc02b590 con 0x7f46e8105780 2024-04-12T05:47:49.012 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.016+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f46e80fd530 con 0x7f46e8105780 2024-04-12T05:47:49.013 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.017+0000 7f46db7fe700 1 -- 172.21.15.194:0/1940659069 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f46dc027030 con 0x7f46e8105780 2024-04-12T05:47:49.013 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:49.013 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 msgr2=0x7f46d403a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 0x7f46d403a970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006fd0 tx=0x7f46e4006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 msgr2=0x7f46e80fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46dc004f40 tx=0x7f46dc005020 comp rx=0 tx=0).stop 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 shutdown_connections 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f46d40384b0 0x7f46d403a970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 --2- 172.21.15.194:0/1940659069 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e8105780 0x7f46e80fc7e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:49.015 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 >> 172.21.15.194:0/1940659069 conn(0x7f46e8075290 msgr2=0x7f46e8067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 shutdown_connections 2024-04-12T05:47:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:49.019+0000 7f46f05dd700 1 -- 172.21.15.194:0/1940659069 wait complete. 2024-04-12T05:47:49.016 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:50.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: Deploying daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1940659069' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:50.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:49 smithi008 ceph-mon[39722]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:50.774 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:50.774 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:51.034 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:51.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:51 smithi008 ceph-mon[39722]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:53.387 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 -- 172.21.15.194:0/2850072827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 msgr2=0x7f3e90100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:53.387 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/2850072827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90100470 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009910 tx=0x7f3e80009c20 comp rx=0 tx=0).stop 2024-04-12T05:47:53.387 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 -- 172.21.15.194:0/2850072827 shutdown_connections 2024-04-12T05:47:53.388 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/2850072827 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90100470 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:53.388 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 -- 172.21.15.194:0/2850072827 >> 172.21.15.194:0/2850072827 conn(0x7f3e90075df0 msgr2=0x7f3e90076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:53.388 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 -- 172.21.15.194:0/2850072827 shutdown_connections 2024-04-12T05:47:53.388 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.391+0000 7f3e94ff0700 1 -- 172.21.15.194:0/2850072827 wait complete. 2024-04-12T05:47:53.389 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.392+0000 7f3e94ff0700 1 Processor -- start 2024-04-12T05:47:53.389 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.392+0000 7f3e94ff0700 1 -- start start 2024-04-12T05:47:53.389 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.392+0000 7f3e94ff0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:53.389 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.392+0000 7f3e94ff0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90196f90 con 0x7f3e90100090 2024-04-12T05:47:53.389 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e8e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e8e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.194:58122/0 (socket says 172.21.15.194:58122) 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e8e59c700 1 -- 172.21.15.194:0/1363564160 learned_addr learned my addr 172.21.15.194:0/1363564160 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e8e59c700 1 -- 172.21.15.194:0/1363564160 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800095f0 con 0x7f3e90100090 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e8e59c700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e80004f40 tx=0x7f3e80005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e8001d070 con 0x7f3e90100090 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3e80009e60 con 0x7f3e90100090 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3e80022ce0 con 0x7f3e90100090 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.393+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e90197190 con 0x7f3e90100090 2024-04-12T05:47:53.390 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.394+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e90197630 con 0x7f3e90100090 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.394+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 17) v1 ==== 45394+0+0 (secure 0 0 0) 0x7f3e8000f460 con 0x7f3e90100090 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.395+0000 7f3e7f7fe700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 0x7f3e7803edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.395+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90103a10 con 0x7f3e90100090 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.395+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3e8004c550 con 0x7f3e90100090 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.395+0000 7f3e8dd9b700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 0x7f3e7803edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:47:53.391 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.395+0000 7f3e8dd9b700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 0x7f3e7803edd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006fd0 tx=0x7f3e84006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:47:53.395 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.400+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3e80020bf0 con 0x7f3e90100090 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:47:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:47:53.681 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.685+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3e900ff340 con 0x7f3e90100090 2024-04-12T05:47:53.682 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.686+0000 7f3e7f7fe700 1 -- 172.21.15.194:0/1363564160 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f3e80027030 con 0x7f3e90100090 2024-04-12T05:47:53.682 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:47:53.682 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:44:57.056630Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 msgr2=0x7f3e7803edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 0x7f3e7803edd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3e84006fd0 tx=0x7f3e84006e40 comp rx=0 tx=0).stop 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 msgr2=0x7f3e90196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3e80004f40 tx=0x7f3e80005020 comp rx=0 tx=0).stop 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 shutdown_connections 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3e7803c910 0x7f3e7803edd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 --2- 172.21.15.194:0/1363564160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3e90100090 0x7f3e90196a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 >> 172.21.15.194:0/1363564160 conn(0x7f3e90075df0 msgr2=0x7f3e900fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 shutdown_connections 2024-04-12T05:47:53.685 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:47:53.689+0000 7f3e94ff0700 1 -- 172.21.15.194:0/1363564160 wait complete. 2024-04-12T05:47:53.686 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-04-12T05:47:54.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:54 smithi008 ceph-mon[39722]: Deploying daemon mon.smithi194 on smithi194 2024-04-12T05:47:54.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:54 smithi008 ceph-mon[39722]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:54.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:54 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1363564160' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:47:56.137 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-12T05:47:56.137 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mon dump -f json 2024-04-12T05:47:56.368 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:47:56.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:56 smithi008 ceph-mon[39722]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:47:57.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:47:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:47:58.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:47:58 smithi194 bash[49702]: 423eb70e605fc48ce3c8a1bdb38f8cac4364849f14cdaf31b6739e6ae7f029f3 2024-04-12T05:47:58.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:47:58 smithi194 systemd[1]: Started Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:48:03.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:48:03.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: mon.smithi008 calling monitor election 2024-04-12T05:48:03.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:48:03.377 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: mon.smithi194 calling monitor election 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: fsmap 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T05:48:03.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: mgrmap e17: smithi008.ixmnlp(active, since 36s) 2024-04-12T05:48:03.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:48:03.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:48:03.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: overall HEALTH_OK 2024-04-12T05:48:03.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:48:03.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:03.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:48:03.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: mon.smithi008 calling monitor election 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: mon.smithi194 calling monitor election 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:03.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: fsmap 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: osdmap e5: 0 total, 0 up, 0 in 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: mgrmap e17: smithi008.ixmnlp(active, since 36s) 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: overall HEALTH_OK 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/3654432252' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:48:03.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 -- 172.21.15.194:0/226854833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 msgr2=0x7fc810005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/226854833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc810005a50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc8240766d0 tx=0x7fc82000fe60 comp rx=0 tx=0).stop 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 -- 172.21.15.194:0/226854833 shutdown_connections 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/226854833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc810005a50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 -- 172.21.15.194:0/226854833 >> 172.21.15.194:0/226854833 conn(0x7fc824075290 msgr2=0x7fc8240756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 -- 172.21.15.194:0/226854833 shutdown_connections 2024-04-12T05:48:03.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.908+0000 7fc82bb0d700 1 -- 172.21.15.194:0/226854833 wait complete. 2024-04-12T05:48:03.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 Processor -- start 2024-04-12T05:48:03.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 -- start start 2024-04-12T05:48:03.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc824197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:03.906 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824198b20 con 0x7fc8100035c0 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc82bb0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824198c90 con 0x7fc824198500 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:51802/0 (socket says 172.21.15.194:51802) 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 -- 172.21.15.194:0/2963565147 learned_addr learned my addr 172.21.15.194:0/2963565147 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 -- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 msgr2=0x7fc82419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 -- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 msgr2=0x7fc82419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8298a9700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc824197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.909+0000 7fc8290a8700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc8298a9700 1 -- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 msgr2=0x7fc82419c970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc8298a9700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc8298a9700 1 -- 172.21.15.194:0/2963565147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82000f8b0 con 0x7fc8100035c0 2024-04-12T05:48:03.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc8298a9700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc824197fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8240766d0 tx=0x7fc820014c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc820012b60 con 0x7fc8100035c0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82419cf70 con 0x7fc8100035c0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc820012cc0 con 0x7fc8100035c0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc820014660 con 0x7fc8100035c0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.910+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82419d4c0 con 0x7fc8100035c0 2024-04-12T05:48:03.908 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.911+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240623c0 con 0x7fc8100035c0 2024-04-12T05:48:03.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.912+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7fc820024050 con 0x7fc8100035c0 2024-04-12T05:48:03.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.912+0000 7fc81affd700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 0x7fc80806ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:03.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.912+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc82008d5e0 con 0x7fc8100035c0 2024-04-12T05:48:03.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.912+0000 7fc8290a8700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 0x7fc80806ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:03.910 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.913+0000 7fc8290a8700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 0x7fc80806ea80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc814009180 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:03.912 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:03.916+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc820058e80 con 0x7fc8100035c0 2024-04-12T05:48:04.180 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.184+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc824199320 con 0x7fc8100035c0 2024-04-12T05:48:04.181 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.185+0000 7fc81affd700 1 -- 172.21.15.194:0/2963565147 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1018 (secure 0 0 0) 0x7fc82002b090 con 0x7fc8100035c0 2024-04-12T05:48:04.181 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-04-12T05:48:04.181 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","modified":"2024-04-12T05:47:58.282219Z","created":"2024-04-12T05:44:57.056630Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi008","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:3300","nonce":0},{"type":"v1","addr":"172.21.15.8:6789","nonce":0}]},"addr":"172.21.15.8:6789/0","public_addr":"172.21.15.8:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi194","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 msgr2=0x7fc80806ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 0x7fc80806ea80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc814009180 tx=0x7fc814008040 comp rx=0 tx=0).stop 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 msgr2=0x7fc824197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc824197fc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8240766d0 tx=0x7fc820014c50 comp rx=0 tx=0).stop 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 shutdown_connections 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc80806c5c0 0x7fc80806ea80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc8100035c0 0x7fc824197fc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:04.183 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 --2- 172.21.15.194:0/2963565147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc824198500 0x7fc82419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:04.184 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 >> 172.21.15.194:0/2963565147 conn(0x7fc824075290 msgr2=0x7fc824107d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:04.184 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.187+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 shutdown_connections 2024-04-12T05:48:04.184 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:48:04.188+0000 7fc82bb0d700 1 -- 172.21.15.194:0/2963565147 wait complete. 2024-04-12T05:48:04.184 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-04-12T05:48:04.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: mgrmap e18: smithi008.ixmnlp(active, since 36s), standbys: smithi194.xbiyaj 2024-04-12T05:48:04.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:48:04.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:48:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/2963565147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:48:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: mgrmap e18: smithi008.ixmnlp(active, since 36s), standbys: smithi194.xbiyaj 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:48:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/2963565147' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-12T05:48:04.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:48:05.325 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-12T05:48:05.325 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph config generate-minimal-conf 2024-04-12T05:48:05.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:05 smithi194 ceph-mon[49771]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:05.573 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:48:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:05 smithi008 ceph-mon[39722]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:07.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 -- 172.21.15.8:0/399577155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c000fef80 msgr2=0x7f6c000ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 --2- 172.21.15.8:0/399577155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c000fef80 0x7f6c000ff360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6be8009910 tx=0x7f6be8009c20 comp rx=0 tx=0).stop 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 -- 172.21.15.8:0/399577155 shutdown_connections 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 --2- 172.21.15.8:0/399577155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c000fef80 0x7f6c000ff360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 -- 172.21.15.8:0/399577155 >> 172.21.15.8:0/399577155 conn(0x7f6c00075df0 msgr2=0x7f6c00076200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 -- 172.21.15.8:0/399577155 shutdown_connections 2024-04-12T05:48:07.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.444+0000 7f6c04f57700 1 -- 172.21.15.8:0/399577155 wait complete. 2024-04-12T05:48:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 Processor -- start 2024-04-12T05:48:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 -- start start 2024-04-12T05:48:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c000fef80 0x7f6c000fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c001a1f50 con 0x7f6c00100440 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6c04f57700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c001a2090 con 0x7f6c000fef80 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6bfdd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6bfdd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51526/0 (socket says 172.21.15.8:51526) 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.445+0000 7f6bfdd9b700 1 -- 172.21.15.8:0/788288037 learned_addr learned my addr 172.21.15.8:0/788288037 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bfe59c700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c000fef80 0x7f6c000fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bfdd9b700 1 -- 172.21.15.8:0/788288037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c000fef80 msgr2=0x7f6c000fff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bfdd9b700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c000fef80 0x7f6c000fff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bfdd9b700 1 -- 172.21.15.8:0/788288037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be80095f0 con 0x7f6c00100440 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bfdd9b700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000b730 tx=0x7f6bf000baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf0004200 con 0x7f6c00100440 2024-04-12T05:48:07.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f6bf0004360 con 0x7f6c00100440 2024-04-12T05:48:07.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf0011b90 con 0x7f6c00100440 2024-04-12T05:48:07.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c001a21d0 con 0x7f6c00100440 2024-04-12T05:48:07.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.446+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c001a2720 con 0x7f6c00100440 2024-04-12T05:48:07.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.447+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c000623c0 con 0x7f6c00100440 2024-04-12T05:48:07.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.447+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7f6bf00036a0 con 0x7f6c00100440 2024-04-12T05:48:07.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.448+0000 7f6bf77fe700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 0x7f6bec06ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:07.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.448+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6bf008a200 con 0x7f6c00100440 2024-04-12T05:48:07.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.448+0000 7f6bfe59c700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 0x7f6bec06ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:07.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.448+0000 7f6bfe59c700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 0x7f6bec06ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6be800b5c0 tx=0x7f6be8005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:07.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.452+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6bf0055b40 con 0x7f6c00100440 2024-04-12T05:48:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:48:07.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:48:07.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:07.641 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.701 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:07.703 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:48:07.703 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:48:07.703 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:07.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.659+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f6c001a2b90 con 0x7f6c00100440 2024-04-12T05:48:07.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.659+0000 7f6bf77fe700 1 -- 172.21.15.8:0/788288037 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+223 (secure 0 0 0) 0x7f6bf00899c0 con 0x7f6c00100440 2024-04-12T05:48:07.704 INFO:teuthology.orchestra.run.smithi008.stdout:# minimal ceph.conf for b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:48:07.705 INFO:teuthology.orchestra.run.smithi008.stdout:[global] 2024-04-12T05:48:07.705 INFO:teuthology.orchestra.run.smithi008.stdout: fsid = b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:48:07.705 INFO:teuthology.orchestra.run.smithi008.stdout: mon_host = [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 msgr2=0x7f6bec06ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 0x7f6bec06ea80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6be800b5c0 tx=0x7f6be8005a90 comp rx=0 tx=0).stop 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 msgr2=0x7f6c001008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000b730 tx=0x7f6bf000baf0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 shutdown_connections 2024-04-12T05:48:07.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6c000fef80 0x7f6c000fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f6bec06c5c0 0x7f6bec06ea80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 --2- 172.21.15.8:0/788288037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6c00100440 0x7f6c001008c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:07.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 >> 172.21.15.8:0/788288037 conn(0x7f6c00075df0 msgr2=0x7f6c001065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:07.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 shutdown_connections 2024-04-12T05:48:07.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:07.661+0000 7f6c04f57700 1 -- 172.21.15.8:0/788288037 wait complete. 2024-04-12T05:48:08.371 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-12T05:48:08.372 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:48:08.372 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.conf 2024-04-12T05:48:08.404 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:48:08.404 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:48:08.472 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:48:08.472 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-04-12T05:48:08.509 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:48:08.509 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:48:08.581 INFO:tasks.cephadm:Deploying OSDs... 2024-04-12T05:48:08.581 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:48:08.581 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T05:48:08.603 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T05:48:08.603 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_1 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: Reconfiguring mon.smithi008 (unknown last config time)... 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: Reconfiguring daemon mon.smithi008 on smithi008 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:08 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/788288037' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:08.664 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T05:48:08.664 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:08.664 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 163768 Links: 1 2024-04-12T05:48:08.664 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:08.665 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:08.665 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-04-12 05:47:18.519370771 +0000 2024-04-12T05:48:08.665 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-04-12 05:38:42.821302307 +0000 2024-04-12T05:48:08.665 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-04-12 05:38:42.821302307 +0000 2024-04-12T05:48:08.665 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2024-04-12T05:48:08.665 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T05:48:08.734 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-04-12T05:48:08.734 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-04-12T05:48:08.734 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 8.501e-05 s, 6.0 MB/s 2024-04-12T05:48:08.735 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T05:48:08.799 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_2 2024-04-12T05:48:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: Reconfiguring mon.smithi008 (unknown last config time)... 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: Reconfiguring daemon mon.smithi008 on smithi008 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:08 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/788288037' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:08.863 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T05:48:08.863 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:08.863 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 165383 Links: 1 2024-04-12T05:48:08.863 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:08.863 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:08.864 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-04-12 05:47:18.519370771 +0000 2024-04-12T05:48:08.864 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-04-12 05:38:43.169298843 +0000 2024-04-12T05:48:08.864 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-04-12 05:38:43.169298843 +0000 2024-04-12T05:48:08.864 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2024-04-12T05:48:08.864 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T05:48:08.934 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-04-12T05:48:08.934 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-04-12T05:48:08.935 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 9.6578e-05 s, 5.3 MB/s 2024-04-12T05:48:08.936 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T05:48:08.999 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_3 2024-04-12T05:48:09.063 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T05:48:09.063 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:09.064 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 169076 Links: 1 2024-04-12T05:48:09.064 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:09.065 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:09.065 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-04-12 05:47:18.519370771 +0000 2024-04-12T05:48:09.065 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-04-12 05:38:43.514295408 +0000 2024-04-12T05:48:09.065 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-04-12 05:38:43.514295408 +0000 2024-04-12T05:48:09.065 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2024-04-12T05:48:09.065 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T05:48:09.132 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-04-12T05:48:09.133 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-04-12T05:48:09.133 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 9.5625e-05 s, 5.4 MB/s 2024-04-12T05:48:09.134 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T05:48:09.194 DEBUG:teuthology.orchestra.run.smithi008:> stat /dev/vg_nvme/lv_4 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Device: 6h/6d Inode: 165452 Links: 1 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 2024-04-12 05:47:18.519370771 +0000 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-04-12 05:38:43.869291874 +0000 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-04-12 05:38:43.869291874 +0000 2024-04-12T05:48:09.252 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2024-04-12T05:48:09.253 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T05:48:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records in 2024-04-12T05:48:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:1+0 records out 2024-04-12T05:48:09.320 INFO:teuthology.orchestra.run.smithi008.stderr:512 bytes copied, 0.000105568 s, 4.8 MB/s 2024-04-12T05:48:09.321 DEBUG:teuthology.orchestra.run.smithi008:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T05:48:09.382 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:48:09.382 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-04-12T05:48:09.404 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-12T05:48:09.405 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-04-12T05:48:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-12T05:48:09.465 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:09.465 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 165153 Links: 1 2024-04-12T05:48:09.465 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:09.465 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:09.466 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-12 05:47:57.222219761 +0000 2024-04-12T05:48:09.466 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-12 05:38:34.586354601 +0000 2024-04-12T05:48:09.466 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-12 05:38:34.586354601 +0000 2024-04-12T05:48:09.466 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2024-04-12T05:48:09.466 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-12T05:48:09.534 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-12T05:48:09.535 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-12T05:48:09.535 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000271767 s, 1.9 MB/s 2024-04-12T05:48:09.536 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-12T05:48:09.597 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 166319 Links: 1 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-12 05:47:57.222219761 +0000 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-12 05:38:34.924347914 +0000 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-12 05:38:34.924347914 +0000 2024-04-12T05:48:09.658 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2024-04-12T05:48:09.659 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-12T05:48:09.726 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-12T05:48:09.726 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-12T05:48:09.726 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000261197 s, 2.0 MB/s 2024-04-12T05:48:09.727 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-12T05:48:09.786 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 172155 Links: 1 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-12 05:47:57.222219761 +0000 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-12 05:38:35.269341089 +0000 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-12 05:38:35.269341089 +0000 2024-04-12T05:48:09.845 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2024-04-12T05:48:09.846 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-12T05:48:09.912 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-12T05:48:09.912 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-12T05:48:09.912 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.00019148 s, 2.7 MB/s 2024-04-12T05:48:09.913 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-12T05:48:09.972 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-04-12T05:48:10.030 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-12T05:48:10.030 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 6h/6d Inode: 170178 Links: 1 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-04-12 05:47:57.222219761 +0000 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-12 05:38:35.598334580 +0000 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-12 05:38:35.598334580 +0000 2024-04-12T05:48:10.031 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2024-04-12T05:48:10.031 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-12T05:48:10.096 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-04-12T05:48:10.096 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-04-12T05:48:10.096 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000210818 s, 2.4 MB/s 2024-04-12T05:48:10.097 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-12T05:48:10.156 INFO:tasks.cephadm:Deploying osd.0 on smithi008 with /dev/vg_nvme/lv_4... 2024-04-12T05:48:10.156 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-12T05:48:10.465 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:48:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: Reconfiguring mgr.smithi008.ixmnlp (unknown last config time)... 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:10 smithi194 ceph-mon[49771]: Reconfiguring daemon mgr.smithi008.ixmnlp on smithi008 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: Reconfiguring mgr.smithi008.ixmnlp (unknown last config time)... 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:48:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:48:10.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:10.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:10 smithi008 ceph-mon[39722]: Reconfiguring daemon mgr.smithi008.ixmnlp on smithi008 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: Reconfiguring ceph-exporter.smithi008 (monmap changed)... 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: Reconfiguring daemon ceph-exporter.smithi008 on smithi008 2024-04-12T05:48:12.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:11 smithi194 ceph-mon[49771]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:12.351 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:12.351 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:12.351 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: Reconfiguring ceph-exporter.smithi008 (monmap changed)... 2024-04-12T05:48:12.351 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:48:12.351 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:12.352 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: Reconfiguring daemon ceph-exporter.smithi008 on smithi008 2024-04-12T05:48:12.352 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:11 smithi008 ceph-mon[39722]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:13.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:13.854 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-12T05:48:13.854 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-12T05:48:13.854 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-04-12T05:48:13.854 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-04-12T05:48:13.855 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00841762 s, 1.2 GB/s 2024-04-12T05:48:13.855 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-04-12T05:48:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: Reconfiguring crash.smithi008 (monmap changed)... 2024-04-12T05:48:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:48:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:14.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: Reconfiguring daemon crash.smithi008 on smithi008 2024-04-12T05:48:14.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:14 smithi008 ceph-mon[39722]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: Reconfiguring crash.smithi008 (monmap changed)... 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: Reconfiguring daemon crash.smithi008 on smithi008 2024-04-12T05:48:14.378 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:14 smithi194 ceph-mon[49771]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:15.438 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi008:vg_nvme/lv_4 2024-04-12T05:48:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:16 smithi008 ceph-mon[39722]: Reconfiguring alertmanager.smithi008 (dependencies changed)... 2024-04-12T05:48:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:16 smithi008 ceph-mon[39722]: Reconfiguring daemon alertmanager.smithi008 on smithi008 2024-04-12T05:48:16.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:16 smithi008 ceph-mon[39722]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:16.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:16.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:16.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:16 smithi194 ceph-mon[49771]: Reconfiguring alertmanager.smithi008 (dependencies changed)... 2024-04-12T05:48:16.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:16 smithi194 ceph-mon[49771]: Reconfiguring daemon alertmanager.smithi008 on smithi008 2024-04-12T05:48:16.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:16 smithi194 ceph-mon[49771]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:16.412 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:48:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:18 smithi194 ceph-mon[49771]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:18 smithi008 ceph-mon[39722]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 -- 172.21.15.8:0/4021237666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 msgr2=0x7f4e54102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 --2- 172.21.15.8:0/4021237666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54102310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009910 tx=0x7f4e48009c20 comp rx=0 tx=0).stop 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 -- 172.21.15.8:0/4021237666 shutdown_connections 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 --2- 172.21.15.8:0/4021237666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54102310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 --2- 172.21.15.8:0/4021237666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e541010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:19.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 -- 172.21.15.8:0/4021237666 >> 172.21.15.8:0/4021237666 conn(0x7f4e540fc230 msgr2=0x7f4e540fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 -- 172.21.15.8:0/4021237666 shutdown_connections 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.900+0000 7f4e5a262700 1 -- 172.21.15.8:0/4021237666 wait complete. 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 Processor -- start 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 -- start start 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e54196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e541970f0 con 0x7f4e54101eb0 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.901+0000 7f4e5a262700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e54197230 con 0x7f4e54100cb0 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48138/0 (socket says 172.21.15.8:48138) 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 -- 172.21.15.8:0/2568156023 learned_addr learned my addr 172.21.15.8:0/2568156023 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e537fe700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e54196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:19.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 -- 172.21.15.8:0/2568156023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 msgr2=0x7f4e54196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e54196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 -- 172.21.15.8:0/2568156023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e480095f0 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e537fe700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e54196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e52ffd700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e4800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e4801d070 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5419bc80 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4e4800f460 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e48021600 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.902+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5419c170 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.903+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e54109770 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.904+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7f4e4800f5d0 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.905+0000 7f4e50ff9700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 0x7f4e3c06e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.905+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4e4808be60 con 0x7f4e54101eb0 2024-04-12T05:48:19.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.905+0000 7f4e537fe700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 0x7f4e3c06e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:19.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.905+0000 7f4e537fe700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 0x7f4e3c06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e54101d10 tx=0x7f4e44008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:19.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:19.909+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4e4805b230 con 0x7f4e54101eb0 2024-04-12T05:48:20.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:20.123+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e54106800 con 0x7f4e3c06c470 2024-04-12T05:48:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:20 smithi008 ceph-mon[39722]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:48:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:48:20.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:20 smithi194 ceph-mon[49771]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:48:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:48:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:21.464 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:21 smithi008 ceph-mon[39722]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:48:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:21 smithi194 ceph-mon[49771]: from='client.14248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:48:22.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:22 smithi008 ceph-mon[39722]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:22.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:22 smithi194 ceph-mon[49771]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:23 smithi008 ceph-mon[39722]: Reconfiguring grafana.smithi008 (dependencies changed)... 2024-04-12T05:48:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:23 smithi008 ceph-mon[39722]: Reconfiguring daemon grafana.smithi008 on smithi008 2024-04-12T05:48:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:23 smithi008 ceph-mon[39722]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:23 smithi194 ceph-mon[49771]: Reconfiguring grafana.smithi008 (dependencies changed)... 2024-04-12T05:48:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:23 smithi194 ceph-mon[49771]: Reconfiguring daemon grafana.smithi008 on smithi008 2024-04-12T05:48:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:23 smithi194 ceph-mon[49771]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3321401704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa2f9673-7e1b-494b-89f2-f87fe4fbc40f"}]: dispatch 2024-04-12T05:48:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3321401704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa2f9673-7e1b-494b-89f2-f87fe4fbc40f"}]': finished 2024-04-12T05:48:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:24 smithi008 ceph-mon[39722]: osdmap e6: 1 total, 0 up, 1 in 2024-04-12T05:48:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:24 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:24 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3321401704' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa2f9673-7e1b-494b-89f2-f87fe4fbc40f"}]: dispatch 2024-04-12T05:48:24.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:24 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3321401704' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa2f9673-7e1b-494b-89f2-f87fe4fbc40f"}]': finished 2024-04-12T05:48:24.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:24 smithi194 ceph-mon[49771]: osdmap e6: 1 total, 0 up, 1 in 2024-04-12T05:48:24.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:24 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:25.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:25 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/928271409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:48:25.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:25 smithi008 ceph-mon[39722]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:25.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:25 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/928271409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:48:25.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:25 smithi194 ceph-mon[49771]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:27.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:28.504 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:28 smithi008 ceph-mon[39722]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:28 smithi194 ceph-mon[49771]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:30.554 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:30 smithi008 ceph-mon[39722]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:30 smithi194 ceph-mon[49771]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:31.537 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:31 smithi008 ceph-mon[39722]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:31 smithi194 ceph-mon[49771]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:34.596 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:34 smithi008 ceph-mon[39722]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:34.596 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:34.596 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:34.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:34 smithi194 ceph-mon[49771]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:34.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:34 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:34.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:34 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:35.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:35 smithi008 ceph-mon[39722]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:48:35.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:35 smithi008 ceph-mon[39722]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:48:35.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T05:48:35.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:35 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:35.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:35 smithi194 ceph-mon[49771]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:48:35.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:35 smithi194 ceph-mon[49771]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:48:35.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:35 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T05:48:35.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:35 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:36.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:36 smithi008 ceph-mon[39722]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:36.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:36 smithi008 ceph-mon[39722]: Deploying daemon osd.0 on smithi008 2024-04-12T05:48:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:36 smithi194 ceph-mon[49771]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:36 smithi194 ceph-mon[49771]: Deploying daemon osd.0 on smithi008 2024-04-12T05:48:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:38 smithi008 ceph-mon[39722]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:38.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:38 smithi194 ceph-mon[49771]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:40 smithi008 ceph-mon[39722]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:48:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:40.695 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:40 smithi194 ceph-mon[49771]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:40.695 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:40.695 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:40.695 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T05:48:40.695 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:41.535 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:41 smithi008 ceph-mon[39722]: Reconfiguring ceph-exporter.smithi194 (monmap changed)... 2024-04-12T05:48:41.536 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:41 smithi008 ceph-mon[39722]: Reconfiguring daemon ceph-exporter.smithi194 on smithi194 2024-04-12T05:48:41.813 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:41 smithi194 ceph-mon[49771]: Reconfiguring ceph-exporter.smithi194 (monmap changed)... 2024-04-12T05:48:41.813 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:41 smithi194 ceph-mon[49771]: Reconfiguring daemon ceph-exporter.smithi194 on smithi194 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:42.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:43.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:43 smithi008 ceph-mon[39722]: Reconfiguring crash.smithi194 (monmap changed)... 2024-04-12T05:48:43.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:43 smithi008 ceph-mon[39722]: Reconfiguring daemon crash.smithi194 on smithi194 2024-04-12T05:48:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:43 smithi194 ceph-mon[49771]: Reconfiguring crash.smithi194 (monmap changed)... 2024-04-12T05:48:43.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:43 smithi194 ceph-mon[49771]: Reconfiguring daemon crash.smithi194 on smithi194 2024-04-12T05:48:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:48:44.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:48:44.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:44.735 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:44.736 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:44.736 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:44.736 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:48:44.736 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:48:44.736 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:45.613 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: Reconfiguring mgr.smithi194.xbiyaj (monmap changed)... 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: Reconfiguring daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:48:45.614 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:45 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:45.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: Reconfiguring mgr.smithi194.xbiyaj (monmap changed)... 2024-04-12T05:48:45.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: Reconfiguring daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:48:45.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:45.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:45.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:48:45.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:48:45.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:45 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:46 smithi008 ceph-mon[39722]: Reconfiguring mon.smithi194 (monmap changed)... 2024-04-12T05:48:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:46 smithi008 ceph-mon[39722]: Reconfiguring daemon mon.smithi194 on smithi194 2024-04-12T05:48:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:46 smithi008 ceph-mon[39722]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:46 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:46 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:46.524 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:46 smithi194 ceph-mon[49771]: Reconfiguring mon.smithi194 (monmap changed)... 2024-04-12T05:48:46.524 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:46 smithi194 ceph-mon[49771]: Reconfiguring daemon mon.smithi194 on smithi194 2024-04-12T05:48:46.524 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:46 smithi194 ceph-mon[49771]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:46.524 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:46 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:46.524 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:46 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi008.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi008.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T05:48:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi008.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi008.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:48:47.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi008.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T05:48:47.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi008.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T05:48:47.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:48:47.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:47 smithi194 ceph-mon[49771]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:48:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:48:47.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi008.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi008.front.sepia.ceph.com:9093"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi008.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi008.front.sepia.ceph.com:3000"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi008.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi008.front.sepia.ceph.com:9095"}]: dispatch 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:48:47.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:47 smithi008 ceph-mon[39722]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:49.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.206+0000 7f4e50ff9700 1 -- 172.21.15.8:0/2568156023 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4e54106800 con 0x7f4e3c06c470 2024-04-12T05:48:49.203 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 0 on host 'smithi008' 2024-04-12T05:48:49.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 msgr2=0x7f4e3c06e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:49.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 0x7f4e3c06e930 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e54101d10 tx=0x7f4e44008040 comp rx=0 tx=0).stop 2024-04-12T05:48:49.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 msgr2=0x7f4e54196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:49.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e4800be20 comp rx=0 tx=0).stop 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 shutdown_connections 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e54100cb0 0x7f4e54196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4e3c06c470 0x7f4e3c06e930 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 --2- 172.21.15.8:0/2568156023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e54101eb0 0x7f4e54196ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 >> 172.21.15.8:0/2568156023 conn(0x7f4e540fc230 msgr2=0x7f4e541050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 shutdown_connections 2024-04-12T05:48:49.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:49.209+0000 7f4e5a262700 1 -- 172.21.15.8:0/2568156023 wait complete. 2024-04-12T05:48:49.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:49.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:49.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:49.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:49.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:49.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:49 smithi008 ceph-mon[39722]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T05:48:50.146 DEBUG:teuthology.orchestra.run.smithi008:osd.0> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.0.service 2024-04-12T05:48:50.149 INFO:tasks.cephadm:Deploying osd.1 on smithi008 with /dev/vg_nvme/lv_3... 2024-04-12T05:48:50.149 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-12T05:48:50.181 INFO:journalctl@ceph.osd.0.smithi008.stdout:-- Logs begin at Fri 2024-04-12 05:32:04 UTC. -- 2024-04-12T05:48:50.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:50.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:50.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:50.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:50.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:50.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:49 smithi194 ceph-mon[49771]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T05:48:50.650 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:48:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:51 smithi194 ceph-mon[49771]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T05:48:51.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:51 smithi194 ceph-mon[49771]: osdmap e7: 1 total, 0 up, 1 in 2024-04-12T05:48:51.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:51.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:51 smithi194 ceph-mon[49771]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:48:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-mon[39722]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T05:48:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-mon[39722]: osdmap e7: 1 total, 0 up, 1 in 2024-04-12T05:48:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-mon[39722]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:48:51.655 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[63659]: 2024-04-12T05:48:51.190+0000 7f4398a2e700 -1 osd.0 0 waiting for initial osdmap 2024-04-12T05:48:51.655 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:48:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[63659]: 2024-04-12T05:48:51.194+0000 7f439201e700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:48:52.481 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:52 smithi008 ceph-mon[39722]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:48:52.481 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:52 smithi008 ceph-mon[39722]: osdmap e8: 1 total, 0 up, 1 in 2024-04-12T05:48:52.481 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:52 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:52.481 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:52 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:52.482 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:52 smithi008 ceph-mon[39722]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:52.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:52 smithi194 ceph-mon[49771]: from='osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:48:52.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:52 smithi194 ceph-mon[49771]: osdmap e8: 1 total, 0 up, 1 in 2024-04-12T05:48:52.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:52 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:52.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:52 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:52.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:52 smithi194 ceph-mon[49771]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00810768 s, 1.3 GB/s 2024-04-12T05:48:53.345 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] boot 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: osdmap e9: 1 total, 1 up, 1 in 2024-04-12T05:48:53.538 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:53.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:48:53.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:48:53.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:53.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: osd.0 [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] boot 2024-04-12T05:48:53.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: osdmap e9: 1 total, 1 up, 1 in 2024-04-12T05:48:53.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:48:54.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:54 smithi008 ceph-mon[39722]: osdmap e10: 1 total, 1 up, 1 in 2024-04-12T05:48:54.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:54 smithi008 ceph-mon[39722]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:54.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:54 smithi194 ceph-mon[49771]: osdmap e10: 1 total, 1 up, 1 in 2024-04-12T05:48:54.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:54 smithi194 ceph-mon[49771]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:54.632 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi008:vg_nvme/lv_3 2024-04-12T05:48:54.913 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:48:56.619 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:56 smithi008 ceph-mon[39722]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:56.619 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:56.619 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:56.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:56 smithi194 ceph-mon[49771]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:56.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:56.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 -- 172.21.15.8:0/2589719527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 msgr2=0x7f3814106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 --2- 172.21.15.8:0/2589719527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f3814106450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3808009910 tx=0x7f3808009c20 comp rx=0 tx=0).stop 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 -- 172.21.15.8:0/2589719527 shutdown_connections 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 --2- 172.21.15.8:0/2589719527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f3814106450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 --2- 172.21.15.8:0/2589719527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 0x7f3814103b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 -- 172.21.15.8:0/2589719527 >> 172.21.15.8:0/2589719527 conn(0x7f38140fb120 msgr2=0x7f38140fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 -- 172.21.15.8:0/2589719527 shutdown_connections 2024-04-12T05:48:57.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 -- 172.21.15.8:0/2589719527 wait complete. 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.186+0000 7f3819e98700 1 Processor -- start 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3819e98700 1 -- start start 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3819e98700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 0x7f3814101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3819e98700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3819e98700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ffc20 con 0x7f3814104060 2024-04-12T05:48:57.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3819e98700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ffd60 con 0x7f3814101730 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60256/0 (socket says 172.21.15.8:60256) 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 -- 172.21.15.8:0/1130306945 learned_addr learned my addr 172.21.15.8:0/1130306945 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f38137fe700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 0x7f3814101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 -- 172.21.15.8:0/1130306945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 msgr2=0x7f3814101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 0x7f3814101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.187+0000 7f3812ffd700 1 -- 172.21.15.8:0/1130306945 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080095f0 con 0x7f3814104060 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3812ffd700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3808006010 tx=0x7f380800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f380801d070 con 0x7f3814104060 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140fffe0 con 0x7f3814104060 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f380800f460 con 0x7f3814104060 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3808021640 con 0x7f3814104060 2024-04-12T05:48:57.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.188+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141004d0 con 0x7f3814104060 2024-04-12T05:48:57.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.189+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38141904d0 con 0x7f3814104060 2024-04-12T05:48:57.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.189+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7f380800f940 con 0x7f3814104060 2024-04-12T05:48:57.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.190+0000 7f3810ff9700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 0x7f37fc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:48:57.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.190+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f380808bc60 con 0x7f3814104060 2024-04-12T05:48:57.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.190+0000 7f38137fe700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 0x7f37fc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:48:57.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.190+0000 7f38137fe700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 0x7f37fc06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3804005950 tx=0x7f38040058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:48:57.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.197+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f380805ad50 con 0x7f3814104060 2024-04-12T05:48:57.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:48:57.417+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f38140611d0 con 0x7f37fc06c460 2024-04-12T05:48:57.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:48:58.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:58 smithi008 ceph-mon[39722]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:58.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:58 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:48:58.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:58 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:48:58.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:58 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:58 smithi194 ceph-mon[49771]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:48:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:58 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:48:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:58 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:48:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:58 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:48:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:48:59 smithi008 ceph-mon[39722]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:48:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:48:59 smithi194 ceph-mon[49771]: from='client.14266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:49:00.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:00 smithi008 ceph-mon[39722]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:00 smithi194 ceph-mon[49771]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi008 2024-04-12T05:49:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi008 to 11878M 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:02.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:01 smithi194 ceph-mon[49771]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi008 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi008 to 11878M 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:01 smithi008 ceph-mon[39722]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:03.246 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:03 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2142325904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad299755-103b-4710-8304-f0e5abed87d4"}]: dispatch 2024-04-12T05:49:03.246 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:03 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2142325904' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad299755-103b-4710-8304-f0e5abed87d4"}]': finished 2024-04-12T05:49:03.246 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:03 smithi008 ceph-mon[39722]: osdmap e11: 2 total, 1 up, 2 in 2024-04-12T05:49:03.246 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:03 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:03.246 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:03 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3160810076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:49:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:03 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2142325904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad299755-103b-4710-8304-f0e5abed87d4"}]: dispatch 2024-04-12T05:49:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:03 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2142325904' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad299755-103b-4710-8304-f0e5abed87d4"}]': finished 2024-04-12T05:49:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:03 smithi194 ceph-mon[49771]: osdmap e11: 2 total, 1 up, 2 in 2024-04-12T05:49:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:03 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:03.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:03 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3160810076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:49:04.570 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:04.570 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:04 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:04.570 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:04 smithi008 ceph-mon[39722]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:04.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:04 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:04 smithi194 ceph-mon[49771]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:06.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:06.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:08.577 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:08 smithi008 ceph-mon[39722]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:08 smithi194 ceph-mon[49771]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:10 smithi008 ceph-mon[39722]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T05:49:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:10 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:10 smithi194 ceph-mon[49771]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T05:49:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:10 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:11.532 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:11 smithi008 ceph-mon[39722]: Deploying daemon osd.1 on smithi008 2024-04-12T05:49:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:11 smithi194 ceph-mon[49771]: Deploying daemon osd.1 on smithi008 2024-04-12T05:49:12.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:12 smithi008 ceph-mon[39722]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:12.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:12 smithi194 ceph-mon[49771]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:14 smithi008 ceph-mon[39722]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:14.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:14 smithi194 ceph-mon[49771]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:16 smithi008 ceph-mon[39722]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:16 smithi194 ceph-mon[49771]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:18 smithi008 ceph-mon[39722]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:18 smithi194 ceph-mon[49771]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:18.826 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 1 on host 'smithi008' 2024-04-12T05:49:18.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.830+0000 7f3810ff9700 1 -- 172.21.15.8:0/1130306945 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f38140611d0 con 0x7f37fc06c460 2024-04-12T05:49:18.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 msgr2=0x7f37fc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:18.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 0x7f37fc06e920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3804005950 tx=0x7f38040058e0 comp rx=0 tx=0).stop 2024-04-12T05:49:18.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 msgr2=0x7f38140ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:18.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3808006010 tx=0x7f380800be20 comp rx=0 tx=0).stop 2024-04-12T05:49:18.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 shutdown_connections 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3814101730 0x7f3814101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f37fc06c460 0x7f37fc06e920 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 --2- 172.21.15.8:0/1130306945 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3814104060 0x7f38140ff6e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 >> 172.21.15.8:0/1130306945 conn(0x7f38140fb120 msgr2=0x7f38140fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 shutdown_connections 2024-04-12T05:49:18.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:18.832+0000 7f3819e98700 1 -- 172.21.15.8:0/1130306945 wait complete. 2024-04-12T05:49:19.468 DEBUG:teuthology.orchestra.run.smithi008:osd.1> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.1.service 2024-04-12T05:49:19.471 INFO:tasks.cephadm:Deploying osd.2 on smithi008 with /dev/vg_nvme/lv_2... 2024-04-12T05:49:19.471 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-12T05:49:19.504 INFO:journalctl@ceph.osd.1.smithi008.stdout:-- Logs begin at Fri 2024-04-12 05:32:04 UTC. -- 2024-04-12T05:49:20.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:20.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:20.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:19 smithi194 ceph-mon[49771]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T05:49:20.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:19 smithi194 ceph-mon[49771]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:20.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:20.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:20.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:19 smithi008 ceph-mon[39722]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T05:49:20.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:19 smithi008 ceph-mon[39722]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:49:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-mon[39722]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T05:49:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-mon[39722]: osdmap e12: 2 total, 1 up, 2 in 2024-04-12T05:49:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-mon[39722]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:49:20.905 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[73600]: 2024-04-12T05:49:20.826+0000 7fa383d65700 -1 osd.1 0 waiting for initial osdmap 2024-04-12T05:49:20.905 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:20 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[73600]: 2024-04-12T05:49:20.830+0000 7fa37c353700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:49:21.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:20 smithi194 ceph-mon[49771]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T05:49:21.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:20 smithi194 ceph-mon[49771]: osdmap e12: 2 total, 1 up, 2 in 2024-04-12T05:49:21.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:21.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:20 smithi194 ceph-mon[49771]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:49:22.062 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:21 smithi008 ceph-mon[39722]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:49:22.062 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:21 smithi008 ceph-mon[39722]: osdmap e13: 2 total, 1 up, 2 in 2024-04-12T05:49:22.062 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:21 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:22.062 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:21 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:22.063 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:21 smithi008 ceph-mon[39722]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:22.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:21 smithi194 ceph-mon[49771]: from='osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:49:22.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:21 smithi194 ceph-mon[49771]: osdmap e13: 2 total, 1 up, 2 in 2024-04-12T05:49:22.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:21 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:22.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:21 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:22.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:21 smithi194 ceph-mon[49771]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr: stderr: 10+0 records in 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr:10+0 records out 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00862194 s, 1.2 GB/s 2024-04-12T05:49:22.852 INFO:teuthology.orchestra.run.smithi008.stderr:--> Zapping successful for: 2024-04-12T05:49:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:22 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:49:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:22 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:49:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:22 smithi008 ceph-mon[39722]: osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] boot 2024-04-12T05:49:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:22 smithi008 ceph-mon[39722]: osdmap e14: 2 total, 2 up, 2 in 2024-04-12T05:49:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:22 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:22 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:49:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:22 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:49:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:22 smithi194 ceph-mon[49771]: osd.1 [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] boot 2024-04-12T05:49:23.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:22 smithi194 ceph-mon[49771]: osdmap e14: 2 total, 2 up, 2 in 2024-04-12T05:49:23.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:22 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:49:23.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:23 smithi008 ceph-mon[39722]: osdmap e15: 2 total, 2 up, 2 in 2024-04-12T05:49:23.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:23 smithi008 ceph-mon[39722]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:23.973 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi008:vg_nvme/lv_2 2024-04-12T05:49:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:23 smithi194 ceph-mon[49771]: osdmap e15: 2 total, 2 up, 2 in 2024-04-12T05:49:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:23 smithi194 ceph-mon[49771]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:24.954 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:49:26.483 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:26 smithi008 ceph-mon[39722]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:26.483 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:26 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:26.483 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:26 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:26 smithi194 ceph-mon[49771]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:26 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:26 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 -- 172.21.15.8:0/401499373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188101ea0 msgr2=0x7ff188102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 --2- 172.21.15.8:0/401499373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188101ea0 0x7ff188102300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009910 tx=0x7ff17c009c20 comp rx=0 tx=0).stop 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 -- 172.21.15.8:0/401499373 shutdown_connections 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 --2- 172.21.15.8:0/401499373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188101ea0 0x7ff188102300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 --2- 172.21.15.8:0/401499373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188100ca0 0x7ff1881010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:26.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 -- 172.21.15.8:0/401499373 >> 172.21.15.8:0/401499373 conn(0x7ff1880fc260 msgr2=0x7ff1880fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:49:26.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 -- 172.21.15.8:0/401499373 shutdown_connections 2024-04-12T05:49:26.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.790+0000 7ff18cad6700 1 -- 172.21.15.8:0/401499373 wait complete. 2024-04-12T05:49:26.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 Processor -- start 2024-04-12T05:49:26.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 -- start start 2024-04-12T05:49:26.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:26.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 0x7ff188073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880736c0 con 0x7ff188100ca0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18cad6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188073800 con 0x7ff188101ea0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.791+0000 7ff18659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48512/0 (socket says 172.21.15.8:48512) 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 -- 172.21.15.8:0/732643056 learned_addr learned my addr 172.21.15.8:0/732643056 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff185d9b700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 0x7ff188073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 -- 172.21.15.8:0/732643056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 msgr2=0x7ff188073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 0x7ff188073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 -- 172.21.15.8:0/732643056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0095f0 con 0x7ff188100ca0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff185d9b700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 0x7ff188073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18659c700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff17800d900 tx=0x7ff17800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:49:26.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1780041f0 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff178004350 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff188073ae0 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff178003cf0 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.792+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18819e460 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.793+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1881096c0 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.795+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7ff178009730 con 0x7ff188100ca0 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.795+0000 7ff1777fe700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 0x7ff17006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:26.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.796+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7ff17808b0c0 con 0x7ff188100ca0 2024-04-12T05:49:26.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.796+0000 7ff185d9b700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 0x7ff17006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:26.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.796+0000 7ff185d9b700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 0x7ff17006e970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00b5c0 tx=0x7ff17c005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:49:26.796 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:26.799+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff17805a060 con 0x7ff188100ca0 2024-04-12T05:49:27.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:27.007+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1881067f0 con 0x7ff17006c4b0 2024-04-12T05:49:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:49:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:49:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:49:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:49:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:28.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:28 smithi194 ceph-mon[49771]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:49:28.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:28 smithi194 ceph-mon[49771]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:28.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:28 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:28.635 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:28 smithi008 ceph-mon[39722]: from='client.14284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi008:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:49:28.635 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:28 smithi008 ceph-mon[39722]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:28.635 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:30.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:30 smithi008 ceph-mon[39722]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:30.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:30 smithi194 ceph-mon[49771]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:32.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi008 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi008 to 5939M 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:32.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:31 smithi194 ceph-mon[49771]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi008 2024-04-12T05:49:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi008 to 5939M 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:32.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:31 smithi008 ceph-mon[39722]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:32 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3763626643' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff12c02f-ab4b-4542-8b53-0c6112a339f6"}]: dispatch 2024-04-12T05:49:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:32 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3763626643' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff12c02f-ab4b-4542-8b53-0c6112a339f6"}]': finished 2024-04-12T05:49:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:32 smithi008 ceph-mon[39722]: osdmap e16: 3 total, 2 up, 3 in 2024-04-12T05:49:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:32 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:32 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3116790488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:49:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:32 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3763626643' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff12c02f-ab4b-4542-8b53-0c6112a339f6"}]: dispatch 2024-04-12T05:49:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:32 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3763626643' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff12c02f-ab4b-4542-8b53-0c6112a339f6"}]': finished 2024-04-12T05:49:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:32 smithi194 ceph-mon[49771]: osdmap e16: 3 total, 2 up, 3 in 2024-04-12T05:49:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:32 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:32 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3116790488' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:49:34.480 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:34 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:34.480 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:34 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:34.480 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:34 smithi194 ceph-mon[49771]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:34.501 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:34.501 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:34 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:34.501 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:34 smithi008 ceph-mon[39722]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:36.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:36.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:36.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:36 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:36.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:36.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:36 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:38 smithi008 ceph-mon[39722]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:38.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:38 smithi194 ceph-mon[49771]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:40 smithi008 ceph-mon[39722]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T05:49:40.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:40 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:40 smithi194 ceph-mon[49771]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T05:49:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:40 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:41.471 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:41 smithi008 ceph-mon[39722]: Deploying daemon osd.2 on smithi008 2024-04-12T05:49:41.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:41 smithi194 ceph-mon[49771]: Deploying daemon osd.2 on smithi008 2024-04-12T05:49:42.545 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:42 smithi008 ceph-mon[39722]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:42.545 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:42 smithi194 ceph-mon[49771]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:44.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:44 smithi008 ceph-mon[39722]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:44.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:44 smithi194 ceph-mon[49771]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:46.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:46 smithi008 ceph-mon[39722]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:46.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:46 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:46.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:46 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:46.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:46 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:46.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:46 smithi194 ceph-mon[49771]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:46.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:46 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:46.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:46 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:46.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:46 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:48 smithi008 ceph-mon[39722]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:48 smithi194 ceph-mon[49771]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:50.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.069+0000 7ff1777fe700 1 -- 172.21.15.8:0/732643056 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff1881067f0 con 0x7ff17006c4b0 2024-04-12T05:49:50.066 INFO:teuthology.orchestra.run.smithi008.stdout:Created osd(s) 2 on host 'smithi008' 2024-04-12T05:49:50.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.071+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 msgr2=0x7ff17006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:50.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.071+0000 7ff18cad6700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 0x7ff17006e970 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00b5c0 tx=0x7ff17c005e20 comp rx=0 tx=0).stop 2024-04-12T05:49:50.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.071+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 msgr2=0x7ff188074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.071+0000 7ff18cad6700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff17800d900 tx=0x7ff17800dcc0 comp rx=0 tx=0).stop 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 shutdown_connections 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff17006c4b0 0x7ff17006e970 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff188100ca0 0x7ff188074b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 --2- 172.21.15.8:0/732643056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff188101ea0 0x7ff188073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 >> 172.21.15.8:0/732643056 conn(0x7ff1880fc260 msgr2=0x7ff1881050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 shutdown_connections 2024-04-12T05:49:50.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:49:50.072+0000 7ff18cad6700 1 -- 172.21.15.8:0/732643056 wait complete. 2024-04-12T05:49:50.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:50 smithi008 ceph-mon[39722]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:50 smithi008 ceph-mon[39722]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T05:49:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:50 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:50 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:50.762 DEBUG:teuthology.orchestra.run.smithi008:osd.2> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.2.service 2024-04-12T05:49:50.765 INFO:tasks.cephadm:Deploying osd.3 on smithi194 with /dev/vg_nvme/lv_4... 2024-04-12T05:49:50.765 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-12T05:49:50.796 INFO:journalctl@ceph.osd.2.smithi008.stdout:-- Logs begin at Fri 2024-04-12 05:32:04 UTC. -- 2024-04-12T05:49:50.798 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:50 smithi194 ceph-mon[49771]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:50.798 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:50 smithi194 ceph-mon[49771]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T05:49:50.798 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:50 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:50.798 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:50 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:50.993 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:49:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:51 smithi194 ceph-mon[49771]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T05:49:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:51 smithi194 ceph-mon[49771]: osdmap e17: 3 total, 2 up, 3 in 2024-04-12T05:49:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:51 smithi194 ceph-mon[49771]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:49:51.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[83405]: 2024-04-12T05:49:51.361+0000 7f50ff6eb700 -1 osd.2 0 waiting for initial osdmap 2024-04-12T05:49:51.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[83405]: 2024-04-12T05:49:51.366+0000 7f50f94dc700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:49:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-mon[39722]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T05:49:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-mon[39722]: osdmap e17: 3 total, 2 up, 3 in 2024-04-12T05:49:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:51 smithi008 ceph-mon[39722]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T05:49:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:52 smithi008 ceph-mon[39722]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:52.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:52 smithi008 ceph-mon[39722]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:49:52.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:52 smithi008 ceph-mon[39722]: osdmap e18: 3 total, 2 up, 3 in 2024-04-12T05:49:52.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:52 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:52.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:52 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:52.664 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:52 smithi194 ceph-mon[49771]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:52.664 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:52 smithi194 ceph-mon[49771]: from='osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]': finished 2024-04-12T05:49:52.664 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:52 smithi194 ceph-mon[49771]: osdmap e18: 3 total, 2 up, 3 in 2024-04-12T05:49:52.664 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:52 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:52.664 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:52 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:53.269 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-12T05:49:53.270 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-12T05:49:53.270 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-12T05:49:53.270 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-12T05:49:53.270 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148592 s, 706 MB/s 2024-04-12T05:49:53.270 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-12T05:49:53.981 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] boot 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: osdmap e19: 3 total, 3 up, 3 in 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:53.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:54.002 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: osd.2 [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] boot 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: osdmap e19: 3 total, 3 up, 3 in 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:54.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:54.265 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:49:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:54 smithi008 ceph-mon[39722]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-12T05:49:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-12T05:49:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:54 smithi008 ceph-mon[39722]: osdmap e20: 3 total, 3 up, 3 in 2024-04-12T05:49:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-12T05:49:55.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:54 smithi194 ceph-mon[49771]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-12T05:49:55.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-12T05:49:55.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-12T05:49:55.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:54 smithi194 ceph-mon[49771]: osdmap e20: 3 total, 3 up, 3 in 2024-04-12T05:49:55.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-12T05:49:55.654 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[86966]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T05:49:55.654 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[86966]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:55.654 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[86966]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:55.654 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[86966]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:55.945 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:55 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-12T05:49:55.945 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:55 smithi194 ceph-mon[49771]: osdmap e21: 3 total, 3 up, 3 in 2024-04-12T05:49:55.945 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:55 smithi194 ceph-mon[49771]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:49:56.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:55 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-12T05:49:56.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:55 smithi008 ceph-mon[39722]: osdmap e21: 3 total, 3 up, 3 in 2024-04-12T05:49:56.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:55 smithi008 ceph-mon[39722]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:49:56.154 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[88860]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T05:49:56.154 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[88860]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:56.154 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[88860]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:56.154 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 05:49:55 smithi008 sudo[88860]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:56.224 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.226+0000 7ff5ec101700 1 -- 172.21.15.194:0/3321615062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e40ff320 msgr2=0x7ff5e40ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.226+0000 7ff5ec101700 1 --2- 172.21.15.194:0/3321615062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e40ff320 0x7ff5e40ff740 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009910 tx=0x7ff5e0009c20 comp rx=0 tx=0).stop 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- 172.21.15.194:0/3321615062 shutdown_connections 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 --2- 172.21.15.194:0/3321615062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e4100520 0x7ff5e4100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 --2- 172.21.15.194:0/3321615062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e40ff320 0x7ff5e40ff740 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- 172.21.15.194:0/3321615062 >> 172.21.15.194:0/3321615062 conn(0x7ff5e40fa8c0 msgr2=0x7ff5e40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- 172.21.15.194:0/3321615062 shutdown_connections 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- 172.21.15.194:0/3321615062 wait complete. 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 Processor -- start 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- start start 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e40ff320 0x7ff5e41062c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419b730 con 0x7ff5e40ff320 2024-04-12T05:49:56.225 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.227+0000 7ff5ec101700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e419b870 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50774/0 (socket says 172.21.15.194:50774) 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 -- 172.21.15.194:0/1758202259 learned_addr learned my addr 172.21.15.194:0/1758202259 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e9e9d700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e40ff320 0x7ff5e41062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 -- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e40ff320 msgr2=0x7ff5e41062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e40ff320 0x7ff5e41062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 -- 172.21.15.194:0/1758202259 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00095f0 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5e969c700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400d900 tx=0x7ff5d400dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d4009900 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff5d400b6e0 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e419bb50 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d400f3c0 con 0x7ff5e4100520 2024-04-12T05:49:56.226 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.228+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e419c0a0 con 0x7ff5e4100520 2024-04-12T05:49:56.227 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.229+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e41085c0 con 0x7ff5e4100520 2024-04-12T05:49:56.227 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.229+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 18) v1 ==== 90254+0+0 (secure 0 0 0) 0x7ff5d4010460 con 0x7ff5e4100520 2024-04-12T05:49:56.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.230+0000 7ff5daffd700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 0x7ff5d006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:49:56.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.230+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3232+0+0 (secure 0 0 0) 0x7ff5d408ac00 con 0x7ff5e4100520 2024-04-12T05:49:56.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.230+0000 7ff5e9e9d700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 0x7ff5d006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:49:56.228 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.230+0000 7ff5e9e9d700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 0x7ff5d006e930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0006010 tx=0x7ff5e000b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:49:56.231 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.235+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff5d4059890 con 0x7ff5e4100520 2024-04-12T05:49:56.427 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.431+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5e4104e70 con 0x7ff5d006c470 2024-04-12T05:49:56.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88872]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T05:49:56.654 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88872]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:56.654 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88872]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:56.654 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88872]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:56.857 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:49:56.861+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7ff5d40561d0 con 0x7ff5e4100520 2024-04-12T05:49:57.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: osdmap e22: 3 total, 3 up, 3 in 2024-04-12T05:49:57.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:56 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:57.154 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88998]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T05:49:57.154 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:57.154 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88998]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:57.154 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 05:49:56 smithi008 sudo[88998]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: osdmap e22: 3 total, 3 up, 3 in 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:49:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:49:57.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:56 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:49:57.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89092]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T05:49:57.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89092]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:57.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89092]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:57.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89092]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:58.024 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89130]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T05:49:58.024 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:58.024 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89130]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:58.024 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 05:49:57 smithi008 sudo[89130]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:58.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T05:49:58.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:58.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:58.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89234]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89264]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T05:49:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89264]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:58 smithi008 sudo[89264]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:58 smithi194 sudo[54323]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:58 smithi194 sudo[54323]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:58 smithi194 sudo[54323]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:58 smithi194 sudo[54323]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 sudo[54326]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 sudo[54326]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 sudo[54326]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 sudo[54326]: pam_unix(sudo:session): session closed for user root 2024-04-12T05:49:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi008 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi008 to 3959M 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: mgrmap e19: smithi008.ixmnlp(active, since 2m), standbys: smithi194.xbiyaj 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:49:59 smithi194 ceph-mon[49771]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi008 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi008 to 3959M 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: mgrmap e19: smithi008.ixmnlp(active, since 2m), standbys: smithi194.xbiyaj 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:49:59.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:49:59 smithi008 ceph-mon[39722]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/281363682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]: dispatch 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]: dispatch 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]': finished 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: osdmap e23: 4 total, 3 up, 4 in 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/202854965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:50:00.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:00 smithi008 ceph-mon[39722]: overall HEALTH_OK 2024-04-12T05:50:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T05:50:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:50:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:50:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-12T05:50:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/281363682' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6feb5af6-bf9e-48a4-bc8d-7cff6420c259"}]': finished 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: osdmap e23: 4 total, 3 up, 4 in 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/202854965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:50:00.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:00 smithi194 ceph-mon[49771]: overall HEALTH_OK 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:02.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:04.607 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:04 smithi194 ceph-mon[49771]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 195 KiB/s wr, 13 op/s 2024-04-12T05:50:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:04 smithi008 ceph-mon[39722]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 195 KiB/s wr, 13 op/s 2024-04-12T05:50:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:06 smithi008 ceph-mon[39722]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s 2024-04-12T05:50:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:06 smithi194 ceph-mon[49771]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 130 KiB/s wr, 9 op/s 2024-04-12T05:50:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T05:50:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:07.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T05:50:07.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:08.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:08 smithi008 ceph-mon[39722]: Deploying daemon osd.3 on smithi194 2024-04-12T05:50:08.651 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:08 smithi008 ceph-mon[39722]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-12T05:50:08.789 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:08 smithi194 ceph-mon[49771]: Deploying daemon osd.3 on smithi194 2024-04-12T05:50:08.790 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:08 smithi194 ceph-mon[49771]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-12T05:50:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:10 smithi194 ceph-mon[49771]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-12T05:50:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:10 smithi008 ceph-mon[39722]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-12T05:50:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:11 smithi194 ceph-mon[49771]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T05:50:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:11 smithi008 ceph-mon[39722]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T05:50:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:14.532 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:14.533 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:14.533 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:14.533 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:14 smithi194 ceph-mon[49771]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T05:50:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:14 smithi008 ceph-mon[39722]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-12T05:50:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:16 smithi008 ceph-mon[39722]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:16.657 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:16 smithi194 ceph-mon[49771]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:17.397 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 3 on host 'smithi194' 2024-04-12T05:50:17.397 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.400+0000 7ff5daffd700 1 -- 172.21.15.194:0/1758202259 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5e4104e70 con 0x7ff5d006c470 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 msgr2=0x7ff5d006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 0x7ff5d006e930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0006010 tx=0x7ff5e000b560 comp rx=0 tx=0).stop 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 msgr2=0x7ff5e419b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400d900 tx=0x7ff5d400dc10 comp rx=0 tx=0).stop 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 shutdown_connections 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff5d006c470 0x7ff5d006e930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff5e40ff320 0x7ff5e41062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 --2- 172.21.15.194:0/1758202259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff5e4100520 0x7ff5e419b160 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 >> 172.21.15.194:0/1758202259 conn(0x7ff5e40fa8c0 msgr2=0x7ff5e4103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:50:17.399 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 shutdown_connections 2024-04-12T05:50:17.400 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:17.403+0000 7ff5ec101700 1 -- 172.21.15.194:0/1758202259 wait complete. 2024-04-12T05:50:17.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:17 smithi008 ceph-mon[39722]: from='osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T05:50:17.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:17 smithi008 ceph-mon[39722]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T05:50:17.814 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:17 smithi194 ceph-mon[49771]: from='osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T05:50:17.814 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:17 smithi194 ceph-mon[49771]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T05:50:18.214 DEBUG:teuthology.orchestra.run.smithi194:osd.3> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.3.service 2024-04-12T05:50:18.216 INFO:tasks.cephadm:Deploying osd.4 on smithi194 with /dev/vg_nvme/lv_3... 2024-04-12T05:50:18.216 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-12T05:50:18.246 INFO:journalctl@ceph.osd.3.smithi194.stdout:-- Logs begin at Fri 2024-04-12 05:32:37 UTC. -- 2024-04-12T05:50:18.570 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[56703]: 2024-04-12T05:50:18.357+0000 7fe11220b700 -1 osd.3 0 waiting for initial osdmap 2024-04-12T05:50:18.570 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[56703]: 2024-04-12T05:50:18.367+0000 7fe10a7f9700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: osdmap e24: 4 total, 3 up, 4 in 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: osdmap e24: 4 total, 3 up, 4 in 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:19.206 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:50:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:19 smithi008 ceph-mon[39722]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:50:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:19 smithi008 ceph-mon[39722]: osdmap e25: 4 total, 3 up, 4 in 2024-04-12T05:50:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:19.680 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:19 smithi194 ceph-mon[49771]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:50:19.681 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:19 smithi194 ceph-mon[49771]: osdmap e25: 4 total, 3 up, 4 in 2024-04-12T05:50:19.681 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:19.681 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] boot 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: osdmap e26: 4 total, 4 up, 4 in 2024-04-12T05:50:20.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: osd.3 [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] boot 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: osdmap e26: 4 total, 4 up, 4 in 2024-04-12T05:50:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:50:21.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:21 smithi194 ceph-mon[49771]: osdmap e27: 4 total, 4 up, 4 in 2024-04-12T05:50:21.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:21 smithi008 ceph-mon[39722]: osdmap e27: 4 total, 4 up, 4 in 2024-04-12T05:50:22.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:22 smithi008 ceph-mon[39722]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:22.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:22 smithi008 ceph-mon[39722]: osdmap e28: 4 total, 4 up, 4 in 2024-04-12T05:50:22.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:22 smithi194 ceph-mon[49771]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:22.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:22 smithi194 ceph-mon[49771]: osdmap e28: 4 total, 4 up, 4 in 2024-04-12T05:50:23.588 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-12T05:50:23.588 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-12T05:50:23.588 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-12T05:50:23.588 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-12T05:50:23.589 INFO:teuthology.orchestra.run.smithi194.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111574 s, 940 MB/s 2024-04-12T05:50:23.589 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 2024-04-12T05:50:23.589 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: osdmap e29: 4 total, 4 up, 4 in 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: osdmap e29: 4 total, 4 up, 4 in 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:24.408 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-04-12T05:50:24.628 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:50:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:24 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi194 2024-04-12T05:50:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:24 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi194 to 14946M 2024-04-12T05:50:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:24 smithi008 ceph-mon[39722]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-12T05:50:24.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:24 smithi008 ceph-mon[39722]: osdmap e30: 4 total, 4 up, 4 in 2024-04-12T05:50:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:24 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi194 2024-04-12T05:50:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:24 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi194 to 14946M 2024-04-12T05:50:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:24 smithi194 ceph-mon[49771]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-12T05:50:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:24 smithi194 ceph-mon[49771]: osdmap e30: 4 total, 4 up, 4 in 2024-04-12T05:50:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:26 smithi194 ceph-mon[49771]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-12T05:50:26.583 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- 172.21.15.194:0/4091689342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 msgr2=0x7f10481023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 --2- 172.21.15.194:0/4091689342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f10481023a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f103c009910 tx=0x7f103c009c20 comp rx=0 tx=0).stop 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- 172.21.15.194:0/4091689342 shutdown_connections 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 --2- 172.21.15.194:0/4091689342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f10481023a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 --2- 172.21.15.194:0/4091689342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10480fef60 0x7f10480ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- 172.21.15.194:0/4091689342 >> 172.21.15.194:0/4091689342 conn(0x7f10480fa8c0 msgr2=0x7f10480fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- 172.21.15.194:0/4091689342 shutdown_connections 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- 172.21.15.194:0/4091689342 wait complete. 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 Processor -- start 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 -- start start 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10480fef60 0x7f1048198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:26.584 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.576+0000 7f104dfbc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f104dfbc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048199170 con 0x7f10480fef60 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f104dfbc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481992b0 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:43798/0 (socket says 172.21.15.194:43798) 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 -- 172.21.15.194:0/1205752408 learned_addr learned my addr 172.21.15.194:0/1205752408 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 -- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10480fef60 msgr2=0x7f1048198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10480fef60 0x7f1048198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 -- 172.21.15.194:0/1205752408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0095f0 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1046ffd700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f103c006010 tx=0x7f103c00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c01d070 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f103c00f460 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104819dd00 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c021640 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.577+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104819e1f0 con 0x7f1048101f40 2024-04-12T05:50:26.585 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.578+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048106020 con 0x7f1048101f40 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.578+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f103c003a60 con 0x7f1048101f40 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.579+0000 7f1044ff9700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.579+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f103c08c470 con 0x7f1048101f40 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.579+0000 7f10477fe700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.580+0000 7f10477fe700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 0x7f103006e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1048068380 tx=0x7f1038009470 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:50:26.586 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.583+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f103c05ad80 con 0x7f1048101f40 2024-04-12T05:50:26.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:26 smithi008 ceph-mon[39722]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-04-12T05:50:26.772 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:26.777+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f10480611d0 con 0x7f103006c4e0 2024-04-12T05:50:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:50:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:50:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:27.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:50:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:50:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:27.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:28.561 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:28 smithi194 ceph-mon[49771]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:50:28.561 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:28 smithi194 ceph-mon[49771]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-12T05:50:28.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:28 smithi008 ceph-mon[39722]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:50:28.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:28 smithi008 ceph-mon[39722]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-12T05:50:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:29 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1440264523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]: dispatch 2024-04-12T05:50:29.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:29 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]: dispatch 2024-04-12T05:50:29.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:29 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/1440264523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]: dispatch 2024-04-12T05:50:29.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:29 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]: dispatch 2024-04-12T05:50:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:30 smithi194 ceph-mon[49771]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-12T05:50:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:30 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]': finished 2024-04-12T05:50:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:30 smithi194 ceph-mon[49771]: osdmap e31: 5 total, 4 up, 5 in 2024-04-12T05:50:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:30 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:30 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/1627813618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:50:30.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:30 smithi008 ceph-mon[39722]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-12T05:50:30.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:30 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79640d26-73e7-4d99-9933-f82e5e6bb487"}]': finished 2024-04-12T05:50:30.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:30 smithi008 ceph-mon[39722]: osdmap e31: 5 total, 4 up, 5 in 2024-04-12T05:50:30.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:30 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:30.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:30 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1627813618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:50:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:31 smithi194 ceph-mon[49771]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:31.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:31 smithi008 ceph-mon[39722]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:34.601 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:34 smithi194 ceph-mon[49771]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:34 smithi008 ceph-mon[39722]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:36.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:36 smithi194 ceph-mon[49771]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:36.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:36 smithi008 ceph-mon[39722]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:37.620 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:37 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T05:50:37.620 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:37 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:37.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:37 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T05:50:37.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:37 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:38 smithi008 ceph-mon[39722]: Deploying daemon osd.4 on smithi194 2024-04-12T05:50:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:38 smithi008 ceph-mon[39722]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:38.683 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:38 smithi194 ceph-mon[49771]: Deploying daemon osd.4 on smithi194 2024-04-12T05:50:38.683 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:38 smithi194 ceph-mon[49771]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:40.482 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:40 smithi194 ceph-mon[49771]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:40 smithi008 ceph-mon[39722]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:41.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:41 smithi194 ceph-mon[49771]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:41.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:41 smithi008 ceph-mon[39722]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:42.723 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:44.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:43 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:44.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:43 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:44.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:43 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:44.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:43 smithi194 ceph-mon[49771]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:43 smithi008 ceph-mon[39722]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:46.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:46 smithi008 ceph-mon[39722]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:46.716 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:46 smithi194 ceph-mon[49771]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:47.344 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.348+0000 7f1044ff9700 1 -- 172.21.15.194:0/1205752408 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f10480611d0 con 0x7f103006c4e0 2024-04-12T05:50:47.345 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 4 on host 'smithi194' 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.350+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 msgr2=0x7f103006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.350+0000 7f104dfbc700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 0x7f103006e9a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1048068380 tx=0x7f1038009470 comp rx=0 tx=0).stop 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 msgr2=0x7f1048198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f103c006010 tx=0x7f103c00be20 comp rx=0 tx=0).stop 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 shutdown_connections 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f103006c4e0 0x7f103006e9a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10480fef60 0x7f1048198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 --2- 172.21.15.194:0/1205752408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1048101f40 0x7f1048198b50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:47.347 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 >> 172.21.15.194:0/1205752408 conn(0x7f10480fa8c0 msgr2=0x7f1048103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:50:47.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 shutdown_connections 2024-04-12T05:50:47.348 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:47.351+0000 7f104dfbc700 1 -- 172.21.15.194:0/1205752408 wait complete. 2024-04-12T05:50:47.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:47 smithi194 ceph-mon[49771]: from='osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T05:50:47.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:47 smithi194 ceph-mon[49771]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T05:50:47.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:47.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:47 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:47.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:47 smithi008 ceph-mon[39722]: from='osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T05:50:47.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:47 smithi008 ceph-mon[39722]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T05:50:47.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:47.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:47 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:48.134 DEBUG:teuthology.orchestra.run.smithi194:osd.4> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.4.service 2024-04-12T05:50:48.136 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_2... 2024-04-12T05:50:48.137 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-12T05:50:48.170 INFO:journalctl@ceph.osd.4.smithi194.stdout:-- Logs begin at Fri 2024-04-12 05:32:37 UTC. -- 2024-04-12T05:50:48.363 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[64703]: 2024-04-12T05:50:48.360+0000 7f6b3bd49700 -1 osd.4 0 waiting for initial osdmap 2024-04-12T05:50:48.406 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:48.406 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: osdmap e32: 5 total, 4 up, 5 in 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:48.407 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:48.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: osdmap e32: 5 total, 4 up, 5 in 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:48.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:48.779 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 05:50:48 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[64703]: 2024-04-12T05:50:48.367+0000 7f6b3633b700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:50:49.099 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:50:49.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:49 smithi008 ceph-mon[39722]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:50:49.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:49 smithi008 ceph-mon[39722]: osdmap e33: 5 total, 4 up, 5 in 2024-04-12T05:50:49.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:49.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:49.661 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:49 smithi194 ceph-mon[49771]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:50:49.661 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:49 smithi194 ceph-mon[49771]: osdmap e33: 5 total, 4 up, 5 in 2024-04-12T05:50:49.661 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:49.661 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:50:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:50:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] boot 2024-04-12T05:50:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: osdmap e34: 5 total, 5 up, 5 in 2024-04-12T05:50:50.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:50 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:50:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:50:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-12T05:50:50.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: osd.4 [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] boot 2024-04-12T05:50:50.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: osdmap e34: 5 total, 5 up, 5 in 2024-04-12T05:50:50.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:50 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:50:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:51 smithi194 ceph-mon[49771]: osdmap e35: 5 total, 5 up, 5 in 2024-04-12T05:50:51.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:51 smithi008 ceph-mon[39722]: osdmap e35: 5 total, 5 up, 5 in 2024-04-12T05:50:52.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:52 smithi008 ceph-mon[39722]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:52.679 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:52 smithi194 ceph-mon[49771]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:53.469 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-12T05:50:53.469 INFO:teuthology.orchestra.run.smithi194.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-12T05:50:53.470 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10+0 records in 2024-04-12T05:50:53.470 INFO:teuthology.orchestra.run.smithi194.stderr:10+0 records out 2024-04-12T05:50:53.470 INFO:teuthology.orchestra.run.smithi194.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129783 s, 808 MB/s 2024-04-12T05:50:53.470 INFO:teuthology.orchestra.run.smithi194.stderr:--> Zapping successful for: 2024-04-12T05:50:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi194 2024-04-12T05:50:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi194 to 7473M 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:53 smithi194 ceph-mon[49771]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi194 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi194 to 7473M 2024-04-12T05:50:54.002 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:50:54.003 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:53 smithi008 ceph-mon[39722]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:54.247 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-04-12T05:50:54.470 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.363+0000 7f600cf7d700 1 -- 172.21.15.194:0/2051645531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 msgr2=0x7f6008101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.363+0000 7f600cf7d700 1 --2- 172.21.15.194:0/2051645531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f6008101270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009910 tx=0x7f5ffc009c20 comp rx=0 tx=0).stop 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 -- 172.21.15.194:0/2051645531 shutdown_connections 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 --2- 172.21.15.194:0/2051645531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f6008101270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 --2- 172.21.15.194:0/2051645531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60080ffc10 0x7f6008100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 -- 172.21.15.194:0/2051645531 >> 172.21.15.194:0/2051645531 conn(0x7f60080fb190 msgr2=0x7f60080fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 -- 172.21.15.194:0/2051645531 shutdown_connections 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 -- 172.21.15.194:0/2051645531 wait complete. 2024-04-12T05:50:56.362 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 Processor -- start 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.364+0000 7f600cf7d700 1 -- start start 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60080ffc10 0x7f6008069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600819b820 con 0x7f60080ffc10 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600819b990 con 0x7f6008100e10 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:44548/0 (socket says 172.21.15.194:44548) 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 -- 172.21.15.194:0/218165613 learned_addr learned my addr 172.21.15.194:0/218165613 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 -- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60080ffc10 msgr2=0x7f6008069100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60080ffc10 0x7f6008069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 -- 172.21.15.194:0/218165613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0095f0 con 0x7f6008100e10 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f6005d9b700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0055d0 tx=0x7f5ffc00be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc01d070 con 0x7f6008100e10 2024-04-12T05:50:56.363 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5ffc00f460 con 0x7f6008100e10 2024-04-12T05:50:56.364 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600819bc10 con 0x7f6008100e10 2024-04-12T05:50:56.364 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ffc017630 con 0x7f6008100e10 2024-04-12T05:50:56.364 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.365+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600819c100 con 0x7f6008100e10 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.366+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008108660 con 0x7f6008100e10 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.367+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f5ffc00f9e0 con 0x7f6008100e10 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.367+0000 7f5ff77fe700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 0x7f5ff006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.367+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f5ffc08c9b0 con 0x7f6008100e10 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.367+0000 7f600659c700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 0x7f5ff006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:50:56.365 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.368+0000 7f600659c700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 0x7f5ff006e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6008100c70 tx=0x7f5ff8006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:50:56.368 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.372+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ffc05b110 con 0x7f6008100e10 2024-04-12T05:50:56.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:50:56.566+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6008105760 con 0x7f5ff006c4e0 2024-04-12T05:50:56.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:56 smithi194 ceph-mon[49771]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:56 smithi008 ceph-mon[39722]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:57.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:50:57.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:50:57.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:57.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:57.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-12T05:50:57.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-12T05:50:57.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:50:57.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:50:58.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:58 smithi194 ceph-mon[49771]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:50:58.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:58 smithi194 ceph-mon[49771]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:58.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:58 smithi008 ceph-mon[39722]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:50:58.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:58 smithi008 ceph-mon[39722]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:50:59.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:59 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1636551793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]: dispatch 2024-04-12T05:50:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:59 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]: dispatch 2024-04-12T05:50:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:59 smithi008 ceph-mon[39722]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]': finished 2024-04-12T05:50:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:59 smithi008 ceph-mon[39722]: osdmap e36: 6 total, 5 up, 6 in 2024-04-12T05:50:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:50:59 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:50:59.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:59 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/1636551793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]: dispatch 2024-04-12T05:50:59.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:59 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]: dispatch 2024-04-12T05:50:59.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:59 smithi194 ceph-mon[49771]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b267a717-3b34-4518-ae46-420004107ef3"}]': finished 2024-04-12T05:50:59.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:59 smithi194 ceph-mon[49771]: osdmap e36: 6 total, 5 up, 6 in 2024-04-12T05:50:59.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:50:59 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:00.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:00 smithi008 ceph-mon[39722]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:00.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/1181829125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:51:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:00 smithi194 ceph-mon[49771]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:00 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/1181829125' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-12T05:51:01.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:01 smithi194 ceph-mon[49771]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:01.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:01 smithi008 ceph-mon[39722]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:04.598 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:04 smithi194 ceph-mon[49771]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:04 smithi008 ceph-mon[39722]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:06 smithi008 ceph-mon[39722]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:06.807 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:06 smithi194 ceph-mon[49771]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T05:51:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:07.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T05:51:07.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:08.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:08 smithi008 ceph-mon[39722]: Deploying daemon osd.5 on smithi194 2024-04-12T05:51:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:08 smithi008 ceph-mon[39722]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:08.785 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:08 smithi194 ceph-mon[49771]: Deploying daemon osd.5 on smithi194 2024-04-12T05:51:08.785 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:08 smithi194 ceph-mon[49771]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:10.567 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:10 smithi194 ceph-mon[49771]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:10 smithi008 ceph-mon[39722]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:11.733 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:11 smithi194 ceph-mon[49771]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:11 smithi008 ceph-mon[39722]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:14.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:14.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:14.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:51:14.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:14 smithi008 ceph-mon[39722]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:14.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:14.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:14.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:51:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:14 smithi194 ceph-mon[49771]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:16 smithi008 ceph-mon[39722]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:16 smithi194 ceph-mon[49771]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:17.433 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.438+0000 7f5ff77fe700 1 -- 172.21.15.194:0/218165613 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6008105760 con 0x7f5ff006c4e0 2024-04-12T05:51:17.434 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2024-04-12T05:51:17.435 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 msgr2=0x7f5ff006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 0x7f5ff006e9a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6008100c70 tx=0x7f5ff8006d20 comp rx=0 tx=0).stop 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 msgr2=0x7f600819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0055d0 tx=0x7f5ffc00be20 comp rx=0 tx=0).stop 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 shutdown_connections 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5ff006c4e0 0x7f5ff006e9a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f60080ffc10 0x7f6008069100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 --2- 172.21.15.194:0/218165613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6008100e10 0x7f600819b250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 >> 172.21.15.194:0/218165613 conn(0x7f60080fb190 msgr2=0x7f6008104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 shutdown_connections 2024-04-12T05:51:17.436 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:17.440+0000 7f600cf7d700 1 -- 172.21.15.194:0/218165613 wait complete. 2024-04-12T05:51:17.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:17 smithi008 ceph-mon[39722]: from='osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T05:51:17.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:17 smithi008 ceph-mon[39722]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T05:51:17.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:17 smithi194 ceph-mon[49771]: from='osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T05:51:17.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:17 smithi194 ceph-mon[49771]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T05:51:18.262 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.5.service 2024-04-12T05:51:18.265 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-12T05:51:18.265 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd stat -f json 2024-04-12T05:51:18.297 INFO:journalctl@ceph.osd.5.smithi194.stdout:-- Logs begin at Fri 2024-04-12 05:32:37 UTC. -- 2024-04-12T05:51:18.516 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:18.570 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[72935]: 2024-04-12T05:51:18.366+0000 7f099d175700 -1 osd.5 0 waiting for initial osdmap 2024-04-12T05:51:18.570 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[72935]: 2024-04-12T05:51:18.373+0000 7f099976b700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T05:51:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: osdmap e37: 6 total, 5 up, 6 in 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T05:51:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: osdmap e37: 6 total, 5 up, 6 in 2024-04-12T05:51:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:51:18.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T05:51:18.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:18.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:19.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:51:19.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: osdmap e38: 6 total, 5 up, 6 in 2024-04-12T05:51:19.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:19.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:19.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:19.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:19 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: osdmap e38: 6 total, 5 up, 6 in 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:19.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:19 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:20.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 -- 172.21.15.8:0/2437434283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b8100e10 msgr2=0x7f48b8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:20.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 --2- 172.21.15.8:0/2437434283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b8100e10 0x7f48b8101270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009910 tx=0x7f48ac009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:20.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 -- 172.21.15.8:0/2437434283 shutdown_connections 2024-04-12T05:51:20.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 --2- 172.21.15.8:0/2437434283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b8100e10 0x7f48b8101270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 --2- 172.21.15.8:0/2437434283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b80ffc10 0x7f48b8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 -- 172.21.15.8:0/2437434283 >> 172.21.15.8:0/2437434283 conn(0x7f48b80fb190 msgr2=0x7f48b80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 -- 172.21.15.8:0/2437434283 shutdown_connections 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.368+0000 7f48bd81d700 1 -- 172.21.15.8:0/2437434283 wait complete. 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 Processor -- start 2024-04-12T05:51:20.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 -- start start 2024-04-12T05:51:20.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:20.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b8100e10 0x7f48b8196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8196f20 con 0x7f48b80ffc10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48bd81d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b8197060 con 0x7f48b8100e10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54668/0 (socket says 172.21.15.8:54668) 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 -- 172.21.15.8:0/861903244 learned_addr learned my addr 172.21.15.8:0/861903244 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b67fc700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b8100e10 0x7f48b8196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 -- 172.21.15.8:0/861903244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b8100e10 msgr2=0x7f48b8196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b8100e10 0x7f48b8196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.369+0000 7f48b6ffd700 1 -- 172.21.15.8:0/861903244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0095f0 con 0x7f48b80ffc10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48b6ffd700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e8f0 tx=0x7f48b000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b0009790 con 0x7f48b80ffc10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b819bb10 con 0x7f48b80ffc10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f48b000c810 con 0x7f48b80ffc10 2024-04-12T05:51:20.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b0003c20 con 0x7f48b80ffc10 2024-04-12T05:51:20.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.370+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b819c060 con 0x7f48b80ffc10 2024-04-12T05:51:20.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.371+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b81085c0 con 0x7f48b80ffc10 2024-04-12T05:51:20.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.372+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f48b000c980 con 0x7f48b80ffc10 2024-04-12T05:51:20.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.372+0000 7f48bc81b700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 0x7f48a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:20.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.372+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f48b0014070 con 0x7f48b80ffc10 2024-04-12T05:51:20.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.372+0000 7f48b67fc700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 0x7f48a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:20.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.372+0000 7f48b67fc700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 0x7f48a006e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009fd0 tx=0x7f48ac005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:20.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.377+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f48b0059da0 con 0x7f48b80ffc10 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: purged_snaps scrub starts 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: purged_snaps scrub ok 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] boot 2024-04-12T05:51:20.442 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: osdmap e39: 6 total, 6 up, 6 in 2024-04-12T05:51:20.443 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:20.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.579+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f48b81087b0 con 0x7f48b80ffc10 2024-04-12T05:51:20.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.579+0000 7f48bc81b700 1 -- 172.21.15.8:0/861903244 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f48b81087b0 con 0x7f48b80ffc10 2024-04-12T05:51:20.575 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 msgr2=0x7f48a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 0x7f48a006e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009fd0 tx=0x7f48ac005fb0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 msgr2=0x7f48b81963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48b000e8f0 tx=0x7f48b000ec00 comp rx=0 tx=0).stop 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 shutdown_connections 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f48a006c4e0 0x7f48a006e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f48b80ffc10 0x7f48b81963c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 --2- 172.21.15.8:0/861903244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48b8100e10 0x7f48b8196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 >> 172.21.15.8:0/861903244 conn(0x7f48b80fb190 msgr2=0x7f48b8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 shutdown_connections 2024-04-12T05:51:20.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:20.581+0000 7f48bd81d700 1 -- 172.21.15.8:0/861903244 wait complete. 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: purged_snaps scrub starts 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: purged_snaps scrub ok 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: osd.5 [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] boot 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: osdmap e39: 6 total, 6 up, 6 in 2024-04-12T05:51:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:51:21.439 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1712901079,"num_in_osds":6,"osd_in_since":1712901059,"num_remapped_pgs":0} 2024-04-12T05:51:21.440 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd dump --format=json 2024-04-12T05:51:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:21 smithi008 ceph-mon[39722]: osdmap e40: 6 total, 6 up, 6 in 2024-04-12T05:51:21.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:21 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/861903244' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-12T05:51:21.680 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:21.701 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:21 smithi194 ceph-mon[49771]: osdmap e40: 6 total, 6 up, 6 in 2024-04-12T05:51:21.701 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:21 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/861903244' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-12T05:51:22.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:22 smithi008 ceph-mon[39722]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:22.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:22 smithi194 ceph-mon[49771]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- 172.21.15.8:0/3559908037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18800ffb40 msgr2=0x7f18800fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 --2- 172.21.15.8:0/3559908037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18800ffb40 0x7f18800fffc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1874009910 tx=0x7f1874009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- 172.21.15.8:0/3559908037 shutdown_connections 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 --2- 172.21.15.8:0/3559908037 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18800ffb40 0x7f18800fffc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 --2- 172.21.15.8:0/3559908037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1880105bf0 0x7f1880105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- 172.21.15.8:0/3559908037 >> 172.21.15.8:0/3559908037 conn(0x7f18800fb610 msgr2=0x7f18800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- 172.21.15.8:0/3559908037 shutdown_connections 2024-04-12T05:51:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- 172.21.15.8:0/3559908037 wait complete. 2024-04-12T05:51:23.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 Processor -- start 2024-04-12T05:51:23.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.572+0000 7f1886381700 1 -- start start 2024-04-12T05:51:23.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18800ffb40 0x7f188019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:23.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:23.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188019d0c0 con 0x7f1880105bf0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1880196570 con 0x7f18800ffb40 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58310/0 (socket says 172.21.15.8:58310) 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 -- 172.21.15.8:0/1303127048 learned_addr learned my addr 172.21.15.8:0/1303127048 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187ffff700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18800ffb40 0x7f188019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 -- 172.21.15.8:0/1303127048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18800ffb40 msgr2=0x7f188019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18800ffb40 0x7f188019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 -- 172.21.15.8:0/1303127048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18740095f0 con 0x7f1880105bf0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187f7fe700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f187400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f187401d070 con 0x7f1880105bf0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f187400f460 con 0x7f1880105bf0 2024-04-12T05:51:23.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18801967f0 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1874021640 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.573+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1880196ce0 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.574+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188004f2e0 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.575+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f187400fa60 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.575+0000 7f187d7fa700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 0x7f186806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.576+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f187408c6b0 con 0x7f1880105bf0 2024-04-12T05:51:23.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.576+0000 7f187ffff700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 0x7f186806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:23.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.576+0000 7f187ffff700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 0x7f186806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1870005950 tx=0x7f1870005cf0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:23.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.580+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f187405ace0 con 0x7f1880105bf0 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: Detected new or changed devices on smithi194 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi194 to 4982M 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:51:23.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:51:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:51:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:23 smithi008 ceph-mon[39722]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:23.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.778+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1880197a80 con 0x7f1880105bf0 2024-04-12T05:51:23.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.779+0000 7f187d7fa700 1 -- 172.21.15.8:0/1303127048 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11262 (secure 0 0 0) 0x7f187405a870 con 0x7f1880105bf0 2024-04-12T05:51:23.775 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:51:23.775 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T05:51:20.399794+0000","last_up_change":"2024-04-12T05:51:19.387464+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6803","nonce":106489579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6805","nonce":106489579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6809","nonce":106489579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6807","nonce":106489579}]},"public_addr":"172.21.15.8:6803/106489579","cluster_addr":"172.21.15.8:6805/106489579","heartbeat_back_addr":"172.21.15.8:6809/106489579","heartbeat_front_addr":"172.21.15.8:6807/106489579","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6811","nonce":618315693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6813","nonce":618315693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6817","nonce":618315693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6815","nonce":618315693}]},"public_addr":"172.21.15.8:6811/618315693","cluster_addr":"172.21.15.8:6813/618315693","heartbeat_back_addr":"172.21.15.8:6817/618315693","heartbeat_front_addr":"172.21.15.8:6815/618315693","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6819","nonce":4265124490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6821","nonce":4265124490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6825","nonce":4265124490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6823","nonce":4265124490}]},"public_addr":"172.21.15.8:6819/4265124490","cluster_addr":"172.21.15.8:6821/4265124490","heartbeat_back_addr":"172.21.15.8:6825/4265124490","heartbeat_front_addr":"172.21.15.8:6823/4265124490","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6801","nonce":3418265960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6803","nonce":3418265960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6807","nonce":3418265960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6805","nonce":3418265960}]},"public_addr":"172.21.15.194:6801/3418265960","cluster_addr":"172.21.15.194:6803/3418265960","heartbeat_back_addr":"172.21.15.194:6807/3418265960","heartbeat_front_addr":"172.21.15.194:6805/3418265960","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6809","nonce":4144829010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6811","nonce":4144829010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6815","nonce":4144829010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6813","nonce":4144829010}]},"public_addr":"172.21.15.194:6809/4144829010","cluster_addr":"172.21.15.194:6811/4144829010","heartbeat_back_addr":"172.21.15.194:6815/4144829010","heartbeat_front_addr":"172.21.15.194:6813/4144829010","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6817","nonce":824210471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6819","nonce":824210471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6823","nonce":824210471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6821","nonce":824210471}]},"public_addr":"172.21.15.194:6817/824210471","cluster_addr":"172.21.15.194:6819/824210471","heartbeat_back_addr":"172.21.15.194:6823/824210471","heartbeat_front_addr":"172.21.15.194:6821/824210471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 msgr2=0x7f186806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 0x7f186806e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1870005950 tx=0x7f1870005cf0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 msgr2=0x7f188019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f187400be20 comp rx=0 tx=0).stop 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 shutdown_connections 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18800ffb40 0x7f188019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f186806c4e0 0x7f186806e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 --2- 172.21.15.8:0/1303127048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1880105bf0 0x7f188019ca30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 >> 172.21.15.8:0/1303127048 conn(0x7f18800fb610 msgr2=0x7f1880068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 shutdown_connections 2024-04-12T05:51:23.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:23.781+0000 7f1886381700 1 -- 172.21.15.8:0/1303127048 wait complete. 2024-04-12T05:51:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi194 2024-04-12T05:51:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi194 to 4982M 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:51:23.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:23 smithi194 ceph-mon[49771]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:24.617 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T05:49:53.787041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T05:51:24.617 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-12T05:51:24.854 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:24.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1303127048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:25.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:24 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1303127048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:26.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:25 smithi194 ceph-mon[49771]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:26.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:25 smithi008 ceph-mon[39722]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 -- 172.21.15.8:0/2956870307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 msgr2=0x7f7a4c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 --2- 172.21.15.8:0/2956870307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c1070b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009910 tx=0x7f7a48009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 -- 172.21.15.8:0/2956870307 shutdown_connections 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 --2- 172.21.15.8:0/2956870307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 0x7f7a4c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 --2- 172.21.15.8:0/2956870307 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c1070b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 -- 172.21.15.8:0/2956870307 >> 172.21.15.8:0/2956870307 conn(0x7f7a4c075670 msgr2=0x7f7a4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 -- 172.21.15.8:0/2956870307 shutdown_connections 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.852+0000 7f7a540ba700 1 -- 172.21.15.8:0/2956870307 wait complete. 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 Processor -- start 2024-04-12T05:51:26.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 -- start start 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 0x7f7a4c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c19d250 con 0x7f7a4c106cd0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a540ba700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c196700 con 0x7f7a4c068580 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a51655700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a51655700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58328/0 (socket says 172.21.15.8:58328) 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.853+0000 7f7a51655700 1 -- 172.21.15.8:0/3402311584 learned_addr learned my addr 172.21.15.8:0/3402311584 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a51e56700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 0x7f7a4c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a51655700 1 -- 172.21.15.8:0/3402311584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 msgr2=0x7f7a4c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a51655700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 0x7f7a4c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a51655700 1 -- 172.21.15.8:0/3402311584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480095f0 con 0x7f7a4c106cd0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a51655700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b4b0 tx=0x7f7a3c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3c011800 con 0x7f7a4c106cd0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c1969e0 con 0x7f7a4c106cd0 2024-04-12T05:51:26.851 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7a3c011df0 con 0x7f7a4c106cd0 2024-04-12T05:51:26.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3c010e40 con 0x7f7a4c106cd0 2024-04-12T05:51:26.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.854+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c196f30 con 0x7f7a4c106cd0 2024-04-12T05:51:26.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.855+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c1091d0 con 0x7f7a4c106cd0 2024-04-12T05:51:26.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.856+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f7a3c010380 con 0x7f7a4c106cd0 2024-04-12T05:51:26.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.856+0000 7f7a42ffd700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 0x7f7a3806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:26.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.856+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7a3c08b0e0 con 0x7f7a4c106cd0 2024-04-12T05:51:26.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.856+0000 7f7a51e56700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 0x7f7a3806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:26.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.856+0000 7f7a51e56700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 0x7f7a3806e950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006010 tx=0x7f7a4801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:26.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:26.860+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a3c059690 con 0x7f7a4c106cd0 2024-04-12T05:51:27.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.054+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a4c1971b0 con 0x7f7a4c106cd0 2024-04-12T05:51:27.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.055+0000 7f7a42ffd700 1 -- 172.21.15.8:0/3402311584 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a3c089e60 con 0x7f7a4c106cd0 2024-04-12T05:51:27.051 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-04-12T05:51:27.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 msgr2=0x7f7a3806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 0x7f7a3806e950 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a48006010 tx=0x7f7a4801a040 comp rx=0 tx=0).stop 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 msgr2=0x7f7a4c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00b4b0 tx=0x7f7a3c00b870 comp rx=0 tx=0).stop 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 shutdown_connections 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c068580 0x7f7a4c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7a3806c490 0x7f7a3806e950 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 --2- 172.21.15.8:0/3402311584 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c106cd0 0x7f7a4c19cbc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 >> 172.21.15.8:0/3402311584 conn(0x7f7a4c075670 msgr2=0x7f7a4c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 shutdown_connections 2024-04-12T05:51:27.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:27.057+0000 7f7a540ba700 1 -- 172.21.15.8:0/3402311584 wait complete. 2024-04-12T05:51:27.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:27 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3402311584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T05:51:27.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:27 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3402311584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T05:51:27.917 INFO:tasks.cephadm:Setting up client nodes... 2024-04-12T05:51:27.917 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-12T05:51:28.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:28 smithi008 ceph-mon[39722]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:28.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:28 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:28.157 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:28.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:28 smithi194 ceph-mon[49771]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:28.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:28 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- 172.21.15.8:0/748612772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 msgr2=0x7f9690101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 --2- 172.21.15.8:0/748612772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690101270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9684009910 tx=0x7f9684009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- 172.21.15.8:0/748612772 shutdown_connections 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 --2- 172.21.15.8:0/748612772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690101270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 --2- 172.21.15.8:0/748612772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96900ffc10 0x7f9690100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- 172.21.15.8:0/748612772 >> 172.21.15.8:0/748612772 conn(0x7f96900fb190 msgr2=0x7f96900fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- 172.21.15.8:0/748612772 shutdown_connections 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- 172.21.15.8:0/748612772 wait complete. 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 Processor -- start 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.244+0000 7f9695b65700 1 -- start start 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96900ffc10 0x7f9690195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9690195fb0 con 0x7f9690100e10 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96901960f0 con 0x7f96900ffc10 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58346/0 (socket says 172.21.15.8:58346) 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 -- 172.21.15.8:0/542765802 learned_addr learned my addr 172.21.15.8:0/542765802 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:30.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 -- 172.21.15.8:0/542765802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96900ffc10 msgr2=0x7f9690195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96900ffc10 0x7f9690195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 -- 172.21.15.8:0/542765802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840095f0 con 0x7f9690100e10 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968effd700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f968400bd70 tx=0x7f968400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968401d070 con 0x7f9690100e10 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9684005940 con 0x7f9690100e10 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969019aaf0 con 0x7f9690100e10 2024-04-12T05:51:30.243 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f968400f460 con 0x7f9690100e10 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.245+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969019afe0 con 0x7f9690100e10 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.246+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9690108660 con 0x7f9690100e10 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.247+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f9684022800 con 0x7f9690100e10 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.247+0000 7f968cff9700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 0x7f967806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.247+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f968408ca60 con 0x7f9690100e10 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.247+0000 7f968f7fe700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 0x7f967806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:30.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.248+0000 7f968f7fe700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 0x7f967806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9690100c70 tx=0x7f9680006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:30.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.252+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f968405b0c0 con 0x7f9690100e10 2024-04-12T05:51:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.536+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f969019b750 con 0x7f9690100e10 2024-04-12T05:51:30.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.539+0000 7f968cff9700 1 -- 172.21.15.8:0/542765802 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f968405ac50 con 0x7f9690100e10 2024-04-12T05:51:30.535 INFO:teuthology.orchestra.run.smithi008.stdout:[client.0] 2024-04-12T05:51:30.535 INFO:teuthology.orchestra.run.smithi008.stdout: key = AQDiyxhmtscMIBAAUPHaoX2+VPhZMmFTHBm5xA== 2024-04-12T05:51:30.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.541+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 msgr2=0x7f967806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:30.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.541+0000 7f9695b65700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 0x7f967806e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9690100c70 tx=0x7f9680006d20 comp rx=0 tx=0).stop 2024-04-12T05:51:30.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.541+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 msgr2=0x7f9690195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:30.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.541+0000 7f9695b65700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f968400bd70 tx=0x7f968400be50 comp rx=0 tx=0).stop 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 shutdown_connections 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f96900ffc10 0x7f9690195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f967806c4e0 0x7f967806e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 --2- 172.21.15.8:0/542765802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9690100e10 0x7f9690195990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 >> 172.21.15.8:0/542765802 conn(0x7f96900fb190 msgr2=0x7f9690104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 shutdown_connections 2024-04-12T05:51:30.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:30.542+0000 7f9695b65700 1 -- 172.21.15.8:0/542765802 wait complete. 2024-04-12T05:51:30.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:30 smithi008 ceph-mon[39722]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:30 smithi194 ceph-mon[49771]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:31.305 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:51:31.305 DEBUG:teuthology.orchestra.run.smithi008:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-12T05:51:31.305 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-12T05:51:31.352 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-12T05:51:31.591 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T05:51:31.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:31 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/542765802' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:31.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:31 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/542765802' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T05:51:31.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:31 smithi008 ceph-mon[39722]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:31 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/542765802' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:31 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/542765802' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T05:51:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:31 smithi194 ceph-mon[49771]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:33.604 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 -- 172.21.15.194:0/1929435721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341017a0 msgr2=0x7f3134103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:33.604 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 --2- 172.21.15.194:0/1929435721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341017a0 0x7f3134103b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3124009910 tx=0x7f3124009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:33.604 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 -- 172.21.15.194:0/1929435721 shutdown_connections 2024-04-12T05:51:33.604 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 --2- 172.21.15.194:0/1929435721 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31341040d0 0x7f31341064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 --2- 172.21.15.194:0/1929435721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341017a0 0x7f3134103b90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 -- 172.21.15.194:0/1929435721 >> 172.21.15.194:0/1929435721 conn(0x7f31340fb150 msgr2=0x7f31340fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 -- 172.21.15.194:0/1929435721 shutdown_connections 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.603+0000 7f3139bc1700 1 -- 172.21.15.194:0/1929435721 wait complete. 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 Processor -- start 2024-04-12T05:51:33.605 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 -- start start 2024-04-12T05:51:33.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31341017a0 0x7f31341963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:33.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:33.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134196f10 con 0x7f31341017a0 2024-04-12T05:51:33.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3139bc1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134197050 con 0x7f31341040d0 2024-04-12T05:51:33.606 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.604+0000 7f3132ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:33.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:47210/0 (socket says 172.21.15.194:47210) 2024-04-12T05:51:33.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 -- 172.21.15.194:0/3204676941 learned_addr learned my addr 172.21.15.194:0/3204676941 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-04-12T05:51:33.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 -- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31341017a0 msgr2=0x7f31341963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:33.607 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31341017a0 0x7f31341963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 -- 172.21.15.194:0/3204676941 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240095f0 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3132ffd700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f312800c8a0 tx=0x7f312800cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312800cea0 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3128004830 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313419bb00 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3128005540 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.605+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313419c050 con 0x7f31341040d0 2024-04-12T05:51:33.608 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.606+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31341096d0 con 0x7f31341040d0 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.607+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f31280049a0 con 0x7f31341040d0 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.607+0000 7f3130ff9700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 0x7f311c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.608+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f312808b5d0 con 0x7f31341040d0 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.608+0000 7f31337fe700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 0x7f311c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.608+0000 7f31337fe700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 0x7f311c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f312400b5c0 tx=0x7f3124005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:33.609 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.612+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3128059be0 con 0x7f31341040d0 2024-04-12T05:51:33.870 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.874+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f313419c6c0 con 0x7f31341040d0 2024-04-12T05:51:33.875 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.879+0000 7f3130ff9700 1 -- 172.21.15.194:0/3204676941 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f312801d0f0 con 0x7f31341040d0 2024-04-12T05:51:33.875 INFO:teuthology.orchestra.run.smithi194.stdout:[client.1] 2024-04-12T05:51:33.875 INFO:teuthology.orchestra.run.smithi194.stdout: key = AQDlyxhmkqdBNBAAHGchX4BQJSR7nDvPLWZ+GQ== 2024-04-12T05:51:33.877 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 msgr2=0x7f311c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:33.877 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 0x7f311c06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f312400b5c0 tx=0x7f3124005960 comp rx=0 tx=0).stop 2024-04-12T05:51:33.877 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 msgr2=0x7f31341968f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:33.877 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f312800c8a0 tx=0x7f312800cc60 comp rx=0 tx=0).stop 2024-04-12T05:51:33.877 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 shutdown_connections 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f311c06c4e0 0x7f311c06e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31341017a0 0x7f31341963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 --2- 172.21.15.194:0/3204676941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31341040d0 0x7f31341968f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 >> 172.21.15.194:0/3204676941 conn(0x7f31340fb150 msgr2=0x7f31340fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 shutdown_connections 2024-04-12T05:51:33.878 INFO:teuthology.orchestra.run.smithi194.stderr:2024-04-12T05:51:33.881+0000 7f3139bc1700 1 -- 172.21.15.194:0/3204676941 wait complete. 2024-04-12T05:51:34.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:34 smithi194 ceph-mon[49771]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:34.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:34 smithi194 ceph-mon[49771]: from='client.? 172.21.15.194:0/3204676941' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:34.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:34 smithi194 ceph-mon[49771]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:34.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:34 smithi194 ceph-mon[49771]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T05:51:34.622 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:51:34.622 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-12T05:51:34.622 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-12T05:51:34.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:34 smithi008 ceph-mon[39722]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:34 smithi008 ceph-mon[39722]: from='client.? 172.21.15.194:0/3204676941' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:34 smithi008 ceph-mon[39722]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-12T05:51:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:34 smithi008 ceph-mon[39722]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-12T05:51:34.662 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-12T05:51:34.662 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-12T05:51:34.662 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph mgr dump --format=json 2024-04-12T05:51:34.913 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:36.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:36 smithi008 ceph-mon[39722]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:36 smithi194 ceph-mon[49771]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:36.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 -- 172.21.15.8:0/1581544891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8101830 msgr2=0x7f87d8105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:36.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/1581544891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8101830 0x7f87d8105880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009910 tx=0x7f87cc009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:36.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 -- 172.21.15.8:0/1581544891 shutdown_connections 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/1581544891 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8101830 0x7f87d8105880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/1581544891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8100e80 0x7f87d8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 -- 172.21.15.8:0/1581544891 >> 172.21.15.8:0/1581544891 conn(0x7f87d80fc730 msgr2=0x7f87d80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 -- 172.21.15.8:0/1581544891 shutdown_connections 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 -- 172.21.15.8:0/1581544891 wait complete. 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.951+0000 7f87ddd0e700 1 Processor -- start 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87ddd0e700 1 -- start start 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87ddd0e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87ddd0e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 0x7f87d819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87ddd0e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d819f2d0 con 0x7f87d8100e80 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87ddd0e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8198810 con 0x7f87d8101830 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:36.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48858/0 (socket says 172.21.15.8:48858) 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 -- 172.21.15.8:0/923185464 learned_addr learned my addr 172.21.15.8:0/923185464 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d6ffd700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 0x7f87d819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 -- 172.21.15.8:0/923185464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 msgr2=0x7f87d819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 0x7f87d819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 -- 172.21.15.8:0/923185464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc0095f0 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d6ffd700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 0x7f87d819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.952+0000 7f87d77fe700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c930 tx=0x7f87c800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c8007ad0 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f87c8007c30 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d8198af0 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c8018630 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d8199040 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.953+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d8105000 con 0x7f87d8100e80 2024-04-12T05:51:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.954+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f87c800f450 con 0x7f87d8100e80 2024-04-12T05:51:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.954+0000 7f87d4ff9700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 0x7f87c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.955+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f87c808b720 con 0x7f87d8100e80 2024-04-12T05:51:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.955+0000 7f87d6ffd700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 0x7f87c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.955+0000 7f87d6ffd700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 0x7f87c006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87d80fcf70 tx=0x7f87cc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:36.959+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f87c8059e00 con 0x7f87d8100e80 2024-04-12T05:51:37.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.211+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f87d804f2e0 con 0x7f87d8100e80 2024-04-12T05:51:37.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.214+0000 7f87d4ff9700 1 -- 172.21.15.8:0/923185464 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173029 (secure 0 0 0) 0x7f87c800f700 con 0x7f87d8100e80 2024-04-12T05:51:37.213 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:51:37.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 msgr2=0x7f87c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:37.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 0x7f87c006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f87d80fcf70 tx=0x7f87cc01a040 comp rx=0 tx=0).stop 2024-04-12T05:51:37.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 msgr2=0x7f87d819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:37.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c930 tx=0x7f87c800ccf0 comp rx=0 tx=0).stop 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 shutdown_connections 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f87c006c4e0 0x7f87c006e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87d8100e80 0x7f87d819e790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 --2- 172.21.15.8:0/923185464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87d8101830 0x7f87d819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 >> 172.21.15.8:0/923185464 conn(0x7f87d80fc730 msgr2=0x7f87d80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 shutdown_connections 2024-04-12T05:51:37.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:37.217+0000 7f87ddd0e700 1 -- 172.21.15.8:0/923185464 wait complete. 2024-04-12T05:51:37.614 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:37 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/923185464' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-12T05:51:37.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:37 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/923185464' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-12T05:51:37.909 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi008.ixmnlp","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6800","nonce":2294526141},{"type":"v1","addr":"172.21.15.8:6801","nonce":2294526141}]},"active_addr":"172.21.15.8:6801/2294526141","active_change":"2024-04-12T05:47:27.259178+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14234,"name":"smithi194.xbiyaj","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.8:8443/","prometheus":"http://172.21.15.8:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3326205956}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":867529145}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":1577264968}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.8:0","nonce":3187654197}]}]} 2024-04-12T05:51:37.913 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-12T05:51:37.914 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-12T05:51:37.914 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd dump --format=json 2024-04-12T05:51:38.152 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:38 smithi008 ceph-mon[39722]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:38.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:38 smithi194 ceph-mon[49771]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1777173815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 msgr2=0x7f7f500fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1777173815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f500fffc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009910 tx=0x7f7f40009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1777173815 shutdown_connections 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1777173815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f500fffc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1777173815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 0x7f7f50105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1777173815 >> 172.21.15.8:0/1777173815 conn(0x7f7f500fb610 msgr2=0x7f7f500fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1777173815 shutdown_connections 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1777173815 wait complete. 2024-04-12T05:51:40.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 Processor -- start 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.134+0000 7f7f57ee7700 1 -- start start 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 0x7f7f5019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5019d0c0 con 0x7f7f500ffb40 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50196570 con 0x7f7f50105bf0 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48876/0 (socket says 172.21.15.8:48876) 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 -- 172.21.15.8:0/1697401397 learned_addr learned my addr 172.21.15.8:0/1697401397 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55482700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 0x7f7f5019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 -- 172.21.15.8:0/1697401397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 msgr2=0x7f7f5019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 0x7f7f5019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 -- 172.21.15.8:0/1697401397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f400095f0 con 0x7f7f500ffb40 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f55c83700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00d8d0 tx=0x7f7f4c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:40.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4c0098a0 con 0x7f7f500ffb40 2024-04-12T05:51:40.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7f4c009e90 con 0x7f7f500ffb40 2024-04-12T05:51:40.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4c00f5f0 con 0x7f7f500ffb40 2024-04-12T05:51:40.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f50196850 con 0x7f7f500ffb40 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.135+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f50196d70 con 0x7f7f500ffb40 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.136+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f500679c0 con 0x7f7f500ffb40 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.137+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f7f4c010460 con 0x7f7f500ffb40 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.137+0000 7f7f46ffd700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 0x7f7f3c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.138+0000 7f7f55482700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 0x7f7f3c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.138+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7f4c08b2d0 con 0x7f7f500ffb40 2024-04-12T05:51:40.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.138+0000 7f7f55482700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 0x7f7f3c06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000b5c0 tx=0x7f7f4001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:40.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.142+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7f4c059900 con 0x7f7f500ffb40 2024-04-12T05:51:40.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.339+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f50197110 con 0x7f7f500ffb40 2024-04-12T05:51:40.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.340+0000 7f7f46ffd700 1 -- 172.21.15.8:0/1697401397 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11262 (secure 0 0 0) 0x7f7f4c010710 con 0x7f7f500ffb40 2024-04-12T05:51:40.336 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:51:40.336 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T05:51:20.399794+0000","last_up_change":"2024-04-12T05:51:19.387464+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6803","nonce":106489579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6805","nonce":106489579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6809","nonce":106489579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6807","nonce":106489579}]},"public_addr":"172.21.15.8:6803/106489579","cluster_addr":"172.21.15.8:6805/106489579","heartbeat_back_addr":"172.21.15.8:6809/106489579","heartbeat_front_addr":"172.21.15.8:6807/106489579","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6811","nonce":618315693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6813","nonce":618315693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6817","nonce":618315693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6815","nonce":618315693}]},"public_addr":"172.21.15.8:6811/618315693","cluster_addr":"172.21.15.8:6813/618315693","heartbeat_back_addr":"172.21.15.8:6817/618315693","heartbeat_front_addr":"172.21.15.8:6815/618315693","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6819","nonce":4265124490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6821","nonce":4265124490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6825","nonce":4265124490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6823","nonce":4265124490}]},"public_addr":"172.21.15.8:6819/4265124490","cluster_addr":"172.21.15.8:6821/4265124490","heartbeat_back_addr":"172.21.15.8:6825/4265124490","heartbeat_front_addr":"172.21.15.8:6823/4265124490","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6801","nonce":3418265960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6803","nonce":3418265960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6807","nonce":3418265960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6805","nonce":3418265960}]},"public_addr":"172.21.15.194:6801/3418265960","cluster_addr":"172.21.15.194:6803/3418265960","heartbeat_back_addr":"172.21.15.194:6807/3418265960","heartbeat_front_addr":"172.21.15.194:6805/3418265960","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6809","nonce":4144829010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6811","nonce":4144829010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6815","nonce":4144829010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6813","nonce":4144829010}]},"public_addr":"172.21.15.194:6809/4144829010","cluster_addr":"172.21.15.194:6811/4144829010","heartbeat_back_addr":"172.21.15.194:6815/4144829010","heartbeat_front_addr":"172.21.15.194:6813/4144829010","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6817","nonce":824210471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6819","nonce":824210471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6823","nonce":824210471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6821","nonce":824210471}]},"public_addr":"172.21.15.194:6817/824210471","cluster_addr":"172.21.15.194:6819/824210471","heartbeat_back_addr":"172.21.15.194:6823/824210471","heartbeat_front_addr":"172.21.15.194:6821/824210471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T05:51:40.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 msgr2=0x7f7f3c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:40.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 0x7f7f3c06e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000b5c0 tx=0x7f7f4001a040 comp rx=0 tx=0).stop 2024-04-12T05:51:40.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 msgr2=0x7f7f5019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:40.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c00d8d0 tx=0x7f7f4c00dbe0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 shutdown_connections 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7f3c06c4e0 0x7f7f3c06e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7f500ffb40 0x7f7f5019c4f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 --2- 172.21.15.8:0/1697401397 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7f50105bf0 0x7f7f5019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 >> 172.21.15.8:0/1697401397 conn(0x7f7f500fb610 msgr2=0x7f7f50068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 shutdown_connections 2024-04-12T05:51:40.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:40.342+0000 7f7f57ee7700 1 -- 172.21.15.8:0/1697401397 wait complete. 2024-04-12T05:51:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:40 smithi008 ceph-mon[39722]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:40 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1697401397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:40 smithi194 ceph-mon[49771]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:40 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1697401397' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:41.097 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-12T05:51:41.098 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd dump --format=json 2024-04-12T05:51:41.336 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:41.819 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:41 smithi008 ceph-mon[39722]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:41.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:41 smithi194 ceph-mon[49771]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:42.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:43.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- 172.21.15.8:0/2862126156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 msgr2=0x7ff93c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:43.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 --2- 172.21.15.8:0/2862126156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c100080 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff928009910 tx=0x7ff928009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:43.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- 172.21.15.8:0/2862126156 shutdown_connections 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 --2- 172.21.15.8:0/2862126156 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c100080 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 --2- 172.21.15.8:0/2862126156 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff93c105bd0 0x7ff93c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- 172.21.15.8:0/2862126156 >> 172.21.15.8:0/2862126156 conn(0x7ff93c075670 msgr2=0x7ff93c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- 172.21.15.8:0/2862126156 shutdown_connections 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- 172.21.15.8:0/2862126156 wait complete. 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 Processor -- start 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.422+0000 7ff9430ed700 1 -- start start 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff9430ed700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff9430ed700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff93c105bd0 0x7ff93c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff9430ed700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c102350 con 0x7ff93c0ffc20 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff9430ed700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c1024c0 con 0x7ff93c105bd0 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43862/0 (socket says 172.21.15.8:43862) 2024-04-12T05:51:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 -- 172.21.15.8:0/4247668282 learned_addr learned my addr 172.21.15.8:0/4247668282 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 -- 172.21.15.8:0/4247668282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff93c105bd0 msgr2=0x7ff93c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff93c105bd0 0x7ff93c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 -- 172.21.15.8:0/4247668282 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9280095f0 con 0x7ff93c0ffc20 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff940e89700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b4b0 tx=0x7ff93000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff930011800 con 0x7ff93c0ffc20 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c1a2aa0 con 0x7ff93c0ffc20 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff930011df0 con 0x7ff93c0ffc20 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.423+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff930010e40 con 0x7ff93c0ffc20 2024-04-12T05:51:43.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.424+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c1a2f10 con 0x7ff93c0ffc20 2024-04-12T05:51:43.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.424+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c04f2e0 con 0x7ff93c0ffc20 2024-04-12T05:51:43.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.425+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7ff930010380 con 0x7ff93c0ffc20 2024-04-12T05:51:43.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.425+0000 7ff939ffb700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 0x7ff92c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:43.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.425+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff93008b0e0 con 0x7ff93c0ffc20 2024-04-12T05:51:43.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.425+0000 7ff93bfff700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 0x7ff92c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:43.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.426+0000 7ff93bfff700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 0x7ff92c06e950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff92800b5c0 tx=0x7ff928005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:43.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.430+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff930055c00 con 0x7ff93c0ffc20 2024-04-12T05:51:43.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.628+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff93c1a3160 con 0x7ff93c0ffc20 2024-04-12T05:51:43.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.629+0000 7ff939ffb700 1 -- 172.21.15.8:0/4247668282 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11262 (secure 0 0 0) 0x7ff930089e60 con 0x7ff93c0ffc20 2024-04-12T05:51:43.625 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:51:43.625 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T05:51:20.399794+0000","last_up_change":"2024-04-12T05:51:19.387464+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6803","nonce":106489579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6805","nonce":106489579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6809","nonce":106489579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6807","nonce":106489579}]},"public_addr":"172.21.15.8:6803/106489579","cluster_addr":"172.21.15.8:6805/106489579","heartbeat_back_addr":"172.21.15.8:6809/106489579","heartbeat_front_addr":"172.21.15.8:6807/106489579","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6811","nonce":618315693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6813","nonce":618315693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6817","nonce":618315693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6815","nonce":618315693}]},"public_addr":"172.21.15.8:6811/618315693","cluster_addr":"172.21.15.8:6813/618315693","heartbeat_back_addr":"172.21.15.8:6817/618315693","heartbeat_front_addr":"172.21.15.8:6815/618315693","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6819","nonce":4265124490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6821","nonce":4265124490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6825","nonce":4265124490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6823","nonce":4265124490}]},"public_addr":"172.21.15.8:6819/4265124490","cluster_addr":"172.21.15.8:6821/4265124490","heartbeat_back_addr":"172.21.15.8:6825/4265124490","heartbeat_front_addr":"172.21.15.8:6823/4265124490","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6801","nonce":3418265960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6803","nonce":3418265960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6807","nonce":3418265960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6805","nonce":3418265960}]},"public_addr":"172.21.15.194:6801/3418265960","cluster_addr":"172.21.15.194:6803/3418265960","heartbeat_back_addr":"172.21.15.194:6807/3418265960","heartbeat_front_addr":"172.21.15.194:6805/3418265960","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6809","nonce":4144829010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6811","nonce":4144829010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6815","nonce":4144829010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6813","nonce":4144829010}]},"public_addr":"172.21.15.194:6809/4144829010","cluster_addr":"172.21.15.194:6811/4144829010","heartbeat_back_addr":"172.21.15.194:6815/4144829010","heartbeat_front_addr":"172.21.15.194:6813/4144829010","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6817","nonce":824210471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6819","nonce":824210471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6823","nonce":824210471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6821","nonce":824210471}]},"public_addr":"172.21.15.194:6817/824210471","cluster_addr":"172.21.15.194:6819/824210471","heartbeat_back_addr":"172.21.15.194:6823/824210471","heartbeat_front_addr":"172.21.15.194:6821/824210471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 msgr2=0x7ff92c06e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 0x7ff92c06e950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff92800b5c0 tx=0x7ff928005960 comp rx=0 tx=0).stop 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 msgr2=0x7ff93c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff93000b4b0 tx=0x7ff93000b870 comp rx=0 tx=0).stop 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 shutdown_connections 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff92c06c490 0x7ff92c06e950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff93c0ffc20 0x7ff93c101840 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 --2- 172.21.15.8:0/4247668282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff93c105bd0 0x7ff93c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 >> 172.21.15.8:0/4247668282 conn(0x7ff93c075670 msgr2=0x7ff93c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 shutdown_connections 2024-04-12T05:51:43.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:43.631+0000 7ff9430ed700 1 -- 172.21.15.8:0/4247668282 wait complete. 2024-04-12T05:51:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:43 smithi194 ceph-mon[49771]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:43.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:43 smithi008 ceph-mon[39722]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:44.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-12T05:51:44.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-12T05:51:44.479 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-12T05:51:44.480 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-12T05:51:44.480 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-12T05:51:44.480 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-12T05:51:44.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:44 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/4247668282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:44.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:44 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/4247668282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:51:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:44.873 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:44.874 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:44.883 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:44.915 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:45.759 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:45 smithi008 ceph-mon[39722]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:45.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:45 smithi194 ceph-mon[49771]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:48.628 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:48 smithi008 ceph-mon[39722]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:48 smithi194 ceph-mon[49771]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 -- 172.21.15.8:0/1140932070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 msgr2=0x7f99fc10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 --2- 172.21.15.8:0/1140932070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc10abf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009910 tx=0x7f99f8009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 -- 172.21.15.8:0/1140932070 shutdown_connections 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 --2- 172.21.15.8:0/1140932070 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc10abf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 --2- 172.21.15.8:0/1140932070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 -- 172.21.15.8:0/1140932070 >> 172.21.15.8:0/1140932070 conn(0x7f99fc0fb610 msgr2=0x7f99fc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 -- 172.21.15.8:0/1140932070 shutdown_connections 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 -- 172.21.15.8:0/1140932070 wait complete. 2024-04-12T05:51:50.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.250+0000 7f9a03290700 1 Processor -- start 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 -- start start 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc19d290 con 0x7f99fc102700 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc196740 con 0x7f99fc101de0 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43884/0 (socket says 172.21.15.8:43884) 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 -- 172.21.15.8:0/2192401213 learned_addr learned my addr 172.21.15.8:0/2192401213 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 -- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 msgr2=0x7f99fc19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0102c700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f80095f0 con 0x7f99fc102700 2024-04-12T05:51:50.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0102c700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a0082b700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99f80092e0 tx=0x7f99f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f801d070 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f99f800f460 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99fc1969c0 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f8005720 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.251+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc196eb0 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.252+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99fc10e000 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.253+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f99f800f5d0 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.253+0000 7f99f27fc700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.253+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f99f808ce00 con 0x7f99fc102700 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.253+0000 7f9a0102c700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:50.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f99f27fc700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99e8074c90 con 0x7f99e80721c0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99f808d260 con 0x7f99fc102700 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f9a0182d700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f9a0102c700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 0x7f99e806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007900 tx=0x7f99ec008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.254+0000 7f9a0182d700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 0x7f99e80745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.255+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== osd.3 v2:172.21.15.194:6800/3418265960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f99e8074c90 con 0x7f99e80721c0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 --> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99fc0679c0 con 0x7f99e80721c0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f99f27fc700 1 -- 172.21.15.8:0/2192401213 <== osd.3 v2:172.21.15.194:6800/3418265960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f99fc0679c0 con 0x7f99e80721c0 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 msgr2=0x7f99e80745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 0x7f99e80745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 msgr2=0x7f99e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:50.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 0x7f99e806e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007900 tx=0x7f99ec008040 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 msgr2=0x7f99fc19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f99f80092e0 tx=0x7f99f800be20 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 shutdown_connections 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:6800/3418265960,v1:172.21.15.194:6801/3418265960] conn(0x7f99e80721c0 0x7f99e80745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99fc101de0 0x7f99fc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f99e806c4e0 0x7f99e806e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 --2- 172.21.15.8:0/2192401213 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f99fc102700 0x7f99fc19cc00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.271+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 >> 172.21.15.8:0/2192401213 conn(0x7f99fc0fb610 msgr2=0x7f99fc105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.272+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 shutdown_connections 2024-04-12T05:51:50.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.272+0000 7f9a03290700 1 -- 172.21.15.8:0/2192401213 wait complete. 2024-04-12T05:51:50.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:50 smithi008 ceph-mon[39722]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:50 smithi194 ceph-mon[49771]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:51.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 -- 172.21.15.8:0/134264068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 msgr2=0x7fd6e8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:51.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 --2- 172.21.15.8:0/134264068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e8105fb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009910 tx=0x7fd6d8009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 -- 172.21.15.8:0/134264068 shutdown_connections 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 --2- 172.21.15.8:0/134264068 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 0x7fd6e8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 --2- 172.21.15.8:0/134264068 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e8105fb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 -- 172.21.15.8:0/134264068 >> 172.21.15.8:0/134264068 conn(0x7fd6e8075670 msgr2=0x7fd6e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 -- 172.21.15.8:0/134264068 shutdown_connections 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.121+0000 7fd6ee558700 1 -- 172.21.15.8:0/134264068 wait complete. 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 Processor -- start 2024-04-12T05:51:51.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 -- start start 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 0x7fd6e819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e819d1f0 con 0x7fd6e8105bd0 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6ee558700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e81966a0 con 0x7fd6e80ffc20 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43878/0 (socket says 172.21.15.8:43878) 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6e77fe700 1 -- 172.21.15.8:0/3349876830 learned_addr learned my addr 172.21.15.8:0/3349876830 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:51.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.122+0000 7fd6e7fff700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 0x7fd6e819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e77fe700 1 -- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 msgr2=0x7fd6e819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e77fe700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 0x7fd6e819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e77fe700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d80095f0 con 0x7fd6e8105bd0 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e77fe700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c8f0 tx=0x7fd6dc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc004a00 con 0x7fd6e8105bd0 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e8196980 con 0x7fd6e8105bd0 2024-04-12T05:51:51.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd6dc005020 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6dc007c40 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.123+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e8196ed0 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.124+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6e810ce80 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.124+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fd6dc005190 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e57fa700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 0x7fd6d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd6dc013070 con 0x7fd6e8105bd0 2024-04-12T05:51:51.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e7fff700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 0x7fd6d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e57fa700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 0x7fd6d00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6d0074c90 con 0x7fd6d00721c0 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6dc08d7f0 con 0x7fd6e8105bd0 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6e7fff700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 0x7fd6d006e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8005f50 tx=0x7fd6d801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.125+0000 7fd6ecaf5700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 0x7fd6d00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:51.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.126+0000 7fd6ecaf5700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 0x7fd6d00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:51.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.126+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== osd.4 v2:172.21.15.194:6808/4144829010 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd6d0074c90 con 0x7fd6d00721c0 2024-04-12T05:51:51.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.142+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 --> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd6e804f2e0 con 0x7fd6d00721c0 2024-04-12T05:51:51.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6e57fa700 1 -- 172.21.15.8:0/3349876830 <== osd.4 v2:172.21.15.194:6808/4144829010 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd6e804f2e0 con 0x7fd6d00721c0 2024-04-12T05:51:51.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 msgr2=0x7fd6d00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 0x7fd6d00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 msgr2=0x7fd6d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 0x7fd6d006e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8005f50 tx=0x7fd6d801a040 comp rx=0 tx=0).stop 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 msgr2=0x7fd6e819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00c8f0 tx=0x7fd6dc00cc00 comp rx=0 tx=0).stop 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 shutdown_connections 2024-04-12T05:51:51.033 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:6808/4144829010,v1:172.21.15.194:6809/4144829010] conn(0x7fd6d00721c0 0x7fd6d00745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6e80ffc20 0x7fd6e819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd6d006c4e0 0x7fd6d006e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 --2- 172.21.15.8:0/3349876830 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6e8105bd0 0x7fd6e819cb60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 >> 172.21.15.8:0/3349876830 conn(0x7fd6e8075670 msgr2=0x7fd6e8068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 shutdown_connections 2024-04-12T05:51:51.034 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:50.143+0000 7fd6ee558700 1 -- 172.21.15.8:0/3349876830 wait complete. 2024-04-12T05:51:51.633 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:51 smithi008 ceph-mon[39722]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:51.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:51 smithi194 ceph-mon[49771]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:52.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- 172.21.15.8:0/3733216078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac105bd0 msgr2=0x7f25ac105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 --2- 172.21.15.8:0/3733216078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac105bd0 0x7f25ac105fb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009910 tx=0x7f25a8009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- 172.21.15.8:0/3733216078 shutdown_connections 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 --2- 172.21.15.8:0/3733216078 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ac0ffc20 0x7f25ac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 --2- 172.21.15.8:0/3733216078 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac105bd0 0x7f25ac105fb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- 172.21.15.8:0/3733216078 >> 172.21.15.8:0/3733216078 conn(0x7f25ac075670 msgr2=0x7f25ac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- 172.21.15.8:0/3733216078 shutdown_connections 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- 172.21.15.8:0/3733216078 wait complete. 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 Processor -- start 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.432+0000 7f25b387e700 1 -- start start 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ac105bd0 0x7f25ac19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:52.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac19d150 con 0x7f25ac0ffc20 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac196600 con 0x7f25ac105bd0 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43904/0 (socket says 172.21.15.8:43904) 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 -- 172.21.15.8:0/2424959485 learned_addr learned my addr 172.21.15.8:0/2424959485 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 -- 172.21.15.8:0/2424959485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ac105bd0 msgr2=0x7f25ac19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ac105bd0 0x7f25ac19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80095f0 con 0x7f25ac0ffc20 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b161a700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a801d070 con 0x7f25ac0ffc20 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac196880 con 0x7f25ac0ffc20 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f25a800f460 con 0x7f25ac0ffc20 2024-04-12T05:51:52.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a8021620 con 0x7f25ac0ffc20 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.433+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac196d70 con 0x7f25ac0ffc20 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.434+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25ac10ce80 con 0x7f25ac0ffc20 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f25a800fa40 con 0x7f25ac0ffc20 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 0x7f259806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f25a808c6e0 con 0x7f25ac0ffc20 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25b0e19700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 0x7f259806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 0x7f25980745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:52.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2598074c90 con 0x7f25980721c0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a808edb0 con 0x7f25ac0ffc20 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.435+0000 7f25b1e1b700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 0x7f25980745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.436+0000 7f25b0e19700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 0x7f259806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f259c005950 tx=0x7f259c0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.436+0000 7f25b1e1b700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 0x7f25980745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.436+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== osd.1 v2:172.21.15.8:6810/618315693 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2598074c90 con 0x7f25980721c0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 --> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25ac04f2e0 con 0x7f25980721c0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25a27fc700 1 -- 172.21.15.8:0/2424959485 <== osd.1 v2:172.21.15.8:6810/618315693 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25ac04f2e0 con 0x7f25980721c0 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 msgr2=0x7f25980745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 0x7f25980745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 msgr2=0x7f259806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 0x7f259806e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f259c005950 tx=0x7f259c0058e0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 msgr2=0x7f25ac19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:52.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25a8006010 tx=0x7f25a800be20 comp rx=0 tx=0).stop 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 shutdown_connections 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6810/618315693,v1:172.21.15.8:6811/618315693] conn(0x7f25980721c0 0x7f25980745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f259806c4e0 0x7f259806e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25ac0ffc20 0x7f25ac19c580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 --2- 172.21.15.8:0/2424959485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25ac105bd0 0x7f25ac19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.452+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 >> 172.21.15.8:0/2424959485 conn(0x7f25ac075670 msgr2=0x7f25ac068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.453+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 shutdown_connections 2024-04-12T05:51:52.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:52.453+0000 7f25b387e700 1 -- 172.21.15.8:0/2424959485 wait complete. 2024-04-12T05:51:53.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1520412412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee8100750 msgr2=0x7f0ee81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:53.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1520412412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee8100750 0x7f0ee81047a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009910 tx=0x7f0edc009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:53.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1520412412 shutdown_connections 2024-04-12T05:51:53.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1520412412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee8100750 0x7f0ee81047a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1520412412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee80ffda0 0x7f0ee8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1520412412 >> 172.21.15.8:0/1520412412 conn(0x7f0ee80fb650 msgr2=0x7f0ee80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1520412412 shutdown_connections 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.420+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1520412412 wait complete. 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.421+0000 7f0eedeb3700 1 Processor -- start 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.421+0000 7f0eedeb3700 1 -- start start 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee8100750 0x7f0ee819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819c130 con 0x7f0ee80ffda0 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee81955e0 con 0x7f0ee8100750 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:53.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59340/0 (socket says 172.21.15.8:59340) 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 -- 172.21.15.8:0/1847078807 learned_addr learned my addr 172.21.15.8:0/1847078807 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee6ffd700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee8100750 0x7f0ee819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 -- 172.21.15.8:0/1847078807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee8100750 msgr2=0x7f0ee819baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee8100750 0x7f0ee819baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc0095f0 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee77fe700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e920 tx=0x7f0ed800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed8009790 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0ed800cba0 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee81958c0 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ed8018650 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.422+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee8195e10 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.423+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ee810ce80 con 0x7f0ee80ffda0 2024-04-12T05:51:53.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.424+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f0ed800cd10 con 0x7f0ee80ffda0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee4ff9700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 0x7f0ed006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee6ffd700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 0x7f0ed006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0ed8014070 con 0x7f0ee80ffda0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee4ff9700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 0x7f0ed00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ed0074c90 con 0x7f0ed00721c0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ed808bfd0 con 0x7f0ee80ffda0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee7fff700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 0x7f0ed00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee6ffd700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 0x7f0ed006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80fbec0 tx=0x7f0edc009f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.425+0000 7f0ee7fff700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 0x7f0ed00745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:53.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.426+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== osd.2 v2:172.21.15.8:6818/4265124490 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ed0074c90 con 0x7f0ed00721c0 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 --> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ee804f2e0 con 0x7f0ed00721c0 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0ee4ff9700 1 -- 172.21.15.8:0/1847078807 <== osd.2 v2:172.21.15.8:6818/4265124490 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ee804f2e0 con 0x7f0ed00721c0 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 msgr2=0x7f0ed00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 0x7f0ed00745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 msgr2=0x7f0ed006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 0x7f0ed006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80fbec0 tx=0x7f0edc009f90 comp rx=0 tx=0).stop 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 msgr2=0x7f0ee819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:53.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e920 tx=0x7f0ed800ece0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 shutdown_connections 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6818/4265124490,v1:172.21.15.8:6819/4265124490] conn(0x7f0ed00721c0 0x7f0ed00745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0ed006c4e0 0x7f0ed006e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ee80ffda0 0x7f0ee819b560 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 --2- 172.21.15.8:0/1847078807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ee8100750 0x7f0ee819baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.442+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 >> 172.21.15.8:0/1847078807 conn(0x7f0ee80fb650 msgr2=0x7f0ee80fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.443+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 shutdown_connections 2024-04-12T05:51:53.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:53.443+0000 7f0eedeb3700 1 -- 172.21.15.8:0/1847078807 wait complete. 2024-04-12T05:51:54.377 INFO:teuthology.orchestra.run.smithi008.stdout:60129542176 2024-04-12T05:51:54.378 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-12T05:51:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:146028888079 2024-04-12T05:51:54.549 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-12T05:51:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:54 smithi008 ceph-mon[39722]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 -- 172.21.15.8:0/773647073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 msgr2=0x7ff2d4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 --2- 172.21.15.8:0/773647073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d4100080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009910 tx=0x7ff2c4009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 -- 172.21.15.8:0/773647073 shutdown_connections 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 --2- 172.21.15.8:0/773647073 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d4100080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 --2- 172.21.15.8:0/773647073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 0x7ff2d4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 -- 172.21.15.8:0/773647073 >> 172.21.15.8:0/773647073 conn(0x7ff2d4075670 msgr2=0x7ff2d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 -- 172.21.15.8:0/773647073 shutdown_connections 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.748+0000 7ff2d976a700 1 -- 172.21.15.8:0/773647073 wait complete. 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 Processor -- start 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 -- start start 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 0x7ff2d419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d419c0f0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d976a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d41955a0 con 0x7ff2d4105bd0 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59352/0 (socket says 172.21.15.8:59352) 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d2ffd700 1 -- 172.21.15.8:0/222917546 learned_addr learned my addr 172.21.15.8:0/222917546 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:54.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.749+0000 7ff2d27fc700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 0x7ff2d419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d2ffd700 1 -- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 msgr2=0x7ff2d419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d2ffd700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 0x7ff2d419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d2ffd700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40095f0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d2ffd700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c8a0 tx=0x7ff2bc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc0078e0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4195880 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff2bc00f450 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2bc00e5e0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.750+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d4195dd0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.751+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2d410ce80 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7ff2bc00f5c0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 0x7ff2c006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff2bc08b3f0 con 0x7ff2d40ffc20 2024-04-12T05:51:54.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2d27fc700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 0x7ff2c006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 0x7ff2c00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2c0074c90 con 0x7ff2c00721c0 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.752+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2bc08b850 con 0x7ff2d40ffc20 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.753+0000 7ff2d27fc700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 0x7ff2c006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40fe2d0 tx=0x7ff2c401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.753+0000 7ff2d37fe700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 0x7ff2c00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:54.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.753+0000 7ff2d37fe700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 0x7ff2c00745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.754+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== osd.5 v2:172.21.15.194:6816/824210471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff2c0074c90 con 0x7ff2c00721c0 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.769+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 --> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff2d404f2e0 con 0x7ff2c00721c0 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2cbfff700 1 -- 172.21.15.8:0/222917546 <== osd.5 v2:172.21.15.194:6816/824210471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff2d404f2e0 con 0x7ff2c00721c0 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 msgr2=0x7ff2c00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 0x7ff2c00745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 msgr2=0x7ff2c006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 0x7ff2c006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40fe2d0 tx=0x7ff2c401a040 comp rx=0 tx=0).stop 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 msgr2=0x7ff2d419b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:54.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc00c8a0 tx=0x7ff2bc00cbb0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 shutdown_connections 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7ff2c006c4e0 0x7ff2c006e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff2d40ffc20 0x7ff2d419b520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:6816/824210471,v1:172.21.15.194:6817/824210471] conn(0x7ff2c00721c0 0x7ff2c00745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 --2- 172.21.15.8:0/222917546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2d4105bd0 0x7ff2d419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.770+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 >> 172.21.15.8:0/222917546 conn(0x7ff2d4075670 msgr2=0x7ff2d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.771+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 shutdown_connections 2024-04-12T05:51:54.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:54.771+0000 7ff2d976a700 1 -- 172.21.15.8:0/222917546 wait complete. 2024-04-12T05:51:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:54 smithi194 ceph-mon[49771]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:55.405 INFO:teuthology.orchestra.run.smithi008.stdout:167503724554 2024-04-12T05:51:55.405 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-12T05:51:56.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:56 smithi008 ceph-mon[39722]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:56.788 INFO:teuthology.orchestra.run.smithi008.stdout:111669149717 2024-04-12T05:51:56.789 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-12T05:51:56.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:56 smithi194 ceph-mon[49771]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:57.185 INFO:teuthology.orchestra.run.smithi008.stdout:81604378652 2024-04-12T05:51:57.185 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-12T05:51:57.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:57.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4164609034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d00ffb40 msgr2=0x7f10d00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:57.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4164609034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d00ffb40 0x7f10d00fffc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009910 tx=0x7f10c0009c20 comp rx=0 tx=0).stop 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4164609034 shutdown_connections 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4164609034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d00ffb40 0x7f10d00fffc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4164609034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10d0105bf0 0x7f10d0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4164609034 >> 172.21.15.8:0/4164609034 conn(0x7f10d00fb610 msgr2=0x7f10d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4164609034 shutdown_connections 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4164609034 wait complete. 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 Processor -- start 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.644+0000 7f10d7d1b700 1 -- start start 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10d00ffb40 0x7f10d019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d019d0d0 con 0x7f10d0105bf0 2024-04-12T05:51:57.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0196580 con 0x7f10d00ffb40 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59374/0 (socket says 172.21.15.8:59374) 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 -- 172.21.15.8:0/4095409189 learned_addr learned my addr 172.21.15.8:0/4095409189 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 -- 172.21.15.8:0/4095409189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10d00ffb40 msgr2=0x7f10d019c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10d00ffb40 0x7f10d019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00095f0 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d52b6700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006010 tx=0x7f10c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10c001d070 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f10c000f460 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d0196800 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10c0021640 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.645+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d0196cf0 con 0x7f10d0105bf0 2024-04-12T05:51:57.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.646+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f10d010ceb0 con 0x7f10d0105bf0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f10c000fa20 con 0x7f10d0105bf0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 0x7f10bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f10c008c780 con 0x7f10d0105bf0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10d5ab7700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 0x7f10bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 0x7f10bc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10bc074c90 con 0x7f10bc0721c0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10c00912a0 con 0x7f10d0105bf0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.647+0000 7f10d62b8700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 0x7f10bc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.648+0000 7f10d5ab7700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 0x7f10bc06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00a990 tx=0x7f10cc005c80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.648+0000 7f10d62b8700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 0x7f10bc0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:51:57.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.648+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== osd.0 v2:172.21.15.8:6802/106489579 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f10bc074c90 con 0x7f10bc0721c0 2024-04-12T05:51:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.664+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 --> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10d00679c0 con 0x7f10bc0721c0 2024-04-12T05:51:57.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.664+0000 7f10c6ffd700 1 -- 172.21.15.8:0/4095409189 <== osd.0 v2:172.21.15.8:6802/106489579 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f10d00679c0 con 0x7f10bc0721c0 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 msgr2=0x7f10bc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 0x7f10bc0745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 msgr2=0x7f10bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 0x7f10bc06e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10cc00a990 tx=0x7f10cc005c80 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 msgr2=0x7f10d019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f10c0006010 tx=0x7f10c000be20 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 shutdown_connections 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6802/106489579,v1:172.21.15.8:6803/106489579] conn(0x7f10bc0721c0 0x7f10bc0745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10d00ffb40 0x7f10d019c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f10bc06c4e0 0x7f10bc06e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 --2- 172.21.15.8:0/4095409189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10d0105bf0 0x7f10d019ca40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:51:57.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 >> 172.21.15.8:0/4095409189 conn(0x7f10d00fb610 msgr2=0x7f10d0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:51:57.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 shutdown_connections 2024-04-12T05:51:57.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:51:57.665+0000 7f10d7d1b700 1 -- 172.21.15.8:0/4095409189 wait complete. 2024-04-12T05:51:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:51:58.464 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:58.465 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:58.471 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:58.478 INFO:teuthology.orchestra.run.smithi008.stdout:38654705704 2024-04-12T05:51:58.478 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-12T05:51:58.522 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:51:58.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:51:58 smithi008 ceph-mon[39722]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:51:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:51:58 smithi194 ceph-mon[49771]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:00.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:00 smithi194 ceph-mon[49771]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:00.641 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:00 smithi008 ceph-mon[39722]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:01.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:01 smithi008 ceph-mon[39722]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:01.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:01 smithi194 ceph-mon[49771]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:02.411 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:03.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 -- 172.21.15.8:0/2972459938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 msgr2=0x7f64e80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 --2- 172.21.15.8:0/2972459938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e80689e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009910 tx=0x7f64d4009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 -- 172.21.15.8:0/2972459938 shutdown_connections 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 --2- 172.21.15.8:0/2972459938 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e80689e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 --2- 172.21.15.8:0/2972459938 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 -- 172.21.15.8:0/2972459938 >> 172.21.15.8:0/2972459938 conn(0x7f64e8075670 msgr2=0x7f64e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 -- 172.21.15.8:0/2972459938 shutdown_connections 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.903+0000 7f64eedf4700 1 -- 172.21.15.8:0/2972459938 wait complete. 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 Processor -- start 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 -- start start 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:03.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e819d290 con 0x7f64e8068580 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64eedf4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e8196740 con 0x7f64e8106cd0 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35056/0 (socket says 172.21.15.8:35056) 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 -- 172.21.15.8:0/3550022072 learned_addr learned my addr 172.21.15.8:0/3550022072 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64e7fff700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 -- 172.21.15.8:0/3550022072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 msgr2=0x7f64e819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64ecb90700 1 -- 172.21.15.8:0/3550022072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40095f0 con 0x7f64e8068580 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.904+0000 7f64e7fff700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64ecb90700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b4b0 tx=0x7f64dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc011800 con 0x7f64e8068580 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f64dc011df0 con 0x7f64e8068580 2024-04-12T05:52:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64dc010e40 con 0x7f64e8068580 2024-04-12T05:52:03.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e8196a20 con 0x7f64e8068580 2024-04-12T05:52:03.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.905+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8196f70 con 0x7f64e8068580 2024-04-12T05:52:03.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.906+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e81091d0 con 0x7f64e8068580 2024-04-12T05:52:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.906+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f64dc010380 con 0x7f64e8068580 2024-04-12T05:52:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.907+0000 7f64e5ffb700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 0x7f64d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.907+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f64dc08aff0 con 0x7f64e8068580 2024-04-12T05:52:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.907+0000 7f64e7fff700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 0x7f64d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.907+0000 7f64e7fff700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 0x7f64d806e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b5c0 tx=0x7f64d4005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:03.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:03.911+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f64dc059620 con 0x7f64e8068580 2024-04-12T05:52:04.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.111+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f64e8197d50 con 0x7f64e8068580 2024-04-12T05:52:04.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.111+0000 7f64e5ffb700 1 -- 172.21.15.8:0/3550022072 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64e8197d50 con 0x7f64e8068580 2024-04-12T05:52:04.107 INFO:teuthology.orchestra.run.smithi008.stdout:146028888081 2024-04-12T05:52:04.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 msgr2=0x7f64d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:04.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 0x7f64d806e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b5c0 tx=0x7f64d4005960 comp rx=0 tx=0).stop 2024-04-12T05:52:04.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 msgr2=0x7f64e819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:04.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00b4b0 tx=0x7f64dc00b870 comp rx=0 tx=0).stop 2024-04-12T05:52:04.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 shutdown_connections 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f64d806c4e0 0x7f64d806e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e8068580 0x7f64e819c6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 --2- 172.21.15.8:0/3550022072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e8106cd0 0x7f64e819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 >> 172.21.15.8:0/3550022072 conn(0x7f64e8075670 msgr2=0x7f64e80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.113+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 shutdown_connections 2024-04-12T05:52:04.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.114+0000 7f64eedf4700 1 -- 172.21.15.8:0/3550022072 wait complete. 2024-04-12T05:52:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:04 smithi008 ceph-mon[39722]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:04 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3550022072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-12T05:52:04.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:04 smithi194 ceph-mon[49771]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:04.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:04 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3550022072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-12T05:52:04.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 -- 172.21.15.8:0/2684988742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 msgr2=0x7fdb9c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:04.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 --2- 172.21.15.8:0/2684988742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c1070b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009910 tx=0x7fdb8c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:04.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 -- 172.21.15.8:0/2684988742 shutdown_connections 2024-04-12T05:52:04.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 --2- 172.21.15.8:0/2684988742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 0x7fdb9c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 --2- 172.21.15.8:0/2684988742 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c1070b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 -- 172.21.15.8:0/2684988742 >> 172.21.15.8:0/2684988742 conn(0x7fdb9c075670 msgr2=0x7fdb9c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:04.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 -- 172.21.15.8:0/2684988742 shutdown_connections 2024-04-12T05:52:04.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.954+0000 7fdba0a84700 1 -- 172.21.15.8:0/2684988742 wait complete. 2024-04-12T05:52:04.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 Processor -- start 2024-04-12T05:52:04.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 -- start start 2024-04-12T05:52:04.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 0x7fdb9c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:04.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c19d250 con 0x7fdb9c106cd0 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdba0a84700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c196700 con 0x7fdb9c068580 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdb99d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdb99d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35076/0 (socket says 172.21.15.8:35076) 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.955+0000 7fdb99d9b700 1 -- 172.21.15.8:0/4199007025 learned_addr learned my addr 172.21.15.8:0/4199007025 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb9a59c700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 0x7fdb9c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb99d9b700 1 -- 172.21.15.8:0/4199007025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 msgr2=0x7fdb9c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb99d9b700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 0x7fdb9c19c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb99d9b700 1 -- 172.21.15.8:0/4199007025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0095f0 con 0x7fdb9c106cd0 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb99d9b700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b4b0 tx=0x7fdb8400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:04.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb84011800 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c1969e0 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdb84011df0 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb84010e40 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.956+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c196f30 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.957+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c1091d0 con 0x7fdb9c106cd0 2024-04-12T05:52:04.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.958+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fdb84010380 con 0x7fdb9c106cd0 2024-04-12T05:52:04.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.958+0000 7fdb937fe700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 0x7fdb8806e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:04.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.958+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdb8408b0e0 con 0x7fdb9c106cd0 2024-04-12T05:52:04.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.958+0000 7fdb9a59c700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 0x7fdb8806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:04.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.959+0000 7fdb9a59c700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 0x7fdb8806e940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006010 tx=0x7fdb8c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:04.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:04.963+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb84059690 con 0x7fdb9c106cd0 2024-04-12T05:52:05.029 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-04-12T05:52:05.029 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:05.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.166+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdb9c04f2e0 con 0x7fdb9c106cd0 2024-04-12T05:52:05.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.167+0000 7fdb937fe700 1 -- 172.21.15.8:0/4199007025 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdb84089e60 con 0x7fdb9c106cd0 2024-04-12T05:52:05.163 INFO:teuthology.orchestra.run.smithi008.stdout:60129542179 2024-04-12T05:52:05.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 msgr2=0x7fdb8806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 0x7fdb8806e940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006010 tx=0x7fdb8c01a040 comp rx=0 tx=0).stop 2024-04-12T05:52:05.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 msgr2=0x7fdb9c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400b4b0 tx=0x7fdb8400b870 comp rx=0 tx=0).stop 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 shutdown_connections 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb9c068580 0x7fdb9c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdb8806c480 0x7fdb8806e940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 --2- 172.21.15.8:0/4199007025 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdb9c106cd0 0x7fdb9c19cbc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 >> 172.21.15.8:0/4199007025 conn(0x7fdb9c075670 msgr2=0x7fdb9c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 shutdown_connections 2024-04-12T05:52:05.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.169+0000 7fdba0a84700 1 -- 172.21.15.8:0/4199007025 wait complete. 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- 172.21.15.8:0/970708562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758102700 msgr2=0x7fb75810abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 --2- 172.21.15.8:0/970708562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758102700 0x7fb75810abf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- 172.21.15.8:0/970708562 shutdown_connections 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 --2- 172.21.15.8:0/970708562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758102700 0x7fb75810abf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 --2- 172.21.15.8:0/970708562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758101de0 0x7fb7581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- 172.21.15.8:0/970708562 >> 172.21.15.8:0/970708562 conn(0x7fb7580fb610 msgr2=0x7fb7580fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- 172.21.15.8:0/970708562 shutdown_connections 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- 172.21.15.8:0/970708562 wait complete. 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 Processor -- start 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.253+0000 7fb75ee69700 1 -- start start 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75ee69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75ee69700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102700 0x7fb75819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75ee69700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75819d190 con 0x7fb758101de0 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75ee69700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb758196640 con 0x7fb758102700 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:05.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35088/0 (socket says 172.21.15.8:35088) 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 -- 172.21.15.8:0/2510436039 learned_addr learned my addr 172.21.15.8:0/2510436039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb757fff700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102700 0x7fb75819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 -- 172.21.15.8:0/2510436039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102700 msgr2=0x7fb75819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102700 0x7fb75819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 -- 172.21.15.8:0/2510436039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400095f0 con 0x7fb758101de0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75cc05700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c8a0 tx=0x7fb74800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb74800cea0 con 0x7fb758101de0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb758196920 con 0x7fb758101de0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb748004830 con 0x7fb758101de0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.254+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb748005540 con 0x7fb758101de0 2024-04-12T05:52:05.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.255+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb758196e70 con 0x7fb758101de0 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.255+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7581082f0 con 0x7fb758101de0 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.256+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fb7480049a0 con 0x7fb758101de0 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.256+0000 7fb755ffb700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 0x7fb74406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.257+0000 7fb757fff700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 0x7fb74406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.257+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb74808b240 con 0x7fb758101de0 2024-04-12T05:52:05.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.257+0000 7fb757fff700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 0x7fb74406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7580fcdf0 tx=0x7fb74001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:05.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.261+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb748059750 con 0x7fb758101de0 2024-04-12T05:52:05.270 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.469+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb7581970f0 con 0x7fb758101de0 2024-04-12T05:52:05.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.470+0000 7fb755ffb700 1 -- 172.21.15.8:0/2510436039 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7480592e0 con 0x7fb758101de0 2024-04-12T05:52:05.466 INFO:teuthology.orchestra.run.smithi008.stdout:111669149720 2024-04-12T05:52:05.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 msgr2=0x7fb74406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 0x7fb74406e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb7580fcdf0 tx=0x7fb74001a040 comp rx=0 tx=0).stop 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 msgr2=0x7fb75819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c8a0 tx=0x7fb74800cc60 comp rx=0 tx=0).stop 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 shutdown_connections 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb74406c4e0 0x7fb74406e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb758101de0 0x7fb75819c5c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 --2- 172.21.15.8:0/2510436039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb758102700 0x7fb75819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 >> 172.21.15.8:0/2510436039 conn(0x7fb7580fb610 msgr2=0x7fb758105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 shutdown_connections 2024-04-12T05:52:05.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:05.472+0000 7fb75ee69700 1 -- 172.21.15.8:0/2510436039 wait complete. 2024-04-12T05:52:05.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:05 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/4199007025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-12T05:52:05.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:05 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/4199007025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-12T05:52:06.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.062+0000 7fd3d274b700 1 -- 172.21.15.8:0/3025244229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc106cd0 msgr2=0x7fd3cc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.062+0000 7fd3d274b700 1 --2- 172.21.15.8:0/3025244229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc1070b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009910 tx=0x7fd3bc009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 -- 172.21.15.8:0/3025244229 shutdown_connections 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 --2- 172.21.15.8:0/3025244229 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc068580 0x7fd3cc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 --2- 172.21.15.8:0/3025244229 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc1070b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 -- 172.21.15.8:0/3025244229 >> 172.21.15.8:0/3025244229 conn(0x7fd3cc075670 msgr2=0x7fd3cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 -- 172.21.15.8:0/3025244229 shutdown_connections 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 -- 172.21.15.8:0/3025244229 wait complete. 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 Processor -- start 2024-04-12T05:52:06.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.063+0000 7fd3d274b700 1 -- start start 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc19d250 con 0x7fd3cc068580 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc196700 con 0x7fd3cc106cd0 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35104/0 (socket says 172.21.15.8:35104) 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 -- 172.21.15.8:0/81883661 learned_addr learned my addr 172.21.15.8:0/81883661 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cb7fe700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 -- 172.21.15.8:0/81883661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc106cd0 msgr2=0x7fd3cc19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 -- 172.21.15.8:0/81883661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc0095f0 con 0x7fd3cc068580 2024-04-12T05:52:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3cbfff700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc006010 tx=0x7fd3bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3bc01d070 con 0x7fd3cc068580 2024-04-12T05:52:06.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd3bc00f460 con 0x7fd3cc068580 2024-04-12T05:52:06.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc196980 con 0x7fd3cc068580 2024-04-12T05:52:06.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3bc021640 con 0x7fd3cc068580 2024-04-12T05:52:06.323 INFO:teuthology.orchestra.run.smithi008.stdout:167503724556 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.064+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc196e70 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.065+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc1091f0 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.066+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fd3bc00fa60 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.066+0000 7fd3c97fa700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 0x7fd3b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.067+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd3bc08cc80 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.067+0000 7fd3cb7fe700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 0x7fd3b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.067+0000 7fd3cb7fe700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 0x7fd3b406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0006fd0 tx=0x7fd3c0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.070+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd3bc05b210 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.269+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd3cc04f2e0 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.269+0000 7fd3c97fa700 1 -- 172.21.15.8:0/81883661 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3bc05ada0 con 0x7fd3cc068580 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 msgr2=0x7fd3b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:06.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 0x7fd3b406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0006fd0 tx=0x7fd3c0008040 comp rx=0 tx=0).stop 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 msgr2=0x7fd3cc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc006010 tx=0x7fd3bc00be20 comp rx=0 tx=0).stop 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 shutdown_connections 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd3b406c4e0 0x7fd3b406e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3cc068580 0x7fd3cc19c680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 --2- 172.21.15.8:0/81883661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3cc106cd0 0x7fd3cc19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 >> 172.21.15.8:0/81883661 conn(0x7fd3cc075670 msgr2=0x7fd3cc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 shutdown_connections 2024-04-12T05:52:06.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:06.271+0000 7fd3d274b700 1 -- 172.21.15.8:0/81883661 wait complete. 2024-04-12T05:52:06.622 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:06 smithi008 ceph-mon[39722]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:06.622 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:06 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2510436039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-12T05:52:06.622 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:06 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/81883661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-12T05:52:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:06 smithi194 ceph-mon[49771]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:06 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2510436039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-12T05:52:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:06 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/81883661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-12T05:52:07.463 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724556 for osd.5 2024-04-12T05:52:07.463 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:08.049 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542176 got 60129542179 for osd.1 2024-04-12T05:52:08.049 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:08.627 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:08 smithi008 ceph-mon[39722]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:08 smithi194 ceph-mon[49771]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:09.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 -- 172.21.15.8:0/3437410468 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0fff50 msgr2=0x7f50cc105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:09.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 --2- 172.21.15.8:0/3437410468 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0fff50 0x7f50cc105840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009910 tx=0x7f50c0009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:09.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 -- 172.21.15.8:0/3437410468 shutdown_connections 2024-04-12T05:52:09.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 --2- 172.21.15.8:0/3437410468 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0fff50 0x7f50cc105840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 --2- 172.21.15.8:0/3437410468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc0ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 -- 172.21.15.8:0/3437410468 >> 172.21.15.8:0/3437410468 conn(0x7f50cc074e30 msgr2=0x7f50cc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 -- 172.21.15.8:0/3437410468 shutdown_connections 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 -- 172.21.15.8:0/3437410468 wait complete. 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.578+0000 7f50d1616700 1 Processor -- start 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50d1616700 1 -- start start 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50d1616700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50d1616700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0fff50 0x7f50cc100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50d1616700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc100fa0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50d1616700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc1010e0 con 0x7f50cc0fff50 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35130/0 (socket says 172.21.15.8:35130) 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 -- 172.21.15.8:0/262972452 learned_addr learned my addr 172.21.15.8:0/262972452 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50ca7fc700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0fff50 0x7f50cc100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 -- 172.21.15.8:0/262972452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0fff50 msgr2=0x7f50cc100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0fff50 0x7f50cc100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.579+0000 7f50caffd700 1 -- 172.21.15.8:0/262972452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00095f0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50caffd700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d900 tx=0x7f50bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc0041f0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc1a2990 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f50bc004cc0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50bc005660 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.580+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc1a2da0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.581+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50cc109200 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.581+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f50bc004750 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.582+0000 7f50b3fff700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 0x7f50b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:09.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.582+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f50bc08b350 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.582+0000 7f50ca7fc700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 0x7f50b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:09.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.582+0000 7f50ca7fc700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 0x7f50b406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50cc101d10 tx=0x7f50c001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:09.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.587+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f50bc055e70 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.790+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50cc108be0 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.790+0000 7f50b3fff700 1 -- 172.21.15.8:0/262972452 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f50bc059490 con 0x7f50cc0ff5a0 2024-04-12T05:52:09.786 INFO:teuthology.orchestra.run.smithi008.stdout:81604378655 2024-04-12T05:52:09.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 msgr2=0x7f50b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 0x7f50b406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50cc101d10 tx=0x7f50c001a040 comp rx=0 tx=0).stop 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 msgr2=0x7f50cc100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00d900 tx=0x7f50bc00dc10 comp rx=0 tx=0).stop 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 shutdown_connections 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f50b406c4e0 0x7f50b406e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50cc0ff5a0 0x7f50cc100520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 --2- 172.21.15.8:0/262972452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50cc0fff50 0x7f50cc100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 >> 172.21.15.8:0/262972452 conn(0x7f50cc074e30 msgr2=0x7f50cc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 shutdown_connections 2024-04-12T05:52:09.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:09.792+0000 7f50d1616700 1 -- 172.21.15.8:0/262972452 wait complete. 2024-04-12T05:52:10.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149717 got 111669149720 for osd.3 2024-04-12T05:52:10.502 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:10 smithi194 ceph-mon[49771]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:10 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/262972452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-12T05:52:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:10 smithi008 ceph-mon[39722]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:10 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/262972452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-12T05:52:10.844 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378652 got 81604378655 for osd.2 2024-04-12T05:52:10.844 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:11.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.371+0000 7f7cd8c60700 1 -- 172.21.15.8:0/3862961499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 msgr2=0x7f7cd4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.371+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/3862961499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd4105fb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009910 tx=0x7f7cc4009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 -- 172.21.15.8:0/3862961499 shutdown_connections 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/3862961499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 0x7f7cd4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/3862961499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd4105fb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 -- 172.21.15.8:0/3862961499 >> 172.21.15.8:0/3862961499 conn(0x7f7cd4075670 msgr2=0x7f7cd4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 -- 172.21.15.8:0/3862961499 shutdown_connections 2024-04-12T05:52:11.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 -- 172.21.15.8:0/3862961499 wait complete. 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 Processor -- start 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.372+0000 7f7cd8c60700 1 -- start start 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 0x7f7cd419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd419d080 con 0x7f7cd4105bd0 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4196610 con 0x7f7cd40ffc20 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35158/0 (socket says 172.21.15.8:35158) 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 -- 172.21.15.8:0/2172479636 learned_addr learned my addr 172.21.15.8:0/2172479636 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd259c700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 0x7f7cd419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 -- 172.21.15.8:0/2172479636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 msgr2=0x7f7cd419c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 0x7f7cd419c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 -- 172.21.15.8:0/2172479636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40095f0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd1d9b700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d8d0 tx=0x7f7cc800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc80098a0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7cc8009e90 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd41968f0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cc800f5f0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.373+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd4196e40 con 0x7f7cd4105bd0 2024-04-12T05:52:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.374+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd404f2e0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.375+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f7cc8010460 con 0x7f7cd4105bd0 2024-04-12T05:52:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.375+0000 7f7cc37fe700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 0x7f7cbc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.375+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7cc808b3b0 con 0x7f7cd4105bd0 2024-04-12T05:52:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.375+0000 7f7cd259c700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 0x7f7cbc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.376+0000 7f7cd259c700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 0x7f7cbc06e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.380+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7cc8059960 con 0x7f7cd4105bd0 2024-04-12T05:52:11.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.581+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7cd4197180 con 0x7f7cd4105bd0 2024-04-12T05:52:11.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.581+0000 7f7cc37fe700 1 -- 172.21.15.8:0/2172479636 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cc8010710 con 0x7f7cd4105bd0 2024-04-12T05:52:11.577 INFO:teuthology.orchestra.run.smithi008.stdout:38654705707 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 msgr2=0x7f7cbc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 0x7f7cbc06e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4006010 tx=0x7f7cc401a040 comp rx=0 tx=0).stop 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 msgr2=0x7f7cd419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cc800d8d0 tx=0x7f7cc800dbe0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 shutdown_connections 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7cd40ffc20 0x7f7cd419c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7cbc06c4e0 0x7f7cbc06e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 --2- 172.21.15.8:0/2172479636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7cd4105bd0 0x7f7cd419ca60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 >> 172.21.15.8:0/2172479636 conn(0x7f7cd4075670 msgr2=0x7f7cd40686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 shutdown_connections 2024-04-12T05:52:11.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:11.583+0000 7f7cd8c60700 1 -- 172.21.15.8:0/2172479636 wait complete. 2024-04-12T05:52:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:11 smithi194 ceph-mon[49771]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:11 smithi008 ceph-mon[39722]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:12.232 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705704 got 38654705707 for osd.0 2024-04-12T05:52:12.232 DEBUG:teuthology.parallel:result is None 2024-04-12T05:52:12.233 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-12T05:52:12.233 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph pg dump --format=json 2024-04-12T05:52:12.475 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:12.754 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:12 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2172479636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-12T05:52:12.754 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:12 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2172479636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-12T05:52:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:13.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:13 smithi194 ceph-mon[49771]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:13.893 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:13 smithi008 ceph-mon[39722]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.534+0000 7fd868002700 1 -- 172.21.15.8:0/3023134613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8600ffc20 msgr2=0x7fd860100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.534+0000 7fd868002700 1 --2- 172.21.15.8:0/3023134613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8600ffc20 0x7fd860100080 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd854009910 tx=0x7fd854009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 -- 172.21.15.8:0/3023134613 shutdown_connections 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 --2- 172.21.15.8:0/3023134613 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd8600ffc20 0x7fd860100080 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 --2- 172.21.15.8:0/3023134613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd860105bd0 0x7fd860105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 -- 172.21.15.8:0/3023134613 >> 172.21.15.8:0/3023134613 conn(0x7fd860075670 msgr2=0x7fd860075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 -- 172.21.15.8:0/3023134613 shutdown_connections 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 -- 172.21.15.8:0/3023134613 wait complete. 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 Processor -- start 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.535+0000 7fd868002700 1 -- start start 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8600ffc20 0x7fd86019c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019d1c0 con 0x7fd860105bd0 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd860196670 con 0x7fd8600ffc20 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55818/0 (socket says 172.21.15.8:55818) 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 -- 172.21.15.8:0/2207626333 learned_addr learned my addr 172.21.15.8:0/2207626333 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd865d9e700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8600ffc20 0x7fd86019c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 -- 172.21.15.8:0/2207626333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8600ffc20 msgr2=0x7fd86019c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:14.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8600ffc20 0x7fd86019c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 -- 172.21.15.8:0/2207626333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540095f0 con 0x7fd860105bd0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd86559d700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd854006010 tx=0x7fd85400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85401d070 con 0x7fd860105bd0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8601968f0 con 0x7fd860105bd0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd854005710 con 0x7fd860105bd0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd85400fdc0 con 0x7fd860105bd0 2024-04-12T05:52:14.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.536+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd860196de0 con 0x7fd860105bd0 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.537+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86004f2e0 con 0x7fd860105bd0 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.538+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fd85400f5f0 con 0x7fd860105bd0 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.538+0000 7fd852ffd700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 0x7fd84c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.538+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd85408c780 con 0x7fd860105bd0 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.538+0000 7fd865d9e700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 0x7fd84c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:14.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.539+0000 7fd865d9e700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 0x7fd84c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005950 tx=0x7fd85c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:14.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.543+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd85405ac90 con 0x7fd860105bd0 2024-04-12T05:52:14.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.740+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd860197070 con 0x7fd84c06c4e0 2024-04-12T05:52:14.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.742+0000 7fd852ffd700 1 -- 172.21.15.8:0/2207626333 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28088 (secure 0 0 0) 0x7fd860197070 con 0x7fd84c06c4e0 2024-04-12T05:52:14.738 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:14.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 msgr2=0x7fd84c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 0x7fd84c06e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd85c005950 tx=0x7fd85c00b410 comp rx=0 tx=0).stop 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 msgr2=0x7fd86019cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd854006010 tx=0x7fd85400bd70 comp rx=0 tx=0).stop 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 shutdown_connections 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8600ffc20 0x7fd86019c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd84c06c4e0 0x7fd84c06e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 --2- 172.21.15.8:0/2207626333 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd860105bd0 0x7fd86019cb30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 >> 172.21.15.8:0/2207626333 conn(0x7fd860075670 msgr2=0x7fd860103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 shutdown_connections 2024-04-12T05:52:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:14.744+0000 7fd868002700 1 -- 172.21.15.8:0/2207626333 wait complete. 2024-04-12T05:52:14.741 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-04-12T05:52:15.558 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-12T05:52:13.311648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002388"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-12T05:51:19.390861+0000","last_change":"2024-04-12T05:50:49.360914+0000","last_active":"2024-04-12T05:51:19.390861+0000","last_peered":"2024-04-12T05:51:19.390861+0000","last_clean":"2024-04-12T05:51:19.390861+0000","last_became_active":"2024-04-12T05:50:49.360590+0000","last_became_peered":"2024-04-12T05:50:49.360590+0000","last_unstale":"2024-04-12T05:51:19.390861+0000","last_undegraded":"2024-04-12T05:51:19.390861+0000","last_fullsized":"2024-04-12T05:51:19.390861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T05:49:53.787408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T05:49:53.787408+0000","last_clean_scrub_stamp":"2024-04-12T05:49:53.787408+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T09:15:26.069909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.5},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.44}]},{"osd":3,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.379}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":3,"up_from":26,"seq":111669149722,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.41599999999999998}]},{"osd":1,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Apr 12 05:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.434}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378655,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Apr 12 05:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Apr 12 05:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.746},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Apr 12 05:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 12 05:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.34100000000000003,"15min":0.33600000000000002},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Apr 12 05:51:56 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.223}]},{"osd":3,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.437}]},{"osd":4,"last update":"Fri Apr 12 05:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46999999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.29399999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27500000000000002}]},{"osd":2,"last update":"Fri Apr 12 05:51:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.25600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38500000000000001,"15min":0.38800000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Fri Apr 12 05:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.497}]},{"osd":4,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T05:52:15.560 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph pg dump --format=json 2024-04-12T05:52:15.798 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:16 smithi008 ceph-mon[39722]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T05:52:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:16 smithi008 ceph-mon[39722]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:16 smithi194 ceph-mon[49771]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T05:52:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:16 smithi194 ceph-mon[49771]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 -- 172.21.15.8:0/1635189423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 msgr2=0x7f4f980689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 --2- 172.21.15.8:0/1635189423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f980689e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009910 tx=0x7f4f8c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 -- 172.21.15.8:0/1635189423 shutdown_connections 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 --2- 172.21.15.8:0/1635189423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f980689e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 --2- 172.21.15.8:0/1635189423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 0x7f4f981070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 -- 172.21.15.8:0/1635189423 >> 172.21.15.8:0/1635189423 conn(0x7f4f98075670 msgr2=0x7f4f98075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 -- 172.21.15.8:0/1635189423 shutdown_connections 2024-04-12T05:52:17.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.590+0000 7f4f9d161700 1 -- 172.21.15.8:0/1635189423 wait complete. 2024-04-12T05:52:17.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.591+0000 7f4f9d161700 1 Processor -- start 2024-04-12T05:52:17.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.591+0000 7f4f9d161700 1 -- start start 2024-04-12T05:52:17.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f9d161700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:17.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f9d161700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 0x7f4f9819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:17.588 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f9d161700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98198900 con 0x7f4f98068580 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f9d161700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f98198a70 con 0x7f4f98106cd0 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55840/0 (socket says 172.21.15.8:55840) 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 -- 172.21.15.8:0/2801639031 learned_addr learned my addr 172.21.15.8:0/2801639031 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f9659c700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 0x7f4f9819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 -- 172.21.15.8:0/2801639031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 msgr2=0x7f4f9819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 0x7f4f9819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 -- 172.21.15.8:0/2801639031 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c0095f0 con 0x7f4f98068580 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f96d9d700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4b0 tx=0x7f4f8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.592+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f88011800 con 0x7f4f98068580 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.593+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f98198d50 con 0x7f4f98068580 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.593+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4f88011df0 con 0x7f4f98068580 2024-04-12T05:52:17.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.593+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f88010e40 con 0x7f4f98068580 2024-04-12T05:52:17.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.593+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f981992a0 con 0x7f4f98068580 2024-04-12T05:52:17.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.593+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9804f2e0 con 0x7f4f98068580 2024-04-12T05:52:17.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.594+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f4f88010380 con 0x7f4f98068580 2024-04-12T05:52:17.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.594+0000 7f4f7ffff700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 0x7f4f8006e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:17.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.595+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4f8808b180 con 0x7f4f98068580 2024-04-12T05:52:17.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.595+0000 7f4f9659c700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 0x7f4f8006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:17.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.595+0000 7f4f9659c700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 0x7f4f8006e950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b5c0 tx=0x7f4f8c005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:17.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.599+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4f88055c00 con 0x7f4f98068580 2024-04-12T05:52:17.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.793+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f981995d0 con 0x7f4f8006c490 2024-04-12T05:52:17.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.794+0000 7f4f7ffff700 1 -- 172.21.15.8:0/2801639031 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28147 (secure 0 0 0) 0x7f4f981995d0 con 0x7f4f8006c490 2024-04-12T05:52:17.791 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.796+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 msgr2=0x7f4f8006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.796+0000 7f4f9d161700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 0x7f4f8006e950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00b5c0 tx=0x7f4f8c005fd0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.796+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 msgr2=0x7f4f9819e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.796+0000 7f4f9d161700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4b0 tx=0x7f4f8800b870 comp rx=0 tx=0).stop 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 shutdown_connections 2024-04-12T05:52:17.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f4f8006c490 0x7f4f8006e950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4f98068580 0x7f4f9819e910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 --2- 172.21.15.8:0/2801639031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f98106cd0 0x7f4f9819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 >> 172.21.15.8:0/2801639031 conn(0x7f4f98075670 msgr2=0x7f4f981002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 shutdown_connections 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:17.797+0000 7f4f9d161700 1 -- 172.21.15.8:0/2801639031 wait complete. 2024-04-12T05:52:17.793 INFO:teuthology.orchestra.run.smithi008.stderr:dumped all 2024-04-12T05:52:18.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:18 smithi008 ceph-mon[39722]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:18.622 INFO:teuthology.orchestra.run.smithi008.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-12T05:52:17.312453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002445"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-12T05:51:19.390861+0000","last_change":"2024-04-12T05:50:49.360914+0000","last_active":"2024-04-12T05:51:19.390861+0000","last_peered":"2024-04-12T05:51:19.390861+0000","last_clean":"2024-04-12T05:51:19.390861+0000","last_became_active":"2024-04-12T05:50:49.360590+0000","last_became_peered":"2024-04-12T05:50:49.360590+0000","last_unstale":"2024-04-12T05:51:19.390861+0000","last_undegraded":"2024-04-12T05:51:19.390861+0000","last_fullsized":"2024-04-12T05:51:19.390861+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-12T05:49:53.787408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-12T05:49:53.787408+0000","last_clean_scrub_stamp":"2024-04-12T05:49:53.787408+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-13T09:15:26.069909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724559,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]}]},{"osd":4,"up_from":34,"seq":146028888084,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.626},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.46300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149723,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.49199999999999999}]},{"osd":1,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.436},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Apr 12 05:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.35599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.35399999999999998,"15min":0.35299999999999998},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Fri Apr 12 05:51:53 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.22900000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Apr 12 05:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Apr 12 05:51:52 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 12 05:51:26 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34200000000000003,"15min":0.33800000000000002},"min":{"1min":0.22600000000000001,"5min":0.187,"15min":0.187},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.34100000000000003,"15min":0.33600000000000002},"min":{"1min":0.23300000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Fri Apr 12 05:51:56 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.49299999999999999,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39600000000000002,"15min":0.39700000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.223}]},{"osd":3,"last update":"Fri Apr 12 05:51:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.437}]},{"osd":4,"last update":"Fri Apr 12 05:51:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 12 05:51:27 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33100000000000002,"15min":0.32600000000000001},"min":{"1min":0.23799999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.46999999999999997,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34100000000000003},"min":{"1min":0.29399999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.27500000000000002}]},{"osd":2,"last update":"Fri Apr 12 05:51:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.36099999999999999},"min":{"1min":0.25600000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.47599999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38500000000000001,"15min":0.38800000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.47999999999999998,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.28799999999999998}]},{"osd":3,"last update":"Fri Apr 12 05:51:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.497}]},{"osd":4,"last update":"Fri Apr 12 05:51:49 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-12T05:52:18.623 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-12T05:52:18.623 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-12T05:52:18.623 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-12T05:52:18.623 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph health --format=json 2024-04-12T05:52:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:18 smithi194 ceph-mon[49771]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:18.863 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:19.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:19 smithi008 ceph-mon[39722]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T05:52:19.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:19 smithi194 ceph-mon[49771]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-12T05:52:20.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:20 smithi008 ceph-mon[39722]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:20 smithi194 ceph-mon[49771]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:20.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.694+0000 7f1c47926700 1 -- 172.21.15.8:0/209522091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 msgr2=0x7f1c401070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.694+0000 7f1c47926700 1 --2- 172.21.15.8:0/209522091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c401070b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009910 tx=0x7f1c3c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.694+0000 7f1c47926700 1 -- 172.21.15.8:0/209522091 shutdown_connections 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.694+0000 7f1c47926700 1 --2- 172.21.15.8:0/209522091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 0x7f1c400689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 --2- 172.21.15.8:0/209522091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c401070b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- 172.21.15.8:0/209522091 >> 172.21.15.8:0/209522091 conn(0x7f1c40075670 msgr2=0x7f1c40075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- 172.21.15.8:0/209522091 shutdown_connections 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- 172.21.15.8:0/209522091 wait complete. 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 Processor -- start 2024-04-12T05:52:20.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- start start 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 0x7f1c4019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4019f360 con 0x7f1c40106cd0 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.695+0000 7f1c47926700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40198810 con 0x7f1c40068580 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55856/0 (socket says 172.21.15.8:55856) 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 -- 172.21.15.8:0/3362575919 learned_addr learned my addr 172.21.15.8:0/3362575919 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c456c2700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 0x7f1c4019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 -- 172.21.15.8:0/3362575919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 msgr2=0x7f1c4019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 0x7f1c4019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 -- 172.21.15.8:0/3362575919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c0095f0 con 0x7f1c40106cd0 2024-04-12T05:52:20.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c44ec1700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000d8d0 tx=0x7f1c3000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c300098a0 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1c30009e90 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c40198af0 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c3000f5f0 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.696+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c40199040 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.697+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4004f2e0 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.698+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f1c30010460 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.698+0000 7f1c367fc700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 0x7f1c2c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.698+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1c3008b370 con 0x7f1c40106cd0 2024-04-12T05:52:20.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.698+0000 7f1c456c2700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 0x7f1c2c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:20.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.699+0000 7f1c456c2700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 0x7f1c2c06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:20.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.703+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1c30059900 con 0x7f1c40106cd0 2024-04-12T05:52:20.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.943+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1c40199d20 con 0x7f1c40106cd0 2024-04-12T05:52:20.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.943+0000 7f1c367fc700 1 -- 172.21.15.8:0/3362575919 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1c30010710 con 0x7f1c40106cd0 2024-04-12T05:52:20.939 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:20.939 INFO:teuthology.orchestra.run.smithi008.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-12T05:52:20.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 msgr2=0x7f1c2c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 0x7f1c2c06e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c006010 tx=0x7f1c3c01a040 comp rx=0 tx=0).stop 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 msgr2=0x7f1c4019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000d8d0 tx=0x7f1c3000dbe0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 shutdown_connections 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c40068580 0x7f1c4019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c2c06c4e0 0x7f1c2c06e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 --2- 172.21.15.8:0/3362575919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c40106cd0 0x7f1c4019ecd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 >> 172.21.15.8:0/3362575919 conn(0x7f1c40075670 msgr2=0x7f1c400feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 shutdown_connections 2024-04-12T05:52:20.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:20.945+0000 7f1c47926700 1 -- 172.21.15.8:0/3362575919 wait complete. 2024-04-12T05:52:21.586 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:21 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3362575919' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-12T05:52:21.586 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:21 smithi008 ceph-mon[39722]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:21.608 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-12T05:52:21.609 INFO:tasks.cephadm:Setup complete, yielding 2024-04-12T05:52:21.609 INFO:teuthology.run_tasks:Running task print... 2024-04-12T05:52:21.619 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-04-12T05:52:21.619 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:21.653 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:21.654 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-12T05:52:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:21 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3362575919' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-12T05:52:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:21 smithi194 ceph-mon[49771]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:21.894 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:23.118 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:22 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:22 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 -- 172.21.15.8:0/985076845 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81017a0 msgr2=0x7f62d8103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 --2- 172.21.15.8:0/985076845 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81017a0 0x7f62d8103b90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009910 tx=0x7f62c8009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 -- 172.21.15.8:0/985076845 shutdown_connections 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 --2- 172.21.15.8:0/985076845 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81040d0 0x7f62d81064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 --2- 172.21.15.8:0/985076845 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81017a0 0x7f62d8103b90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 -- 172.21.15.8:0/985076845 >> 172.21.15.8:0/985076845 conn(0x7f62d80fb190 msgr2=0x7f62d80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 -- 172.21.15.8:0/985076845 shutdown_connections 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.690+0000 7f62dffa0700 1 -- 172.21.15.8:0/985076845 wait complete. 2024-04-12T05:52:23.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 Processor -- start 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 -- start start 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 0x7f62d8196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8196fe0 con 0x7f62d81040d0 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dffa0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8197120 con 0x7f62d81017a0 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38112/0 (socket says 172.21.15.8:38112) 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 -- 172.21.15.8:0/2260535590 learned_addr learned my addr 172.21.15.8:0/2260535590 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62ddd3c700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 0x7f62d8196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 -- 172.21.15.8:0/2260535590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 msgr2=0x7f62d8196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 0x7f62d8196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62dd53b700 1 -- 172.21.15.8:0/2260535590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c80095f0 con 0x7f62d81040d0 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.691+0000 7f62ddd3c700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 0x7f62d8196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:52:23.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62dd53b700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b4b0 tx=0x7f62d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62d4011800 con 0x7f62d81040d0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d819bbd0 con 0x7f62d81040d0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f62d4011df0 con 0x7f62d81040d0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62d4010e40 con 0x7f62d81040d0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.692+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d819c120 con 0x7f62d81040d0 2024-04-12T05:52:23.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.693+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8109740 con 0x7f62d81040d0 2024-04-12T05:52:23.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.693+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f62d4010380 con 0x7f62d81040d0 2024-04-12T05:52:23.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.694+0000 7f62ceffd700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 0x7f62c406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:23.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.694+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f62d408b030 con 0x7f62d81040d0 2024-04-12T05:52:23.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.694+0000 7f62ddd3c700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 0x7f62c406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:23.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.694+0000 7f62ddd3c700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 0x7f62c406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62c80059d0 tx=0x7f62c8005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:23.693 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.698+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f62d4059660 con 0x7f62d81040d0 2024-04-12T05:52:23.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.896+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f62d8109900 con 0x7f62d81040d0 2024-04-12T05:52:23.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.899+0000 7f62ceffd700 1 -- 172.21.15.8:0/2260535590 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f62d40591f0 con 0x7f62d81040d0 2024-04-12T05:52:23.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 msgr2=0x7f62c406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:23.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 0x7f62c406e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f62c80059d0 tx=0x7f62c8005920 comp rx=0 tx=0).stop 2024-04-12T05:52:23.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 msgr2=0x7f62d81969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b4b0 tx=0x7f62d400b870 comp rx=0 tx=0).stop 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 shutdown_connections 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62d81017a0 0x7f62d8196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f62c406c4e0 0x7f62c406e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 --2- 172.21.15.8:0/2260535590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62d81040d0 0x7f62d81969c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 >> 172.21.15.8:0/2260535590 conn(0x7f62d80fb190 msgr2=0x7f62d80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 shutdown_connections 2024-04-12T05:52:23.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:23.902+0000 7f62dffa0700 1 -- 172.21.15.8:0/2260535590 wait complete. 2024-04-12T05:52:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:23 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:24.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:23 smithi008 ceph-mon[39722]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:23 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:24.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:23 smithi194 ceph-mon[49771]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:24.590 INFO:teuthology.run_tasks:Running task print... 2024-04-12T05:52:24.599 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-12T05:52:24.599 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:24.608 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:24.609 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph orch status' 2024-04-12T05:52:24.848 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:25.146 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2260535590' entity='client.admin' 2024-04-12T05:52:25.147 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:24 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:25.147 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:24 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:25.147 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:24 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:25.147 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:24 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:25.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:24 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2260535590' entity='client.admin' 2024-04-12T05:52:25.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:24 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:25.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:24 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:25.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:24 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:25.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:24 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:26.050 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:25 smithi008 ceph-mon[39722]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:26.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:25 smithi194 ceph-mon[49771]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 -- 172.21.15.8:0/1056568435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c1017f0 msgr2=0x7fde4c103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 --2- 172.21.15.8:0/1056568435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c1017f0 0x7fde4c103be0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fde40009910 tx=0x7fde40009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 -- 172.21.15.8:0/1056568435 shutdown_connections 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 --2- 172.21.15.8:0/1056568435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c104120 0x7fde4c106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 --2- 172.21.15.8:0/1056568435 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c1017f0 0x7fde4c103be0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 -- 172.21.15.8:0/1056568435 >> 172.21.15.8:0/1056568435 conn(0x7fde4c0fb160 msgr2=0x7fde4c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 -- 172.21.15.8:0/1056568435 shutdown_connections 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.618+0000 7fde531b0700 1 -- 172.21.15.8:0/1056568435 wait complete. 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 Processor -- start 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 -- start start 2024-04-12T05:52:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c1017f0 0x7fde4c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0ffce0 con 0x7fde4c104120 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde531b0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0ffe20 con 0x7fde4c1017f0 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38132/0 (socket says 172.21.15.8:38132) 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 -- 172.21.15.8:0/1989445675 learned_addr learned my addr 172.21.15.8:0/1989445675 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde50f4c700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c1017f0 0x7fde4c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 -- 172.21.15.8:0/1989445675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c1017f0 msgr2=0x7fde4c101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c1017f0 0x7fde4c101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 -- 172.21.15.8:0/1989445675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400095f0 con 0x7fde4c104120 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.619+0000 7fde4bfff700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde3800b4b0 tx=0x7fde3800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.620+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde38011800 con 0x7fde4c104120 2024-04-12T05:52:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.620+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fde38011df0 con 0x7fde4c104120 2024-04-12T05:52:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.620+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c100100 con 0x7fde4c104120 2024-04-12T05:52:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.620+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde38010e40 con 0x7fde4c104120 2024-04-12T05:52:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.620+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c19e870 con 0x7fde4c104120 2024-04-12T05:52:26.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.621+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c190580 con 0x7fde4c104120 2024-04-12T05:52:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.621+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fde38010380 con 0x7fde4c104120 2024-04-12T05:52:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.622+0000 7fde49ffb700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 0x7fde3c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.622+0000 7fde50f4c700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 0x7fde3c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.622+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fde3808aff0 con 0x7fde4c104120 2024-04-12T05:52:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.622+0000 7fde50f4c700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 0x7fde3c06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde400059d0 tx=0x7fde40005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:26.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.626+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fde38059620 con 0x7fde4c104120 2024-04-12T05:52:26.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.840+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde4c02cf50 con 0x7fde3c06c4e0 2024-04-12T05:52:26.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.841+0000 7fde49ffb700 1 -- 172.21.15.8:0/1989445675 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fde4c02cf50 con 0x7fde3c06c4e0 2024-04-12T05:52:26.836 INFO:teuthology.orchestra.run.smithi008.stdout:Backend: cephadm 2024-04-12T05:52:26.837 INFO:teuthology.orchestra.run.smithi008.stdout:Available: Yes 2024-04-12T05:52:26.837 INFO:teuthology.orchestra.run.smithi008.stdout:Paused: No 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 msgr2=0x7fde3c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 0x7fde3c06e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fde400059d0 tx=0x7fde40005920 comp rx=0 tx=0).stop 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 msgr2=0x7fde4c0ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde3800b4b0 tx=0x7fde3800b870 comp rx=0 tx=0).stop 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 shutdown_connections 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fde4c1017f0 0x7fde4c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fde3c06c4e0 0x7fde3c06e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 --2- 172.21.15.8:0/1989445675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fde4c104120 0x7fde4c0ff7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 >> 172.21.15.8:0/1989445675 conn(0x7fde4c0fb160 msgr2=0x7fde4c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 shutdown_connections 2024-04-12T05:52:26.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:26.843+0000 7fde531b0700 1 -- 172.21.15.8:0/1989445675 wait complete. 2024-04-12T05:52:27.607 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:27.631 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-12T05:52:27.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:27.871 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:28.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:28 smithi008 ceph-mon[39722]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:28.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:28 smithi008 ceph-mon[39722]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:28 smithi194 ceph-mon[49771]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:28 smithi194 ceph-mon[49771]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 -- 172.21.15.8:0/1365254062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 msgr2=0x7f1244102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 --2- 172.21.15.8:0/1365254062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244102310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1234009910 tx=0x7f1234009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:29.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 -- 172.21.15.8:0/1365254062 shutdown_connections 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 --2- 172.21.15.8:0/1365254062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244102310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 --2- 172.21.15.8:0/1365254062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 0x7f12441010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.777+0000 7f124ba55700 1 -- 172.21.15.8:0/1365254062 >> 172.21.15.8:0/1365254062 conn(0x7f12440fc230 msgr2=0x7f12440fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 -- 172.21.15.8:0/1365254062 shutdown_connections 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 -- 172.21.15.8:0/1365254062 wait complete. 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 Processor -- start 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 -- start start 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 0x7f12441964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244197030 con 0x7f1244101eb0 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.778+0000 7f124ba55700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244197170 con 0x7f1244100cb0 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:29.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38164/0 (socket says 172.21.15.8:38164) 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 -- 172.21.15.8:0/1395456411 learned_addr learned my addr 172.21.15.8:0/1395456411 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f12497f1700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 0x7f12441964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 -- 172.21.15.8:0/1395456411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 msgr2=0x7f12441964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 0x7f12441964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 -- 172.21.15.8:0/1395456411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340095f0 con 0x7f1244101eb0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f1248ff0700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1234006010 tx=0x7f123400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123401d070 con 0x7f1244101eb0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124419bbc0 con 0x7f1244101eb0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1234022470 con 0x7f1244101eb0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123400f690 con 0x7f1244101eb0 2024-04-12T05:52:29.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.779+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124419c0b0 con 0x7f1244101eb0 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.780+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244109770 con 0x7f1244101eb0 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.782+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f1234022980 con 0x7f1244101eb0 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.782+0000 7f123a7fc700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 0x7f123006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.782+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f123408d3c0 con 0x7f1244101eb0 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.782+0000 7f12497f1700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 0x7f123006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.782+0000 7f12497f1700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 0x7f123006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:29.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.786+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f123405b9f0 con 0x7f1244101eb0 2024-04-12T05:52:29.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:29.991+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1244106800 con 0x7f123006c4e0 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.000+0000 7f123a7fc700 1 -- 172.21.15.8:0/1395456411 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f1244106800 con 0x7f123006c4e0 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (4m) 2m ago 5m 26.6M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (6m) 2m ago 6m 7025k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (4m) 71s ago 4m 6945k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (5m) 2m ago 5m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (4m) 71s ago 4m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (3m) 2m ago 5m 78.2M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:52:29.996 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:9283,8765,8443 running (7m) 2m ago 7m 481M - 18.2.1 5be31c24972a 545bb7b78dec 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (4m) 71s ago 4m 441M - 18.2.1 5be31c24972a 324447c2f903 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (7m) 2m ago 7m 45.2M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (4m) 71s ago 4m 40.2M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (5m) 2m ago 5m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (4m) 71s ago 4m 17.9M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (3m) 2m ago 3m 41.5M 3959M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (3m) 2m ago 3m 40.5M 3959M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (2m) 2m ago 2m 39.5M 3959M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (2m) 71s ago 2m 40.7M 4982M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (107s) 71s ago 107s 41.2M 4982M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:52:29.997 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (77s) 71s ago 76s 10.8M 4982M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:52:29.998 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (3m) 2m ago 5m 39.5M - 2.43.0 a07b618ecd1d c0932eae2f67 2024-04-12T05:52:29.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 msgr2=0x7f123006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:29.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 0x7f123006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1240006fd0 tx=0x7f1240008040 comp rx=0 tx=0).stop 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 msgr2=0x7f1244196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1234006010 tx=0x7f123400be20 comp rx=0 tx=0).stop 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 shutdown_connections 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1244100cb0 0x7f12441964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f123006c4e0 0x7f123006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 --2- 172.21.15.8:0/1395456411 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1244101eb0 0x7f1244196a10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 >> 172.21.15.8:0/1395456411 conn(0x7f12440fc230 msgr2=0x7f12441050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 shutdown_connections 2024-04-12T05:52:29.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:30.002+0000 7f124ba55700 1 -- 172.21.15.8:0/1395456411 wait complete. 2024-04-12T05:52:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:30 smithi008 ceph-mon[39722]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:30.716 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-12T05:52:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:30 smithi194 ceph-mon[49771]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:30.951 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:31.463 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:31 smithi008 ceph-mon[39722]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:31 smithi194 ceph-mon[49771]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:32.594 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:32 smithi008 ceph-mon[39722]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:32.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:32 smithi194 ceph-mon[49771]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:32.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- 172.21.15.8:0/1865477631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 msgr2=0x7fe298106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/1865477631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe298106500 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe288009910 tx=0x7fe288009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- 172.21.15.8:0/1865477631 shutdown_connections 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/1865477631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe298106500 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/1865477631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- 172.21.15.8:0/1865477631 >> 172.21.15.8:0/1865477631 conn(0x7fe2980fb150 msgr2=0x7fe2980fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:32.898 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- 172.21.15.8:0/1865477631 shutdown_connections 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- 172.21.15.8:0/1865477631 wait complete. 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 Processor -- start 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 -- start start 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.896+0000 7fe2a02c2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe2a02c2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:32.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe2a02c2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298196ef0 con 0x7fe298104110 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe2a02c2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe298197030 con 0x7fe2981017e0 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38188/0 (socket says 172.21.15.8:38188) 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 -- 172.21.15.8:0/4049618097 learned_addr learned my addr 172.21.15.8:0/4049618097 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29e05e700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:32.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 -- 172.21.15.8:0/4049618097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 msgr2=0x7fe298196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 -- 172.21.15.8:0/4049618097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880095f0 con 0x7fe298104110 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29e05e700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe29d85d700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe288006010 tx=0x7fe28800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe28801d070 con 0x7fe298104110 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fe28800f460 con 0x7fe298104110 2024-04-12T05:52:32.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29819ba80 con 0x7fe298104110 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe288021640 con 0x7fe298104110 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.897+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29819bf70 con 0x7fe298104110 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.898+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2981096e0 con 0x7fe298104110 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.899+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fe28800f940 con 0x7fe298104110 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.899+0000 7fe28f7fe700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 0x7fe28406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:32.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.899+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe28808c730 con 0x7fe298104110 2024-04-12T05:52:32.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.899+0000 7fe29e05e700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 0x7fe28406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:32.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.900+0000 7fe29e05e700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 0x7fe28406e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe294005fd0 tx=0x7fe294005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:32.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:32.904+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe28805ad60 con 0x7fe298104110 2024-04-12T05:52:33.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.109+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe298104d80 con 0x7fe28406c4e0 2024-04-12T05:52:33.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.113+0000 7fe28f7fe700 1 -- 172.21.15.8:0/4049618097 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7fe298104d80 con 0x7fe28406c4e0 2024-04-12T05:52:33.109 INFO:teuthology.orchestra.run.smithi008.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-12T05:52:33.109 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-04-12T05:52:33.109 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter 2/2 2m ago 6m * 2024-04-12T05:52:33.109 INFO:teuthology.orchestra.run.smithi008.stdout:crash 2/2 2m ago 6m * 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:mgr 2/2 2m ago 6m count:2 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:mon 2/2 2m ago 5m smithi008:172.21.15.8=smithi008;smithi194:172.21.15.194=smithi194;count:2 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:osd 6 2m ago - 2024-04-12T05:52:33.110 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 msgr2=0x7fe28406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 0x7fe28406e9a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe294005fd0 tx=0x7fe294005ee0 comp rx=0 tx=0).stop 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 msgr2=0x7fe2981968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe288006010 tx=0x7fe28800be20 comp rx=0 tx=0).stop 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 shutdown_connections 2024-04-12T05:52:33.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe2981017e0 0x7fe298196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:33.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fe28406c4e0 0x7fe28406e9a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:33.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 --2- 172.21.15.8:0/4049618097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe298104110 0x7fe2981968d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:33.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 >> 172.21.15.8:0/4049618097 conn(0x7fe2980fb150 msgr2=0x7fe2980fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:33.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 shutdown_connections 2024-04-12T05:52:33.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:33.115+0000 7fe2a02c2700 1 -- 172.21.15.8:0/4049618097 wait complete. 2024-04-12T05:52:33.858 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-12T05:52:34.099 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:34 smithi008 ceph-mon[39722]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:34.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:34 smithi008 ceph-mon[39722]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:34 smithi194 ceph-mon[49771]: from='client.14446 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:34 smithi194 ceph-mon[49771]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:35.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 -- 172.21.15.8:0/4210428705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c100e10 msgr2=0x7fd37c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:35.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 --2- 172.21.15.8:0/4210428705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c100e10 0x7fd37c101270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd36c009910 tx=0x7fd36c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:35.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 -- 172.21.15.8:0/4210428705 shutdown_connections 2024-04-12T05:52:35.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 --2- 172.21.15.8:0/4210428705 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c100e10 0x7fd37c101270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 --2- 172.21.15.8:0/4210428705 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c0ffc10 0x7fd37c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 -- 172.21.15.8:0/4210428705 >> 172.21.15.8:0/4210428705 conn(0x7fd37c0fb190 msgr2=0x7fd37c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 -- 172.21.15.8:0/4210428705 shutdown_connections 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.905+0000 7fd383440700 1 -- 172.21.15.8:0/4210428705 wait complete. 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 Processor -- start 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 -- start start 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c100e10 0x7fd37c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c196f90 con 0x7fd37c0ffc10 2024-04-12T05:52:35.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd383440700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c19da60 con 0x7fd37c100e10 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38790/0 (socket says 172.21.15.8:38790) 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 -- 172.21.15.8:0/1620011291 learned_addr learned my addr 172.21.15.8:0/1620011291 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3809db700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c100e10 0x7fd37c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 -- 172.21.15.8:0/1620011291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c100e10 msgr2=0x7fd37c19d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:35.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c100e10 0x7fd37c19d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.906+0000 7fd3811dc700 1 -- 172.21.15.8:0/1620011291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0095f0 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd3811dc700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d930 tx=0x7fd37800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37800b920 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd378010460 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c19dd10 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd37800f5d0 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.907+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c19e260 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.908+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c1085c0 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.908+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fd3780105d0 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.909+0000 7fd3727fc700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 0x7fd36806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.909+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd37808b0c0 con 0x7fd37c0ffc10 2024-04-12T05:52:35.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.909+0000 7fd3809db700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 0x7fd36806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:35.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.909+0000 7fd3809db700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 0x7fd36806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd37c069660 tx=0x7fd36c00b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:35.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:35.913+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd378059670 con 0x7fd37c0ffc10 2024-04-12T05:52:36.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.116+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37c105760 con 0x7fd36806c4e0 2024-04-12T05:52:36.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.117+0000 7fd3727fc700 1 -- 172.21.15.8:0/1620011291 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7fd37c105760 con 0x7fd36806c4e0 2024-04-12T05:52:36.113 INFO:teuthology.orchestra.run.smithi008.stdout:HOST ADDR LABELS STATUS 2024-04-12T05:52:36.113 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 172.21.15.8 2024-04-12T05:52:36.113 INFO:teuthology.orchestra.run.smithi008.stdout:smithi194 172.21.15.194 2024-04-12T05:52:36.113 INFO:teuthology.orchestra.run.smithi008.stdout:2 hosts in cluster 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 msgr2=0x7fd36806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 0x7fd36806e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd37c069660 tx=0x7fd36c00b560 comp rx=0 tx=0).stop 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 msgr2=0x7fd37c0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d930 tx=0x7fd37800dc40 comp rx=0 tx=0).stop 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 shutdown_connections 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fd36806c4e0 0x7fd36806e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd37c0ffc10 0x7fd37c0690e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 --2- 172.21.15.8:0/1620011291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd37c100e10 0x7fd37c19d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 >> 172.21.15.8:0/1620011291 conn(0x7fd37c0fb190 msgr2=0x7fd37c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 shutdown_connections 2024-04-12T05:52:36.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:36.119+0000 7fd383440700 1 -- 172.21.15.8:0/1620011291 wait complete. 2024-04-12T05:52:36.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:36 smithi008 ceph-mon[39722]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:36 smithi194 ceph-mon[49771]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:36.880 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-12T05:52:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:37.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:37 smithi008 ceph-mon[39722]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:37.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:37 smithi194 ceph-mon[49771]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:38 smithi008 ceph-mon[39722]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:38.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:38 smithi194 ceph-mon[49771]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 -- 172.21.15.8:0/2236925540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 msgr2=0x7fc06c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 --2- 172.21.15.8:0/2236925540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c101270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009910 tx=0x7fc05c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 -- 172.21.15.8:0/2236925540 shutdown_connections 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 --2- 172.21.15.8:0/2236925540 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c101270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 --2- 172.21.15.8:0/2236925540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 0x7fc06c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 -- 172.21.15.8:0/2236925540 >> 172.21.15.8:0/2236925540 conn(0x7fc06c0fb190 msgr2=0x7fc06c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.055+0000 7fc073735700 1 -- 172.21.15.8:0/2236925540 shutdown_connections 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 -- 172.21.15.8:0/2236925540 wait complete. 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 Processor -- start 2024-04-12T05:52:39.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 -- start start 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 0x7fc06c103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c102730 con 0x7fc06c100e10 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc073735700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c102870 con 0x7fc06c0ffc10 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.056+0000 7fc070cd0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38818/0 (socket says 172.21.15.8:38818) 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 -- 172.21.15.8:0/94315026 learned_addr learned my addr 172.21.15.8:0/94315026 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc0714d1700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 0x7fc06c103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 -- 172.21.15.8:0/94315026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 msgr2=0x7fc06c103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 0x7fc06c103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 -- 172.21.15.8:0/94315026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0095f0 con 0x7fc06c100e10 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc070cd0700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00bd70 tx=0x7fc05c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc05c01d070 con 0x7fc06c100e10 2024-04-12T05:52:39.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc05c00f460 con 0x7fc06c100e10 2024-04-12T05:52:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc05c021640 con 0x7fc06c100e10 2024-04-12T05:52:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c102af0 con 0x7fc06c100e10 2024-04-12T05:52:39.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.057+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c102fe0 con 0x7fc06c100e10 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.058+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c108660 con 0x7fc06c100e10 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.059+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fc05c00f9b0 con 0x7fc06c100e10 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.059+0000 7fc0627fc700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 0x7fc05806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.059+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc05c08c6f0 con 0x7fc06c100e10 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.059+0000 7fc0714d1700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 0x7fc05806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:39.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.060+0000 7fc0714d1700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 0x7fc05806e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc06c100c70 tx=0x7fc068009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:39.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.064+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc05c05ad20 con 0x7fc06c100e10 2024-04-12T05:52:39.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.268+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc06c068c90 con 0x7fc05806c4e0 2024-04-12T05:52:39.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.269+0000 7fc0627fc700 1 -- 172.21.15.8:0/94315026 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fc06c068c90 con 0x7fc05806c4e0 2024-04-12T05:52:39.265 INFO:teuthology.orchestra.run.smithi008.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-12T05:52:39.265 INFO:teuthology.orchestra.run.smithi008.stdout:smithi008 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476004A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-12T05:52:39.265 INFO:teuthology.orchestra.run.smithi008.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 76s ago Has a FileSystem, LVM detected 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 msgr2=0x7fc05806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 0x7fc05806e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc06c100c70 tx=0x7fc068009190 comp rx=0 tx=0).stop 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 msgr2=0x7fc06c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00bd70 tx=0x7fc05c00bda0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 shutdown_connections 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc06c0ffc10 0x7fc06c103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc05806c4e0 0x7fc05806e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 --2- 172.21.15.8:0/94315026 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc06c100e10 0x7fc06c1021f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 >> 172.21.15.8:0/94315026 conn(0x7fc06c0fb190 msgr2=0x7fc06c068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 shutdown_connections 2024-04-12T05:52:39.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:39.271+0000 7fc073735700 1 -- 172.21.15.8:0/94315026 wait complete. 2024-04-12T05:52:39.975 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:39.986 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:39.986 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-12T05:52:40.223 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:40 smithi008 ceph-mon[39722]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:40 smithi008 ceph-mon[39722]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:40 smithi194 ceph-mon[49771]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:40 smithi194 ceph-mon[49771]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:41.637 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:41 smithi008 ceph-mon[39722]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:41.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:41 smithi194 ceph-mon[49771]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 -- 172.21.15.8:0/2604576781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c101800 msgr2=0x7fa39c103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 --2- 172.21.15.8:0/2604576781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c101800 0x7fa39c103bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa398009910 tx=0x7fa398009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 -- 172.21.15.8:0/2604576781 shutdown_connections 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 --2- 172.21.15.8:0/2604576781 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c104130 0x7fa39c106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 --2- 172.21.15.8:0/2604576781 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c101800 0x7fa39c103bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:41.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 -- 172.21.15.8:0/2604576781 >> 172.21.15.8:0/2604576781 conn(0x7fa39c0fb190 msgr2=0x7fa39c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:41.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 -- 172.21.15.8:0/2604576781 shutdown_connections 2024-04-12T05:52:41.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.892+0000 7fa3a3372700 1 -- 172.21.15.8:0/2604576781 wait complete. 2024-04-12T05:52:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 Processor -- start 2024-04-12T05:52:41.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 -- start start 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c101800 0x7fa39c101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0ffcf0 con 0x7fa39c104130 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a3372700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0ffe30 con 0x7fa39c101800 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a090d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a090d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38842/0 (socket says 172.21.15.8:38842) 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.893+0000 7fa3a090d700 1 -- 172.21.15.8:0/3124862870 learned_addr learned my addr 172.21.15.8:0/3124862870 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a110e700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c101800 0x7fa39c101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a090d700 1 -- 172.21.15.8:0/3124862870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c101800 msgr2=0x7fa39c101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a090d700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c101800 0x7fa39c101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a090d700 1 -- 172.21.15.8:0/3124862870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980095f0 con 0x7fa39c104130 2024-04-12T05:52:41.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a090d700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d8d0 tx=0x7fa38c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c0098a0 con 0x7fa39c104130 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa38c009e90 con 0x7fa39c104130 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c100110 con 0x7fa39c104130 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c00f5f0 con 0x7fa39c104130 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.894+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c19e420 con 0x7fa39c104130 2024-04-12T05:52:41.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.895+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c1904d0 con 0x7fa39c104130 2024-04-12T05:52:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.896+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fa38c010460 con 0x7fa39c104130 2024-04-12T05:52:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.896+0000 7fa3927fc700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 0x7fa38806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.896+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa38c08b3b0 con 0x7fa39c104130 2024-04-12T05:52:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.896+0000 7fa3a110e700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 0x7fa38806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:41.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.896+0000 7fa3a110e700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 0x7fa38806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa398006010 tx=0x7fa39800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:41.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:41.900+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa38c059960 con 0x7fa39c104130 2024-04-12T05:52:42.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:42.126+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39c0611d0 con 0x7fa38806c4e0 2024-04-12T05:52:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:43 smithi194 ceph-mon[49771]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:43 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-12T05:52:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:43 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:43.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:43 smithi008 ceph-mon[39722]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:52:43.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-12T05:52:43.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:43 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:43.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.641+0000 7fa3927fc700 1 -- 172.21.15.8:0/3124862870 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa39c0611d0 con 0x7fa38806c4e0 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 msgr2=0x7fa38806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 0x7fa38806e9a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa398006010 tx=0x7fa39800b540 comp rx=0 tx=0).stop 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 msgr2=0x7fa39c0ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00d8d0 tx=0x7fa38c00dbe0 comp rx=0 tx=0).stop 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 shutdown_connections 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa39c101800 0x7fa39c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa38806c4e0 0x7fa38806e9a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 --2- 172.21.15.8:0/3124862870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa39c104130 0x7fa39c0ff7b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 >> 172.21.15.8:0/3124862870 conn(0x7fa39c0fb190 msgr2=0x7fa39c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:43.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 shutdown_connections 2024-04-12T05:52:43.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:43.643+0000 7fa3a3372700 1 -- 172.21.15.8:0/3124862870 wait complete. 2024-04-12T05:52:43.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:43 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[39699]: 2024-04-12T05:52:43.615+0000 7f8e58e90700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T05:52:44.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-12T05:52:44.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: osdmap e41: 6 total, 6 up, 6 in 2024-04-12T05:52:44.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-12T05:52:44.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: pgmap v187: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:44.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: osdmap e42: 6 total, 6 up, 6 in 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: osdmap e43: 6 total, 6 up, 6 in 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: fsmap cephfs:0 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:44.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:44.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:44 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: osdmap e41: 6 total, 6 up, 6 in 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: pgmap v187: 33 pgs: 10 creating+peering, 1 active+clean, 22 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: osdmap e42: 6 total, 6 up, 6 in 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: osdmap e43: 6 total, 6 up, 6 in 2024-04-12T05:52:44.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: fsmap cephfs:0 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:44.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:44 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:44.471 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T05:52:45.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:45 smithi194 ceph-mon[49771]: Saving service mds.cephfs spec with placement count:4 2024-04-12T05:52:45.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:45 smithi194 ceph-mon[49771]: Deploying daemon mds.cephfs.smithi008.rhdhsr on smithi008 2024-04-12T05:52:45.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:45 smithi194 ceph-mon[49771]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T05:52:45.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:45 smithi194 ceph-mon[49771]: osdmap e44: 6 total, 6 up, 6 in 2024-04-12T05:52:45.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:45 smithi008 ceph-mon[39722]: Saving service mds.cephfs spec with placement count:4 2024-04-12T05:52:45.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:45 smithi008 ceph-mon[39722]: Deploying daemon mds.cephfs.smithi008.rhdhsr on smithi008 2024-04-12T05:52:45.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:45 smithi008 ceph-mon[39722]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-12T05:52:45.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:45 smithi008 ceph-mon[39722]: osdmap e44: 6 total, 6 up, 6 in 2024-04-12T05:52:45.738 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:46.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:46 smithi194 ceph-mon[49771]: pgmap v191: 65 pgs: 1 creating+activating, 15 creating+peering, 6 active+clean, 43 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:46.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:46 smithi194 ceph-mon[49771]: osdmap e45: 6 total, 6 up, 6 in 2024-04-12T05:52:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:46 smithi008 ceph-mon[39722]: pgmap v191: 65 pgs: 1 creating+activating, 15 creating+peering, 6 active+clean, 43 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:46 smithi008 ceph-mon[39722]: osdmap e45: 6 total, 6 up, 6 in 2024-04-12T05:52:47.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:47 smithi194 ceph-mon[49771]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T05:52:47.391 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:47 smithi008 ceph-mon[39722]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-12T05:52:48.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1541292646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 msgr2=0x7fc6cc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1541292646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc101270 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009910 tx=0x7fc6bc009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1541292646 shutdown_connections 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1541292646 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc101270 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1541292646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 0x7fc6cc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1541292646 >> 172.21.15.8:0/1541292646 conn(0x7fc6cc0fb190 msgr2=0x7fc6cc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1541292646 shutdown_connections 2024-04-12T05:52:48.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.015+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1541292646 wait complete. 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 Processor -- start 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 -- start start 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 0x7fc6cc1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc195e20 con 0x7fc6cc100e10 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d3f92700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc195f60 con 0x7fc6cc0ffc10 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41156/0 (socket says 172.21.15.8:41156) 2024-04-12T05:52:48.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 -- 172.21.15.8:0/1276402375 learned_addr learned my addr 172.21.15.8:0/1276402375 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d1d2e700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 0x7fc6cc1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 -- 172.21.15.8:0/1276402375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 msgr2=0x7fc6cc1952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 0x7fc6cc1952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 -- 172.21.15.8:0/1276402375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0095f0 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.016+0000 7fc6d152d700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00bde0 tx=0x7fc6bc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.017+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc01f070 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.017+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc6bc005dc0 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.017+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc068580 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.017+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc003b10 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.017+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc068a10 con 0x7fc6cc100e10 2024-04-12T05:52:48.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.018+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc1085c0 con 0x7fc6cc100e10 2024-04-12T05:52:48.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.018+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fc6bc0048e0 con 0x7fc6cc100e10 2024-04-12T05:52:48.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.019+0000 7fc6c2ffd700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 0x7fc6b806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:48.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.019+0000 7fc6d1d2e700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 0x7fc6b806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:48.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.019+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fc6bc08f160 con 0x7fc6cc100e10 2024-04-12T05:52:48.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.019+0000 7fc6d1d2e700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 0x7fc6b806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc100c70 tx=0x7fc6c8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:48.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.024+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6bc05d3d0 con 0x7fc6cc100e10 2024-04-12T05:52:48.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.273+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc6cc1087b0 con 0x7fc6cc100e10 2024-04-12T05:52:48.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.274+0000 7fc6c2ffd700 1 -- 172.21.15.8:0/1276402375 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1262 (secure 0 0 0) 0x7fc6bc063030 con 0x7fc6cc100e10 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:e4 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 4 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T05:52:48.061713+0000 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T05:52:48.271 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 0 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 1 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:in 0 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14462} 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T05:52:48.272 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 0 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14462} state up:creating seq 1 addr [v2:172.21.15.8:6826/2911956822,v1:172.21.15.8:6827/2911956822] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:48.273 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 msgr2=0x7fc6b806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 0x7fc6b806e9a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc100c70 tx=0x7fc6c8009260 comp rx=0 tx=0).stop 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 msgr2=0x7fc6cc195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc00bde0 tx=0x7fc6bc00be10 comp rx=0 tx=0).stop 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 shutdown_connections 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc0ffc10 0x7fc6cc1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc6b806c4e0 0x7fc6b806e9a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 --2- 172.21.15.8:0/1276402375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc100e10 0x7fc6cc195800 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:48.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 >> 172.21.15.8:0/1276402375 conn(0x7fc6cc0fb190 msgr2=0x7fc6cc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:48.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 shutdown_connections 2024-04-12T05:52:48.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:48.276+0000 7fc6d3f92700 1 -- 172.21.15.8:0/1276402375 wait complete. 2024-04-12T05:52:48.275 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 4 2024-04-12T05:52:48.297 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: pgmap v193: 65 pgs: 1 creating+activating, 15 creating+peering, 34 active+clean, 15 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:48.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:48 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: pgmap v193: 65 pgs: 1 creating+activating, 15 creating+peering, 34 active+clean, 15 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:48 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:48.943 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:48.952 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:48.953 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-12T05:52:49.199 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:49.293 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: Deploying daemon mds.cephfs.smithi194.uzfpyu on smithi194 2024-04-12T05:52:49.293 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6826/2911956822,v1:172.21.15.8:6827/2911956822] up:boot 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: daemon mds.cephfs.smithi008.rhdhsr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: Cluster is now healthy 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: fsmap cephfs:0 1 up:standby 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:creating} 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: daemon mds.cephfs.smithi008.rhdhsr is now active in filesystem cephfs as rank 0 2024-04-12T05:52:49.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:49 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1276402375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:52:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: Deploying daemon mds.cephfs.smithi194.uzfpyu on smithi194 2024-04-12T05:52:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6826/2911956822,v1:172.21.15.8:6827/2911956822] up:boot 2024-04-12T05:52:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: daemon mds.cephfs.smithi008.rhdhsr assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: Cluster is now healthy 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: fsmap cephfs:0 1 up:standby 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:creating} 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: daemon mds.cephfs.smithi008.rhdhsr is now active in filesystem cephfs as rank 0 2024-04-12T05:52:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:49 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1276402375' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:52:50.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:50 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6826/2911956822,v1:172.21.15.8:6827/2911956822] up:active 2024-04-12T05:52:50.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:50 smithi008 ceph-mon[39722]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 2024-04-12T05:52:50.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:50 smithi008 ceph-mon[39722]: pgmap v194: 65 pgs: 1 creating+activating, 64 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 3 op/s 2024-04-12T05:52:50.317 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:50 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6826/2911956822,v1:172.21.15.8:6827/2911956822] up:active 2024-04-12T05:52:50.317 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:50 smithi194 ceph-mon[49771]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 2024-04-12T05:52:50.317 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:50 smithi194 ceph-mon[49771]: pgmap v194: 65 pgs: 1 creating+activating, 64 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 3 op/s 2024-04-12T05:52:51.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 -- 172.21.15.8:0/283422207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 msgr2=0x7f5ecc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:51.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/283422207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc101270 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc009910 tx=0x7f5ebc009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 -- 172.21.15.8:0/283422207 shutdown_connections 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/283422207 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc101270 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/283422207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 0x7f5ecc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.381+0000 7f5ed0b36700 1 -- 172.21.15.8:0/283422207 >> 172.21.15.8:0/283422207 conn(0x7f5ecc0fb190 msgr2=0x7f5ecc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.382+0000 7f5ed0b36700 1 -- 172.21.15.8:0/283422207 shutdown_connections 2024-04-12T05:52:51.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.382+0000 7f5ed0b36700 1 -- 172.21.15.8:0/283422207 wait complete. 2024-04-12T05:52:51.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.382+0000 7f5ed0b36700 1 Processor -- start 2024-04-12T05:52:51.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.382+0000 7f5ed0b36700 1 -- start start 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 0x7f5ecc1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc196f20 con 0x7f5ecc100e10 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc197060 con 0x7f5ecc0ffc10 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41182/0 (socket says 172.21.15.8:41182) 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 -- 172.21.15.8:0/3133152643 learned_addr learned my addr 172.21.15.8:0/3133152643 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5eca59c700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 0x7f5ecc1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 -- 172.21.15.8:0/3133152643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 msgr2=0x7f5ecc1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 0x7f5ecc1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 -- 172.21.15.8:0/3133152643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc0095f0 con 0x7f5ecc100e10 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec9d9b700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc006010 tx=0x7f5ebc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ebc01d070 con 0x7f5ecc100e10 2024-04-12T05:52:51.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5ebc005dc0 con 0x7f5ecc100e10 2024-04-12T05:52:51.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc19bab0 con 0x7f5ecc100e10 2024-04-12T05:52:51.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ebc021410 con 0x7f5ecc100e10 2024-04-12T05:52:51.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.383+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc19bfa0 con 0x7f5ecc100e10 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.385+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc1085c0 con 0x7f5ecc100e10 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.385+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f5ebc003a60 con 0x7f5ecc100e10 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.385+0000 7f5ec37fe700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 0x7f5eb806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.386+0000 7f5eca59c700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 0x7f5eb806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.386+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f5ebc08c9f0 con 0x7f5ecc100e10 2024-04-12T05:52:51.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.386+0000 7f5eca59c700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 0x7f5eb806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc100c70 tx=0x7f5eb400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:51.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.390+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ebc08ee60 con 0x7f5ecc100e10 2024-04-12T05:52:51.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:51.638+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f5ecc19c630 con 0x7f5ecc100e10 2024-04-12T05:52:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: Deploying daemon mds.cephfs.smithi008.lmvepz on smithi008 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:boot 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 1 up:standby 2024-04-12T05:52:51.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:51 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:52:51.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: Deploying daemon mds.cephfs.smithi008.lmvepz on smithi008 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:boot 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 1 up:standby 2024-04-12T05:52:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:51 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:52:52.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.498+0000 7f5ec37fe700 1 -- 172.21.15.8:0/3133152643 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v7) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5ebc05ace0 con 0x7f5ecc100e10 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 msgr2=0x7f5eb806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 0x7f5eb806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc100c70 tx=0x7f5eb400b410 comp rx=0 tx=0).stop 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 msgr2=0x7f5ecc196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc006010 tx=0x7f5ebc00bd70 comp rx=0 tx=0).stop 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 shutdown_connections 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ecc0ffc10 0x7f5ecc1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f5eb806c4e0 0x7f5eb806e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 --2- 172.21.15.8:0/3133152643 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ecc100e10 0x7f5ecc196900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.501+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 >> 172.21.15.8:0/3133152643 conn(0x7f5ecc0fb190 msgr2=0x7f5ecc068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.502+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 shutdown_connections 2024-04-12T05:52:52.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:52.502+0000 7f5ed0b36700 1 -- 172.21.15.8:0/3133152643 wait complete. 2024-04-12T05:52:52.531 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:52 smithi008 ceph-mon[39722]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-12T05:52:52.532 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:52 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3133152643' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-12T05:52:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:52 smithi194 ceph-mon[49771]: pgmap v195: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-12T05:52:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:52 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3133152643' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-12T05:52:53.281 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:53.292 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:53.292 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-12T05:52:53.542 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3133152643' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: daemon mds.cephfs.smithi194.uzfpyu assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 1 up:standby 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:creating} 2024-04-12T05:52:53.543 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:53 smithi008 ceph-mon[39722]: daemon mds.cephfs.smithi194.uzfpyu is now active in filesystem cephfs as rank 1 2024-04-12T05:52:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-12T05:52:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3133152643' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-12T05:52:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: daemon mds.cephfs.smithi194.uzfpyu assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-12T05:52:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T05:52:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-12T05:52:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: fsmap cephfs:1 {0=cephfs.smithi008.rhdhsr=up:active} 1 up:standby 2024-04-12T05:52:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:creating} 2024-04-12T05:52:53.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:53 smithi194 ceph-mon[49771]: daemon mds.cephfs.smithi194.uzfpyu is now active in filesystem cephfs as rank 1 2024-04-12T05:52:53.897 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:54.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:active 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:54.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 6 op/s 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:active 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-12T05:52:54.772 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: Deploying daemon mds.cephfs.smithi194.gbfdvc on smithi194 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: Cluster is now healthy 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:boot 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby 2024-04-12T05:52:55.611 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:55 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: Deploying daemon mds.cephfs.smithi194.gbfdvc on smithi194 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: Cluster is now healthy 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:boot 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby 2024-04-12T05:52:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:55 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:52:55.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 -- 172.21.15.8:0/978089105 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54101eb0 msgr2=0x7f0f54102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 --2- 172.21.15.8:0/978089105 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54101eb0 0x7f0f54102310 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009910 tx=0x7f0f3c009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 -- 172.21.15.8:0/978089105 shutdown_connections 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 --2- 172.21.15.8:0/978089105 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54101eb0 0x7f0f54102310 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 --2- 172.21.15.8:0/978089105 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54100cb0 0x7f0f541010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.903+0000 7f0f59fab700 1 -- 172.21.15.8:0/978089105 >> 172.21.15.8:0/978089105 conn(0x7f0f540fc230 msgr2=0x7f0f540fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.904+0000 7f0f59fab700 1 -- 172.21.15.8:0/978089105 shutdown_connections 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.904+0000 7f0f59fab700 1 -- 172.21.15.8:0/978089105 wait complete. 2024-04-12T05:52:55.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.904+0000 7f0f59fab700 1 Processor -- start 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f59fab700 1 -- start start 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f59fab700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f59fab700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54101eb0 0x7f0f54196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f59fab700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54196f50 con 0x7f0f54100cb0 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f59fab700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54197090 con 0x7f0f54101eb0 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53542/0 (socket says 172.21.15.8:53542) 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 -- 172.21.15.8:0/2259497939 learned_addr learned my addr 172.21.15.8:0/2259497939 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f52ffd700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54101eb0 0x7f0f54196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 -- 172.21.15.8:0/2259497939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54101eb0 msgr2=0x7f0f54196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:55.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54101eb0 0x7f0f54196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 -- 172.21.15.8:0/2259497939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c0095f0 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f537fe700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c8f0 tx=0x7f0f4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.905+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f44007a70 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.906+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0f44007bd0 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.906+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f44018630 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.906+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5419bb40 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.906+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5419c090 con 0x7f0f54100cb0 2024-04-12T05:52:55.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.906+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f541096d0 con 0x7f0f54100cb0 2024-04-12T05:52:55.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.907+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f0f4400f450 con 0x7f0f54100cb0 2024-04-12T05:52:55.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.908+0000 7f0f50ff9700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 0x7f0f4006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:55.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.908+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f0f4408b880 con 0x7f0f54100cb0 2024-04-12T05:52:55.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.908+0000 7f0f52ffd700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 0x7f0f4006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:55.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.908+0000 7f0f52ffd700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 0x7f0f4006e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0059d0 tx=0x7f0f3c005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:55.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:55.912+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0f44059a50 con 0x7f0f54100cb0 2024-04-12T05:52:56.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.154+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f0f5419c830 con 0x7f0f54100cb0 2024-04-12T05:52:56.505 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.510+0000 7f0f50ff9700 1 -- 172.21.15.8:0/2259497939 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v11) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0f440595e0 con 0x7f0f54100cb0 2024-04-12T05:52:56.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 msgr2=0x7f0f4006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:56.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 0x7f0f4006e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c0059d0 tx=0x7f0f3c005920 comp rx=0 tx=0).stop 2024-04-12T05:52:56.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 msgr2=0x7f0f541963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:56.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c8f0 tx=0x7f0f4400ccb0 comp rx=0 tx=0).stop 2024-04-12T05:52:56.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 shutdown_connections 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0f4006c4e0 0x7f0f4006e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f54100cb0 0x7f0f541963f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 --2- 172.21.15.8:0/2259497939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f54101eb0 0x7f0f54196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 >> 172.21.15.8:0/2259497939 conn(0x7f0f540fc230 msgr2=0x7f0f541050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 shutdown_connections 2024-04-12T05:52:56.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:56.513+0000 7f0f59fab700 1 -- 172.21.15.8:0/2259497939 wait complete. 2024-04-12T05:52:56.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:56 smithi194 ceph-mon[49771]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-12T05:52:56.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:56 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2259497939' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-12T05:52:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:56 smithi008 ceph-mon[39722]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-12T05:52:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:56 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2259497939' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-12T05:52:57.169 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:52:57.185 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:52:57.186 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-12T05:52:57.437 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:52:57.775 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2259497939' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:57.776 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2259497939' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:57.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:52:57.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: pgmap v198: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-04-12T05:52:57.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:52:57.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:52:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:58 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] up:boot 2024-04-12T05:52:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:58 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:52:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:58 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:52:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:58 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:52:58.835 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:58 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] up:boot 2024-04-12T05:52:58.835 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:58 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:52:58.836 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:58 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:52:58.836 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:58 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:standby-replay 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: Dropping low affinity active daemon mds.cephfs.smithi008.rhdhsr in favor of higher affinity standby. 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: Replacing daemon mds.cephfs.smithi008.rhdhsr as rank 0 with standby daemon mds.cephfs.smithi008.lmvepz 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: osdmap e46: 6 total, 6 up, 6 in 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:replay,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 2024-04-12T05:52:59.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:52:59 smithi008 ceph-mon[39722]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 614 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.660+0000 7f18f2ace700 1 -- 172.21.15.8:0/235616109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 msgr2=0x7f18ec103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.660+0000 7f18f2ace700 1 --2- 172.21.15.8:0/235616109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec103b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009910 tx=0x7f18e0009c20 comp rx=0 tx=0).stop 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- 172.21.15.8:0/235616109 shutdown_connections 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 --2- 172.21.15.8:0/235616109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 0x7f18ec1064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 --2- 172.21.15.8:0/235616109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec103b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:59.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- 172.21.15.8:0/235616109 >> 172.21.15.8:0/235616109 conn(0x7f18ec0fb190 msgr2=0x7f18ec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:52:59.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- 172.21.15.8:0/235616109 shutdown_connections 2024-04-12T05:52:59.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- 172.21.15.8:0/235616109 wait complete. 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 Processor -- start 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- start start 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 0x7f18ec0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0ffc90 con 0x7f18ec1017a0 2024-04-12T05:52:59.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f2ace700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec0ffdd0 con 0x7f18ec1040d0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.661+0000 7f18f086a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53578/0 (socket says 172.21.15.8:53578) 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 -- 172.21.15.8:0/3340970151 learned_addr learned my addr 172.21.15.8:0/3340970151 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18ebfff700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 0x7f18ec0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 -- 172.21.15.8:0/3340970151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 msgr2=0x7f18ec0ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 0x7f18ec0ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 -- 172.21.15.8:0/3340970151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e00095f0 con 0x7f18ec1017a0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f086a700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006010 tx=0x7f18e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e001d070 con 0x7f18ec1017a0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec100050 con 0x7f18ec1017a0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f18e0022470 con 0x7f18ec1017a0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e000f690 con 0x7f18ec1017a0 2024-04-12T05:52:59.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.662+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec100540 con 0x7f18ec1017a0 2024-04-12T05:52:59.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.663+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec1904d0 con 0x7f18ec1017a0 2024-04-12T05:52:59.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.664+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f18e0022980 con 0x7f18ec1017a0 2024-04-12T05:52:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.664+0000 7f18e9ffb700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 0x7f18dc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:52:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.664+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f18e008d0d0 con 0x7f18ec1017a0 2024-04-12T05:52:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.664+0000 7f18ebfff700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 0x7f18dc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:52:59.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.665+0000 7f18ebfff700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 0x7f18dc06e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005ea0 tx=0x7f18d8005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:52:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.669+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f18e005b400 con 0x7f18ec1017a0 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:standby-replay 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: Dropping low affinity active daemon mds.cephfs.smithi008.rhdhsr in favor of higher affinity standby. 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: Replacing daemon mds.cephfs.smithi008.rhdhsr as rank 0 with standby daemon mds.cephfs.smithi008.lmvepz 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: osdmap e46: 6 total, 6 up, 6 in 2024-04-12T05:52:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:replay,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 2024-04-12T05:52:59.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:52:59 smithi194 ceph-mon[49771]: pgmap v200: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 614 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-12T05:52:59.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:52:59.913+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f18ec02cc70 con 0x7f18ec1017a0 2024-04-12T05:53:00.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.542+0000 7f18e9ffb700 1 -- 172.21.15.8:0/3340970151 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f18e005af90 con 0x7f18ec1017a0 2024-04-12T05:53:00.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 msgr2=0x7f18dc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 0x7f18dc06e9a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f18d8005ea0 tx=0x7f18d8005e30 comp rx=0 tx=0).stop 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 msgr2=0x7f18ec101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f18e0006010 tx=0x7f18e000be20 comp rx=0 tx=0).stop 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 shutdown_connections 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f18dc06c4e0 0x7f18dc06e9a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18ec1017a0 0x7f18ec101100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 --2- 172.21.15.8:0/3340970151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18ec1040d0 0x7f18ec0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 >> 172.21.15.8:0/3340970151 conn(0x7f18ec0fb190 msgr2=0x7f18ec0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 shutdown_connections 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:00.544+0000 7f18f2ace700 1 -- 172.21.15.8:0/3340970151 wait complete. 2024-04-12T05:53:00.540 INFO:teuthology.orchestra.run.smithi008.stderr:inline data disabled 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] up:boot 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:resolve 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:resolve,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:00 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3340970151' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-12T05:53:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] up:boot 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:resolve 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:resolve,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:00 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3340970151' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-12T05:53:01.575 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T05:53:01.585 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:53:01.585 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T05:53:01.820 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:01 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:reconnect 2024-04-12T05:53:01.820 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:01 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3340970151' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-12T05:53:01.820 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:01 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:active 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:01 smithi008 ceph-mon[39722]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:reconnect,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:01 smithi008 ceph-mon[39722]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:01 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:reconnect 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:01 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3340970151' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:01 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] up:active 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:01 smithi194 ceph-mon[49771]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:reconnect,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:01.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:01 smithi194 ceph-mon[49771]: pgmap v201: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-12T05:53:02.237 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:02 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:rejoin 2024-04-12T05:53:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:02 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] up:standby-replay 2024-04-12T05:53:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:02 smithi194 ceph-mon[49771]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:rejoin,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:02 smithi194 ceph-mon[49771]: daemon mds.cephfs.smithi008.lmvepz is now active in filesystem cephfs as rank 0 2024-04-12T05:53:02.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:02 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:rejoin 2024-04-12T05:53:02.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:02 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] up:standby-replay 2024-04-12T05:53:02.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:02 smithi008 ceph-mon[39722]: fsmap cephfs:2/2 {0=cephfs.smithi008.lmvepz=up:rejoin,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:02.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:02 smithi008 ceph-mon[39722]: daemon mds.cephfs.smithi008.lmvepz is now active in filesystem cephfs as rank 0 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: Cluster is now healthy 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:active 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:53:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:03 smithi194 ceph-mon[49771]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.4 KiB/s rd, 819 B/s wr, 8 op/s 2024-04-12T05:53:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T05:53:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: Cluster is now healthy 2024-04-12T05:53:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: mds.? [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] up:active 2024-04-12T05:53:03.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 1 up:standby-replay 1 up:standby 2024-04-12T05:53:03.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:53:03.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:03 smithi008 ceph-mon[39722]: pgmap v202: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.4 KiB/s rd, 819 B/s wr, 8 op/s 2024-04-12T05:53:05.317 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 -- 172.21.15.8:0/3505058808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce8100e10 msgr2=0x7fdce8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 --2- 172.21.15.8:0/3505058808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce8100e10 0x7fdce8101270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009910 tx=0x7fdcdc009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 -- 172.21.15.8:0/3505058808 shutdown_connections 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 --2- 172.21.15.8:0/3505058808 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce8100e10 0x7fdce8101270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 --2- 172.21.15.8:0/3505058808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce80ffc10 0x7fdce8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 -- 172.21.15.8:0/3505058808 >> 172.21.15.8:0/3505058808 conn(0x7fdce80fb190 msgr2=0x7fdce80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 -- 172.21.15.8:0/3505058808 shutdown_connections 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.320+0000 7fdceffde700 1 -- 172.21.15.8:0/3505058808 wait complete. 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 Processor -- start 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 -- start start 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce8100e10 0x7fdce8195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:05.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8195fb0 con 0x7fdce80ffc10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdceffde700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce81960f0 con 0x7fdce8100e10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdcedd7a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdcedd7a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36262/0 (socket says 172.21.15.8:36262) 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdcedd7a700 1 -- 172.21.15.8:0/1427138486 learned_addr learned my addr 172.21.15.8:0/1427138486 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.321+0000 7fdced579700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce8100e10 0x7fdce8195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcedd7a700 1 -- 172.21.15.8:0/1427138486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce8100e10 msgr2=0x7fdce8195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcedd7a700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce8100e10 0x7fdce8195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcedd7a700 1 -- 172.21.15.8:0/1427138486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0095f0 con 0x7fdce80ffc10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcedd7a700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdce400b480 tx=0x7fdce400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce40118c0 con 0x7fdce80ffc10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdce4010380 con 0x7fdce80ffc10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdce400f510 con 0x7fdce80ffc10 2024-04-12T05:53:05.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce819ab50 con 0x7fdce80ffc10 2024-04-12T05:53:05.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.322+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce819b0a0 con 0x7fdce80ffc10 2024-04-12T05:53:05.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.323+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce8108660 con 0x7fdce80ffc10 2024-04-12T05:53:05.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.324+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fdce40104f0 con 0x7fdce80ffc10 2024-04-12T05:53:05.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.324+0000 7fdcdaffd700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 0x7fdcd406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:05.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.324+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdce408b510 con 0x7fdce80ffc10 2024-04-12T05:53:05.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.324+0000 7fdced579700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 0x7fdcd406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:05.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.324+0000 7fdced579700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 0x7fdcd406e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0059d0 tx=0x7fdcdc005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:05.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.329+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdce4059720 con 0x7fdce80ffc10 2024-04-12T05:53:05.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.609+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdce8108850 con 0x7fdce80ffc10 2024-04-12T05:53:05.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.609+0000 7fdcdaffd700 1 -- 172.21.15.8:0/1427138486 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1845 (secure 0 0 0) 0x7fdce4015750 con 0x7fdce80ffc10 2024-04-12T05:53:05.605 INFO:teuthology.orchestra.run.smithi008.stdout:e21 2024-04-12T05:53:05.605 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 21 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T05:53:02.557535+0000 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T05:53:05.606 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T05:53:05.607 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:53:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 msgr2=0x7fdcd406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 0x7fdcd406e9a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0059d0 tx=0x7fdcdc005920 comp rx=0 tx=0).stop 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 msgr2=0x7fdce8195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdce400b480 tx=0x7fdce400b840 comp rx=0 tx=0).stop 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 shutdown_connections 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdcd406c4e0 0x7fdcd406e9a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdce80ffc10 0x7fdce8195450 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 --2- 172.21.15.8:0/1427138486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdce8100e10 0x7fdce8195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.611+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 >> 172.21.15.8:0/1427138486 conn(0x7fdce80fb190 msgr2=0x7fdce8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.612+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 shutdown_connections 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:05.612+0000 7fdceffde700 1 -- 172.21.15.8:0/1427138486 wait complete. 2024-04-12T05:53:05.609 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2024-04-12T05:53:06.481 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-12T05:53:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.1 KiB/s rd, 819 B/s wr, 11 op/s 2024-04-12T05:53:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:53:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:53:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1427138486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:53:06.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: pgmap v203: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.1 KiB/s rd, 819 B/s wr, 11 op/s 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1427138486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:53:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:53:07.340 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:08.645 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:08 smithi008 ceph-mon[39722]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-12T05:53:08.645 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:08 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:08.645 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:08 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:08 smithi194 ceph-mon[49771]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 11 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-12T05:53:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:08 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:08 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- 172.21.15.8:0/1312645006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80101eb0 msgr2=0x7f0b80102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 --2- 172.21.15.8:0/1312645006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80101eb0 0x7f0b80102310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009910 tx=0x7f0b6c009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- 172.21.15.8:0/1312645006 shutdown_connections 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 --2- 172.21.15.8:0/1312645006 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80101eb0 0x7f0b80102310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 --2- 172.21.15.8:0/1312645006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80100cb0 0x7f0b801010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- 172.21.15.8:0/1312645006 >> 172.21.15.8:0/1312645006 conn(0x7f0b800fc230 msgr2=0x7f0b800fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:10.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- 172.21.15.8:0/1312645006 shutdown_connections 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- 172.21.15.8:0/1312645006 wait complete. 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 Processor -- start 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- start start 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80101eb0 0x7f0b80196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80196f40 con 0x7f0b80100cb0 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.025+0000 7f0b86e83700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b80197080 con 0x7f0b80101eb0 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36268/0 (socket says 172.21.15.8:36268) 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 -- 172.21.15.8:0/376068800 learned_addr learned my addr 172.21.15.8:0/376068800 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b7ffff700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80101eb0 0x7f0b80196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 -- 172.21.15.8:0/376068800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80101eb0 msgr2=0x7f0b80196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:10.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80101eb0 0x7f0b80196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 -- 172.21.15.8:0/376068800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0095f0 con 0x7f0b80100cb0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b84c1f700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400d900 tx=0x7f0b7400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b740041f0 con 0x7f0b80100cb0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0b74004350 con 0x7f0b80100cb0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8019bb30 con 0x7f0b80100cb0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b74003bf0 con 0x7f0b80100cb0 2024-04-12T05:53:10.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.026+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8019c080 con 0x7f0b80100cb0 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.028+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b801096d0 con 0x7f0b80100cb0 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.028+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f0b74009730 con 0x7f0b80100cb0 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.028+0000 7f0b7dffb700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 0x7f0b7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.028+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0b7408b730 con 0x7f0b80100cb0 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.028+0000 7f0b7ffff700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 0x7f0b7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:10.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.029+0000 7f0b7ffff700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 0x7f0b7006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b5c0 tx=0x7f0b6c005ee0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:10.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.033+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0b740599c0 con 0x7f0b80100cb0 2024-04-12T05:53:10.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.284+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b801098c0 con 0x7f0b80100cb0 2024-04-12T05:53:10.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.284+0000 7f0b7dffb700 1 -- 172.21.15.8:0/376068800 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4767 (secure 0 0 0) 0x7f0b74059550 con 0x7f0b80100cb0 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 msgr2=0x7f0b7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 0x7f0b7006e9a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00b5c0 tx=0x7f0b6c005ee0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 msgr2=0x7f0b801963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400d900 tx=0x7f0b7400dc10 comp rx=0 tx=0).stop 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 shutdown_connections 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f0b7006c4e0 0x7f0b7006e9a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b80100cb0 0x7f0b801963e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 --2- 172.21.15.8:0/376068800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b80101eb0 0x7f0b80196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 >> 172.21.15.8:0/376068800 conn(0x7f0b800fc230 msgr2=0x7f0b801050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:10.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 shutdown_connections 2024-04-12T05:53:10.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:10.286+0000 7f0b86e83700 1 -- 172.21.15.8:0/376068800 wait complete. 2024-04-12T05:53:10.283 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2024-04-12T05:53:10.292 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:53:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:10 smithi008 ceph-mon[39722]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-12T05:53:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:10 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/376068800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T05:53:10.730 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:10 smithi194 ceph-mon[49771]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.7 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-12T05:53:10.730 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:10 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/376068800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T05:53:11.455 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-12T05:53:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:11 smithi194 ceph-mon[49771]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.6 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-12T05:53:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:11 smithi008 ceph-mon[39722]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.6 KiB/s rd, 0 B/s wr, 11 op/s 2024-04-12T05:53:12.352 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:13.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:13 smithi194 ceph-mon[49771]: pgmap v207: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T05:53:13.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:13 smithi008 ceph-mon[39722]: pgmap v207: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-12T05:53:14.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3024067457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 msgr2=0x7f3c44100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:14.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3024067457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44100030 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009910 tx=0x7f3c40009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3024067457 shutdown_connections 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3024067457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c44100e10 0x7f3c44101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3024067457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44100030 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3024067457 >> 172.21.15.8:0/3024067457 conn(0x7f3c440fb190 msgr2=0x7f3c440fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3024067457 shutdown_connections 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:14.999+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3024067457 wait complete. 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 Processor -- start 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 -- start start 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:14.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c44100e10 0x7f3c44196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c44197080 con 0x7f3c440ffc10 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4c68b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c441971c0 con 0x7f3c44100e10 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58628/0 (socket says 172.21.15.8:58628) 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 -- 172.21.15.8:0/3596582304 learned_addr learned my addr 172.21.15.8:0/3596582304 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 -- 172.21.15.8:0/3596582304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c44100e10 msgr2=0x7f3c44196a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c44100e10 0x7f3c44196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 -- 172.21.15.8:0/3596582304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400095f0 con 0x7f3c440ffc10 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.000+0000 7f3c4a427700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c400092e0 tx=0x7f3c4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.001+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c4001d070 con 0x7f3c440ffc10 2024-04-12T05:53:14.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.001+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3c4000f460 con 0x7f3c440ffc10 2024-04-12T05:53:14.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.001+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4419bc10 con 0x7f3c440ffc10 2024-04-12T05:53:14.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.001+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c40021640 con 0x7f3c440ffc10 2024-04-12T05:53:14.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.001+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4419c100 con 0x7f3c440ffc10 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.002+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c44108640 con 0x7f3c440ffc10 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.002+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f3c4000f9e0 con 0x7f3c440ffc10 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.003+0000 7f3c3b7fe700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 0x7f3c3006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.003+0000 7f3c49c26700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 0x7f3c3006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.003+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3c4008cc10 con 0x7f3c440ffc10 2024-04-12T05:53:15.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.003+0000 7f3c49c26700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 0x7f3c3006e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c340097b0 tx=0x7f3c34006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:15.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.007+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3c4005ae20 con 0x7f3c440ffc10 2024-04-12T05:53:15.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.311+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f3c44108830 con 0x7f3c440ffc10 2024-04-12T05:53:15.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.311+0000 7f3c3b7fe700 1 -- 172.21.15.8:0/3596582304 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v21) v1 ==== 78+0+83 (secure 0 0 0) 0x7f3c4005a9b0 con 0x7f3c440ffc10 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 msgr2=0x7f3c3006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 0x7f3c3006e9a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c340097b0 tx=0x7f3c34006d20 comp rx=0 tx=0).stop 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 msgr2=0x7f3c44196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3c400092e0 tx=0x7f3c4000be20 comp rx=0 tx=0).stop 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 shutdown_connections 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f3c3006c4e0 0x7f3c3006e9a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3c440ffc10 0x7f3c44196520 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 --2- 172.21.15.8:0/3596582304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3c44100e10 0x7f3c44196a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 >> 172.21.15.8:0/3596582304 conn(0x7f3c440fb190 msgr2=0x7f3c44068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.313+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 shutdown_connections 2024-04-12T05:53:15.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:15.314+0000 7f3c4c68b700 1 -- 172.21.15.8:0/3596582304 wait complete. 2024-04-12T05:53:15.319 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:53:16.188 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-12T05:53:16.207 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T05:53:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 -- 172.21.15.8:0/1250174527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 msgr2=0x7f7320103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 --2- 172.21.15.8:0/1250174527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f7320103990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7310009910 tx=0x7f7310009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 -- 172.21.15.8:0/1250174527 shutdown_connections 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 --2- 172.21.15.8:0/1250174527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 0x7f73201062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 --2- 172.21.15.8:0/1250174527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f7320103990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 -- 172.21.15.8:0/1250174527 >> 172.21.15.8:0/1250174527 conn(0x7f73200faf70 msgr2=0x7f73200fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 -- 172.21.15.8:0/1250174527 shutdown_connections 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.353+0000 7f73261fd700 1 -- 172.21.15.8:0/1250174527 wait complete. 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 Processor -- start 2024-04-12T05:53:16.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 -- start start 2024-04-12T05:53:16.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 0x7f7320196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320196d50 con 0x7f73201015a0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f73261fd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320196e90 con 0x7f7320103ed0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58644/0 (socket says 172.21.15.8:58644) 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 -- 172.21.15.8:0/2114457527 learned_addr learned my addr 172.21.15.8:0/2114457527 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731effd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 0x7f7320196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 -- 172.21.15.8:0/2114457527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 msgr2=0x7f7320196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 0x7f7320196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.354+0000 7f731f7fe700 1 -- 172.21.15.8:0/2114457527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100095f0 con 0x7f73201015a0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f731f7fe700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f731000b5c0 tx=0x7f731000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:16.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f731001d070 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732019b8d0 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f731000f460 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7310021640 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.355+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732019be50 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.356+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7320190350 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.356+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f731000fa20 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.357+0000 7f731cff9700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 0x7f730c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.357+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f731008cb20 con 0x7f73201015a0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.357+0000 7f731effd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 0x7f730c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.357+0000 7f731effd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 0x7f730c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7308005950 tx=0x7f730800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:16.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.361+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f731005ad30 con 0x7f73201015a0 2024-04-12T05:53:16.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:53:16.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:53:16.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3596582304' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-12T05:53:16.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:16 smithi008 ceph-mon[39722]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-12T05:53:16.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.563+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f732002cc70 con 0x7f73201015a0 2024-04-12T05:53:16.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.564+0000 7f731cff9700 1 -- 172.21.15.8:0/2114457527 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15063 (secure 0 0 0) 0x7f731005a8c0 con 0x7f73201015a0 2024-04-12T05:53:16.560 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:53:16.560 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":46,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T05:52:58.525975+0000","last_up_change":"2024-04-12T05:51:19.387464+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T05:52:42.555620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T05:52:43.040716+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6803","nonce":106489579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6805","nonce":106489579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6809","nonce":106489579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":106489579},{"type":"v1","addr":"172.21.15.8:6807","nonce":106489579}]},"public_addr":"172.21.15.8:6803/106489579","cluster_addr":"172.21.15.8:6805/106489579","heartbeat_back_addr":"172.21.15.8:6809/106489579","heartbeat_front_addr":"172.21.15.8:6807/106489579","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6811","nonce":618315693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6813","nonce":618315693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6817","nonce":618315693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":618315693},{"type":"v1","addr":"172.21.15.8:6815","nonce":618315693}]},"public_addr":"172.21.15.8:6811/618315693","cluster_addr":"172.21.15.8:6813/618315693","heartbeat_back_addr":"172.21.15.8:6817/618315693","heartbeat_front_addr":"172.21.15.8:6815/618315693","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6819","nonce":4265124490}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6821","nonce":4265124490}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6825","nonce":4265124490}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":4265124490},{"type":"v1","addr":"172.21.15.8:6823","nonce":4265124490}]},"public_addr":"172.21.15.8:6819/4265124490","cluster_addr":"172.21.15.8:6821/4265124490","heartbeat_back_addr":"172.21.15.8:6825/4265124490","heartbeat_front_addr":"172.21.15.8:6823/4265124490","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6801","nonce":3418265960}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6803","nonce":3418265960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6807","nonce":3418265960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":3418265960},{"type":"v1","addr":"172.21.15.194:6805","nonce":3418265960}]},"public_addr":"172.21.15.194:6801/3418265960","cluster_addr":"172.21.15.194:6803/3418265960","heartbeat_back_addr":"172.21.15.194:6807/3418265960","heartbeat_front_addr":"172.21.15.194:6805/3418265960","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6809","nonce":4144829010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6811","nonce":4144829010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6815","nonce":4144829010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":4144829010},{"type":"v1","addr":"172.21.15.194:6813","nonce":4144829010}]},"public_addr":"172.21.15.194:6809/4144829010","cluster_addr":"172.21.15.194:6811/4144829010","heartbeat_back_addr":"172.21.15.194:6815/4144829010","heartbeat_front_addr":"172.21.15.194:6813/4144829010","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6817","nonce":824210471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6819","nonce":824210471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6823","nonce":824210471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":824210471},{"type":"v1","addr":"172.21.15.194:6821","nonce":824210471}]},"public_addr":"172.21.15.194:6817/824210471","cluster_addr":"172.21.15.194:6819/824210471","heartbeat_back_addr":"172.21.15.194:6823/824210471","heartbeat_front_addr":"172.21.15.194:6821/824210471","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.8:6827/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6826/2911956822":"2024-04-13T05:52:58.525944+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T05:53:16.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 msgr2=0x7f730c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 0x7f730c06e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7308005950 tx=0x7f730800b410 comp rx=0 tx=0).stop 2024-04-12T05:53:16.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 msgr2=0x7f73201961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f731000b5c0 tx=0x7f731000bd70 comp rx=0 tx=0).stop 2024-04-12T05:53:16.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 shutdown_connections 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f730c06c4e0 0x7f730c06e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f73201015a0 0x7f73201961f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 --2- 172.21.15.8:0/2114457527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7320103ed0 0x7f7320196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 >> 172.21.15.8:0/2114457527 conn(0x7f73200faf70 msgr2=0x7f73200fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 shutdown_connections 2024-04-12T05:53:16.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.566+0000 7f73261fd700 1 -- 172.21.15.8:0/2114457527 wait complete. 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3596582304' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-12T05:53:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:16 smithi194 ceph-mon[49771]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-12T05:53:16.575 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T05:49:53.787041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T05:52:42.555620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T05:52:43.040716+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T05:53:16.575 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T05:53:16.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 -- 172.21.15.8:0/1638549261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 msgr2=0x7f658c1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 --2- 172.21.15.8:0/1638549261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1039d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f657c009910 tx=0x7f657c009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 -- 172.21.15.8:0/1638549261 shutdown_connections 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 --2- 172.21.15.8:0/1638549261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 0x7f658c106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 --2- 172.21.15.8:0/1638549261 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1039d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 -- 172.21.15.8:0/1638549261 >> 172.21.15.8:0/1638549261 conn(0x7f658c0faf90 msgr2=0x7f658c0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 -- 172.21.15.8:0/1638549261 shutdown_connections 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.710+0000 7f6591447700 1 -- 172.21.15.8:0/1638549261 wait complete. 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 Processor -- start 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 -- start start 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 0x7f658c196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c196cb0 con 0x7f658c1015e0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f6591447700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c196df0 con 0x7f658c103f10 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58662/0 (socket says 172.21.15.8:58662) 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 -- 172.21.15.8:0/1673164542 learned_addr learned my addr 172.21.15.8:0/1673164542 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658a7fc700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 0x7f658c196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 -- 172.21.15.8:0/1673164542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 msgr2=0x7f658c196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 0x7f658c196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.711+0000 7f658affd700 1 -- 172.21.15.8:0/1673164542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c0095f0 con 0x7f658c1015e0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f658affd700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f657c00bd70 tx=0x7f657c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c01d070 con 0x7f658c1015e0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c19b840 con 0x7f658c1015e0 2024-04-12T05:53:16.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f657c005940 con 0x7f658c1015e0 2024-04-12T05:53:16.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c00f460 con 0x7f658c1015e0 2024-04-12T05:53:16.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.712+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c19bdc0 con 0x7f658c1015e0 2024-04-12T05:53:16.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.713+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c190330 con 0x7f658c1015e0 2024-04-12T05:53:16.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.713+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f657c022800 con 0x7f658c1015e0 2024-04-12T05:53:16.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.714+0000 7f6573fff700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 0x7f657406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:16.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.714+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f657c08cef0 con 0x7f658c1015e0 2024-04-12T05:53:16.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.714+0000 7f658a7fc700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 0x7f657406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:16.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.714+0000 7f658a7fc700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 0x7f657406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6580005ea0 tx=0x7f6580005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:16.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.718+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f657c05b1b0 con 0x7f658c1015e0 2024-04-12T05:53:16.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.919+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f658c061a40 con 0x7f658c1015e0 2024-04-12T05:53:16.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.919+0000 7f6573fff700 1 -- 172.21.15.8:0/1673164542 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f657c027020 con 0x7f658c1015e0 2024-04-12T05:53:16.915 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 msgr2=0x7f657406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 0x7f657406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6580005ea0 tx=0x7f6580005e30 comp rx=0 tx=0).stop 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 msgr2=0x7f658c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f657c00bd70 tx=0x7f657c00be50 comp rx=0 tx=0).stop 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 shutdown_connections 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f657406c4e0 0x7f657406e9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f658c1015e0 0x7f658c1961e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 --2- 172.21.15.8:0/1673164542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f658c103f10 0x7f658c196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:16.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 >> 172.21.15.8:0/1673164542 conn(0x7f658c0faf90 msgr2=0x7f658c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:16.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.921+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 shutdown_connections 2024-04-12T05:53:16.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:16.922+0000 7f6591447700 1 -- 172.21.15.8:0/1673164542 wait complete. 2024-04-12T05:53:16.929 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T05:53:17.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 -- 172.21.15.8:0/1055068956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 msgr2=0x7f1c80103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 --2- 172.21.15.8:0/1055068956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c80103940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009910 tx=0x7f1c70009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:17.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 -- 172.21.15.8:0/1055068956 shutdown_connections 2024-04-12T05:53:17.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 --2- 172.21.15.8:0/1055068956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 0x7f1c80106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 --2- 172.21.15.8:0/1055068956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c80103940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 -- 172.21.15.8:0/1055068956 >> 172.21.15.8:0/1055068956 conn(0x7f1c800faf40 msgr2=0x7f1c800fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 -- 172.21.15.8:0/1055068956 shutdown_connections 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.062+0000 7f1c86121700 1 -- 172.21.15.8:0/1055068956 wait complete. 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 Processor -- start 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 -- start start 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 0x7f1c801966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80196c80 con 0x7f1c80101550 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c86121700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80196dc0 con 0x7f1c80103e80 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c7f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58678/0 (socket says 172.21.15.8:58678) 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c7f7fe700 1 -- 172.21.15.8:0/3989772991 learned_addr learned my addr 172.21.15.8:0/3989772991 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.063+0000 7f1c7effd700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 0x7f1c801966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7f7fe700 1 -- 172.21.15.8:0/3989772991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 msgr2=0x7f1c801966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7f7fe700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 0x7f1c801966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7f7fe700 1 -- 172.21.15.8:0/3989772991 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c700095f0 con 0x7f1c80101550 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7f7fe700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000bd70 tx=0x7f1c7000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7001d070 con 0x7f1c80101550 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1c7000f460 con 0x7f1c80101550 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8019b810 con 0x7f1c80101550 2024-04-12T05:53:17.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c70021640 con 0x7f1c80101550 2024-04-12T05:53:17.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.064+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8019bd90 con 0x7f1c80101550 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.065+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c80190300 con 0x7f1c80101550 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.066+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f1c7000f9b0 con 0x7f1c80101550 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.066+0000 7f1c7cff9700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 0x7f1c6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.066+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1c7008cba0 con 0x7f1c80101550 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.066+0000 7f1c7effd700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 0x7f1c6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.066+0000 7f1c7effd700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 0x7f1c6806e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ac50 tx=0x7f1c7400a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.070+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1c7005adb0 con 0x7f1c80101550 2024-04-12T05:53:17.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.272+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1c8002cc70 con 0x7f1c80101550 2024-04-12T05:53:17.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.272+0000 7f1c7cff9700 1 -- 172.21.15.8:0/3989772991 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1c7005a940 con 0x7f1c80101550 2024-04-12T05:53:17.268 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 msgr2=0x7f1c6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 0x7f1c6806e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ac50 tx=0x7f1c7400a380 comp rx=0 tx=0).stop 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 msgr2=0x7f1c801961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c7000bd70 tx=0x7f1c7000bda0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 shutdown_connections 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1c6806c4e0 0x7f1c6806e9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1c80101550 0x7f1c801961b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 --2- 172.21.15.8:0/3989772991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1c80103e80 0x7f1c801966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.274+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 >> 172.21.15.8:0/3989772991 conn(0x7f1c800faf40 msgr2=0x7f1c800fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:17.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.275+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 shutdown_connections 2024-04-12T05:53:17.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.275+0000 7f1c86121700 1 -- 172.21.15.8:0/3989772991 wait complete. 2024-04-12T05:53:17.281 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:17 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2114457527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:53:17.281 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:17 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1673164542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T05:53:17.282 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T05:53:17.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- 172.21.15.8:0/1179395824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 msgr2=0x7f7c200ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 --2- 172.21.15.8:0/1179395824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c200ffe00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009910 tx=0x7f7c08009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- 172.21.15.8:0/1179395824 shutdown_connections 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 --2- 172.21.15.8:0/1179395824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 0x7f7c20101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 --2- 172.21.15.8:0/1179395824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c200ffe00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- 172.21.15.8:0/1179395824 >> 172.21.15.8:0/1179395824 conn(0x7f7c200faf30 msgr2=0x7f7c200fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- 172.21.15.8:0/1179395824 shutdown_connections 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- 172.21.15.8:0/1179395824 wait complete. 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 Processor -- start 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.416+0000 7f7c25868700 1 -- start start 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 0x7f7c20101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c201024f0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20102630 con 0x7f7c20100bd0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58688/0 (socket says 172.21.15.8:58688) 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 -- 172.21.15.8:0/691877861 learned_addr learned my addr 172.21.15.8:0/691877861 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1e7fc700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 0x7f7c20101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 -- 172.21.15.8:0/691877861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 msgr2=0x7f7c20101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 0x7f7c20101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 -- 172.21.15.8:0/691877861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080095f0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c1effd700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c08000c00 tx=0x7f7c0800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c0801d070 con 0x7f7c200ff9e0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7c08005dc0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c201028b0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c08003b10 con 0x7f7c200ff9e0 2024-04-12T05:53:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.417+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20102dd0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.418+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c20108410 con 0x7f7c200ff9e0 2024-04-12T05:53:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.419+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f7c080048e0 con 0x7f7c200ff9e0 2024-04-12T05:53:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.419+0000 7f7c24866700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 0x7f7c0c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.419+0000 7f7c1e7fc700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 0x7f7c0c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.419+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c0808cf90 con 0x7f7c200ff9e0 2024-04-12T05:53:17.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.420+0000 7f7c1e7fc700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 0x7f7c0c06e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009bf0 tx=0x7f7c10009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.424+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7c0805b220 con 0x7f7c200ff9e0 2024-04-12T05:53:17.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:17 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2114457527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T05:53:17.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:17 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1673164542' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T05:53:17.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.623+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f7c2019e740 con 0x7f7c200ff9e0 2024-04-12T05:53:17.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.623+0000 7f7c24866700 1 -- 172.21.15.8:0/691877861 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7c08026020 con 0x7f7c200ff9e0 2024-04-12T05:53:17.619 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T05:53:17.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 msgr2=0x7f7c0c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 0x7f7c0c06e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009bf0 tx=0x7f7c10009260 comp rx=0 tx=0).stop 2024-04-12T05:53:17.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 msgr2=0x7f7c20103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7c08000c00 tx=0x7f7c0800be90 comp rx=0 tx=0).stop 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 shutdown_connections 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7c0c06c4e0 0x7f7c0c06e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7c200ff9e0 0x7f7c20103970 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 --2- 172.21.15.8:0/691877861 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7c20100bd0 0x7f7c20101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 >> 172.21.15.8:0/691877861 conn(0x7f7c200faf30 msgr2=0x7f7c20068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 shutdown_connections 2024-04-12T05:53:17.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.626+0000 7f7c25868700 1 -- 172.21.15.8:0/691877861 wait complete. 2024-04-12T05:53:17.634 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T05:53:17.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.763+0000 7fdabe489700 1 -- 172.21.15.8:0/4177386357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab81015c0 msgr2=0x7fdab81039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.763+0000 7fdabe489700 1 --2- 172.21.15.8:0/4177386357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab81015c0 0x7fdab81039b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009910 tx=0x7fdaa8009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 -- 172.21.15.8:0/4177386357 shutdown_connections 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 --2- 172.21.15.8:0/4177386357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab8103ef0 0x7fdab81062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 --2- 172.21.15.8:0/4177386357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab81015c0 0x7fdab81039b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 -- 172.21.15.8:0/4177386357 >> 172.21.15.8:0/4177386357 conn(0x7fdab80faf30 msgr2=0x7fdab80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 -- 172.21.15.8:0/4177386357 shutdown_connections 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 -- 172.21.15.8:0/4177386357 wait complete. 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 Processor -- start 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.764+0000 7fdabe489700 1 -- start start 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab81015c0 0x7fdab8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80ffab0 con 0x7fdab8103ef0 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab80ffbf0 con 0x7fdab81015c0 2024-04-12T05:53:17.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58710/0 (socket says 172.21.15.8:58710) 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 -- 172.21.15.8:0/1986172935 learned_addr learned my addr 172.21.15.8:0/1986172935 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab7fff700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab81015c0 0x7fdab8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 -- 172.21.15.8:0/1986172935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab81015c0 msgr2=0x7fdab8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab81015c0 0x7fdab8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 -- 172.21.15.8:0/1986172935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80095f0 con 0x7fdab8103ef0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab77fe700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d8d0 tx=0x7fdaac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac009940 con 0x7fdab8103ef0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdaac010460 con 0x7fdab8103ef0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab80ffed0 con 0x7fdab8103ef0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaac00f5f0 con 0x7fdab8103ef0 2024-04-12T05:53:17.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.765+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab819e1e0 con 0x7fdab8103ef0 2024-04-12T05:53:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.766+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab8190310 con 0x7fdab8103ef0 2024-04-12T05:53:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.767+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fdaac0105d0 con 0x7fdab8103ef0 2024-04-12T05:53:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.767+0000 7fdab57fa700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 0x7fdaa006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:17.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.767+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdaac08b8c0 con 0x7fdab8103ef0 2024-04-12T05:53:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.767+0000 7fdab7fff700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 0x7fdaa006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.768+0000 7fdab7fff700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 0x7fdaa006e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8006010 tx=0x7fdaa800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:17.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:17.772+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdaac059ad0 con 0x7fdab8103ef0 2024-04-12T05:53:18.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.016+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdab802d100 con 0x7fdab8103ef0 2024-04-12T05:53:18.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.016+0000 7fdab57fa700 1 -- 172.21.15.8:0/1986172935 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4767 (secure 0 0 0) 0x7fdaac010880 con 0x7fdab8103ef0 2024-04-12T05:53:18.012 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:53:18.012 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T05:53:02.557535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.8:6827/534481461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":534481461},{"type":"v1","addr":"172.21.15.8:6827","nonce":534481461}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.194:6827/3772682938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3772682938},{"type":"v1","addr":"172.21.15.194:6827","nonce":3772682938}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.018+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 msgr2=0x7fdaa006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.018+0000 7fdabe489700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 0x7fdaa006e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8006010 tx=0x7fdaa800b540 comp rx=0 tx=0).stop 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.018+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 msgr2=0x7fdab80ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.018+0000 7fdabe489700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00d8d0 tx=0x7fdaac00dc90 comp rx=0 tx=0).stop 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 shutdown_connections 2024-04-12T05:53:18.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdab81015c0 0x7fdab8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdaa006c4e0 0x7fdaa006e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 --2- 172.21.15.8:0/1986172935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdab8103ef0 0x7fdab80ff570 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 >> 172.21.15.8:0/1986172935 conn(0x7fdab80faf30 msgr2=0x7fdab80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 shutdown_connections 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:18.019+0000 7fdabe489700 1 -- 172.21.15.8:0/1986172935 wait complete. 2024-04-12T05:53:18.015 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 21 2024-04-12T05:53:18.027 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 21, 'max_mds': 2, 'flags': 50} 2024-04-12T05:53:18.027 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-12T05:53:18.052 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-12T05:53:18.052 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-12T05:53:18.052 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-12T05:53:18.052 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T05:53:18.052 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:18.053 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-12T05:53:18.053 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-12T05:53:18.053 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:18.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:18.053 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-04-12T05:53:18.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:18.072 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link delete ceph-brx 2024-04-12T05:53:18.143 INFO:teuthology.orchestra.run.smithi008.stderr:Cannot find device "ceph-brx" 2024-04-12T05:53:18.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:53:18.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:18.145 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-04-12T05:53:18.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:18.169 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link delete ceph-brx 2024-04-12T05:53:18.248 INFO:teuthology.orchestra.run.smithi194.stderr:Cannot find device "ceph-brx" 2024-04-12T05:53:18.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T05:53:18.250 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-12T05:53:18.250 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph fs ls 2024-04-12T05:53:18.275 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:18 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3989772991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T05:53:18.276 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:18 smithi008 ceph-mon[39722]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-12T05:53:18.276 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:18 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/691877861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T05:53:18.276 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:18 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1986172935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T05:53:18.484 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:18 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3989772991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T05:53:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:18 smithi194 ceph-mon[49771]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.2 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-12T05:53:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:18 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/691877861' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T05:53:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:18 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1986172935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T05:53:20.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/2541944190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14106cd0 msgr2=0x7fdf141070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:20.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/2541944190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14106cd0 0x7fdf141070b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009910 tx=0x7fdf04009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/2541944190 shutdown_connections 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/2541944190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14068580 0x7fdf140689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/2541944190 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14106cd0 0x7fdf141070b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/2541944190 >> 172.21.15.8:0/2541944190 conn(0x7fdf14075670 msgr2=0x7fdf14075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/2541944190 shutdown_connections 2024-04-12T05:53:20.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.390+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/2541944190 wait complete. 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 Processor -- start 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 -- start start 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14106cd0 0x7fdf1419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1419d130 con 0x7fdf14068580 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf1b7d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14196630 con 0x7fdf14106cd0 2024-04-12T05:53:20.390 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58716/0 (socket says 172.21.15.8:58716) 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 -- 172.21.15.8:0/901785920 learned_addr learned my addr 172.21.15.8:0/901785920 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 -- 172.21.15.8:0/901785920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14106cd0 msgr2=0x7fdf1419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf18d6f700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14106cd0 0x7fdf1419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14106cd0 0x7fdf1419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.391+0000 7fdf19570700 1 -- 172.21.15.8:0/901785920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040095f0 con 0x7fdf14068580 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf19570700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf040092e0 tx=0x7fdf0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:20.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0401d070 con 0x7fdf14068580 2024-04-12T05:53:20.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fdf04005e00 con 0x7fdf14068580 2024-04-12T05:53:20.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0400fe50 con 0x7fdf14068580 2024-04-12T05:53:20.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf141968b0 con 0x7fdf14068580 2024-04-12T05:53:20.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.392+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf14196da0 con 0x7fdf14068580 2024-04-12T05:53:20.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.393+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf141091f0 con 0x7fdf14068580 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.394+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fdf0400f6f0 con 0x7fdf14068580 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.394+0000 7fdf0a7fc700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.394+0000 7fdf18d6f700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.394+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdf0408ce40 con 0x7fdf14068580 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.394+0000 7fdf18d6f700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005950 tx=0x7fdf100058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:20.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.398+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdf0405b0d0 con 0x7fdf14068580 2024-04-12T05:53:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:20 smithi008 ceph-mon[39722]: pgmap v210: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T05:53:20.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.640+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf1404f2e0 con 0x7fdf14068580 2024-04-12T05:53:20.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.641+0000 7fdf0a7fc700 1 -- 172.21.15.8:0/901785920 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+83 (secure 0 0 0) 0x7fdf0405ac60 con 0x7fdf14068580 2024-04-12T05:53:20.636 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 msgr2=0x7fdf0006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005950 tx=0x7fdf100058e0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 msgr2=0x7fdf1419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf040092e0 tx=0x7fdf0400be20 comp rx=0 tx=0).stop 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 shutdown_connections 2024-04-12T05:53:20.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fdf0006c4e0 0x7fdf0006e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf14068580 0x7fdf1419c560 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 --2- 172.21.15.8:0/901785920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf14106cd0 0x7fdf1419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:20.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 >> 172.21.15.8:0/901785920 conn(0x7fdf14075670 msgr2=0x7fdf140fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:20.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 shutdown_connections 2024-04-12T05:53:20.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:20.643+0000 7fdf1b7d4700 1 -- 172.21.15.8:0/901785920 wait complete. 2024-04-12T05:53:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:20 smithi194 ceph-mon[49771]: pgmap v210: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-12T05:53:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:21 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/901785920' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T05:53:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:21 smithi008 ceph-mon[39722]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi008.front.sepia.ceph.com 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T05:53:21.704 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T05:53:21.705 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-12T05:53:21.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:21.705 DEBUG:teuthology.orchestra.run.smithi008:> ip addr 2024-04-12T05:53:21.767 INFO:teuthology.orchestra.run.smithi008.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T05:53:21.768 INFO:teuthology.orchestra.run.smithi008.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T05:53:21.768 INFO:teuthology.orchestra.run.smithi008.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T05:53:21.768 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:21.768 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 ::1/128 scope host 2024-04-12T05:53:21.768 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2e brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f0 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether 0c:c4:7a:6b:fe:2f brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:21.769 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp7s0f1 2024-04-12T05:53:21.770 INFO:teuthology.orchestra.run.smithi008.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T05:53:21.770 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:fe brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:21.770 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f0 2024-04-12T05:53:21.770 INFO:teuthology.orchestra.run.smithi008.stdout: inet 172.21.15.8/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-12T05:53:21.770 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft 41936sec preferred_lft 41936sec 2024-04-12T05:53:21.771 INFO:teuthology.orchestra.run.smithi008.stdout: inet6 fe80::ae1f:6bff:fef8:25fe/64 scope link 2024-04-12T05:53:21.771 INFO:teuthology.orchestra.run.smithi008.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:21.771 INFO:teuthology.orchestra.run.smithi008.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:21.771 INFO:teuthology.orchestra.run.smithi008.stdout: link/ether ac:1f:6b:f8:25:ff brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:21.771 INFO:teuthology.orchestra.run.smithi008.stdout: altname enp1s0f1 2024-04-12T05:53:21.772 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T05:53:21.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:21.772 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-04-12T05:53:21.772 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-04-12T05:53:21.773 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add name ceph-brx type bridge 2024-04-12T05:53:21.773 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr flush dev ceph-brx 2024-04-12T05:53:21.773 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set ceph-brx up 2024-04-12T05:53:21.773 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T05:53:21.773 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-04-12T05:53:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:21 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/901785920' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T05:53:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:21 smithi194 ceph-mon[49771]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:53:21.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:21.944 DEBUG:teuthology.orchestra.run.smithi008:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T05:53:21.973 INFO:teuthology.orchestra.run.smithi008.stdout:1 2024-04-12T05:53:21.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:21.975 DEBUG:teuthology.orchestra.run.smithi008:> route 2024-04-12T05:53:22.054 INFO:teuthology.orchestra.run.smithi008.stdout:Kernel IP routing table 2024-04-12T05:53:22.054 INFO:teuthology.orchestra.run.smithi008.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T05:53:22.054 INFO:teuthology.orchestra.run.smithi008.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-12T05:53:22.054 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-12T05:53:22.054 INFO:teuthology.orchestra.run.smithi008.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T05:53:22.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-12T05:53:22.055 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-04-12T05:53:22.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.369 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list 2024-04-12T05:53:22.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.388 DEBUG:teuthology.orchestra.run.smithi008:> ip netns list-id 2024-04-12T05:53:22.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.447 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-04-12T05:53:22.447 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-04-12T05:53:22.447 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-12T05:53:22.447 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-12T05:53:22.447 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-04-12T05:53:22.544 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-12T05:53:22.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-12T05:53:22.544 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-12T05:53:22.545 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-12T05:53:22.545 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-04-12T05:53:22.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:22.743 DEBUG:teuthology.orchestra.run.smithi008:> (cd / && exec bash -c ' 2024-04-12T05:53:22.743 DEBUG:teuthology.orchestra.run.smithi008:> set -e 2024-04-12T05:53:22.743 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set brx.0 up 2024-04-12T05:53:22.743 DEBUG:teuthology.orchestra.run.smithi008:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T05:53:22.743 DEBUG:teuthology.orchestra.run.smithi008:> ') 2024-04-12T05:53:22.842 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-12T05:53:22.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T05:53:22.842 DEBUG:teuthology.orchestra.run.smithi008:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:22.862 INFO:teuthology.orchestra.run.smithi008.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-12T05:53:22.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T05:53:22.862 DEBUG:teuthology.orchestra.run.smithi008:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:22.920 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2024-04-12T05:53:22.989 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-12T05:53:22.989 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-12T05:53:23.058 INFO:teuthology.orchestra.run.smithi008.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-12T05:53:23.059 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-12T05:53:23.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:23.059 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2024-04-12T05:53:23.118 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-12T05:53:23.119 DEBUG:teuthology.orchestra.run.smithi008:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-12T05:53:23.160 DEBUG:teuthology.orchestra.run.smithi008:> sudo modprobe fuse 2024-04-12T05:53:23.188 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-12T05:53:23.188 DEBUG:teuthology.orchestra.run.smithi008:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-12T05:53:23.256 INFO:teuthology.orchestra.run.smithi008.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-12T05:53:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-12T05:53:23.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:23.257 DEBUG:teuthology.orchestra.run.smithi008:> ls /sys/fs/fuse/connections 2024-04-12T05:53:23.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi008.stderr:2024-04-12T05:53:23.274+0000 7f80f9c185c0 -1 init, newargv = 0x5619fcdf8370 newargc=15 2024-04-12T05:53:23.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi008.stderr:ceph-fuse[101505]: starting ceph client 2024-04-12T05:53:23.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi008.stderr:ceph-fuse[101505]: starting fuse 2024-04-12T05:53:23.290 INFO:teuthology.orchestra.run.smithi008.stdout:173 2024-04-12T05:53:23.290 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [173] 2024-04-12T05:53:23.290 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-04-12T05:53:23.290 DEBUG:teuthology.orchestra.run.smithi008:> import glob 2024-04-12T05:53:23.290 DEBUG:teuthology.orchestra.run.smithi008:> import re 2024-04-12T05:53:23.290 DEBUG:teuthology.orchestra.run.smithi008:> import os 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> import subprocess 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> def _find_admin_socket(client_name): 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> files = glob.glob(asok_path) 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> # Given a non-glob path, it better be there 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> if "*" not in asok_path: 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> assert(len(files) == 1) 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> return files[0] 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> for f in files: 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-12T05:53:23.291 DEBUG:teuthology.orchestra.run.smithi008:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> contents = proc_f.read() 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> if mountpoint in contents: 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> return f 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> print(_find_admin_socket("client.0")) 2024-04-12T05:53:23.292 DEBUG:teuthology.orchestra.run.smithi008:> ' 2024-04-12T05:53:23.420 INFO:teuthology.orchestra.run.smithi008.stdout:/var/run/ceph/ceph-client.0.101505.asok 2024-04-12T05:53:23.602 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101505.asok 2024-04-12T05:53:23.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:23.603 DEBUG:teuthology.orchestra.run.smithi008:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101505.asok status 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "metadata": { 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "entity_id": "0", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "hostname": "smithi008", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "pid": "101505", 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "root": "/" 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_count": 0, 2024-04-12T05:53:23.765 INFO:teuthology.orchestra.run.smithi008.stdout: "dentry_pinned_count": 0, 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "id": 14532, 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "inst": { 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "name": { 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "client", 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "num": 14532 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 530473219 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": { 2024-04-12T05:53:23.766 INFO:teuthology.orchestra.run.smithi008.stdout: "type": "v1", 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "addr": "192.168.0.1:0", 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "nonce": 530473219 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "inst_str": "client.14532 192.168.0.1:0/530473219", 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "addr_str": "192.168.0.1:0/530473219", 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "inode_count": 1, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "mds_epoch": 21, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch": 46, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "osd_epoch_barrier": 0, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "blocklisted": false, 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout: "fs_name": "cephfs" 2024-04-12T05:53:23.767 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:53:23.775 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- ceph fs ls 2024-04-12T05:53:24.008 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:24.511 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:24 smithi008 ceph-mon[39722]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:53:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:24 smithi194 ceph-mon[49771]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:53:26.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.023+0000 7fb9aca2d700 1 -- 172.21.15.8:0/3025593217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a80ffc20 msgr2=0x7fb9a8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.023+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/3025593217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a80ffc20 0x7fb9a8100080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb990009910 tx=0x7fb990009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 -- 172.21.15.8:0/3025593217 shutdown_connections 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/3025593217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a80ffc20 0x7fb9a8100080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/3025593217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a8105bd0 0x7fb9a8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 -- 172.21.15.8:0/3025593217 >> 172.21.15.8:0/3025593217 conn(0x7fb9a8075670 msgr2=0x7fb9a8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 -- 172.21.15.8:0/3025593217 shutdown_connections 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 -- 172.21.15.8:0/3025593217 wait complete. 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 Processor -- start 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.024+0000 7fb9aca2d700 1 -- start start 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a80ffc20 0x7fb9a819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:26.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a819c0f0 con 0x7fb9a8105bd0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81955a0 con 0x7fb9a80ffc20 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49118/0 (socket says 172.21.15.8:49118) 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 -- 172.21.15.8:0/2637780487 learned_addr learned my addr 172.21.15.8:0/2637780487 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a659c700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a80ffc20 0x7fb9a819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 -- 172.21.15.8:0/2637780487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a80ffc20 msgr2=0x7fb9a819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a80ffc20 0x7fb9a819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 -- 172.21.15.8:0/2637780487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9900095f0 con 0x7fb9a8105bd0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9a5d9b700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb990006010 tx=0x7fb99000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb99001d070 con 0x7fb9a8105bd0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb99000f460 con 0x7fb9a8105bd0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb990021640 con 0x7fb9a8105bd0 2024-04-12T05:53:26.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a8195820 con 0x7fb9a8105bd0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.025+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a8195cc0 con 0x7fb9a8105bd0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.026+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a81080d0 con 0x7fb9a8105bd0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.027+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fb99000fa20 con 0x7fb9a8105bd0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.028+0000 7fb99f7fe700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 0x7fb99406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.028+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb99008cf80 con 0x7fb9a8105bd0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.028+0000 7fb9a659c700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 0x7fb99406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.028+0000 7fb9a659c700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 0x7fb99406e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb998005950 tx=0x7fb9980058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:26.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.031+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb99005b170 con 0x7fb9a8105bd0 2024-04-12T05:53:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.273+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb9a804f2e0 con 0x7fb9a8105bd0 2024-04-12T05:53:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.274+0000 7fb99f7fe700 1 -- 172.21.15.8:0/2637780487 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+83 (secure 0 0 0) 0x7fb99005ad00 con 0x7fb9a8105bd0 2024-04-12T05:53:26.269 INFO:teuthology.orchestra.run.smithi008.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-12T05:53:26.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 msgr2=0x7fb99406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:26.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 0x7fb99406e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb998005950 tx=0x7fb9980058e0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 msgr2=0x7fb9a819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb990006010 tx=0x7fb99000be20 comp rx=0 tx=0).stop 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 shutdown_connections 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9a80ffc20 0x7fb9a819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb99406c530 0x7fb99406e9f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 --2- 172.21.15.8:0/2637780487 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9a8105bd0 0x7fb9a819ba60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 >> 172.21.15.8:0/2637780487 conn(0x7fb9a8075670 msgr2=0x7fb9a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 shutdown_connections 2024-04-12T05:53:26.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:26.276+0000 7fb9aca2d700 1 -- 172.21.15.8:0/2637780487 wait complete. 2024-04-12T05:53:26.649 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:26 smithi008 ceph-mon[39722]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:53:26.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:26 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2637780487' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T05:53:26.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:26 smithi194 ceph-mon[49771]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:53:26.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:26 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2637780487' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-12T05:53:27.021 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-12T05:53:27.021 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-12T05:53:27.021 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi194.front.sepia.ceph.com 2024-04-12T05:53:27.021 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-12T05:53:27.021 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:27.022 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-12T05:53:27.022 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-12T05:53:27.022 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-12T05:53:27.022 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-12T05:53:27.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.022 DEBUG:teuthology.orchestra.run.smithi194:> ip addr 2024-04-12T05:53:27.045 INFO:teuthology.orchestra.run.smithi194.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-12T05:53:27.045 INFO:teuthology.orchestra.run.smithi194.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-12T05:53:27.045 INFO:teuthology.orchestra.run.smithi194.stdout: inet 127.0.0.1/8 scope host lo 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 ::1/128 scope host 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:fe brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f0 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:d9:3a:ff brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: altname enp7s0f1 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a4 brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: link/ether 0c:c4:7a:88:7e:a5 brd ff:ff:ff:ff:ff:ff 2024-04-12T05:53:27.046 INFO:teuthology.orchestra.run.smithi194.stdout: inet 172.21.15.194/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-12T05:53:27.047 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft 41965sec preferred_lft 41965sec 2024-04-12T05:53:27.047 INFO:teuthology.orchestra.run.smithi194.stdout: inet6 fe80::ec4:7aff:fe88:7ea5/64 scope link 2024-04-12T05:53:27.047 INFO:teuthology.orchestra.run.smithi194.stdout: valid_lft forever preferred_lft forever 2024-04-12T05:53:27.047 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-12T05:53:27.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec bash -c ' 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link add name ceph-brx type bridge 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip addr flush dev ceph-brx 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set ceph-brx up 2024-04-12T05:53:27.047 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-12T05:53:27.048 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-04-12T05:53:27.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.233 DEBUG:teuthology.orchestra.run.smithi194:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-12T05:53:27.260 INFO:teuthology.orchestra.run.smithi194.stdout:1 2024-04-12T05:53:27.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.262 DEBUG:teuthology.orchestra.run.smithi194:> route 2024-04-12T05:53:27.372 INFO:teuthology.orchestra.run.smithi194.stdout:Kernel IP routing table 2024-04-12T05:53:27.372 INFO:teuthology.orchestra.run.smithi194.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-12T05:53:27.372 INFO:teuthology.orchestra.run.smithi194.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-12T05:53:27.373 INFO:teuthology.orchestra.run.smithi194.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-12T05:53:27.373 INFO:teuthology.orchestra.run.smithi194.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-12T05:53:27.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec bash -c ' 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-12T05:53:27.373 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-04-12T05:53:27.414 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:27.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.629 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list 2024-04-12T05:53:27.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:27.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.691 DEBUG:teuthology.orchestra.run.smithi194:> ip netns list-id 2024-04-12T05:53:27.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.749 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec bash -c ' 2024-04-12T05:53:27.749 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-04-12T05:53:27.749 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-12T05:53:27.750 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-12T05:53:27.750 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-04-12T05:53:27.845 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-12T05:53:27.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec bash -c ' 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-12T05:53:27.845 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-04-12T05:53:28.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:28.059 DEBUG:teuthology.orchestra.run.smithi194:> (cd / && exec bash -c ' 2024-04-12T05:53:28.059 DEBUG:teuthology.orchestra.run.smithi194:> set -e 2024-04-12T05:53:28.060 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set brx.0 up 2024-04-12T05:53:28.060 DEBUG:teuthology.orchestra.run.smithi194:> sudo ip link set dev brx.0 master ceph-brx 2024-04-12T05:53:28.060 DEBUG:teuthology.orchestra.run.smithi194:> ') 2024-04-12T05:53:28.157 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-12T05:53:28.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T05:53:28.157 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:28.174 INFO:teuthology.orchestra.run.smithi194.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-12T05:53:28.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-12T05:53:28.175 DEBUG:teuthology.orchestra.run.smithi194:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:28.232 DEBUG:teuthology.orchestra.run.smithi194:> sudo modprobe fuse 2024-04-12T05:53:28.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-12T05:53:28.300 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-12T05:53:28.366 INFO:teuthology.orchestra.run.smithi194.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-12T05:53:28.367 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-12T05:53:28.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:28.368 DEBUG:teuthology.orchestra.run.smithi194:> ls /sys/fs/fuse/connections 2024-04-12T05:53:28.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-12T05:53:28.426 DEBUG:teuthology.orchestra.run.smithi194:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-12T05:53:28.468 DEBUG:teuthology.orchestra.run.smithi194:> sudo modprobe fuse 2024-04-12T05:53:28.493 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-12T05:53:28.493 DEBUG:teuthology.orchestra.run.smithi194:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-04-12T05:53:28.561 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-04-12T05:53:28.561 INFO:teuthology.orchestra.run.smithi194.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-04-12T05:53:28.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:28.562 DEBUG:teuthology.orchestra.run.smithi194:> ls /sys/fs/fuse/connections 2024-04-12T05:53:28.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi194.stderr:2024-04-12T05:53:28.584+0000 7f42d093f5c0 -1 init, newargv = 0x55f27bf7f2a0 newargc=15 2024-04-12T05:53:28.579 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi194.stderr:ceph-fuse[81014]: starting ceph client 2024-04-12T05:53:28.586 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi194.stderr:ceph-fuse[81014]: starting fuse 2024-04-12T05:53:28.597 INFO:teuthology.orchestra.run.smithi194.stdout:137 2024-04-12T05:53:28.597 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [137] 2024-04-12T05:53:28.597 DEBUG:teuthology.orchestra.run.smithi194:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> import glob 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> import re 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> import os 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> import subprocess 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> def _find_admin_socket(client_name): 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> files = glob.glob(asok_path) 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> # Given a non-glob path, it better be there 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> if "*" not in asok_path: 2024-04-12T05:53:28.598 DEBUG:teuthology.orchestra.run.smithi194:> assert(len(files) == 1) 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> return files[0] 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> for f in files: 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> contents = proc_f.read() 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> if mountpoint in contents: 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> return f 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> print(_find_admin_socket("client.1")) 2024-04-12T05:53:28.599 DEBUG:teuthology.orchestra.run.smithi194:> ' 2024-04-12T05:53:28.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:28 smithi008 ceph-mon[39722]: pgmap v214: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-12T05:53:28.662 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:28 smithi194 ceph-mon[49771]: pgmap v214: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-12T05:53:28.720 INFO:teuthology.orchestra.run.smithi194.stdout:/var/run/ceph/ceph-client.1.81014.asok 2024-04-12T05:53:28.903 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.81014.asok 2024-04-12T05:53:28.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:28.904 DEBUG:teuthology.orchestra.run.smithi194:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.81014.asok status 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "metadata": { 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "entity_id": "1", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "hostname": "smithi194", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "pid": "81014", 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "root": "/" 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-12T05:53:29.060 INFO:teuthology.orchestra.run.smithi194.stdout: "dentry_count": 0, 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "dentry_pinned_count": 0, 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "id": 24331, 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "inst": { 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "name": { 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "type": "client", 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "num": 24331 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "addr": { 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "type": "v1", 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "addr": "192.168.0.1:0", 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: "nonce": 4244588227 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-04-12T05:53:29.061 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "addr": { 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "type": "v1", 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "addr": "192.168.0.1:0", 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "nonce": 4244588227 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "inst_str": "client.24331 192.168.0.1:0/4244588227", 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "addr_str": "192.168.0.1:0/4244588227", 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "inode_count": 1, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_epoch": 21, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_epoch": 46, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_epoch_barrier": 0, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "blocklisted": false, 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout: "fs_name": "cephfs" 2024-04-12T05:53:29.062 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-04-12T05:53:29.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:29.069 DEBUG:teuthology.orchestra.run.smithi008:> stat --file-system '--printf=%T 2024-04-12T05:53:29.070 DEBUG:teuthology.orchestra.run.smithi008:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:29.095 INFO:teuthology.orchestra.run.smithi008.stdout:fuseblk 2024-04-12T05:53:29.095 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:29.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:29.095 DEBUG:teuthology.orchestra.run.smithi008:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:29.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:29.173 DEBUG:teuthology.orchestra.run.smithi194:> stat --file-system '--printf=%T 2024-04-12T05:53:29.173 DEBUG:teuthology.orchestra.run.smithi194:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:29.192 INFO:teuthology.orchestra.run.smithi194.stdout:fuseblk 2024-04-12T05:53:29.192 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:29.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T05:53:29.192 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:30.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:30 smithi008 ceph-mon[39722]: pgmap v215: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-12T05:53:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:30 smithi194 ceph-mon[49771]: pgmap v215: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-12T05:53:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:31 smithi194 ceph-mon[49771]: pgmap v216: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-12T05:53:31.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:31 smithi008 ceph-mon[39722]: pgmap v216: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-12T05:53:33.779 INFO:teuthology.run_tasks:Running task print... 2024-04-12T05:53:33.789 INFO:teuthology.task.print:**** done client 2024-04-12T05:53:33.789 INFO:teuthology.run_tasks:Running task parallel... 2024-04-12T05:53:33.830 INFO:teuthology.task.parallel:starting parallel... 2024-04-12T05:53:33.831 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-12T05:53:33.831 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T05:53:33.831 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:53:33.832 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-12T05:53:33.832 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-12T05:53:33.832 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-12T05:53:33.836 INFO:tasks.workunit:Pulling workunits from ref d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:53:33.836 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-12T05:53:33.836 INFO:tasks.workunit:timeout=3h 2024-04-12T05:53:33.836 INFO:tasks.workunit:cleanup=True 2024-04-12T05:53:33.836 DEBUG:teuthology.orchestra.run.smithi008:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Device: adh/173d Inode: 1 Links: 2 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Modify: 2024-04-12 05:52:48.068029928 +0000 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout:Change: 2024-04-12 05:53:29.175441564 +0000 2024-04-12T05:53:33.861 INFO:teuthology.orchestra.run.smithi008.stdout: Birth: - 2024-04-12T05:53:33.862 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-12T05:53:33.862 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-12T05:53:33.937 DEBUG:teuthology.orchestra.run.smithi194:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 89h/137d Inode: 1 Links: 3 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-04-12 05:53:33.936129944 +0000 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-04-12 05:53:33.936129944 +0000 2024-04-12T05:53:33.962 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: - 2024-04-12T05:53:33.963 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-12T05:53:33.963 DEBUG:teuthology.orchestra.run.smithi194:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-12T05:53:34.042 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:53:34.042 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:53:34.062 INFO:tasks.workunit.client.0.smithi008.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-12T05:53:34.090 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:34.103 INFO:tasks.workunit.client.1.smithi194.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-12T05:53:34.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:34 smithi008 ceph-mon[39722]: pgmap v217: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:34 smithi194 ceph-mon[49771]: pgmap v217: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:36.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 -- 172.21.15.8:0/416357139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 msgr2=0x7fb0241064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/416357139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241064c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb020009910 tx=0x7fb020009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 -- 172.21.15.8:0/416357139 shutdown_connections 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/416357139 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241064c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/416357139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 0x7fb024103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 -- 172.21.15.8:0/416357139 >> 172.21.15.8:0/416357139 conn(0x7fb0240fb190 msgr2=0x7fb0240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 -- 172.21.15.8:0/416357139 shutdown_connections 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.184+0000 7fb02b6b2700 1 -- 172.21.15.8:0/416357139 wait complete. 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 Processor -- start 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 -- start start 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 0x7fb024196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:36.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024196fe0 con 0x7fb0241040d0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb02b6b2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024197120 con 0x7fb0241017a0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb028c4d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb028c4d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33864/0 (socket says 172.21.15.8:33864) 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.185+0000 7fb028c4d700 1 -- 172.21.15.8:0/3020847885 learned_addr learned my addr 172.21.15.8:0/3020847885 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb02944e700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 0x7fb024196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb028c4d700 1 -- 172.21.15.8:0/3020847885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 msgr2=0x7fb024196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb028c4d700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 0x7fb024196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb028c4d700 1 -- 172.21.15.8:0/3020847885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200095f0 con 0x7fb0241040d0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb028c4d700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb020006010 tx=0x7fb02000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02001d070 con 0x7fb0241040d0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02419bb70 con 0x7fb0241040d0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb020022470 con 0x7fb0241040d0 2024-04-12T05:53:36.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02000f690 con 0x7fb0241040d0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.186+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02419c060 con 0x7fb0241040d0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.187+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024109740 con 0x7fb0241040d0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.187+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fb020022980 con 0x7fb0241040d0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.188+0000 7fb0167fc700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 0x7fb01006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.188+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb02008cfa0 con 0x7fb0241040d0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.188+0000 7fb02944e700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 0x7fb01006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.188+0000 7fb02944e700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 0x7fb01006e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:36.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.193+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb02005b2d0 con 0x7fb0241040d0 2024-04-12T05:53:36.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.388+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fb02419c9e0 con 0x7fb0241040d0 2024-04-12T05:53:36.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.389+0000 7fb0167fc700 1 -- 172.21.15.8:0/3020847885 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb02005ae60 con 0x7fb0241040d0 2024-04-12T05:53:36.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 msgr2=0x7fb01006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:36.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 0x7fb01006e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb018006fd0 tx=0x7fb018008040 comp rx=0 tx=0).stop 2024-04-12T05:53:36.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 msgr2=0x7fb0241969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb020006010 tx=0x7fb02000be20 comp rx=0 tx=0).stop 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 shutdown_connections 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0241017a0 0x7fb024196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fb01006c4e0 0x7fb01006e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 --2- 172.21.15.8:0/3020847885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0241040d0 0x7fb0241969c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 >> 172.21.15.8:0/3020847885 conn(0x7fb0240fb190 msgr2=0x7fb0240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 shutdown_connections 2024-04-12T05:53:36.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:36.391+0000 7fb02b6b2700 1 -- 172.21.15.8:0/3020847885 wait complete. 2024-04-12T05:53:36.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:36 smithi008 ceph-mon[39722]: pgmap v218: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:36 smithi194 ceph-mon[49771]: pgmap v218: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:37.212 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-12T05:53:37.496 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:38 smithi008 ceph-mon[39722]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-12T05:53:38.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:38 smithi194 ceph-mon[49771]: pgmap v219: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-12T05:53:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 -- 172.21.15.8:0/1846171983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 msgr2=0x7efd3c0ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 --2- 172.21.15.8:0/1846171983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c0ff9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efd3400b0a0 tx=0x7efd3400b3b0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 -- 172.21.15.8:0/1846171983 shutdown_connections 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 --2- 172.21.15.8:0/1846171983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 0x7efd3c100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 --2- 172.21.15.8:0/1846171983 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c0ff9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 -- 172.21.15.8:0/1846171983 >> 172.21.15.8:0/1846171983 conn(0x7efd3c0fb120 msgr2=0x7efd3c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 -- 172.21.15.8:0/1846171983 shutdown_connections 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.584+0000 7efd414e7700 1 -- 172.21.15.8:0/1846171983 wait complete. 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 Processor -- start 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 -- start start 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 0x7efd3c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c199130 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd414e7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c199270 con 0x7efd3c0fff00 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33872/0 (socket says 172.21.15.8:33872) 2024-04-12T05:53:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 -- 172.21.15.8:0/519069372 learned_addr learned my addr 172.21.15.8:0/519069372 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3a7fc700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 0x7efd3c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 -- 172.21.15.8:0/519069372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 msgr2=0x7efd3c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 0x7efd3c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.585+0000 7efd3affd700 1 -- 172.21.15.8:0/519069372 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009d00 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd3affd700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efd34000f80 tx=0x7efd3400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3400e040 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7efd34003de0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c19dcc0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3401d7c0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.586+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c19e1b0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.587+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c10b8c0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.588+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7efd34019070 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.588+0000 7efd33fff700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 0x7efd2806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.588+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efd3408cfe0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.588+0000 7efd3a7fc700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 0x7efd2806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:39.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.589+0000 7efd3a7fc700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 0x7efd2806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd24005950 tx=0x7efd240058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:39.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.592+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7efd3405b2d0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.809+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7efd3c19e7c0 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.809+0000 7efd33fff700 1 -- 172.21.15.8:0/519069372 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7efd3405ae60 con 0x7efd3c0ff5a0 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.811+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 msgr2=0x7efd2806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.811+0000 7efd414e7700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 0x7efd2806e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd24005950 tx=0x7efd240058e0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.811+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 msgr2=0x7efd3c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.811+0000 7efd414e7700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efd34000f80 tx=0x7efd3400bdc0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 shutdown_connections 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7efd2806c4e0 0x7efd2806e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd3c0ff5a0 0x7efd3c1985d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 --2- 172.21.15.8:0/519069372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd3c0fff00 0x7efd3c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:39.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 >> 172.21.15.8:0/519069372 conn(0x7efd3c0fb120 msgr2=0x7efd3c107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:39.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 shutdown_connections 2024-04-12T05:53:39.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:39.812+0000 7efd414e7700 1 -- 172.21.15.8:0/519069372 wait complete. 2024-04-12T05:53:40.614 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:40 smithi008 ceph-mon[39722]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-12T05:53:40.635 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-12T05:53:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:40 smithi194 ceph-mon[49771]: pgmap v220: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-12T05:53:40.891 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:41.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:41 smithi194 ceph-mon[49771]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-12T05:53:41.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:41 smithi008 ceph-mon[39722]: pgmap v221: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-12T05:53:42.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 -- 172.21.15.8:0/1756559087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 msgr2=0x7f00401010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 --2- 172.21.15.8:0/1756559087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401010d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0030009910 tx=0x7f0030009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 -- 172.21.15.8:0/1756559087 shutdown_connections 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 --2- 172.21.15.8:0/1756559087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 0x7f0040102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 --2- 172.21.15.8:0/1756559087 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401010d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 -- 172.21.15.8:0/1756559087 >> 172.21.15.8:0/1756559087 conn(0x7f00400fc230 msgr2=0x7f00400fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 -- 172.21.15.8:0/1756559087 shutdown_connections 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.047+0000 7f0046186700 1 -- 172.21.15.8:0/1756559087 wait complete. 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 Processor -- start 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 -- start start 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 0x7f0040196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040196f40 con 0x7f0040100cb0 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f0046186700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040197080 con 0x7f0040101eb0 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f003f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f003f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33880/0 (socket says 172.21.15.8:33880) 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.048+0000 7f003f7fe700 1 -- 172.21.15.8:0/25541997 learned_addr learned my addr 172.21.15.8:0/25541997 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003effd700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 0x7f0040196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003f7fe700 1 -- 172.21.15.8:0/25541997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 msgr2=0x7f0040196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003f7fe700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 0x7f0040196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003f7fe700 1 -- 172.21.15.8:0/25541997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300095f0 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003f7fe700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f003000bda0 tx=0x7f003000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003001d070 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004019bad0 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f003000f460 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0030022650 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.049+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004019bfc0 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.050+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00401096d0 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.051+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f003000fa20 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.051+0000 7f003cff9700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 0x7f002806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.051+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f003008cbe0 con 0x7f0040100cb0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.051+0000 7f003effd700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 0x7f002806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.052+0000 7f003effd700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 0x7f002806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f003400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:43.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.060+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f003005adf0 con 0x7f0040100cb0 2024-04-12T05:53:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.264+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f0040109890 con 0x7f0040100cb0 2024-04-12T05:53:43.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.265+0000 7f003cff9700 1 -- 172.21.15.8:0/25541997 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0040109890 con 0x7f0040100cb0 2024-04-12T05:53:43.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 msgr2=0x7f002806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:43.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 0x7f002806e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f003400b410 comp rx=0 tx=0).stop 2024-04-12T05:53:43.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 msgr2=0x7f00401963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:43.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f003000bda0 tx=0x7f003000bdd0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 shutdown_connections 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f002806c4e0 0x7f002806e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0040100cb0 0x7f00401963e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 --2- 172.21.15.8:0/25541997 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0040101eb0 0x7f0040196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 >> 172.21.15.8:0/25541997 conn(0x7f00400fc230 msgr2=0x7f00401050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 shutdown_connections 2024-04-12T05:53:43.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:43.267+0000 7f0046186700 1 -- 172.21.15.8:0/25541997 wait complete. 2024-04-12T05:53:43.650 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:43 smithi008 ceph-mon[39722]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-12T05:53:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:43 smithi194 ceph-mon[49771]: pgmap v222: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-12T05:53:43.987 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-12T05:53:44.246 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:53:46.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:46 smithi008 ceph-mon[39722]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:46.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 -- 172.21.15.8:0/4022079011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241017a0 msgr2=0x7f4624103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:46.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 --2- 172.21.15.8:0/4022079011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241017a0 0x7f4624103b90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4620009910 tx=0x7f4620009c20 comp rx=0 tx=0).stop 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 -- 172.21.15.8:0/4022079011 shutdown_connections 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 --2- 172.21.15.8:0/4022079011 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241040d0 0x7f46241064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 --2- 172.21.15.8:0/4022079011 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241017a0 0x7f4624103b90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 -- 172.21.15.8:0/4022079011 >> 172.21.15.8:0/4022079011 conn(0x7f46240fb190 msgr2=0x7f46240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 -- 172.21.15.8:0/4022079011 shutdown_connections 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.471+0000 7f462b8a0700 1 -- 172.21.15.8:0/4022079011 wait complete. 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 Processor -- start 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 -- start start 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 0x7f4624101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240ffc90 con 0x7f46241040d0 2024-04-12T05:53:46.492 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462b8a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240ffdd0 con 0x7f46241017a0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57930/0 (socket says 172.21.15.8:57930) 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 -- 172.21.15.8:0/1479352204 learned_addr learned my addr 172.21.15.8:0/1479352204 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462963c700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 0x7f4624101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 -- 172.21.15.8:0/1479352204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 msgr2=0x7f4624101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 0x7f4624101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 -- 172.21.15.8:0/1479352204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200095f0 con 0x7f46241040d0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f462963c700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 0x7f4624101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f4628e3b700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f461400c8a0 tx=0x7f461400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.472+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461400cea0 con 0x7f46241040d0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.473+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4614004830 con 0x7f46241040d0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.473+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46241000b0 con 0x7f46241040d0 2024-04-12T05:53:46.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.473+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4614005540 con 0x7f46241040d0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.473+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462419e3f0 con 0x7f46241040d0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.473+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46241904d0 con 0x7f46241040d0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.474+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f46140049a0 con 0x7f46241040d0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.475+0000 7f461a7fc700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 0x7f461006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.475+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f461408b8a0 con 0x7f46241040d0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.475+0000 7f462963c700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 0x7f461006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.475+0000 7f462963c700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 0x7f461006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f46200059d0 tx=0x7f4620005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:53:46.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.481+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4614059bd0 con 0x7f46241040d0 2024-04-12T05:53:46.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:53:46.717+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f4624061280 con 0x7f461006c4e0 2024-04-12T05:53:46.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:46 smithi194 ceph-mon[49771]: pgmap v223: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:48.380 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:48 smithi008 ceph-mon[39722]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:53:48.381 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:48 smithi008 ceph-mon[39722]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-12T05:53:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:48 smithi194 ceph-mon[49771]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:53:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:48 smithi194 ceph-mon[49771]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-12T05:53:50.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:50 smithi008 ceph-mon[39722]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-12T05:53:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:50 smithi194 ceph-mon[49771]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-12T05:53:51.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:51 smithi194 ceph-mon[49771]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-12T05:53:51.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:51 smithi008 ceph-mon[39722]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-12T05:53:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:54 smithi008 ceph-mon[39722]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:54 smithi194 ceph-mon[49771]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-12T05:53:56.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:56 smithi194 ceph-mon[49771]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-12T05:53:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:56 smithi008 ceph-mon[39722]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-12T05:53:57.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:57.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:53:58.387 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:53:58 smithi008 ceph-mon[39722]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-12T05:53:58.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:53:58 smithi194 ceph-mon[49771]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-12T05:54:00.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:00 smithi008 ceph-mon[39722]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-12T05:54:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:00 smithi194 ceph-mon[49771]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-12T05:54:01.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:01 smithi194 ceph-mon[49771]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:01.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:01 smithi008 ceph-mon[39722]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:04.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:04 smithi008 ceph-mon[39722]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:04.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:04 smithi194 ceph-mon[49771]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:06.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:06 smithi194 ceph-mon[49771]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:06 smithi008 ceph-mon[39722]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:08.391 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:08 smithi008 ceph-mon[39722]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:08.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:08 smithi194 ceph-mon[49771]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:10 smithi008 ceph-mon[39722]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:10.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:10 smithi194 ceph-mon[49771]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:11 smithi194 ceph-mon[49771]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:11 smithi008 ceph-mon[39722]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:13.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:13 smithi194 ceph-mon[49771]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:13.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:13 smithi008 ceph-mon[39722]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:15.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:15 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:15.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:15 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:16 smithi194 ceph-mon[49771]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:54:16.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:16 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:16 smithi008 ceph-mon[39722]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:54:16.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:16 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:17.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.315+0000 7f461a7fc700 1 -- 172.21.15.8:0/1479352204 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f4624061280 con 0x7f461006c4e0 2024-04-12T05:54:17.311 INFO:teuthology.orchestra.run.smithi008.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 msgr2=0x7f461006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 0x7f461006e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f46200059d0 tx=0x7f4620005920 comp rx=0 tx=0).stop 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 msgr2=0x7f46240ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f461400c8a0 tx=0x7f461400cc60 comp rx=0 tx=0).stop 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 shutdown_connections 2024-04-12T05:54:17.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46241017a0 0x7f4624101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:17.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f461006c4e0 0x7f461006e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:17.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 --2- 172.21.15.8:0/1479352204 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46241040d0 0x7f46240ff750 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:17.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 >> 172.21.15.8:0/1479352204 conn(0x7f46240fb190 msgr2=0x7f46240fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:17.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 shutdown_connections 2024-04-12T05:54:17.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:17.318+0000 7f462b8a0700 1 -- 172.21.15.8:0/1479352204 wait complete. 2024-04-12T05:54:18.132 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:54:18.315 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:18 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:54:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:18 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:19.093 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:54:19.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:19 smithi194 ceph-mon[49771]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:54:19.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:19 smithi008 ceph-mon[39722]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:54:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:20 smithi194 ceph-mon[49771]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:20.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:20 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:20 smithi008 ceph-mon[39722]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:54:20.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:20 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 -- 172.21.15.8:0/311789225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc0ff320 msgr2=0x7fafcc0ff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 --2- 172.21.15.8:0/311789225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc0ff320 0x7fafcc0ff740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fafc800aa20 tx=0x7fafc800ad30 comp rx=0 tx=0).stop 2024-04-12T05:54:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 -- 172.21.15.8:0/311789225 shutdown_connections 2024-04-12T05:54:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 --2- 172.21.15.8:0/311789225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc100520 0x7fafcc100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 --2- 172.21.15.8:0/311789225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc0ff320 0x7fafcc0ff740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 -- 172.21.15.8:0/311789225 >> 172.21.15.8:0/311789225 conn(0x7fafcc0fa8c0 msgr2=0x7fafcc0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 -- 172.21.15.8:0/311789225 shutdown_connections 2024-04-12T05:54:21.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.648+0000 7fafd39dc700 1 -- 172.21.15.8:0/311789225 wait complete. 2024-04-12T05:54:21.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 Processor -- start 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 -- start start 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 0x7fafcc0723c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc0729e0 con 0x7fafcc100520 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd39dc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafcc1a1f70 con 0x7fafcc0ff320 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:45030/0 (socket says 172.21.15.8:45030) 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 -- 172.21.15.8:0/3803864505 learned_addr learned my addr 172.21.15.8:0/3803864505 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd0f77700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 0x7fafcc0723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 -- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 msgr2=0x7fafcc0723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 0x7fafcc0723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd1778700 1 -- 172.21.15.8:0/3803864505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc800a700 con 0x7fafcc0ff320 2024-04-12T05:54:21.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.649+0000 7fafd0f77700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 0x7fafcc0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:54:21.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafd1778700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a3f0 tx=0x7fafc800af40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:21.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc8005000 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafcc1a2110 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafcc1a47c0 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fafc8005160 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.650+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafc8016760 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.651+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb0005320 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.652+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fafc800e450 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.652+0000 7fafc27fc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 0x7fafb806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.652+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fafc8012070 con 0x7fafcc0ff320 2024-04-12T05:54:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.652+0000 7fafd0f77700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 0x7fafb806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:21.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.653+0000 7fafd0f77700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 0x7fafb806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafcc072f70 tx=0x7fafbc009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:21 smithi008 ceph-mon[39722]: Upgrade: Target is version 18.2.2-1187-gd540ebac (reef) 2024-04-12T05:54:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:21 smithi008 ceph-mon[39722]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3'] 2024-04-12T05:54:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:21 smithi008 ceph-mon[39722]: Upgrade: Need to upgrade myself (mgr.smithi008.ixmnlp) 2024-04-12T05:54:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:21 smithi008 ceph-mon[39722]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 on smithi194 2024-04-12T05:54:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.661+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fafc8057cb0 con 0x7fafcc0ff320 2024-04-12T05:54:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:21 smithi194 ceph-mon[49771]: Upgrade: Target is version 18.2.2-1187-gd540ebac (reef) 2024-04-12T05:54:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:21 smithi194 ceph-mon[49771]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3'] 2024-04-12T05:54:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:21 smithi194 ceph-mon[49771]: Upgrade: Need to upgrade myself (mgr.smithi008.ixmnlp) 2024-04-12T05:54:21.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:21 smithi194 ceph-mon[49771]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 on smithi194 2024-04-12T05:54:21.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.945+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb0000bf0 con 0x7fafb806c4e0 2024-04-12T05:54:21.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.947+0000 7fafc27fc700 1 -- 172.21.15.8:0/3803864505 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fafb0000bf0 con 0x7fafb806c4e0 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 msgr2=0x7fafb806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 0x7fafb806e9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafcc072f70 tx=0x7fafbc009260 comp rx=0 tx=0).stop 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 msgr2=0x7fafcc071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fafc800a3f0 tx=0x7fafc800af40 comp rx=0 tx=0).stop 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 shutdown_connections 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fafcc0ff320 0x7fafcc071e80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fafb806c4e0 0x7fafb806e9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 --2- 172.21.15.8:0/3803864505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fafcc100520 0x7fafcc0723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 >> 172.21.15.8:0/3803864505 conn(0x7fafcc0fa8c0 msgr2=0x7fafcc103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 shutdown_connections 2024-04-12T05:54:21.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:21.949+0000 7fafd39dc700 1 -- 172.21.15.8:0/3803864505 wait complete. 2024-04-12T05:54:21.959 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:54:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 -- 172.21.15.8:0/1487935898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 msgr2=0x7f1cd4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/1487935898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4101270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009910 tx=0x7f1cc8009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 -- 172.21.15.8:0/1487935898 shutdown_connections 2024-04-12T05:54:22.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/1487935898 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4101270 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/1487935898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 -- 172.21.15.8:0/1487935898 >> 172.21.15.8:0/1487935898 conn(0x7f1cd40fb190 msgr2=0x7f1cd40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 -- 172.21.15.8:0/1487935898 shutdown_connections 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.079+0000 7f1cd9b84700 1 -- 172.21.15.8:0/1487935898 wait complete. 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 Processor -- start 2024-04-12T05:54:22.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 -- start start 2024-04-12T05:54:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4072380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40729a0 con 0x7f1cd4100e10 2024-04-12T05:54:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.080+0000 7f1cd9b84700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd419ff50 con 0x7f1cd40ffc10 2024-04-12T05:54:22.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:45058/0 (socket says 172.21.15.8:45058) 2024-04-12T05:54:22.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 -- 172.21.15.8:0/220407959 learned_addr learned my addr 172.21.15.8:0/220407959 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:22.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd2ffd700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4072380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 -- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 msgr2=0x7f1cd4072380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4072380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 -- 172.21.15.8:0/220407959 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80095f0 con 0x7f1cd40ffc10 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd37fe700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e8f0 tx=0x7f1cc400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.081+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc4009790 con 0x7f1cd40ffc10 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.082+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd41a0230 con 0x7f1cd40ffc10 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.082+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd41a0750 con 0x7f1cd40ffc10 2024-04-12T05:54:22.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.082+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1cc40044b0 con 0x7f1cd40ffc10 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.082+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cc4003d30 con 0x7f1cd40ffc10 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.083+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc0005320 con 0x7f1cd40ffc10 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.083+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f1cc400cba0 con 0x7f1cd40ffc10 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.084+0000 7f1cd0ff9700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 0x7f1cbc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.084+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1cc4014070 con 0x7f1cd40ffc10 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.084+0000 7f1cd2ffd700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 0x7f1cbc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.085+0000 7f1cd2ffd700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 0x7f1cbc06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cd419ded0 tx=0x7f1cc8009f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:22.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.092+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1cc4059d50 con 0x7f1cd40ffc10 2024-04-12T05:54:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.462+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cc0000bf0 con 0x7f1cbc06c4e0 2024-04-12T05:54:22.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.464+0000 7f1cd0ff9700 1 -- 172.21.15.8:0/220407959 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f1cc0000bf0 con 0x7f1cbc06c4e0 2024-04-12T05:54:22.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 msgr2=0x7f1cbc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 0x7f1cbc06e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cd419ded0 tx=0x7f1cc8009f90 comp rx=0 tx=0).stop 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 msgr2=0x7f1cd4071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400e8f0 tx=0x7f1cc400ecb0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 shutdown_connections 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cd40ffc10 0x7f1cd4071e40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f1cbc06c4e0 0x7f1cbc06e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 --2- 172.21.15.8:0/220407959 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1cd4100e10 0x7f1cd4072380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 >> 172.21.15.8:0/220407959 conn(0x7f1cd40fb190 msgr2=0x7f1cd4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:22.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 shutdown_connections 2024-04-12T05:54:22.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.466+0000 7f1cd9b84700 1 -- 172.21.15.8:0/220407959 wait complete. 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 -- 172.21.15.8:0/3859770592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 msgr2=0x7fc438102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 --2- 172.21.15.8:0/3859770592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc438102300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc428009910 tx=0x7fc428009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 -- 172.21.15.8:0/3859770592 shutdown_connections 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 --2- 172.21.15.8:0/3859770592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc438102300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 --2- 172.21.15.8:0/3859770592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc4381010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 -- 172.21.15.8:0/3859770592 >> 172.21.15.8:0/3859770592 conn(0x7fc4380fc260 msgr2=0x7fc4380fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 -- 172.21.15.8:0/3859770592 shutdown_connections 2024-04-12T05:54:22.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.600+0000 7fc43fc87700 1 -- 172.21.15.8:0/3859770592 wait complete. 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 Processor -- start 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 -- start start 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc4380723d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380729f0 con 0x7fc438101ea0 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.601+0000 7fc43fc87700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4381a41c0 con 0x7fc438100ca0 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:45066/0 (socket says 172.21.15.8:45066) 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 -- 172.21.15.8:0/1454681780 learned_addr learned my addr 172.21.15.8:0/1454681780 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43d222700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc4380723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 -- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 msgr2=0x7fc4380723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc4380723d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 -- 172.21.15.8:0/1454681780 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280095f0 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43da23700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e8f0 tx=0x7fc43400ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc434009790 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4381a43c0 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fc43400c810 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.602+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc434003c20 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.603+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4381a4890 con 0x7fc438100ca0 2024-04-12T05:54:22.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.603+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4381096c0 con 0x7fc438100ca0 2024-04-12T05:54:22.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.605+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fc43400c980 con 0x7fc438100ca0 2024-04-12T05:54:22.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.605+0000 7fc42effd700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 0x7fc42406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:22.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.605+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc43403a080 con 0x7fc438100ca0 2024-04-12T05:54:22.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.605+0000 7fc43d222700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 0x7fc42406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:22.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.606+0000 7fc43d222700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 0x7fc42406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4381a0130 tx=0x7fc42800b5a0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:22.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.613+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc43405a110 con 0x7fc438100ca0 2024-04-12T05:54:22.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:22 smithi008 ceph-mon[39722]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:22.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:22 smithi194 ceph-mon[49771]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:22.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.905+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4381067f0 con 0x7fc42406c4e0 2024-04-12T05:54:22.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.922+0000 7fc42effd700 1 -- 172.21.15.8:0/1454681780 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fc4381067f0 con 0x7fc42406c4e0 2024-04-12T05:54:22.917 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:54:22.917 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (6m) 67s ago 7m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:54:22.917 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (7m) 67s ago 7m 7478k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (6m) 74s ago 6m 7474k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (7m) 67s ago 7m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (6m) 74s ago 6m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (5m) 67s ago 7m 86.1M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (89s) 67s ago 89s 15.0M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (96s) 67s ago 95s 12.8M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (86s) 74s ago 85s 16.1M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (93s) 74s ago 92s 18.4M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:9283,8765,8443 running (9m) 67s ago 9m 489M - 18.2.1 5be31c24972a 545bb7b78dec 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (6m) 74s ago 6m 442M - 18.2.1 5be31c24972a 324447c2f903 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (9m) 67s ago 9m 46.9M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (6m) 74s ago 6m 36.2M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:54:22.918 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (7m) 67s ago 7m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (6m) 74s ago 6m 17.8M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (5m) 67s ago 5m 48.1M 3959M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (5m) 67s ago 5m 48.8M 3959M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (4m) 67s ago 4m 45.8M 3959M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (4m) 74s ago 4m 46.1M 4982M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (3m) 74s ago 3m 46.4M 4982M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (3m) 74s ago 3m 44.0M 4982M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:54:22.919 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (5m) 67s ago 7m 56.7M - 2.43.0 a07b618ecd1d c0932eae2f67 2024-04-12T05:54:22.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 msgr2=0x7fc42406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 0x7fc42406e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4381a0130 tx=0x7fc42800b5a0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 msgr2=0x7fc438071e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc43400e8f0 tx=0x7fc43400ec00 comp rx=0 tx=0).stop 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 shutdown_connections 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc438100ca0 0x7fc438071e90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fc42406c4e0 0x7fc42406e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 --2- 172.21.15.8:0/1454681780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc438101ea0 0x7fc4380723d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 >> 172.21.15.8:0/1454681780 conn(0x7fc4380fc260 msgr2=0x7fc4381050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 shutdown_connections 2024-04-12T05:54:22.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:22.925+0000 7fc43fc87700 1 -- 172.21.15.8:0/1454681780 wait complete. 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 -- 172.21.15.8:0/1921862726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 msgr2=0x7f2904100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 --2- 172.21.15.8:0/1921862726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904100030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009910 tx=0x7f28f4009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 -- 172.21.15.8:0/1921862726 shutdown_connections 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 --2- 172.21.15.8:0/1921862726 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 0x7f2904101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 --2- 172.21.15.8:0/1921862726 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904100030 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 -- 172.21.15.8:0/1921862726 >> 172.21.15.8:0/1921862726 conn(0x7f29040fb190 msgr2=0x7f29040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 -- 172.21.15.8:0/1921862726 shutdown_connections 2024-04-12T05:54:23.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.053+0000 7f2909df3700 1 -- 172.21.15.8:0/1921862726 wait complete. 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 Processor -- start 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 -- start start 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 0x7f2904195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904195e60 con 0x7f29040ffc10 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.054+0000 7f2909df3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904195fa0 con 0x7f2904100e10 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46596/0 (socket says 172.21.15.8:46596) 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 -- 172.21.15.8:0/3683751260 learned_addr learned my addr 172.21.15.8:0/3683751260 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:23.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f2902ffd700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 0x7f2904195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 -- 172.21.15.8:0/3683751260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 msgr2=0x7f2904195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 0x7f2904195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 -- 172.21.15.8:0/3683751260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40095f0 con 0x7f29040ffc10 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f29037fe700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f401d070 con 0x7f29040ffc10 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f28f40059e0 con 0x7f29040ffc10 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290406a8f0 con 0x7f29040ffc10 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.055+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f4003980 con 0x7f29040ffc10 2024-04-12T05:54:23.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.056+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290406ad80 con 0x7f29040ffc10 2024-04-12T05:54:23.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.057+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29041085c0 con 0x7f29040ffc10 2024-04-12T05:54:23.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.058+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f28f40041f0 con 0x7f29040ffc10 2024-04-12T05:54:23.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.058+0000 7f2900ff9700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 0x7f28ec06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.058+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f28f408cea0 con 0x7f29040ffc10 2024-04-12T05:54:23.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.058+0000 7f2902ffd700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 0x7f28ec06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.054 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.059+0000 7f2902ffd700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 0x7f28ec06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009c60 tx=0x7f28f8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:23.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.066+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f28f405b0b0 con 0x7f29040ffc10 2024-04-12T05:54:23.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.376+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f290406b990 con 0x7f29040ffc10 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.377+0000 7f2900ff9700 1 -- 172.21.15.8:0/3683751260 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f28f405ac40 con 0x7f29040ffc10 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:23.372 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:54:23.373 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:54:23.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 msgr2=0x7f28ec06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 0x7f28ec06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009c60 tx=0x7f28f8009260 comp rx=0 tx=0).stop 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 msgr2=0x7f2904195300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f400bde0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 shutdown_connections 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f28ec06c4e0 0x7f28ec06e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f29040ffc10 0x7f2904195300 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 --2- 172.21.15.8:0/3683751260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2904100e10 0x7f2904195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 >> 172.21.15.8:0/3683751260 conn(0x7f29040fb190 msgr2=0x7f2904104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 shutdown_connections 2024-04-12T05:54:23.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.379+0000 7f2909df3700 1 -- 172.21.15.8:0/3683751260 wait complete. 2024-04-12T05:54:23.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 -- 172.21.15.8:0/2426359283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 msgr2=0x7fa1040ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/2426359283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1040ff9b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa100009910 tx=0x7fa100009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:23.485 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 -- 172.21.15.8:0/2426359283 shutdown_connections 2024-04-12T05:54:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/2426359283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/2426359283 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1040ff9b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 -- 172.21.15.8:0/2426359283 >> 172.21.15.8:0/2426359283 conn(0x7fa1040fb190 msgr2=0x7fa1040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 -- 172.21.15.8:0/2426359283 shutdown_connections 2024-04-12T05:54:23.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 -- 172.21.15.8:0/2426359283 wait complete. 2024-04-12T05:54:23.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.490+0000 7fa10b8f2700 1 Processor -- start 2024-04-12T05:54:23.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10b8f2700 1 -- start start 2024-04-12T05:54:23.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10b8f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1041985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10b8f2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10b8f2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104199120 con 0x7fa1040ff590 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10b8f2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104199260 con 0x7fa1040ffef0 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:58270/0 (socket says 172.21.15.8:58270) 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 -- 172.21.15.8:0/3655961328 learned_addr learned my addr 172.21.15.8:0/3655961328 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa10968e700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1041985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 -- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 msgr2=0x7fa1041985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1041985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.491+0000 7fa108e8d700 1 -- 172.21.15.8:0/3655961328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1000095f0 con 0x7fa1040ffef0 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa108e8d700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e920 tx=0x7fa0f400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:23.488 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f400c9d0 con 0x7fa1040ffef0 2024-04-12T05:54:23.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10419dd10 con 0x7fa1040ffef0 2024-04-12T05:54:23.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa0f4010430 con 0x7fa1040ffef0 2024-04-12T05:54:23.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f4018630 con 0x7fa1040ffef0 2024-04-12T05:54:23.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.492+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10419e260 con 0x7fa1040ffef0 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.493+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10410b8b0 con 0x7fa1040ffef0 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.494+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7fa0f40105a0 con 0x7fa1040ffef0 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.494+0000 7fa0fa7fc700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 0x7fa0f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.494+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa0f4014070 con 0x7fa1040ffef0 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.494+0000 7fa10968e700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 0x7fa0f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:23.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.495+0000 7fa10968e700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 0x7fa0f006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa100009fd0 tx=0x7fa100005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:23.496 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.501+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa0f40520e0 con 0x7fa1040ffef0 2024-04-12T05:54:23.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:23 smithi008 ceph-mon[39722]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:23.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.724+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1041092e0 con 0x7fa0f006c4e0 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.726+0000 7fa0fa7fc700 1 -- 172.21.15.8:0/3655961328 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fa1041092e0 con 0x7fa0f006c4e0 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "0/2 daemons upgraded", 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 image on host smithi194", 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:54:23.721 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:54:23.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 msgr2=0x7fa0f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 0x7fa0f006e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa100009fd0 tx=0x7fa100005fb0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 msgr2=0x7fa104198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400e920 tx=0x7fa0f400ec30 comp rx=0 tx=0).stop 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 shutdown_connections 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7fa0f006c4e0 0x7fa0f006e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1040ff590 0x7fa1041985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 --2- 172.21.15.8:0/3655961328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1040ffef0 0x7fa104198b00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 >> 172.21.15.8:0/3655961328 conn(0x7fa1040fb190 msgr2=0x7fa104107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 shutdown_connections 2024-04-12T05:54:23.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:23.728+0000 7fa10b8f2700 1 -- 172.21.15.8:0/3655961328 wait complete. 2024-04-12T05:54:23.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:23 smithi194 ceph-mon[49771]: from='client.24345 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:24 smithi008 ceph-mon[39722]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:24 smithi008 ceph-mon[39722]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:24 smithi008 ceph-mon[39722]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:24 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3683751260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:54:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:24 smithi194 ceph-mon[49771]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:24 smithi194 ceph-mon[49771]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:24 smithi194 ceph-mon[49771]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:24 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3683751260' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:54:25.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:25 smithi008 ceph-mon[39722]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:25.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:25 smithi194 ceph-mon[49771]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:26.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:26 smithi008 ceph-mon[39722]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:26.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:26 smithi194 ceph-mon[49771]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:27.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:27 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:27.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:27 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:28.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:28 smithi008 ceph-mon[39722]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:28 smithi194 ceph-mon[49771]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:30 smithi008 ceph-mon[39722]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:30.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:30 smithi194 ceph-mon[49771]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:31.604 INFO:tasks.workunit.client.0.smithi008.stderr:Updating files: 96% (11712/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-04-12T05:54:31.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:31 smithi194 ceph-mon[49771]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:31.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:31 smithi008 ceph-mon[39722]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr:Note: switching to 'd540ebaca6b131a1dd560e7f69e024b133bbaa42'. 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr:state without impacting any branches by switching back to a branch. 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.375 INFO:tasks.workunit.client.0.smithi008.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: git switch -c 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr:Or undo this operation with: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: git switch - 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr: 2024-04-12T05:54:32.376 INFO:tasks.workunit.client.0.smithi008.stderr:HEAD is now at d540ebaca6b Merge pull request #56672 from batrick/wip-65294-reef 2024-04-12T05:54:32.381 DEBUG:teuthology.orchestra.run.smithi008:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-12T05:54:32.411 INFO:tasks.workunit.client.0.smithi008.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-12T05:54:32.414 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-12T05:54:32.414 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-12T05:54:32.936 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-12T05:54:33.001 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-12T05:54:33.041 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-12T05:54:33.042 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-12T05:54:33.042 INFO:tasks.workunit.client.0.smithi008.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-12T05:54:33.080 INFO:tasks.workunit.client.0.smithi008.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-12T05:54:33.084 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T05:54:33.084 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-12T05:54:33.143 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-12T05:54:33.144 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-12T05:54:33.144 DEBUG:teuthology.orchestra.run.smithi008:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d540ebaca6b131a1dd560e7f69e024b133bbaa42 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-12T05:54:33.214 INFO:tasks.workunit.client.0.smithi008.stderr:+ mkdir -p fsstress 2024-04-12T05:54:33.215 INFO:tasks.workunit.client.0.smithi008.stderr:+ pushd fsstress 2024-04-12T05:54:33.217 INFO:tasks.workunit.client.0.smithi008.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T05:54:33.217 INFO:tasks.workunit.client.0.smithi008.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-12T05:54:33.779 INFO:tasks.workunit.client.0.smithi008.stderr:+ tar xzf ltp-full.tgz 2024-04-12T05:54:34.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:34 smithi008 ceph-mon[39722]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:34 smithi194 ceph-mon[49771]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:54:36.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:36 smithi008 ceph-mon[39722]: pgmap v248: 65 pgs: 65 active+clean; 644 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-04-12T05:54:36.700 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:36 smithi194 ceph-mon[49771]: pgmap v248: 65 pgs: 65 active+clean; 644 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 4 op/s 2024-04-12T05:54:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:38 smithi008 ceph-mon[39722]: pgmap v249: 65 pgs: 65 active+clean; 8.8 MiB data, 181 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 719 KiB/s wr, 27 op/s 2024-04-12T05:54:38.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:38 smithi194 ceph-mon[49771]: pgmap v249: 65 pgs: 65 active+clean; 8.8 MiB data, 181 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 719 KiB/s wr, 27 op/s 2024-04-12T05:54:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:40 smithi008 ceph-mon[39722]: pgmap v250: 65 pgs: 65 active+clean; 15 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-04-12T05:54:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:40 smithi194 ceph-mon[49771]: pgmap v250: 65 pgs: 65 active+clean; 15 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 2.9 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-04-12T05:54:41.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:41 smithi008 ceph-mon[39722]: pgmap v251: 65 pgs: 65 active+clean; 21 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-04-12T05:54:41.773 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:41 smithi194 ceph-mon[49771]: pgmap v251: 65 pgs: 65 active+clean; 21 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-04-12T05:54:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:42 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:42 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:43.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:43 smithi194 ceph-mon[49771]: pgmap v252: 65 pgs: 65 active+clean; 26 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-04-12T05:54:43.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:43 smithi008 ceph-mon[39722]: pgmap v252: 65 pgs: 65 active+clean; 26 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-04-12T05:54:46.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:46 smithi008 ceph-mon[39722]: pgmap v253: 65 pgs: 65 active+clean; 28 MiB data, 277 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-04-12T05:54:46.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:46 smithi194 ceph-mon[49771]: pgmap v253: 65 pgs: 65 active+clean; 28 MiB data, 277 MiB used, 536 GiB / 536 GiB avail; 342 KiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-04-12T05:54:48.401 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:48 smithi008 ceph-mon[39722]: pgmap v254: 65 pgs: 65 active+clean; 34 MiB data, 301 MiB used, 536 GiB / 536 GiB avail; 520 KiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-04-12T05:54:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:48 smithi194 ceph-mon[49771]: pgmap v254: 65 pgs: 65 active+clean; 34 MiB data, 301 MiB used, 536 GiB / 536 GiB avail; 520 KiB/s rd, 2.9 MiB/s wr, 272 op/s 2024-04-12T05:54:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:50 smithi194 ceph-mon[49771]: pgmap v255: 65 pgs: 65 active+clean; 42 MiB data, 341 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-12T05:54:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:50 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:54:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:50 smithi008 ceph-mon[39722]: pgmap v255: 65 pgs: 65 active+clean; 42 MiB data, 341 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-12T05:54:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:50 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:54:51.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:51 smithi194 ceph-mon[49771]: pgmap v256: 65 pgs: 65 active+clean; 48 MiB data, 382 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-04-12T05:54:51.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:51 smithi008 ceph-mon[39722]: pgmap v256: 65 pgs: 65 active+clean; 48 MiB data, 382 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 314 op/s 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 -- 172.21.15.8:0/172833879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c0ffc10 msgr2=0x7f238c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 --2- 172.21.15.8:0/172833879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c0ffc10 0x7f238c100030 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2388009910 tx=0x7f2388009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 -- 172.21.15.8:0/172833879 shutdown_connections 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 --2- 172.21.15.8:0/172833879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c100e10 0x7f238c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 --2- 172.21.15.8:0/172833879 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c0ffc10 0x7f238c100030 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:53.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.862+0000 7f239335f700 1 -- 172.21.15.8:0/172833879 >> 172.21.15.8:0/172833879 conn(0x7f238c0fb190 msgr2=0x7f238c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 -- 172.21.15.8:0/172833879 shutdown_connections 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 -- 172.21.15.8:0/172833879 wait complete. 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 Processor -- start 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 -- start start 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c0ffc10 0x7f238c0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c19b730 con 0x7f238c100e10 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.863+0000 7f239335f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c19b870 con 0x7f238c0ffc10 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47854/0 (socket says 172.21.15.8:47854) 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 -- 172.21.15.8:0/1157010799 learned_addr learned my addr 172.21.15.8:0/1157010799 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23910fb700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c0ffc10 0x7f238c0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 -- 172.21.15.8:0/1157010799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c0ffc10 msgr2=0x7f238c0690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c0ffc10 0x7f238c0690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:53.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 -- 172.21.15.8:0/1157010799 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009520 con 0x7f238c100e10 2024-04-12T05:54:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23908fa700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f237c00eab0 tx=0x7f237c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237c00cba0 con 0x7f238c100e10 2024-04-12T05:54:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f237c0044b0 con 0x7f238c100e10 2024-04-12T05:54:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23880095f0 con 0x7f238c100e10 2024-04-12T05:54:53.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.864+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c19bf10 con 0x7f238c100e10 2024-04-12T05:54:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.865+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237c005020 con 0x7f238c100e10 2024-04-12T05:54:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.865+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370005320 con 0x7f238c100e10 2024-04-12T05:54:53.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.866+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f237c004020 con 0x7f238c100e10 2024-04-12T05:54:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.867+0000 7f23827fc700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 0x7f237806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.867+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f237c014070 con 0x7f238c100e10 2024-04-12T05:54:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.867+0000 7f23910fb700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 0x7f237806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:53.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.867+0000 7f23910fb700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 0x7f237806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2388009fd0 tx=0x7f2388005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:53.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:53.872+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f237c059a00 con 0x7f238c100e10 2024-04-12T05:54:54.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.172+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2370000bf0 con 0x7f237806c4e0 2024-04-12T05:54:54.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.173+0000 7f23827fc700 1 -- 172.21.15.8:0/1157010799 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f2370000bf0 con 0x7f237806c4e0 2024-04-12T05:54:54.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 msgr2=0x7f237806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 0x7f237806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2388009fd0 tx=0x7f2388005fd0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 msgr2=0x7f238c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f237c00eab0 tx=0x7f237c00edc0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 shutdown_connections 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f238c0ffc10 0x7f238c0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f237806c4e0 0x7f237806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 --2- 172.21.15.8:0/1157010799 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f238c100e10 0x7f238c19b160 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 >> 172.21.15.8:0/1157010799 conn(0x7f238c0fb190 msgr2=0x7f238c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.175+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 shutdown_connections 2024-04-12T05:54:54.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.176+0000 7f239335f700 1 -- 172.21.15.8:0/1157010799 wait complete. 2024-04-12T05:54:54.185 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/2724492009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 msgr2=0x7f7dbc0ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/2724492009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc0ffe90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009910 tx=0x7f7dac009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/2724492009 shutdown_connections 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/2724492009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 0x7f7dbc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/2724492009 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc0ffe90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/2724492009 >> 172.21.15.8:0/2724492009 conn(0x7f7dbc0fafd0 msgr2=0x7f7dbc0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:54.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/2724492009 shutdown_connections 2024-04-12T05:54:54.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.288+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/2724492009 wait complete. 2024-04-12T05:54:54.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 Processor -- start 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 -- start start 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 0x7f7dbc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc196d70 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dc1d8f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc196eb0 con 0x7f7dbc100c70 2024-04-12T05:54:54.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dbb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dbb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47874/0 (socket says 172.21.15.8:47874) 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dbb7fe700 1 -- 172.21.15.8:0/1502731444 learned_addr learned my addr 172.21.15.8:0/1502731444 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.289+0000 7f7dbaffd700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 0x7f7dbc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dbb7fe700 1 -- 172.21.15.8:0/1502731444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 msgr2=0x7f7dbc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dbb7fe700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 0x7f7dbc196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dbb7fe700 1 -- 172.21.15.8:0/1502731444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac0095f0 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dbb7fe700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00bd70 tx=0x7f7dac00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dac01d070 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7dac005dc0 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc19b900 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dac021410 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.290+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc19be70 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.291+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc108490 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.292+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f7dac00f460 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.292+0000 7f7db8ff9700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 0x7f7da406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.292+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7dac08cab0 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.292+0000 7f7dbaffd700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 0x7f7da406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.293+0000 7f7dbaffd700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 0x7f7da406e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7db0005950 tx=0x7f7db000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:54.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.297+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7dac05acc0 con 0x7f7dbc0ffa70 2024-04-12T05:54:54.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.604+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dbc068c90 con 0x7f7da406c4e0 2024-04-12T05:54:54.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.605+0000 7f7db8ff9700 1 -- 172.21.15.8:0/1502731444 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f7dbc068c90 con 0x7f7da406c4e0 2024-04-12T05:54:54.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 msgr2=0x7f7da406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 0x7f7da406e9a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7db0005950 tx=0x7f7db000b410 comp rx=0 tx=0).stop 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 msgr2=0x7f7dbc196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00bd70 tx=0x7f7dac00bda0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 shutdown_connections 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f7da406c4e0 0x7f7da406e9a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7dbc0ffa70 0x7f7dbc196210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 --2- 172.21.15.8:0/1502731444 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7dbc100c70 0x7f7dbc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 >> 172.21.15.8:0/1502731444 conn(0x7f7dbc0fafd0 msgr2=0x7f7dbc068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 shutdown_connections 2024-04-12T05:54:54.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.607+0000 7f7dc1d8f700 1 -- 172.21.15.8:0/1502731444 wait complete. 2024-04-12T05:54:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:54 smithi008 ceph-mon[39722]: pgmap v257: 65 pgs: 65 active+clean; 52 MiB data, 396 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-04-12T05:54:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:54:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:54:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:54 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.733+0000 7f39b12bf700 1 -- 172.21.15.8:0/738107620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1040d0 msgr2=0x7f39ac1064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.733+0000 7f39b12bf700 1 --2- 172.21.15.8:0/738107620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1040d0 0x7f39ac1064c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009910 tx=0x7f39a0009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 -- 172.21.15.8:0/738107620 shutdown_connections 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 --2- 172.21.15.8:0/738107620 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1040d0 0x7f39ac1064c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 --2- 172.21.15.8:0/738107620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1017a0 0x7f39ac103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 -- 172.21.15.8:0/738107620 >> 172.21.15.8:0/738107620 conn(0x7f39ac0fb190 msgr2=0x7f39ac0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:54.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 -- 172.21.15.8:0/738107620 shutdown_connections 2024-04-12T05:54:54.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 -- 172.21.15.8:0/738107620 wait complete. 2024-04-12T05:54:54.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.734+0000 7f39b12bf700 1 Processor -- start 2024-04-12T05:54:54.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39b12bf700 1 -- start start 2024-04-12T05:54:54.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39b12bf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39b12bf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1040d0 0x7f39ac0ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39b12bf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0ffc90 con 0x7f39ac1017a0 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39b12bf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0ffdd0 con 0x7f39ac1040d0 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47890/0 (socket says 172.21.15.8:47890) 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 -- 172.21.15.8:0/2677230380 learned_addr learned my addr 172.21.15.8:0/2677230380 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aa7fc700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1040d0 0x7f39ac0ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 -- 172.21.15.8:0/2677230380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1040d0 msgr2=0x7f39ac0ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1040d0 0x7f39ac0ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.735+0000 7f39aaffd700 1 -- 172.21.15.8:0/2677230380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00095f0 con 0x7f39ac1017a0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f39aaffd700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e920 tx=0x7f399c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399c00c9d0 con 0x7f39ac1017a0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f399c010430 con 0x7f39ac1017a0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac1000b0 con 0x7f39ac1017a0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399c018630 con 0x7f39ac1017a0 2024-04-12T05:54:54.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.736+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac19e420 con 0x7f39ac1017a0 2024-04-12T05:54:54.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.737+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac1904d0 con 0x7f39ac1017a0 2024-04-12T05:54:54.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.738+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f399c0105a0 con 0x7f39ac1017a0 2024-04-12T05:54:54.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.738+0000 7f3993fff700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 0x7f399406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:54.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.739+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f399c014070 con 0x7f39ac1017a0 2024-04-12T05:54:54.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.739+0000 7f39aa7fc700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 0x7f399406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:54.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.739+0000 7f39aa7fc700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 0x7f399406e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009fd0 tx=0x7f39a000b5a0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:54.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.743+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f399c05a230 con 0x7f39ac1017a0 2024-04-12T05:54:54.757 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:54 smithi194 ceph-mon[49771]: pgmap v257: 65 pgs: 65 active+clean; 52 MiB data, 396 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 289 op/s 2024-04-12T05:54:54.757 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:54:54.757 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:54:54.757 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:54:54.757 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:54 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:54:54.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.976+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f39ac0611d0 con 0x7f399406c4e0 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.989+0000 7f3993fff700 1 -- 172.21.15.8:0/2677230380 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f39ac0611d0 con 0x7f399406c4e0 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (6m) 99s ago 8m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (8m) 99s ago 8m 7478k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (7m) 106s ago 7m 7474k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (8m) 99s ago 8m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (7m) 106s ago 7m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (6m) 99s ago 7m 86.1M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (2m) 99s ago 2m 15.0M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:54:54.984 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (2m) 99s ago 2m 12.8M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (118s) 106s ago 117s 16.1M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (2m) 106s ago 2m 18.4M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:9283,8765,8443 running (9m) 99s ago 9m 489M - 18.2.1 5be31c24972a 545bb7b78dec 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (7m) 106s ago 7m 442M - 18.2.1 5be31c24972a 324447c2f903 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (9m) 99s ago 9m 46.9M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (6m) 106s ago 6m 36.2M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (8m) 99s ago 8m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (7m) 106s ago 7m 17.8M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (6m) 99s ago 6m 48.1M 3959M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (5m) 99s ago 5m 48.8M 3959M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (5m) 99s ago 5m 45.8M 3959M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (4m) 106s ago 4m 46.1M 4982M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (4m) 106s ago 4m 46.4M 4982M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (3m) 106s ago 3m 44.0M 4982M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:54:54.985 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (6m) 99s ago 7m 56.7M - 2.43.0 a07b618ecd1d c0932eae2f67 2024-04-12T05:54:54.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 msgr2=0x7f399406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 0x7f399406e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009fd0 tx=0x7f39a000b5a0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 msgr2=0x7f39ac101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:54.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f399c00e920 tx=0x7f399c00ec30 comp rx=0 tx=0).stop 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 shutdown_connections 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f399406c4e0 0x7f399406e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f39ac1017a0 0x7f39ac101100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 --2- 172.21.15.8:0/2677230380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39ac1040d0 0x7f39ac0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 >> 172.21.15.8:0/2677230380 conn(0x7f39ac0fb190 msgr2=0x7f39ac0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 shutdown_connections 2024-04-12T05:54:54.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:54.991+0000 7f39b12bf700 1 -- 172.21.15.8:0/2677230380 wait complete. 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 -- 172.21.15.8:0/706454363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1017b0 msgr2=0x7f332c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 --2- 172.21.15.8:0/706454363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1017b0 0x7f332c103ba0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f331c009910 tx=0x7f331c009c20 comp rx=0 tx=0).stop 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 -- 172.21.15.8:0/706454363 shutdown_connections 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 --2- 172.21.15.8:0/706454363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1040e0 0x7f332c1064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 --2- 172.21.15.8:0/706454363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1017b0 0x7f332c103ba0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 -- 172.21.15.8:0/706454363 >> 172.21.15.8:0/706454363 conn(0x7f332c0fb160 msgr2=0x7f332c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 -- 172.21.15.8:0/706454363 shutdown_connections 2024-04-12T05:54:55.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.128+0000 7f333150e700 1 -- 172.21.15.8:0/706454363 wait complete. 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 Processor -- start 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 -- start start 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1017b0 0x7f332c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c197240 con 0x7f332c1040e0 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.129+0000 7f333150e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c197380 con 0x7f332c1017b0 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47906/0 (socket says 172.21.15.8:47906) 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 -- 172.21.15.8:0/1549967035 learned_addr learned my addr 172.21.15.8:0/1549967035 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332affd700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1017b0 0x7f332c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 -- 172.21.15.8:0/1549967035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1017b0 msgr2=0x7f332c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1017b0 0x7f332c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 -- 172.21.15.8:0/1549967035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0095f0 con 0x7f332c1040e0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f332a7fc700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f332000d8d0 tx=0x7f332000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33200098a0 con 0x7f332c1040e0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c19be30 con 0x7f332c1040e0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3320009e90 con 0x7f332c1040e0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f332000f5f0 con 0x7f332c1040e0 2024-04-12T05:54:55.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.130+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c19c350 con 0x7f332c1040e0 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.132+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c1904f0 con 0x7f332c1040e0 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.133+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f3320010460 con 0x7f332c1040e0 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.133+0000 7f3313fff700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 0x7f331406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.133+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f332008b7c0 con 0x7f332c1040e0 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.133+0000 7f332affd700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 0x7f331406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.134+0000 7f332affd700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 0x7f331406e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f331c006010 tx=0x7f331c00b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:55.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.138+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3320059a60 con 0x7f332c1040e0 2024-04-12T05:54:55.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.488+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f332c068af0 con 0x7f332c1040e0 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.489+0000 7f3313fff700 1 -- 172.21.15.8:0/1549967035 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f3320016240 con 0x7f332c1040e0 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:54:55.484 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:54:55.485 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:54:55.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 msgr2=0x7f331406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 0x7f331406e9a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f331c006010 tx=0x7f331c00b560 comp rx=0 tx=0).stop 2024-04-12T05:54:55.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 msgr2=0x7f332c196c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f332000d8d0 tx=0x7f332000dbe0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 shutdown_connections 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f332c1017b0 0x7f332c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f331406c4e0 0x7f331406e9a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 --2- 172.21.15.8:0/1549967035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f332c1040e0 0x7f332c196c20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 >> 172.21.15.8:0/1549967035 conn(0x7f332c0fb160 msgr2=0x7f332c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 shutdown_connections 2024-04-12T05:54:55.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.491+0000 7f333150e700 1 -- 172.21.15.8:0/1549967035 wait complete. 2024-04-12T05:54:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.620+0000 7f9911f01700 1 -- 172.21.15.8:0/173142349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 msgr2=0x7f990c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.620+0000 7f9911f01700 1 --2- 172.21.15.8:0/173142349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c100030 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009870 tx=0x7f98fc009b80 comp rx=0 tx=0).stop 2024-04-12T05:54:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 -- 172.21.15.8:0/173142349 shutdown_connections 2024-04-12T05:54:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 --2- 172.21.15.8:0/173142349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 --2- 172.21.15.8:0/173142349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c100030 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 -- 172.21.15.8:0/173142349 >> 172.21.15.8:0/173142349 conn(0x7f990c0fb190 msgr2=0x7f990c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 -- 172.21.15.8:0/173142349 shutdown_connections 2024-04-12T05:54:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 -- 172.21.15.8:0/173142349 wait complete. 2024-04-12T05:54:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 Processor -- start 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.621+0000 7f9911f01700 1 -- start start 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f9911f01700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f9911f01700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c106210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f9911f01700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c19a090 con 0x7f990c100e10 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f9911f01700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c19a1d0 con 0x7f990c0ffc10 2024-04-12T05:54:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990b7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:60530/0 (socket says 172.21.15.8:60530) 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990b7fe700 1 -- 172.21.15.8:0/1816119777 learned_addr learned my addr 172.21.15.8:0/1816119777 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990affd700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990affd700 1 -- 172.21.15.8:0/1816119777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 msgr2=0x7f990c107bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990affd700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.622+0000 7f990affd700 1 -- 172.21.15.8:0/1816119777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009520 con 0x7f990c100e10 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f990b7fe700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f990affd700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c106210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f990000e940 tx=0x7f990000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990000c990 con 0x7f990c100e10 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c19a4b0 con 0x7f990c100e10 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f9900010430 con 0x7f990c100e10 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9900018610 con 0x7f990c100e10 2024-04-12T05:54:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.623+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c19aa00 con 0x7f990c100e10 2024-04-12T05:54:55.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.624+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c1096c0 con 0x7f990c100e10 2024-04-12T05:54:55.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.625+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 19) v1 ==== 90310+0+0 (secure 0 0 0) 0x7f99000105a0 con 0x7f990c100e10 2024-04-12T05:54:55.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.625+0000 7f9908ff9700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 0x7f98f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:54:55.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.626+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9900014070 con 0x7f990c100e10 2024-04-12T05:54:55.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.626+0000 7f990b7fe700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 0x7f98f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:54:55.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.626+0000 7f990b7fe700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 0x7f98f406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009f50 tx=0x7f98fc019040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:54:55.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.631+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f99000520e0 con 0x7f990c100e10 2024-04-12T05:54:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:55 smithi008 ceph-mon[39722]: Upgrade: Updating mgr.smithi194.xbiyaj 2024-04-12T05:54:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:55 smithi008 ceph-mon[39722]: Deploying daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:54:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:55 smithi008 ceph-mon[39722]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:55 smithi194 ceph-mon[49771]: Upgrade: Updating mgr.smithi194.xbiyaj 2024-04-12T05:54:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:55 smithi194 ceph-mon[49771]: Deploying daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:54:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:55 smithi194 ceph-mon[49771]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:55.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.871+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 --> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f990c19a640 con 0x7f98f406c530 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.872+0000 7f9908ff9700 1 -- 172.21.15.8:0/1816119777 <== mgr.14213 v2:172.21.15.8:6800/2294526141 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f990c19a640 con 0x7f98f406c530 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "0/2 daemons upgraded", 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mgr daemons", 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:54:55.868 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.874+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 msgr2=0x7f98f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.874+0000 7f9911f01700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 0x7f98f406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009f50 tx=0x7f98fc019040 comp rx=0 tx=0).stop 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.874+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 msgr2=0x7f990c106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.874+0000 7f9911f01700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c106210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f990000e940 tx=0x7f990000ec50 comp rx=0 tx=0).stop 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 shutdown_connections 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f990c0ffc10 0x7f990c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:6800/2294526141,v1:172.21.15.8:6801/2294526141] conn(0x7f98f406c530 0x7f98f406e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 --2- 172.21.15.8:0/1816119777 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f990c100e10 0x7f990c106210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:54:55.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 >> 172.21.15.8:0/1816119777 conn(0x7f990c0fb190 msgr2=0x7f990c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:54:55.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 shutdown_connections 2024-04-12T05:54:55.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:54:55.875+0000 7f9911f01700 1 -- 172.21.15.8:0/1816119777 wait complete. 2024-04-12T05:54:56.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:56 smithi194 ceph-mon[49771]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:56.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:56 smithi194 ceph-mon[49771]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:56.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:56 smithi194 ceph-mon[49771]: pgmap v258: 65 pgs: 65 active+clean; 52 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-04-12T05:54:56.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:56 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1549967035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:54:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:56 smithi008 ceph-mon[39722]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:56 smithi008 ceph-mon[39722]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:56 smithi008 ceph-mon[39722]: pgmap v258: 65 pgs: 65 active+clean; 52 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-04-12T05:54:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:56 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1549967035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:54:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:57 smithi194 ceph-mon[49771]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:57.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:57 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:57.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:57 smithi008 ceph-mon[39722]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:54:57.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:57 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:54:58.414 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:54:58 smithi194 ceph-mon[49771]: pgmap v259: 65 pgs: 65 active+clean; 57 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-04-12T05:54:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:54:58 smithi008 ceph-mon[39722]: pgmap v259: 65 pgs: 65 active+clean; 57 MiB data, 432 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-04-12T05:55:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:00 smithi194 ceph-mon[49771]: pgmap v260: 65 pgs: 65 active+clean; 64 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 276 op/s 2024-04-12T05:55:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:00 smithi008 ceph-mon[39722]: pgmap v260: 65 pgs: 65 active+clean; 64 MiB data, 460 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 276 op/s 2024-04-12T05:55:02.157 INFO:tasks.workunit.client.1.smithi194.stderr:Updating files: 83% (10061/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-04-12T05:55:02.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:02 smithi194 ceph-mon[49771]: pgmap v261: 65 pgs: 65 active+clean; 67 MiB data, 481 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-04-12T05:55:02.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:02.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:02.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:02 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:Note: switching to 'd540ebaca6b131a1dd560e7f69e024b133bbaa42'. 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:state without impacting any branches by switching back to a branch. 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-12T05:55:02.626 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: git switch -c 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr:Or undo this operation with: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: git switch - 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr: 2024-04-12T05:55:02.627 INFO:tasks.workunit.client.1.smithi194.stderr:HEAD is now at d540ebaca6b Merge pull request #56672 from batrick/wip-65294-reef 2024-04-12T05:55:02.632 DEBUG:teuthology.orchestra.run.smithi194:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-12T05:55:02.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:02 smithi008 ceph-mon[39722]: pgmap v261: 65 pgs: 65 active+clean; 67 MiB data, 481 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 268 op/s 2024-04-12T05:55:02.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:02.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:02.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:02 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:02.714 INFO:tasks.workunit.client.1.smithi194.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-12T05:55:02.716 INFO:tasks.workunit.client.1.smithi194.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-12T05:55:02.716 INFO:tasks.workunit.client.1.smithi194.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-12T05:55:04.168 INFO:tasks.workunit.client.1.smithi194.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-12T05:55:04.315 INFO:tasks.workunit.client.1.smithi194.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-12T05:55:04.359 INFO:tasks.workunit.client.1.smithi194.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-12T05:55:04.361 INFO:tasks.workunit.client.1.smithi194.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-12T05:55:04.361 INFO:tasks.workunit.client.1.smithi194.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-12T05:55:04.404 INFO:tasks.workunit.client.1.smithi194.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-12T05:55:04.408 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-04-12T05:55:04.408 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-12T05:55:04.468 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-12T05:55:04.469 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-12T05:55:04.469 DEBUG:teuthology.orchestra.run.smithi194:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d540ebaca6b131a1dd560e7f69e024b133bbaa42 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-12T05:55:04.537 INFO:tasks.workunit.client.1.smithi194.stderr:+ mkdir -p fsstress 2024-04-12T05:55:04.540 INFO:tasks.workunit.client.1.smithi194.stderr:+ pushd fsstress 2024-04-12T05:55:04.541 INFO:tasks.workunit.client.1.smithi194.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T05:55:04.541 INFO:tasks.workunit.client.1.smithi194.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-12T05:55:04.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:04 smithi194 ceph-mon[49771]: pgmap v262: 65 pgs: 65 active+clean; 72 MiB data, 489 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 246 op/s 2024-04-12T05:55:04.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:04 smithi008 ceph-mon[39722]: pgmap v262: 65 pgs: 65 active+clean; 72 MiB data, 489 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 246 op/s 2024-04-12T05:55:05.379 INFO:tasks.workunit.client.1.smithi194.stderr:+ tar xzf ltp-full.tgz 2024-04-12T05:55:06.539 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:06 smithi194 ceph-mon[49771]: pgmap v263: 65 pgs: 65 active+clean; 77 MiB data, 493 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-04-12T05:55:06.540 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:06.540 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:06 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:06 smithi008 ceph-mon[39722]: pgmap v263: 65 pgs: 65 active+clean; 77 MiB data, 493 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 231 op/s 2024-04-12T05:55:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:06.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:06 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:07.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:07 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:07 smithi194 ceph-mon[49771]: pgmap v264: 65 pgs: 65 active+clean; 88 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-04-12T05:55:07.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:07.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:07 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:07.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:07 smithi008 ceph-mon[39722]: pgmap v264: 65 pgs: 65 active+clean; 88 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 270 op/s 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: pgmap v265: 65 pgs: 65 active+clean; 98 MiB data, 521 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj restarted 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:55:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:55:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: pgmap v265: 65 pgs: 65 active+clean; 98 MiB data, 521 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-04-12T05:55:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj restarted 2024-04-12T05:55:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:55:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:55:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:55:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:55:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:55:11.702 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:11 smithi008 ceph-mon[39722]: mgrmap e20: smithi008.ixmnlp(active, since 7m), standbys: smithi194.xbiyaj 2024-04-12T05:55:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:11 smithi194 ceph-mon[49771]: mgrmap e20: smithi008.ixmnlp(active, since 7m), standbys: smithi194.xbiyaj 2024-04-12T05:55:12.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:12 smithi194 ceph-mon[49771]: pgmap v266: 65 pgs: 65 active+clean; 111 MiB data, 567 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 352 op/s 2024-04-12T05:55:12.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:12 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:12 smithi008 ceph-mon[39722]: pgmap v266: 65 pgs: 65 active+clean; 111 MiB data, 567 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 352 op/s 2024-04-12T05:55:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:12 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:14.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:55:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr fail", "who": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: osdmap e47: 6 total, 6 up, 6 in 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "mgr fail", "who": "smithi008.ixmnlp"}]': finished 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: mgrmap e21: smithi194.xbiyaj(active, starting, since 0.00748214s) 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:55:14.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:55:14.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:55:14.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:55:14.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: Manager daemon smithi194.xbiyaj is now available 2024-04-12T05:55:14.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/trash_purge_schedule"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/trash_purge_schedule"}]: dispatch 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: Deploying cephadm binary to smithi194 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: [12/Apr/2024:05:55:13] ENGINE Bus STARTING 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: Deploying cephadm binary to smithi008 2024-04-12T05:55:14.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:14 smithi194 ceph-mon[49771]: [12/Apr/2024:05:55:14] ENGINE Serving on https://172.21.15.194:7150 2024-04-12T05:55:14.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr fail", "who": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: osdmap e47: 6 total, 6 up, 6 in 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.14213 172.21.15.8:0/233527925' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "mgr fail", "who": "smithi008.ixmnlp"}]': finished 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: mgrmap e21: smithi194.xbiyaj(active, starting, since 0.00748214s) 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:55:14.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:55:14.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: Manager daemon smithi194.xbiyaj is now available 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/trash_purge_schedule"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi194.xbiyaj/trash_purge_schedule"}]: dispatch 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: Deploying cephadm binary to smithi194 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: [12/Apr/2024:05:55:13] ENGINE Bus STARTING 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: Deploying cephadm binary to smithi008 2024-04-12T05:55:14.656 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:14 smithi008 ceph-mon[39722]: [12/Apr/2024:05:55:14] ENGINE Serving on https://172.21.15.194:7150 2024-04-12T05:55:15.373 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:15 smithi008 ceph-mon[39722]: [12/Apr/2024:05:55:14] ENGINE Serving on http://172.21.15.194:8765 2024-04-12T05:55:15.373 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:15 smithi008 ceph-mon[39722]: [12/Apr/2024:05:55:14] ENGINE Bus STARTED 2024-04-12T05:55:15.374 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:15 smithi008 ceph-mon[39722]: mgrmap e22: smithi194.xbiyaj(active, since 1.00851s) 2024-04-12T05:55:15.374 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:15 smithi008 ceph-mon[39722]: pgmap v3: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:15.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:15 smithi194 ceph-mon[49771]: [12/Apr/2024:05:55:14] ENGINE Serving on http://172.21.15.194:8765 2024-04-12T05:55:15.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:15 smithi194 ceph-mon[49771]: [12/Apr/2024:05:55:14] ENGINE Bus STARTED 2024-04-12T05:55:15.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:15 smithi194 ceph-mon[49771]: mgrmap e22: smithi194.xbiyaj(active, since 1.00851s) 2024-04-12T05:55:15.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:15 smithi194 ceph-mon[49771]: pgmap v3: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:16.057 INFO:tasks.workunit.client.0.smithi008.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-12T05:55:16.067 INFO:tasks.workunit.client.0.smithi008.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T05:55:16.067 INFO:tasks.workunit.client.0.smithi008.stderr:+ make 2024-04-12T05:55:16.134 INFO:tasks.workunit.client.0.smithi008.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-12T05:55:16.624 INFO:tasks.workunit.client.0.smithi008.stderr:++ readlink -f fsstress 2024-04-12T05:55:16.626 INFO:tasks.workunit.client.0.smithi008.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-12T05:55:16.626 INFO:tasks.workunit.client.0.smithi008.stderr:+ popd 2024-04-12T05:55:16.627 INFO:tasks.workunit.client.0.smithi008.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-12T05:55:16.627 INFO:tasks.workunit.client.0.smithi008.stderr:+ popd 2024-04-12T05:55:16.627 INFO:tasks.workunit.client.0.smithi008.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-12T05:55:16.628 INFO:tasks.workunit.client.0.smithi008.stderr:++ mktemp -d -p . 2024-04-12T05:55:16.630 INFO:tasks.workunit.client.0.smithi008.stderr:+ T=./tmp.TkQo1dS5dN 2024-04-12T05:55:16.630 INFO:tasks.workunit.client.0.smithi008.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.TkQo1dS5dN -l 1 -n 1000 -p 10 -v 2024-04-12T05:55:16.634 INFO:tasks.workunit.client.0.smithi008.stdout:seed = 1712392523 2024-04-12T05:55:16.641 INFO:tasks.workunit.client.0.smithi008.stdout:0/0: creat f0 x:0 0 0 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/0: dwrite - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/1: readlink - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/2: dwrite - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/3: write - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/4: fdatasync - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/5: read - no filename 2024-04-12T05:55:16.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/6: fdatasync - no filename 2024-04-12T05:55:16.643 INFO:tasks.workunit.client.0.smithi008.stdout:0/1: write f0 [515085,7935] 0 2024-04-12T05:55:16.643 INFO:tasks.workunit.client.0.smithi008.stdout:2/0: write - no filename 2024-04-12T05:55:16.643 INFO:tasks.workunit.client.0.smithi008.stdout:2/1: dread - no filename 2024-04-12T05:55:16.644 INFO:tasks.workunit.client.0.smithi008.stdout:2/2: read - no filename 2024-04-12T05:55:16.644 INFO:tasks.workunit.client.0.smithi008.stdout:2/3: dread - no filename 2024-04-12T05:55:16.644 INFO:tasks.workunit.client.0.smithi008.stdout:1/7: symlink l0 0 2024-04-12T05:55:16.644 INFO:tasks.workunit.client.0.smithi008.stdout:1/8: dread - no filename 2024-04-12T05:55:16.645 INFO:tasks.workunit.client.0.smithi008.stdout:1/9: dread - no filename 2024-04-12T05:55:16.645 INFO:tasks.workunit.client.0.smithi008.stdout:3/0: link - no file 2024-04-12T05:55:16.645 INFO:tasks.workunit.client.0.smithi008.stdout:3/1: dwrite - no filename 2024-04-12T05:55:16.646 INFO:tasks.workunit.client.0.smithi008.stdout:0/2: fdatasync f0 0 2024-04-12T05:55:16.647 INFO:tasks.workunit.client.0.smithi008.stdout:4/0: rmdir - no directory 2024-04-12T05:55:16.647 INFO:tasks.workunit.client.0.smithi008.stdout:2/4: creat f0 x:0 0 0 2024-04-12T05:55:16.648 INFO:tasks.workunit.client.0.smithi008.stdout:1/10: rename l0 to l1 0 2024-04-12T05:55:16.649 INFO:tasks.workunit.client.0.smithi008.stdout:1/11: fdatasync - no filename 2024-04-12T05:55:16.650 INFO:tasks.workunit.client.0.smithi008.stdout:5/0: truncate - no filename 2024-04-12T05:55:16.650 INFO:tasks.workunit.client.0.smithi008.stdout:3/2: mknod c0 0 2024-04-12T05:55:16.653 INFO:tasks.workunit.client.0.smithi008.stdout:4/1: symlink l0 0 2024-04-12T05:55:16.653 INFO:tasks.workunit.client.0.smithi008.stdout:6/0: rmdir - no directory 2024-04-12T05:55:16.654 INFO:tasks.workunit.client.0.smithi008.stdout:4/2: stat l0 0 2024-04-12T05:55:16.654 INFO:tasks.workunit.client.0.smithi008.stdout:3/3: creat f1 x:0 0 0 2024-04-12T05:55:16.656 INFO:tasks.workunit.client.0.smithi008.stdout:7/0: write - no filename 2024-04-12T05:55:16.656 INFO:tasks.workunit.client.0.smithi008.stdout:5/1: mkdir d0 0 2024-04-12T05:55:16.656 INFO:tasks.workunit.client.0.smithi008.stdout:5/2: truncate - no filename 2024-04-12T05:55:16.657 INFO:tasks.workunit.client.0.smithi008.stdout:4/3: unlink l0 0 2024-04-12T05:55:16.658 INFO:tasks.workunit.client.0.smithi008.stdout:4/4: dread - no filename 2024-04-12T05:55:16.658 INFO:tasks.workunit.client.0.smithi008.stdout:4/5: chown . 177 1 2024-04-12T05:55:16.658 INFO:tasks.workunit.client.0.smithi008.stdout:4/6: dwrite - no filename 2024-04-12T05:55:16.661 INFO:tasks.workunit.client.0.smithi008.stdout:8/0: dwrite - no filename 2024-04-12T05:55:16.661 INFO:tasks.workunit.client.0.smithi008.stdout:8/1: dwrite - no filename 2024-04-12T05:55:16.661 INFO:tasks.workunit.client.0.smithi008.stdout:6/1: mkdir d0 0 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:5/3: mknod d0/c1 0 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:5/4: fsync - no filename 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:7/1: getdents . 0 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:7/2: dread - no filename 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:9/0: dread - no filename 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:9/1: dread - no filename 2024-04-12T05:55:16.664 INFO:tasks.workunit.client.0.smithi008.stdout:9/2: write - no filename 2024-04-12T05:55:16.665 INFO:tasks.workunit.client.0.smithi008.stdout:9/3: rename - no filename 2024-04-12T05:55:16.666 INFO:tasks.workunit.client.0.smithi008.stdout:5/5: mkdir d0/d2 0 2024-04-12T05:55:16.668 INFO:tasks.workunit.client.0.smithi008.stdout:8/2: symlink l0 0 2024-04-12T05:55:16.669 INFO:tasks.workunit.client.0.smithi008.stdout:6/2: getdents d0 0 2024-04-12T05:55:16.669 INFO:tasks.workunit.client.0.smithi008.stdout:7/3: getdents . 0 2024-04-12T05:55:16.670 INFO:tasks.workunit.client.0.smithi008.stdout:9/4: mknod c0 0 2024-04-12T05:55:16.672 INFO:tasks.workunit.client.0.smithi008.stdout:8/3: rename l0 to l1 0 2024-04-12T05:55:16.672 INFO:tasks.workunit.client.0.smithi008.stdout:8/4: dread - no filename 2024-04-12T05:55:16.672 INFO:tasks.workunit.client.0.smithi008.stdout:8/5: write - no filename 2024-04-12T05:55:16.675 INFO:tasks.workunit.client.0.smithi008.stdout:5/6: creat d0/d2/f3 x:0 0 0 2024-04-12T05:55:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:5/7: chown d0/d2/f3 3749376 1 2024-04-12T05:55:16.676 INFO:tasks.workunit.client.0.smithi008.stdout:8/6: mknod c2 0 2024-04-12T05:55:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:7/4: creat f0 x:0 0 0 2024-04-12T05:55:16.678 INFO:tasks.workunit.client.0.smithi008.stdout:7/5: chown f0 11 1 2024-04-12T05:55:16.680 INFO:tasks.workunit.client.0.smithi008.stdout:5/8: creat d0/d2/f4 x:0 0 0 2024-04-12T05:55:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:6/3: rmdir d0 0 2024-04-12T05:55:16.681 INFO:tasks.workunit.client.0.smithi008.stdout:5/9: truncate d0/d2/f3 759248 0 2024-04-12T05:55:16.682 INFO:tasks.workunit.client.0.smithi008.stdout:7/6: mkdir d1 0 2024-04-12T05:55:16.683 INFO:tasks.workunit.client.0.smithi008.stdout:6/4: creat f1 x:0 0 0 2024-04-12T05:55:16.685 INFO:tasks.workunit.client.0.smithi008.stdout:5/10: dread d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:16.686 INFO:tasks.workunit.client.0.smithi008.stdout:6/5: creat f2 x:0 0 0 2024-04-12T05:55:16.687 INFO:tasks.workunit.client.0.smithi008.stdout:6/6: write f2 [590660,1521] 0 2024-04-12T05:55:16.696 INFO:tasks.workunit.client.0.smithi008.stdout:5/11: dwrite d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:16.708 INFO:tasks.workunit.client.0.smithi008.stdout:5/12: dwrite d0/d2/f4 [0,4194304] 0 2024-04-12T05:55:16.714 INFO:tasks.workunit.client.0.smithi008.stdout:5/13: creat d0/d2/f5 x:0 0 0 2024-04-12T05:55:16.718 INFO:tasks.workunit.client.0.smithi008.stdout:5/14: dread d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:16.779 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:16 smithi008 ceph-mon[39722]: pgmap v4: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:16.779 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:16 smithi008 ceph-mon[39722]: mgrmap e23: smithi194.xbiyaj(active, since 3s) 2024-04-12T05:55:16.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:16 smithi194 ceph-mon[49771]: pgmap v4: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:16.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:16 smithi194 ceph-mon[49771]: mgrmap e23: smithi194.xbiyaj(active, since 3s) 2024-04-12T05:55:16.868 INFO:tasks.workunit.client.0.smithi008.stdout:2/5: getdents . 0 2024-04-12T05:55:16.868 INFO:tasks.workunit.client.0.smithi008.stdout:2/6: read - f0 zero size 2024-04-12T05:55:16.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/12: getdents . 0 2024-04-12T05:55:16.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/13: write - no filename 2024-04-12T05:55:16.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/14: truncate - no filename 2024-04-12T05:55:16.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/15: write - no filename 2024-04-12T05:55:16.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/16: truncate - no filename 2024-04-12T05:55:16.877 INFO:tasks.workunit.client.0.smithi008.stdout:3/4: fsync f1 0 2024-04-12T05:55:16.880 INFO:tasks.workunit.client.0.smithi008.stdout:2/7: dwrite f0 [0,4194304] 0 2024-04-12T05:55:16.882 INFO:tasks.workunit.client.0.smithi008.stdout:1/17: getdents . 0 2024-04-12T05:55:16.884 INFO:tasks.workunit.client.0.smithi008.stdout:3/5: mknod c2 0 2024-04-12T05:55:16.884 INFO:tasks.workunit.client.0.smithi008.stdout:3/6: readlink - no filename 2024-04-12T05:55:16.888 INFO:tasks.workunit.client.0.smithi008.stdout:2/8: symlink l1 0 2024-04-12T05:55:16.893 INFO:tasks.workunit.client.0.smithi008.stdout:3/7: mknod c3 0 2024-04-12T05:55:16.896 INFO:tasks.workunit.client.0.smithi008.stdout:7/7: fsync f0 0 2024-04-12T05:55:16.897 INFO:tasks.workunit.client.0.smithi008.stdout:7/8: truncate f0 411094 0 2024-04-12T05:55:16.899 INFO:tasks.workunit.client.0.smithi008.stdout:1/18: rename l1 to l2 0 2024-04-12T05:55:16.899 INFO:tasks.workunit.client.0.smithi008.stdout:1/19: dread - no filename 2024-04-12T05:55:16.901 INFO:tasks.workunit.client.0.smithi008.stdout:8/7: chown l1 24334254 1 2024-04-12T05:55:16.901 INFO:tasks.workunit.client.0.smithi008.stdout:8/8: read - no filename 2024-04-12T05:55:16.904 INFO:tasks.workunit.client.0.smithi008.stdout:3/8: mknod c4 0 2024-04-12T05:55:16.906 INFO:tasks.workunit.client.0.smithi008.stdout:6/7: truncate f2 379460 0 2024-04-12T05:55:16.914 INFO:tasks.workunit.client.0.smithi008.stdout:5/15: write d0/d2/f4 [4365978,50031] 0 2024-04-12T05:55:16.915 INFO:tasks.workunit.client.0.smithi008.stdout:3/9: dwrite f1 [0,4194304] 0 2024-04-12T05:55:16.916 INFO:tasks.workunit.client.0.smithi008.stdout:8/9: symlink l3 0 2024-04-12T05:55:16.925 INFO:tasks.workunit.client.0.smithi008.stdout:7/9: symlink d1/l2 0 2024-04-12T05:55:16.928 INFO:tasks.workunit.client.0.smithi008.stdout:7/10: dread f0 [0,4194304] 0 2024-04-12T05:55:16.929 INFO:tasks.workunit.client.0.smithi008.stdout:5/16: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:16.930 INFO:tasks.workunit.client.0.smithi008.stdout:5/17: write d0/d2/f5 [2662088,78033] 0 2024-04-12T05:55:16.931 INFO:tasks.workunit.client.0.smithi008.stdout:8/10: creat f4 x:0 0 0 2024-04-12T05:55:16.932 INFO:tasks.workunit.client.0.smithi008.stdout:7/11: write f0 [1300538,31057] 0 2024-04-12T05:55:16.935 INFO:tasks.workunit.client.0.smithi008.stdout:6/8: chown f2 1637988408 1 2024-04-12T05:55:16.940 INFO:tasks.workunit.client.0.smithi008.stdout:7/12: dread f0 [0,4194304] 0 2024-04-12T05:55:16.945 INFO:tasks.workunit.client.0.smithi008.stdout:5/18: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:16.961 INFO:tasks.workunit.client.0.smithi008.stdout:8/11: creat f5 x:0 0 0 2024-04-12T05:55:16.962 INFO:tasks.workunit.client.0.smithi008.stdout:8/12: dread - f5 zero size 2024-04-12T05:55:16.964 INFO:tasks.workunit.client.0.smithi008.stdout:7/13: creat d1/f3 x:0 0 0 2024-04-12T05:55:16.966 INFO:tasks.workunit.client.0.smithi008.stdout:5/19: rmdir d0 39 2024-04-12T05:55:16.968 INFO:tasks.workunit.client.0.smithi008.stdout:3/10: link c3 c5 0 2024-04-12T05:55:16.973 INFO:tasks.workunit.client.0.smithi008.stdout:7/14: creat d1/f4 x:0 0 0 2024-04-12T05:55:16.974 INFO:tasks.workunit.client.0.smithi008.stdout:7/15: read - d1/f3 zero size 2024-04-12T05:55:16.975 INFO:tasks.workunit.client.0.smithi008.stdout:5/20: stat d0 0 2024-04-12T05:55:16.976 INFO:tasks.workunit.client.0.smithi008.stdout:5/21: fsync d0/d2/f4 0 2024-04-12T05:55:16.977 INFO:tasks.workunit.client.0.smithi008.stdout:3/11: creat f6 x:0 0 0 2024-04-12T05:55:16.981 INFO:tasks.workunit.client.0.smithi008.stdout:7/16: mkdir d1/d5 0 2024-04-12T05:55:16.985 INFO:tasks.workunit.client.0.smithi008.stdout:5/22: mknod d0/c6 0 2024-04-12T05:55:16.985 INFO:tasks.workunit.client.0.smithi008.stdout:5/23: chown d0 2 1 2024-04-12T05:55:16.987 INFO:tasks.workunit.client.0.smithi008.stdout:5/24: read d0/d2/f4 [4341082,69842] 0 2024-04-12T05:55:16.988 INFO:tasks.workunit.client.0.smithi008.stdout:8/13: link l3 l6 0 2024-04-12T05:55:16.990 INFO:tasks.workunit.client.0.smithi008.stdout:8/14: chown f5 1218770 1 2024-04-12T05:55:16.996 INFO:tasks.workunit.client.0.smithi008.stdout:5/25: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:17.003 INFO:tasks.workunit.client.0.smithi008.stdout:7/17: dwrite f0 [0,4194304] 0 2024-04-12T05:55:17.005 INFO:tasks.workunit.client.0.smithi008.stdout:7/18: write f0 [2100881,118500] 0 2024-04-12T05:55:17.007 INFO:tasks.workunit.client.0.smithi008.stdout:5/26: rename d0/c6 to d0/d2/c7 0 2024-04-12T05:55:17.012 INFO:tasks.workunit.client.0.smithi008.stdout:7/19: dread f0 [0,4194304] 0 2024-04-12T05:55:17.023 INFO:tasks.workunit.client.0.smithi008.stdout:7/20: dwrite d1/f4 [0,4194304] 0 2024-04-12T05:55:17.029 INFO:tasks.workunit.client.0.smithi008.stdout:7/21: rename d1/f4 to d1/f6 0 2024-04-12T05:55:17.096 INFO:tasks.workunit.client.0.smithi008.stdout:1/20: getdents . 0 2024-04-12T05:55:17.106 INFO:tasks.workunit.client.0.smithi008.stdout:1/21: unlink l2 0 2024-04-12T05:55:17.106 INFO:tasks.workunit.client.0.smithi008.stdout:1/22: write - no filename 2024-04-12T05:55:17.106 INFO:tasks.workunit.client.0.smithi008.stdout:1/23: write - no filename 2024-04-12T05:55:17.106 INFO:tasks.workunit.client.0.smithi008.stdout:1/24: fdatasync - no filename 2024-04-12T05:55:17.107 INFO:tasks.workunit.client.0.smithi008.stdout:2/9: truncate f0 3770130 0 2024-04-12T05:55:17.112 INFO:tasks.workunit.client.0.smithi008.stdout:1/25: creat f3 x:0 0 0 2024-04-12T05:55:17.113 INFO:tasks.workunit.client.0.smithi008.stdout:1/26: stat f3 0 2024-04-12T05:55:17.116 INFO:tasks.workunit.client.0.smithi008.stdout:1/27: write f3 [60025,1442] 0 2024-04-12T05:55:17.117 INFO:tasks.workunit.client.0.smithi008.stdout:1/28: chown f3 156927092 1 2024-04-12T05:55:17.127 INFO:tasks.workunit.client.0.smithi008.stdout:1/29: dwrite f3 [0,4194304] 0 2024-04-12T05:55:17.146 INFO:tasks.workunit.client.0.smithi008.stdout:3/12: getdents . 0 2024-04-12T05:55:17.148 INFO:tasks.workunit.client.0.smithi008.stdout:8/15: getdents . 0 2024-04-12T05:55:17.154 INFO:tasks.workunit.client.0.smithi008.stdout:3/13: dread f1 [0,4194304] 0 2024-04-12T05:55:17.158 INFO:tasks.workunit.client.0.smithi008.stdout:3/14: creat f7 x:0 0 0 2024-04-12T05:55:17.158 INFO:tasks.workunit.client.0.smithi008.stdout:8/16: dwrite f5 [0,4194304] 0 2024-04-12T05:55:17.159 INFO:tasks.workunit.client.0.smithi008.stdout:8/17: fsync f4 0 2024-04-12T05:55:17.161 INFO:tasks.workunit.client.0.smithi008.stdout:3/15: chown c3 6 1 2024-04-12T05:55:17.161 INFO:tasks.workunit.client.0.smithi008.stdout:3/16: fsync f6 0 2024-04-12T05:55:17.162 INFO:tasks.workunit.client.0.smithi008.stdout:3/17: fdatasync f7 0 2024-04-12T05:55:17.164 INFO:tasks.workunit.client.0.smithi008.stdout:3/18: dread - f6 zero size 2024-04-12T05:55:17.166 INFO:tasks.workunit.client.0.smithi008.stdout:5/27: write d0/d2/f3 [2560478,49914] 0 2024-04-12T05:55:17.167 INFO:tasks.workunit.client.0.smithi008.stdout:3/19: chown c2 1682 1 2024-04-12T05:55:17.168 INFO:tasks.workunit.client.0.smithi008.stdout:3/20: fdatasync f6 0 2024-04-12T05:55:17.169 INFO:tasks.workunit.client.0.smithi008.stdout:5/28: symlink d0/d2/l8 0 2024-04-12T05:55:17.171 INFO:tasks.workunit.client.0.smithi008.stdout:3/21: mknod c8 0 2024-04-12T05:55:17.171 INFO:tasks.workunit.client.0.smithi008.stdout:7/22: truncate f0 3965376 0 2024-04-12T05:55:17.173 INFO:tasks.workunit.client.0.smithi008.stdout:3/22: chown c2 4 1 2024-04-12T05:55:17.175 INFO:tasks.workunit.client.0.smithi008.stdout:5/29: dread d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:17.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/23: link d1/l2 d1/d5/l7 0 2024-04-12T05:55:17.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/24: dread - d1/f3 zero size 2024-04-12T05:55:17.178 INFO:tasks.workunit.client.0.smithi008.stdout:5/30: chown d0/d2/c7 10782490 1 2024-04-12T05:55:17.182 INFO:tasks.workunit.client.0.smithi008.stdout:7/25: rename d1/d5/l7 to d1/l8 0 2024-04-12T05:55:17.185 INFO:tasks.workunit.client.0.smithi008.stdout:7/26: symlink d1/l9 0 2024-04-12T05:55:17.188 INFO:tasks.workunit.client.0.smithi008.stdout:5/31: dwrite d0/d2/f4 [4194304,4194304] 0 2024-04-12T05:55:17.190 INFO:tasks.workunit.client.0.smithi008.stdout:5/32: write d0/d2/f3 [4021854,56049] 0 2024-04-12T05:55:17.191 INFO:tasks.workunit.client.0.smithi008.stdout:5/33: stat d0/d2/l8 0 2024-04-12T05:55:17.193 INFO:tasks.workunit.client.0.smithi008.stdout:5/34: creat d0/d2/f9 x:0 0 0 2024-04-12T05:55:17.221 INFO:tasks.workunit.client.0.smithi008.stdout:6/9: link f2 f3 0 2024-04-12T05:55:17.228 INFO:tasks.workunit.client.0.smithi008.stdout:1/30: fsync f3 0 2024-04-12T05:55:17.228 INFO:tasks.workunit.client.0.smithi008.stdout:1/31: stat f3 0 2024-04-12T05:55:17.229 INFO:tasks.workunit.client.0.smithi008.stdout:2/10: dwrite f0 [0,4194304] 0 2024-04-12T05:55:17.232 INFO:tasks.workunit.client.0.smithi008.stdout:2/11: write f0 [117029,79284] 0 2024-04-12T05:55:17.237 INFO:tasks.workunit.client.0.smithi008.stdout:1/32: dwrite f3 [0,4194304] 0 2024-04-12T05:55:17.238 INFO:tasks.workunit.client.0.smithi008.stdout:1/33: write f3 [2308978,109937] 0 2024-04-12T05:55:17.260 INFO:tasks.workunit.client.0.smithi008.stdout:8/18: truncate f5 720796 0 2024-04-12T05:55:17.272 INFO:tasks.workunit.client.0.smithi008.stdout:7/27: dread f0 [0,4194304] 0 2024-04-12T05:55:17.272 INFO:tasks.workunit.client.0.smithi008.stdout:8/19: dwrite f4 [0,4194304] 0 2024-04-12T05:55:17.272 INFO:tasks.workunit.client.0.smithi008.stdout:8/20: rmdir - no directory 2024-04-12T05:55:17.274 INFO:tasks.workunit.client.0.smithi008.stdout:5/35: rmdir d0/d2 39 2024-04-12T05:55:17.415 INFO:tasks.workunit.client.0.smithi008.stdout:2/12: mknod c2 0 2024-04-12T05:55:17.417 INFO:tasks.workunit.client.0.smithi008.stdout:1/34: mkdir d4 0 2024-04-12T05:55:17.420 INFO:tasks.workunit.client.0.smithi008.stdout:7/28: fsync f0 0 2024-04-12T05:55:17.422 INFO:tasks.workunit.client.0.smithi008.stdout:8/21: rename f4 to f7 0 2024-04-12T05:55:17.422 INFO:tasks.workunit.client.0.smithi008.stdout:2/13: dread f0 [0,4194304] 0 2024-04-12T05:55:17.426 INFO:tasks.workunit.client.0.smithi008.stdout:1/35: dwrite f3 [0,4194304] 0 2024-04-12T05:55:17.428 INFO:tasks.workunit.client.0.smithi008.stdout:7/29: mkdir d1/d5/da 0 2024-04-12T05:55:17.429 INFO:tasks.workunit.client.0.smithi008.stdout:7/30: write f0 [2428016,122108] 0 2024-04-12T05:55:17.429 INFO:tasks.workunit.client.0.smithi008.stdout:2/14: symlink l3 0 2024-04-12T05:55:17.435 INFO:tasks.workunit.client.0.smithi008.stdout:8/22: rename c2 to c8 0 2024-04-12T05:55:17.435 INFO:tasks.workunit.client.0.smithi008.stdout:8/23: rmdir - no directory 2024-04-12T05:55:17.436 INFO:tasks.workunit.client.0.smithi008.stdout:5/36: fdatasync d0/d2/f5 0 2024-04-12T05:55:17.436 INFO:tasks.workunit.client.0.smithi008.stdout:2/15: read f0 [692625,125245] 0 2024-04-12T05:55:17.438 INFO:tasks.workunit.client.0.smithi008.stdout:7/31: creat d1/fb x:0 0 0 2024-04-12T05:55:17.438 INFO:tasks.workunit.client.0.smithi008.stdout:8/24: fdatasync f7 0 2024-04-12T05:55:17.439 INFO:tasks.workunit.client.0.smithi008.stdout:7/32: read d1/f6 [3679949,17362] 0 2024-04-12T05:55:17.440 INFO:tasks.workunit.client.0.smithi008.stdout:1/36: mknod d4/c5 0 2024-04-12T05:55:17.441 INFO:tasks.workunit.client.0.smithi008.stdout:1/37: chown f3 446 1 2024-04-12T05:55:17.441 INFO:tasks.workunit.client.0.smithi008.stdout:7/33: dread - d1/fb zero size 2024-04-12T05:55:17.442 INFO:tasks.workunit.client.0.smithi008.stdout:8/25: rename f5 to f9 0 2024-04-12T05:55:17.445 INFO:tasks.workunit.client.0.smithi008.stdout:1/38: rmdir d4 39 2024-04-12T05:55:17.446 INFO:tasks.workunit.client.0.smithi008.stdout:2/16: dread f0 [0,4194304] 0 2024-04-12T05:55:17.447 INFO:tasks.workunit.client.0.smithi008.stdout:5/37: creat d0/fa x:0 0 0 2024-04-12T05:55:17.449 INFO:tasks.workunit.client.0.smithi008.stdout:8/26: write f9 [1079968,37770] 0 2024-04-12T05:55:17.449 INFO:tasks.workunit.client.0.smithi008.stdout:5/38: chown d0/fa 16948960 1 2024-04-12T05:55:17.452 INFO:tasks.workunit.client.0.smithi008.stdout:7/34: rename f0 to d1/d5/da/fc 0 2024-04-12T05:55:17.452 INFO:tasks.workunit.client.0.smithi008.stdout:2/17: symlink l4 0 2024-04-12T05:55:17.453 INFO:tasks.workunit.client.0.smithi008.stdout:7/35: stat d1/d5 0 2024-04-12T05:55:17.458 INFO:tasks.workunit.client.0.smithi008.stdout:7/36: write d1/f6 [3984041,79877] 0 2024-04-12T05:55:17.459 INFO:tasks.workunit.client.0.smithi008.stdout:7/37: read - d1/f3 zero size 2024-04-12T05:55:17.462 INFO:tasks.workunit.client.0.smithi008.stdout:5/39: link d0/d2/c7 d0/d2/cb 0 2024-04-12T05:55:17.463 INFO:tasks.workunit.client.0.smithi008.stdout:5/40: dread - d0/fa zero size 2024-04-12T05:55:17.465 INFO:tasks.workunit.client.0.smithi008.stdout:7/38: link d1/d5/da/fc d1/d5/fd 0 2024-04-12T05:55:17.468 INFO:tasks.workunit.client.0.smithi008.stdout:5/41: mkdir d0/dc 0 2024-04-12T05:55:17.471 INFO:tasks.workunit.client.0.smithi008.stdout:7/39: write d1/d5/fd [2514801,103795] 0 2024-04-12T05:55:17.473 INFO:tasks.workunit.client.0.smithi008.stdout:7/40: write d1/d5/fd [2286058,42875] 0 2024-04-12T05:55:17.473 INFO:tasks.workunit.client.0.smithi008.stdout:5/42: dread d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:17.475 INFO:tasks.workunit.client.0.smithi008.stdout:7/41: fdatasync d1/f3 0 2024-04-12T05:55:17.477 INFO:tasks.workunit.client.0.smithi008.stdout:7/42: chown d1/d5/da/fc 33 1 2024-04-12T05:55:17.478 INFO:tasks.workunit.client.0.smithi008.stdout:5/43: mkdir d0/dd 0 2024-04-12T05:55:17.482 INFO:tasks.workunit.client.0.smithi008.stdout:5/44: creat d0/d2/fe x:0 0 0 2024-04-12T05:55:17.483 INFO:tasks.workunit.client.0.smithi008.stdout:5/45: readlink d0/d2/l8 0 2024-04-12T05:55:17.483 INFO:tasks.workunit.client.0.smithi008.stdout:7/43: dread d1/f6 [0,4194304] 0 2024-04-12T05:55:17.484 INFO:tasks.workunit.client.0.smithi008.stdout:7/44: rename d1/d5 to d1/d5/de 22 2024-04-12T05:55:17.487 INFO:tasks.workunit.client.0.smithi008.stdout:7/45: mkdir d1/d5/da/df 0 2024-04-12T05:55:17.489 INFO:tasks.workunit.client.0.smithi008.stdout:5/46: dread d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:17.491 INFO:tasks.workunit.client.0.smithi008.stdout:7/46: mknod d1/d5/c10 0 2024-04-12T05:55:17.492 INFO:tasks.workunit.client.0.smithi008.stdout:7/47: write d1/d5/da/fc [2780608,107116] 0 2024-04-12T05:55:17.495 INFO:tasks.workunit.client.0.smithi008.stdout:5/47: link d0/d2/f9 d0/dc/ff 0 2024-04-12T05:55:17.497 INFO:tasks.workunit.client.0.smithi008.stdout:7/48: mkdir d1/d5/da/d11 0 2024-04-12T05:55:17.508 INFO:tasks.workunit.client.0.smithi008.stdout:5/48: dwrite d0/d2/f4 [4194304,4194304] 0 2024-04-12T05:55:17.514 INFO:tasks.workunit.client.0.smithi008.stdout:5/49: symlink d0/dc/l10 0 2024-04-12T05:55:17.520 INFO:tasks.workunit.client.0.smithi008.stdout:5/50: write d0/d2/f3 [2543945,122897] 0 2024-04-12T05:55:17.521 INFO:tasks.workunit.client.0.smithi008.stdout:5/51: write d0/d2/f5 [4695763,18539] 0 2024-04-12T05:55:17.529 INFO:tasks.workunit.client.0.smithi008.stdout:2/18: truncate f0 1237206 0 2024-04-12T05:55:17.531 INFO:tasks.workunit.client.0.smithi008.stdout:2/19: chown l1 305659 1 2024-04-12T05:55:17.531 INFO:tasks.workunit.client.0.smithi008.stdout:8/27: write f7 [4352140,2684] 0 2024-04-12T05:55:17.533 INFO:tasks.workunit.client.0.smithi008.stdout:5/52: mkdir d0/dd/d11 0 2024-04-12T05:55:17.539 INFO:tasks.workunit.client.0.smithi008.stdout:1/39: truncate f3 3061246 0 2024-04-12T05:55:17.542 INFO:tasks.workunit.client.0.smithi008.stdout:7/49: fsync d1/d5/fd 0 2024-04-12T05:55:17.542 INFO:tasks.workunit.client.0.smithi008.stdout:5/53: dwrite d0/d2/f3 [0,4194304] 0 2024-04-12T05:55:17.545 INFO:tasks.workunit.client.0.smithi008.stdout:7/50: write d1/d5/da/fc [446873,117375] 0 2024-04-12T05:55:17.545 INFO:tasks.workunit.client.0.smithi008.stdout:1/40: mkdir d4/d6 0 2024-04-12T05:55:17.548 INFO:tasks.workunit.client.0.smithi008.stdout:7/51: read d1/f6 [774272,49992] 0 2024-04-12T05:55:17.550 INFO:tasks.workunit.client.0.smithi008.stdout:5/54: write d0/d2/fe [789401,100874] 0 2024-04-12T05:55:17.554 INFO:tasks.workunit.client.0.smithi008.stdout:7/52: rename d1/d5/fd to d1/f12 0 2024-04-12T05:55:17.556 INFO:tasks.workunit.client.0.smithi008.stdout:6/10: write f3 [347378,57684] 0 2024-04-12T05:55:17.559 INFO:tasks.workunit.client.0.smithi008.stdout:5/55: getdents d0 0 2024-04-12T05:55:17.560 INFO:tasks.workunit.client.0.smithi008.stdout:5/56: fsync d0/fa 0 2024-04-12T05:55:17.561 INFO:tasks.workunit.client.0.smithi008.stdout:2/20: write f0 [992861,47515] 0 2024-04-12T05:55:17.563 INFO:tasks.workunit.client.0.smithi008.stdout:7/53: dread d1/f12 [0,4194304] 0 2024-04-12T05:55:17.566 INFO:tasks.workunit.client.0.smithi008.stdout:2/21: creat f5 x:0 0 0 2024-04-12T05:55:17.566 INFO:tasks.workunit.client.0.smithi008.stdout:7/54: rename d1/d5 to d1/d5/da/df/d13 22 2024-04-12T05:55:17.568 INFO:tasks.workunit.client.0.smithi008.stdout:2/22: write f5 [324214,58057] 0 2024-04-12T05:55:17.570 INFO:tasks.workunit.client.0.smithi008.stdout:7/55: chown d1/l2 188689148 1 2024-04-12T05:55:17.570 INFO:tasks.workunit.client.0.smithi008.stdout:2/23: mkdir d6 0 2024-04-12T05:55:17.572 INFO:tasks.workunit.client.0.smithi008.stdout:5/57: dwrite d0/d2/f4 [0,4194304] 0 2024-04-12T05:55:17.574 INFO:tasks.workunit.client.0.smithi008.stdout:2/24: dread f5 [0,4194304] 0 2024-04-12T05:55:17.576 INFO:tasks.workunit.client.0.smithi008.stdout:5/58: creat d0/f12 x:0 0 0 2024-04-12T05:55:17.576 INFO:tasks.workunit.client.0.smithi008.stdout:2/25: chown f0 3860 1 2024-04-12T05:55:17.577 INFO:tasks.workunit.client.0.smithi008.stdout:7/56: link d1/d5/c10 d1/d5/c14 0 2024-04-12T05:55:17.578 INFO:tasks.workunit.client.0.smithi008.stdout:7/57: read - d1/fb zero size 2024-04-12T05:55:17.579 INFO:tasks.workunit.client.0.smithi008.stdout:2/26: symlink d6/l7 0 2024-04-12T05:55:17.580 INFO:tasks.workunit.client.0.smithi008.stdout:7/58: write d1/f12 [3661714,96614] 0 2024-04-12T05:55:17.588 INFO:tasks.workunit.client.0.smithi008.stdout:5/59: write d0/d2/f4 [8556861,124212] 0 2024-04-12T05:55:17.590 INFO:tasks.workunit.client.0.smithi008.stdout:2/27: dwrite f0 [0,4194304] 0 2024-04-12T05:55:17.590 INFO:tasks.workunit.client.0.smithi008.stdout:5/60: chown d0/f12 535965 1 2024-04-12T05:55:17.594 INFO:tasks.workunit.client.0.smithi008.stdout:2/28: fdatasync f5 0 2024-04-12T05:55:17.595 INFO:tasks.workunit.client.0.smithi008.stdout:7/59: dwrite d1/f12 [0,4194304] 0 2024-04-12T05:55:17.595 INFO:tasks.workunit.client.0.smithi008.stdout:2/29: write f0 [1121140,22609] 0 2024-04-12T05:55:17.597 INFO:tasks.workunit.client.0.smithi008.stdout:5/61: mkdir d0/dc/d13 0 2024-04-12T05:55:17.598 INFO:tasks.workunit.client.0.smithi008.stdout:2/30: dread f5 [0,4194304] 0 2024-04-12T05:55:17.601 INFO:tasks.workunit.client.0.smithi008.stdout:2/31: symlink d6/l8 0 2024-04-12T05:55:17.604 INFO:tasks.workunit.client.0.smithi008.stdout:7/60: dread d1/f6 [0,4194304] 0 2024-04-12T05:55:17.606 INFO:tasks.workunit.client.0.smithi008.stdout:7/61: stat d1/l8 0 2024-04-12T05:55:17.610 INFO:tasks.workunit.client.0.smithi008.stdout:2/32: dread f0 [0,4194304] 0 2024-04-12T05:55:17.611 INFO:tasks.workunit.client.0.smithi008.stdout:5/62: dwrite d0/fa [0,4194304] 0 2024-04-12T05:55:17.616 INFO:tasks.workunit.client.0.smithi008.stdout:2/33: creat d6/f9 x:0 0 0 2024-04-12T05:55:17.618 INFO:tasks.workunit.client.0.smithi008.stdout:2/34: mknod d6/ca 0 2024-04-12T05:55:17.624 INFO:tasks.workunit.client.0.smithi008.stdout:7/62: dwrite d1/f6 [4194304,4194304] 0 2024-04-12T05:55:17.628 INFO:tasks.workunit.client.0.smithi008.stdout:5/63: write d0/dc/ff [267385,123190] 0 2024-04-12T05:55:17.630 INFO:tasks.workunit.client.0.smithi008.stdout:5/64: stat d0/d2/f3 0 2024-04-12T05:55:17.633 INFO:tasks.workunit.client.0.smithi008.stdout:7/63: rename d1/d5/da/df to d1/d15 0 2024-04-12T05:55:17.634 INFO:tasks.workunit.client.0.smithi008.stdout:2/35: getdents d6 0 2024-04-12T05:55:17.634 INFO:tasks.workunit.client.0.smithi008.stdout:2/36: readlink d6/l7 0 2024-04-12T05:55:17.635 INFO:tasks.workunit.client.0.smithi008.stdout:2/37: chown l1 1816151 1 2024-04-12T05:55:17.636 INFO:tasks.workunit.client.0.smithi008.stdout:2/38: read f5 [162055,52674] 0 2024-04-12T05:55:17.637 INFO:tasks.workunit.client.0.smithi008.stdout:5/65: link d0/d2/f9 d0/dc/f14 0 2024-04-12T05:55:17.638 INFO:tasks.workunit.client.0.smithi008.stdout:2/39: fdatasync f0 0 2024-04-12T05:55:17.639 INFO:tasks.workunit.client.0.smithi008.stdout:7/64: dread d1/f6 [0,4194304] 0 2024-04-12T05:55:17.640 INFO:tasks.workunit.client.0.smithi008.stdout:2/40: dread f5 [0,4194304] 0 2024-04-12T05:55:17.641 INFO:tasks.workunit.client.0.smithi008.stdout:5/66: rename d0/d2/f9 to d0/dc/d13/f15 0 2024-04-12T05:55:17.643 INFO:tasks.workunit.client.0.smithi008.stdout:2/41: mkdir d6/db 0 2024-04-12T05:55:17.644 INFO:tasks.workunit.client.0.smithi008.stdout:2/42: fdatasync f0 0 2024-04-12T05:55:17.647 INFO:tasks.workunit.client.0.smithi008.stdout:5/67: symlink d0/dc/d13/l16 0 2024-04-12T05:55:17.648 INFO:tasks.workunit.client.0.smithi008.stdout:2/43: write d6/f9 [65693,26661] 0 2024-04-12T05:55:17.650 INFO:tasks.workunit.client.0.smithi008.stdout:5/68: write d0/d2/f5 [863138,91982] 0 2024-04-12T05:55:17.653 INFO:tasks.workunit.client.0.smithi008.stdout:7/65: dwrite d1/f12 [4194304,4194304] 0 2024-04-12T05:55:17.659 INFO:tasks.workunit.client.0.smithi008.stdout:5/69: link d0/fa d0/d2/f17 0 2024-04-12T05:55:17.663 INFO:tasks.workunit.client.0.smithi008.stdout:5/70: rename d0 to d0/dd/d18 22 2024-04-12T05:55:17.663 INFO:tasks.workunit.client.0.smithi008.stdout:2/44: dwrite d6/f9 [0,4194304] 0 2024-04-12T05:55:17.664 INFO:tasks.workunit.client.0.smithi008.stdout:2/45: chown d6/f9 14 1 2024-04-12T05:55:17.664 INFO:tasks.workunit.client.0.smithi008.stdout:7/66: dread d1/f6 [0,4194304] 0 2024-04-12T05:55:17.666 INFO:tasks.workunit.client.0.smithi008.stdout:2/46: symlink d6/lc 0 2024-04-12T05:55:17.672 INFO:tasks.workunit.client.0.smithi008.stdout:5/71: link d0/dc/f14 d0/dd/d11/f19 0 2024-04-12T05:55:17.672 INFO:tasks.workunit.client.0.smithi008.stdout:7/67: dread d1/f6 [4194304,4194304] 0 2024-04-12T05:55:17.676 INFO:tasks.workunit.client.0.smithi008.stdout:7/68: write d1/d5/da/fc [2928822,125063] 0 2024-04-12T05:55:17.678 INFO:tasks.workunit.client.0.smithi008.stdout:2/47: dwrite d6/f9 [0,4194304] 0 2024-04-12T05:55:17.680 INFO:tasks.workunit.client.0.smithi008.stdout:7/69: symlink d1/l16 0 2024-04-12T05:55:17.682 INFO:tasks.workunit.client.0.smithi008.stdout:2/48: write f5 [492649,104559] 0 2024-04-12T05:55:17.685 INFO:tasks.workunit.client.0.smithi008.stdout:5/72: write d0/dc/d13/f15 [1003044,7501] 0 2024-04-12T05:55:17.690 INFO:tasks.workunit.client.0.smithi008.stdout:5/73: getdents d0 0 2024-04-12T05:55:17.690 INFO:tasks.workunit.client.0.smithi008.stdout:2/49: dread f0 [0,4194304] 0 2024-04-12T05:55:17.691 INFO:tasks.workunit.client.0.smithi008.stdout:7/70: getdents d1 0 2024-04-12T05:55:17.693 INFO:tasks.workunit.client.0.smithi008.stdout:5/74: mkdir d0/dc/d13/d1a 0 2024-04-12T05:55:17.695 INFO:tasks.workunit.client.0.smithi008.stdout:7/71: mkdir d1/d5/da/d11/d17 0 2024-04-12T05:55:17.699 INFO:tasks.workunit.client.0.smithi008.stdout:5/75: rename d0/d2/f3 to d0/d2/f1b 0 2024-04-12T05:55:17.700 INFO:tasks.workunit.client.0.smithi008.stdout:7/72: mkdir d1/d5/da/d11/d17/d18 0 2024-04-12T05:55:17.701 INFO:tasks.workunit.client.0.smithi008.stdout:7/73: dread - d1/f3 zero size 2024-04-12T05:55:17.702 INFO:tasks.workunit.client.0.smithi008.stdout:5/76: read d0/dd/d11/f19 [34061,111104] 0 2024-04-12T05:55:17.702 INFO:tasks.workunit.client.0.smithi008.stdout:2/50: dwrite d6/f9 [0,4194304] 0 2024-04-12T05:55:17.702 INFO:tasks.workunit.client.0.smithi008.stdout:7/74: chown d1/d5/da/d11 182341486 1 2024-04-12T05:55:17.703 INFO:tasks.workunit.client.0.smithi008.stdout:2/51: chown c2 1 1 2024-04-12T05:55:17.705 INFO:tasks.workunit.client.0.smithi008.stdout:5/77: rmdir d0/dd/d11 39 2024-04-12T05:55:17.707 INFO:tasks.workunit.client.0.smithi008.stdout:7/75: getdents d1/d5/da/d11/d17/d18 0 2024-04-12T05:55:17.711 INFO:tasks.workunit.client.0.smithi008.stdout:7/76: creat d1/d15/f19 x:0 0 0 2024-04-12T05:55:17.715 INFO:tasks.workunit.client.0.smithi008.stdout:2/52: dwrite f0 [0,4194304] 0 2024-04-12T05:55:17.717 INFO:tasks.workunit.client.0.smithi008.stdout:5/78: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:17.719 INFO:tasks.workunit.client.0.smithi008.stdout:2/53: truncate f5 657967 0 2024-04-12T05:55:17.723 INFO:tasks.workunit.client.0.smithi008.stdout:2/54: read d6/f9 [2076939,36185] 0 2024-04-12T05:55:17.726 INFO:tasks.workunit.client.0.smithi008.stdout:7/77: dwrite d1/f12 [4194304,4194304] 0 2024-04-12T05:55:17.737 INFO:tasks.workunit.client.0.smithi008.stdout:7/78: write d1/d15/f19 [906434,108679] 0 2024-04-12T05:55:17.739 INFO:tasks.workunit.client.0.smithi008.stdout:5/79: dwrite d0/dc/f14 [0,4194304] 0 2024-04-12T05:55:17.743 INFO:tasks.workunit.client.0.smithi008.stdout:2/55: getdents d6 0 2024-04-12T05:55:17.743 INFO:tasks.workunit.client.0.smithi008.stdout:2/56: fsync f0 0 2024-04-12T05:55:17.747 INFO:tasks.workunit.client.0.smithi008.stdout:5/80: mkdir d0/dc/d13/d1c 0 2024-04-12T05:55:17.751 INFO:tasks.workunit.client.0.smithi008.stdout:7/79: fsync d1/f12 0 2024-04-12T05:55:17.753 INFO:tasks.workunit.client.0.smithi008.stdout:2/57: creat d6/db/fd x:0 0 0 2024-04-12T05:55:17.759 INFO:tasks.workunit.client.0.smithi008.stdout:5/81: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:17.765 INFO:tasks.workunit.client.0.smithi008.stdout:5/82: mkdir d0/dc/d1d 0 2024-04-12T05:55:17.767 INFO:tasks.workunit.client.0.smithi008.stdout:2/58: dwrite f5 [0,4194304] 0 2024-04-12T05:55:17.776 INFO:tasks.workunit.client.0.smithi008.stdout:5/83: write d0/d2/f1b [1757523,58282] 0 2024-04-12T05:55:17.777 INFO:tasks.workunit.client.0.smithi008.stdout:2/59: creat d6/db/fe x:0 0 0 2024-04-12T05:55:17.778 INFO:tasks.workunit.client.0.smithi008.stdout:2/60: dread - d6/db/fe zero size 2024-04-12T05:55:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:5/84: rename d0/dc/f14 to d0/dd/f1e 0 2024-04-12T05:55:17.787 INFO:tasks.workunit.client.0.smithi008.stdout:2/61: dwrite d6/db/fe [0,4194304] 0 2024-04-12T05:55:17.799 INFO:tasks.workunit.client.0.smithi008.stdout:5/85: mkdir d0/dc/d1d/d1f 0 2024-04-12T05:55:17.803 INFO:tasks.workunit.client.0.smithi008.stdout:2/62: dwrite d6/f9 [4194304,4194304] 0 2024-04-12T05:55:17.805 INFO:tasks.workunit.client.0.smithi008.stdout:5/86: mknod d0/d2/c20 0 2024-04-12T05:55:17.811 INFO:tasks.workunit.client.0.smithi008.stdout:2/63: unlink d6/db/fe 0 2024-04-12T05:55:17.817 INFO:tasks.workunit.client.0.smithi008.stdout:2/64: rmdir d6 39 2024-04-12T05:55:17.820 INFO:tasks.workunit.client.0.smithi008.stdout:5/87: dwrite d0/dd/f1e [0,4194304] 0 2024-04-12T05:55:17.828 INFO:tasks.workunit.client.0.smithi008.stdout:5/88: creat d0/d2/f21 x:0 0 0 2024-04-12T05:55:17.828 INFO:tasks.workunit.client.0.smithi008.stdout:5/89: stat d0/c1 0 2024-04-12T05:55:17.829 INFO:tasks.workunit.client.0.smithi008.stdout:2/65: fsync d6/db/fd 0 2024-04-12T05:55:17.833 INFO:tasks.workunit.client.0.smithi008.stdout:5/90: rmdir d0/dc/d1d 39 2024-04-12T05:55:17.834 INFO:tasks.workunit.client.0.smithi008.stdout:5/91: chown d0/dd/d11 29 1 2024-04-12T05:55:17.847 INFO:tasks.workunit.client.0.smithi008.stdout:5/92: link d0/d2/f17 d0/dc/d13/d1a/f22 0 2024-04-12T05:55:17.860 INFO:tasks.workunit.client.0.smithi008.stdout:5/93: truncate d0/dc/d13/d1a/f22 2535485 0 2024-04-12T05:55:17.867 INFO:tasks.workunit.client.0.smithi008.stdout:5/94: mknod d0/dc/d1d/c23 0 2024-04-12T05:55:17.868 INFO:tasks.workunit.client.0.smithi008.stdout:5/95: chown d0/d2/fe 63136590 1 2024-04-12T05:55:17.873 INFO:tasks.workunit.client.0.smithi008.stdout:5/96: unlink d0/d2/c20 0 2024-04-12T05:55:17.873 INFO:tasks.workunit.client.0.smithi008.stdout:2/66: getdents d6/db 0 2024-04-12T05:55:17.881 INFO:tasks.workunit.client.0.smithi008.stdout:2/67: dwrite f5 [0,4194304] 0 2024-04-12T05:55:17.887 INFO:tasks.workunit.client.0.smithi008.stdout:2/68: creat d6/ff x:0 0 0 2024-04-12T05:55:17.890 INFO:tasks.workunit.client.0.smithi008.stdout:5/97: rmdir d0/dc/d1d/d1f 0 2024-04-12T05:55:17.891 INFO:tasks.workunit.client.0.smithi008.stdout:5/98: chown d0/f12 8 1 2024-04-12T05:55:17.895 INFO:tasks.workunit.client.0.smithi008.stdout:2/69: dwrite f5 [0,4194304] 0 2024-04-12T05:55:17.898 INFO:tasks.workunit.client.0.smithi008.stdout:5/99: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:17.906 INFO:tasks.workunit.client.0.smithi008.stdout:2/70: mkdir d6/db/d10 0 2024-04-12T05:55:17.909 INFO:tasks.workunit.client.0.smithi008.stdout:5/100: unlink d0/d2/fe 0 2024-04-12T05:55:17.912 INFO:tasks.workunit.client.0.smithi008.stdout:2/71: unlink f5 0 2024-04-12T05:55:17.915 INFO:tasks.workunit.client.0.smithi008.stdout:5/101: dread d0/dd/d11/f19 [0,4194304] 0 2024-04-12T05:55:17.917 INFO:tasks.workunit.client.0.smithi008.stdout:2/72: symlink d6/l11 0 2024-04-12T05:55:17.919 INFO:tasks.workunit.client.0.smithi008.stdout:2/73: write d6/f9 [3832417,78509] 0 2024-04-12T05:55:17.919 INFO:tasks.workunit.client.0.smithi008.stdout:2/74: readlink d6/l8 0 2024-04-12T05:55:17.923 INFO:tasks.workunit.client.0.smithi008.stdout:5/102: creat d0/dc/d13/d1a/f24 x:0 0 0 2024-04-12T05:55:17.929 INFO:tasks.workunit.client.0.smithi008.stdout:2/75: dwrite d6/ff [0,4194304] 0 2024-04-12T05:55:17.930 INFO:tasks.workunit.client.0.smithi008.stdout:5/103: symlink d0/dc/d13/d1c/l25 0 2024-04-12T05:55:17.934 INFO:tasks.workunit.client.0.smithi008.stdout:2/76: symlink d6/l12 0 2024-04-12T05:55:17.939 INFO:tasks.workunit.client.0.smithi008.stdout:2/77: mkdir d6/db/d10/d13 0 2024-04-12T05:55:17.940 INFO:tasks.workunit.client.0.smithi008.stdout:5/104: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:17.945 INFO:tasks.workunit.client.0.smithi008.stdout:2/78: creat d6/db/f14 x:0 0 0 2024-04-12T05:55:17.951 INFO:tasks.workunit.client.0.smithi008.stdout:2/79: creat d6/f15 x:0 0 0 2024-04-12T05:55:17.952 INFO:tasks.workunit.client.0.smithi008.stdout:2/80: fdatasync d6/f9 0 2024-04-12T05:55:17.953 INFO:tasks.workunit.client.0.smithi008.stdout:2/81: dread - d6/db/fd zero size 2024-04-12T05:55:17.956 INFO:tasks.workunit.client.0.smithi008.stdout:5/105: rename d0/dc/d13/d1c to d0/dc/d26 0 2024-04-12T05:55:17.958 INFO:tasks.workunit.client.0.smithi008.stdout:5/106: chown d0/dd/d11/f19 162 1 2024-04-12T05:55:17.963 INFO:tasks.workunit.client.0.smithi008.stdout:2/82: dwrite d6/db/f14 [0,4194304] 0 2024-04-12T05:55:17.964 INFO:tasks.workunit.client.0.smithi008.stdout:2/83: stat d6/db/d10/d13 0 2024-04-12T05:55:17.966 INFO:tasks.workunit.client.0.smithi008.stdout:5/107: dread d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:17.969 INFO:tasks.workunit.client.0.smithi008.stdout:5/108: write d0/d2/f1b [1836796,43663] 0 2024-04-12T05:55:17.969 INFO:tasks.workunit.client.0.smithi008.stdout:2/84: link d6/f15 d6/f16 0 2024-04-12T05:55:17.971 INFO:tasks.workunit.client.0.smithi008.stdout:2/85: symlink d6/db/d10/l17 0 2024-04-12T05:55:17.973 INFO:tasks.workunit.client.0.smithi008.stdout:5/109: rename d0/fa to d0/dc/d1d/f27 0 2024-04-12T05:55:17.974 INFO:tasks.workunit.client.0.smithi008.stdout:5/110: chown d0/d2/c7 5 1 2024-04-12T05:55:17.978 INFO:tasks.workunit.client.0.smithi008.stdout:5/111: dread d0/dd/f1e [0,4194304] 0 2024-04-12T05:55:17.980 INFO:tasks.workunit.client.0.smithi008.stdout:5/112: symlink d0/dc/d26/l28 0 2024-04-12T05:55:17.981 INFO:tasks.workunit.client.0.smithi008.stdout:5/113: read d0/dd/d11/f19 [1852524,63147] 0 2024-04-12T05:55:17.982 INFO:tasks.workunit.client.0.smithi008.stdout:5/114: chown d0/dc/d13 5 1 2024-04-12T05:55:17.984 INFO:tasks.workunit.client.0.smithi008.stdout:5/115: truncate d0/dc/ff 2527480 0 2024-04-12T05:55:17.986 INFO:tasks.workunit.client.0.smithi008.stdout:5/116: unlink d0/d2/l8 0 2024-04-12T05:55:17.986 INFO:tasks.workunit.client.0.smithi008.stdout:5/117: chown d0/dd 265819039 1 2024-04-12T05:55:18.001 INFO:tasks.workunit.client.0.smithi008.stdout:5/118: dwrite d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:18.002 INFO:tasks.workunit.client.0.smithi008.stdout:5/119: chown d0/dd/d11 1231414 1 2024-04-12T05:55:18.008 INFO:tasks.workunit.client.0.smithi008.stdout:5/120: mkdir d0/d29 0 2024-04-12T05:55:18.016 INFO:tasks.workunit.client.0.smithi008.stdout:5/121: dwrite d0/dc/d13/d1a/f24 [0,4194304] 0 2024-04-12T05:55:18.018 INFO:tasks.workunit.client.0.smithi008.stdout:5/122: write d0/f12 [4240010,127664] 0 2024-04-12T05:55:18.024 INFO:tasks.workunit.client.0.smithi008.stdout:5/123: link d0/d2/f17 d0/dc/f2a 0 2024-04-12T05:55:18.025 INFO:tasks.workunit.client.0.smithi008.stdout:5/124: fsync d0/dc/ff 0 2024-04-12T05:55:18.030 INFO:tasks.workunit.client.0.smithi008.stdout:5/125: rename d0/dc/d1d/f27 to d0/dc/d1d/f2b 0 2024-04-12T05:55:18.037 INFO:tasks.workunit.client.0.smithi008.stdout:5/126: dwrite d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:18.042 INFO:tasks.workunit.client.0.smithi008.stdout:5/127: link d0/dc/ff d0/d2/f2c 0 2024-04-12T05:55:18.047 INFO:tasks.workunit.client.0.smithi008.stdout:5/128: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:18.048 INFO:tasks.workunit.client.0.smithi008.stdout:5/129: chown d0/dc/d26/l25 4 1 2024-04-12T05:55:18.050 INFO:tasks.workunit.client.0.smithi008.stdout:5/130: creat d0/dc/d13/f2d x:0 0 0 2024-04-12T05:55:18.052 INFO:tasks.workunit.client.0.smithi008.stdout:5/131: symlink d0/dc/d26/l2e 0 2024-04-12T05:55:18.054 INFO:tasks.workunit.client.0.smithi008.stdout:5/132: creat d0/dc/d26/f2f x:0 0 0 2024-04-12T05:55:18.064 INFO:tasks.workunit.client.0.smithi008.stdout:5/133: dwrite d0/d2/f2c [0,4194304] 0 2024-04-12T05:55:18.785 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:18 smithi008 ceph-mon[39722]: pgmap v5: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:18 smithi194 ceph-mon[49771]: pgmap v5: 65 pgs: 65 active+clean; 129 MiB data, 664 MiB used, 536 GiB / 536 GiB avail 2024-04-12T05:55:19.521 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:19 smithi008 ceph-mon[39722]: mgrmap e24: smithi194.xbiyaj(active, since 5s) 2024-04-12T05:55:19.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:19 smithi194 ceph-mon[49771]: mgrmap e24: smithi194.xbiyaj(active, since 5s) 2024-04-12T05:55:20.868 INFO:tasks.workunit.client.0.smithi008.stdout:0/3: sync 2024-04-12T05:55:20.870 INFO:tasks.workunit.client.0.smithi008.stdout:0/4: dread f0 [0,4194304] 0 2024-04-12T05:55:20.876 INFO:tasks.workunit.client.0.smithi008.stdout:0/5: dwrite f0 [0,4194304] 0 2024-04-12T05:55:20.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/6: dwrite f0 [0,4194304] 0 2024-04-12T05:55:20.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/7: chown f0 4751145 1 2024-04-12T05:55:20.901 INFO:tasks.workunit.client.0.smithi008.stdout:0/8: dwrite f0 [0,4194304] 0 2024-04-12T05:55:20.903 INFO:tasks.workunit.client.0.smithi008.stdout:0/9: write f0 [2099700,111328] 0 2024-04-12T05:55:20.911 INFO:tasks.workunit.client.0.smithi008.stdout:0/10: creat f1 x:0 0 0 2024-04-12T05:55:20.913 INFO:tasks.workunit.client.0.smithi008.stdout:0/11: mkdir d2 0 2024-04-12T05:55:20.914 INFO:tasks.workunit.client.0.smithi008.stdout:0/12: dread - f1 zero size 2024-04-12T05:55:20.915 INFO:tasks.workunit.client.0.smithi008.stdout:0/13: write f1 [179300,80560] 0 2024-04-12T05:55:20.915 INFO:tasks.workunit.client.0.smithi008.stdout:0/14: rename d2 to d2/d3 22 2024-04-12T05:55:20.918 INFO:tasks.workunit.client.0.smithi008.stdout:0/15: read f1 [66610,57940] 0 2024-04-12T05:55:20.932 INFO:tasks.workunit.client.0.smithi008.stdout:0/16: fsync f1 0 2024-04-12T05:55:20.934 INFO:tasks.workunit.client.0.smithi008.stdout:0/17: mknod d2/c4 0 2024-04-12T05:55:20.935 INFO:tasks.workunit.client.0.smithi008.stdout:0/18: read f0 [1210782,30951] 0 2024-04-12T05:55:20.937 INFO:tasks.workunit.client.0.smithi008.stdout:0/19: creat d2/f5 x:0 0 0 2024-04-12T05:55:20.939 INFO:tasks.workunit.client.0.smithi008.stdout:0/20: write f0 [4943694,22017] 0 2024-04-12T05:55:20.940 INFO:tasks.workunit.client.0.smithi008.stdout:0/21: chown f0 2508753 1 2024-04-12T05:55:20.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/22: symlink d2/l6 0 2024-04-12T05:55:20.948 INFO:tasks.workunit.client.0.smithi008.stdout:0/23: getdents d2 0 2024-04-12T05:55:20.949 INFO:tasks.workunit.client.0.smithi008.stdout:0/24: truncate f1 498367 0 2024-04-12T05:55:20.951 INFO:tasks.workunit.client.0.smithi008.stdout:0/25: creat d2/f7 x:0 0 0 2024-04-12T05:55:20.951 INFO:tasks.workunit.client.0.smithi008.stdout:0/26: dread - d2/f5 zero size 2024-04-12T05:55:20.954 INFO:tasks.workunit.client.0.smithi008.stdout:0/27: rename d2/f7 to d2/f8 0 2024-04-12T05:55:20.957 INFO:tasks.workunit.client.0.smithi008.stdout:0/28: dread f1 [0,4194304] 0 2024-04-12T05:55:20.959 INFO:tasks.workunit.client.0.smithi008.stdout:0/29: mknod d2/c9 0 2024-04-12T05:55:20.959 INFO:tasks.workunit.client.0.smithi008.stdout:0/30: write d2/f5 [954214,107821] 0 2024-04-12T05:55:20.965 INFO:tasks.workunit.client.0.smithi008.stdout:0/31: rmdir d2 39 2024-04-12T05:55:20.968 INFO:tasks.workunit.client.0.smithi008.stdout:0/32: rename d2/f8 to d2/fa 0 2024-04-12T05:55:20.968 INFO:tasks.workunit.client.0.smithi008.stdout:0/33: chown f0 676383 1 2024-04-12T05:55:20.970 INFO:tasks.workunit.client.0.smithi008.stdout:0/34: fdatasync f1 0 2024-04-12T05:55:20.971 INFO:tasks.workunit.client.0.smithi008.stdout:0/35: rmdir d2 39 2024-04-12T05:55:20.975 INFO:tasks.workunit.client.0.smithi008.stdout:0/36: link d2/c4 d2/cb 0 2024-04-12T05:55:20.977 INFO:tasks.workunit.client.0.smithi008.stdout:0/37: creat d2/fc x:0 0 0 2024-04-12T05:55:20.979 INFO:tasks.workunit.client.0.smithi008.stdout:0/38: chown d2/cb 1287 1 2024-04-12T05:55:21.107 INFO:tasks.workunit.client.0.smithi008.stdout:3/23: sync 2024-04-12T05:55:21.107 INFO:tasks.workunit.client.0.smithi008.stdout:9/5: sync 2024-04-12T05:55:21.107 INFO:tasks.workunit.client.0.smithi008.stdout:9/6: write - no filename 2024-04-12T05:55:21.107 INFO:tasks.workunit.client.0.smithi008.stdout:9/7: read - no filename 2024-04-12T05:55:21.108 INFO:tasks.workunit.client.0.smithi008.stdout:1/41: sync 2024-04-12T05:55:21.108 INFO:tasks.workunit.client.0.smithi008.stdout:4/7: sync 2024-04-12T05:55:21.109 INFO:tasks.workunit.client.0.smithi008.stdout:4/8: mkdir d1 0 2024-04-12T05:55:21.110 INFO:tasks.workunit.client.0.smithi008.stdout:4/9: write - no filename 2024-04-12T05:55:21.110 INFO:tasks.workunit.client.0.smithi008.stdout:4/10: write - no filename 2024-04-12T05:55:21.112 INFO:tasks.workunit.client.0.smithi008.stdout:9/8: creat f1 x:0 0 0 2024-04-12T05:55:21.114 INFO:tasks.workunit.client.0.smithi008.stdout:4/11: creat d1/f2 x:0 0 0 2024-04-12T05:55:21.115 INFO:tasks.workunit.client.0.smithi008.stdout:4/12: truncate d1/f2 914097 0 2024-04-12T05:55:21.116 INFO:tasks.workunit.client.0.smithi008.stdout:9/9: creat f2 x:0 0 0 2024-04-12T05:55:21.116 INFO:tasks.workunit.client.0.smithi008.stdout:3/24: getdents . 0 2024-04-12T05:55:21.117 INFO:tasks.workunit.client.0.smithi008.stdout:1/42: dread f3 [0,4194304] 0 2024-04-12T05:55:21.118 INFO:tasks.workunit.client.0.smithi008.stdout:3/25: dread - f7 zero size 2024-04-12T05:55:21.118 INFO:tasks.workunit.client.0.smithi008.stdout:9/10: mkdir d3 0 2024-04-12T05:55:21.125 INFO:tasks.workunit.client.0.smithi008.stdout:9/11: rename f2 to d3/f4 0 2024-04-12T05:55:21.128 INFO:tasks.workunit.client.0.smithi008.stdout:3/26: dwrite f6 [0,4194304] 0 2024-04-12T05:55:21.130 INFO:tasks.workunit.client.0.smithi008.stdout:1/43: write f3 [776878,102964] 0 2024-04-12T05:55:21.133 INFO:tasks.workunit.client.0.smithi008.stdout:9/12: write d3/f4 [286322,64345] 0 2024-04-12T05:55:21.135 INFO:tasks.workunit.client.0.smithi008.stdout:1/44: write f3 [3049626,99232] 0 2024-04-12T05:55:21.139 INFO:tasks.workunit.client.0.smithi008.stdout:1/45: mkdir d4/d7 0 2024-04-12T05:55:21.139 INFO:tasks.workunit.client.0.smithi008.stdout:1/46: chown d4 11740824 1 2024-04-12T05:55:21.142 INFO:tasks.workunit.client.0.smithi008.stdout:9/13: dread d3/f4 [0,4194304] 0 2024-04-12T05:55:21.143 INFO:tasks.workunit.client.0.smithi008.stdout:1/47: creat d4/d6/f8 x:0 0 0 2024-04-12T05:55:21.143 INFO:tasks.workunit.client.0.smithi008.stdout:3/27: dwrite f1 [0,4194304] 0 2024-04-12T05:55:21.145 INFO:tasks.workunit.client.0.smithi008.stdout:1/48: write f3 [556093,81059] 0 2024-04-12T05:55:21.147 INFO:tasks.workunit.client.0.smithi008.stdout:3/28: mkdir d9 0 2024-04-12T05:55:21.149 INFO:tasks.workunit.client.0.smithi008.stdout:9/14: rename d3/f4 to d3/f5 0 2024-04-12T05:55:21.151 INFO:tasks.workunit.client.0.smithi008.stdout:1/49: link d4/d6/f8 d4/f9 0 2024-04-12T05:55:21.151 INFO:tasks.workunit.client.0.smithi008.stdout:3/29: symlink d9/la 0 2024-04-12T05:55:21.152 INFO:tasks.workunit.client.0.smithi008.stdout:9/15: creat d3/f6 x:0 0 0 2024-04-12T05:55:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:21 smithi008 ceph-mon[39722]: pgmap v6: 65 pgs: 65 active+clean; 147 MiB data, 712 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 6.8 MiB/s wr, 59 op/s 2024-04-12T05:55:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:21 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:21 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:21 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:21 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.165 INFO:tasks.workunit.client.0.smithi008.stdout:9/16: dwrite d3/f6 [0,4194304] 0 2024-04-12T05:55:21.165 INFO:tasks.workunit.client.0.smithi008.stdout:9/17: truncate d3/f6 4921743 0 2024-04-12T05:55:21.165 INFO:tasks.workunit.client.0.smithi008.stdout:1/50: rmdir d4/d6 39 2024-04-12T05:55:21.168 INFO:tasks.workunit.client.0.smithi008.stdout:9/18: creat d3/f7 x:0 0 0 2024-04-12T05:55:21.169 INFO:tasks.workunit.client.0.smithi008.stdout:9/19: dread - f1 zero size 2024-04-12T05:55:21.169 INFO:tasks.workunit.client.0.smithi008.stdout:3/30: truncate f1 3854164 0 2024-04-12T05:55:21.172 INFO:tasks.workunit.client.0.smithi008.stdout:3/31: mkdir d9/db 0 2024-04-12T05:55:21.173 INFO:tasks.workunit.client.0.smithi008.stdout:9/20: write d3/f6 [4224018,124436] 0 2024-04-12T05:55:21.176 INFO:tasks.workunit.client.0.smithi008.stdout:9/21: symlink d3/l8 0 2024-04-12T05:55:21.179 INFO:tasks.workunit.client.0.smithi008.stdout:9/22: rename f1 to d3/f9 0 2024-04-12T05:55:21.182 INFO:tasks.workunit.client.0.smithi008.stdout:3/32: dread f1 [0,4194304] 0 2024-04-12T05:55:21.183 INFO:tasks.workunit.client.0.smithi008.stdout:9/23: creat d3/fa x:0 0 0 2024-04-12T05:55:21.187 INFO:tasks.workunit.client.0.smithi008.stdout:9/24: creat d3/fb x:0 0 0 2024-04-12T05:55:21.189 INFO:tasks.workunit.client.0.smithi008.stdout:1/51: dwrite d4/d6/f8 [0,4194304] 0 2024-04-12T05:55:21.190 INFO:tasks.workunit.client.0.smithi008.stdout:9/25: dread - d3/fb zero size 2024-04-12T05:55:21.192 INFO:tasks.workunit.client.0.smithi008.stdout:3/33: creat d9/db/fc x:0 0 0 2024-04-12T05:55:21.197 INFO:tasks.workunit.client.0.smithi008.stdout:9/26: creat d3/fc x:0 0 0 2024-04-12T05:55:21.198 INFO:tasks.workunit.client.0.smithi008.stdout:3/34: mkdir d9/dd 0 2024-04-12T05:55:21.201 INFO:tasks.workunit.client.0.smithi008.stdout:9/27: truncate d3/fa 900552 0 2024-04-12T05:55:21.201 INFO:tasks.workunit.client.0.smithi008.stdout:1/52: dwrite d4/d6/f8 [4194304,4194304] 0 2024-04-12T05:55:21.203 INFO:tasks.workunit.client.0.smithi008.stdout:1/53: rename d4 to d4/d7/da 22 2024-04-12T05:55:21.205 INFO:tasks.workunit.client.0.smithi008.stdout:9/28: write d3/f5 [713153,30730] 0 2024-04-12T05:55:21.210 INFO:tasks.workunit.client.0.smithi008.stdout:1/54: creat d4/d6/fb x:0 0 0 2024-04-12T05:55:21.211 INFO:tasks.workunit.client.0.smithi008.stdout:9/29: rename d3/fc to d3/fd 0 2024-04-12T05:55:21.211 INFO:tasks.workunit.client.0.smithi008.stdout:3/35: dread f1 [0,4194304] 0 2024-04-12T05:55:21.213 INFO:tasks.workunit.client.0.smithi008.stdout:1/55: creat d4/d7/fc x:0 0 0 2024-04-12T05:55:21.214 INFO:tasks.workunit.client.0.smithi008.stdout:3/36: truncate f1 4797553 0 2024-04-12T05:55:21.220 INFO:tasks.workunit.client.0.smithi008.stdout:1/56: unlink d4/c5 0 2024-04-12T05:55:21.221 INFO:tasks.workunit.client.0.smithi008.stdout:1/57: fdatasync f3 0 2024-04-12T05:55:21.223 INFO:tasks.workunit.client.0.smithi008.stdout:3/37: dread f6 [0,4194304] 0 2024-04-12T05:55:21.226 INFO:tasks.workunit.client.0.smithi008.stdout:9/30: dwrite d3/fd [0,4194304] 0 2024-04-12T05:55:21.229 INFO:tasks.workunit.client.0.smithi008.stdout:3/38: rmdir d9/db 39 2024-04-12T05:55:21.229 INFO:tasks.workunit.client.0.smithi008.stdout:3/39: chown c0 242 1 2024-04-12T05:55:21.232 INFO:tasks.workunit.client.0.smithi008.stdout:9/31: mknod d3/ce 0 2024-04-12T05:55:21.232 INFO:tasks.workunit.client.0.smithi008.stdout:3/40: truncate d9/db/fc 945771 0 2024-04-12T05:55:21.232 INFO:tasks.workunit.client.0.smithi008.stdout:9/32: readlink d3/l8 0 2024-04-12T05:55:21.235 INFO:tasks.workunit.client.0.smithi008.stdout:9/33: mkdir d3/df 0 2024-04-12T05:55:21.235 INFO:tasks.workunit.client.0.smithi008.stdout:9/34: chown d3 23593 1 2024-04-12T05:55:21.237 INFO:tasks.workunit.client.0.smithi008.stdout:3/41: link c5 d9/dd/ce 0 2024-04-12T05:55:21.239 INFO:tasks.workunit.client.0.smithi008.stdout:9/35: mkdir d3/df/d10 0 2024-04-12T05:55:21.242 INFO:tasks.workunit.client.0.smithi008.stdout:1/58: truncate d4/f9 251428 0 2024-04-12T05:55:21.243 INFO:tasks.workunit.client.0.smithi008.stdout:3/42: mknod d9/db/cf 0 2024-04-12T05:55:21.246 INFO:tasks.workunit.client.0.smithi008.stdout:9/36: dwrite d3/f6 [0,4194304] 0 2024-04-12T05:55:21.248 INFO:tasks.workunit.client.0.smithi008.stdout:3/43: write f1 [339266,65625] 0 2024-04-12T05:55:21.253 INFO:tasks.workunit.client.0.smithi008.stdout:5/134: sync 2024-04-12T05:55:21.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/28: sync 2024-04-12T05:55:21.253 INFO:tasks.workunit.client.0.smithi008.stdout:2/86: sync 2024-04-12T05:55:21.253 INFO:tasks.workunit.client.0.smithi008.stdout:7/80: sync 2024-04-12T05:55:21.253 INFO:tasks.workunit.client.0.smithi008.stdout:6/11: sync 2024-04-12T05:55:21.254 INFO:tasks.workunit.client.0.smithi008.stdout:7/81: readlink d1/l2 0 2024-04-12T05:55:21.256 INFO:tasks.workunit.client.0.smithi008.stdout:8/29: fsync f9 0 2024-04-12T05:55:21.257 INFO:tasks.workunit.client.0.smithi008.stdout:5/135: readlink d0/dc/d13/l16 0 2024-04-12T05:55:21.259 INFO:tasks.workunit.client.0.smithi008.stdout:6/12: truncate f1 261112 0 2024-04-12T05:55:21.261 INFO:tasks.workunit.client.0.smithi008.stdout:2/87: symlink d6/db/d10/l18 0 2024-04-12T05:55:21.262 INFO:tasks.workunit.client.0.smithi008.stdout:3/44: mkdir d9/d10 0 2024-04-12T05:55:21.283 INFO:tasks.workunit.client.0.smithi008.stdout:5/136: mknod d0/dd/c30 0 2024-04-12T05:55:21.283 INFO:tasks.workunit.client.0.smithi008.stdout:6/13: write f3 [717812,105609] 0 2024-04-12T05:55:21.283 INFO:tasks.workunit.client.0.smithi008.stdout:8/30: rename l3 to la 0 2024-04-12T05:55:21.288 INFO:tasks.workunit.client.0.smithi008.stdout:9/37: dwrite d3/f7 [0,4194304] 0 2024-04-12T05:55:21.304 INFO:tasks.workunit.client.0.smithi008.stdout:2/88: dread f0 [0,4194304] 0 2024-04-12T05:55:21.309 INFO:tasks.workunit.client.0.smithi008.stdout:7/82: dwrite d1/f6 [0,4194304] 0 2024-04-12T05:55:21.310 INFO:tasks.workunit.client.0.smithi008.stdout:1/59: dwrite d4/d6/f8 [0,4194304] 0 2024-04-12T05:55:21.312 INFO:tasks.workunit.client.0.smithi008.stdout:2/89: write f0 [4336384,69855] 0 2024-04-12T05:55:21.316 INFO:tasks.workunit.client.0.smithi008.stdout:3/45: dwrite d9/db/fc [0,4194304] 0 2024-04-12T05:55:21.317 INFO:tasks.workunit.client.0.smithi008.stdout:5/137: dread d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:21.318 INFO:tasks.workunit.client.0.smithi008.stdout:2/90: fdatasync d6/ff 0 2024-04-12T05:55:21.322 INFO:tasks.workunit.client.0.smithi008.stdout:6/14: dwrite f1 [0,4194304] 0 2024-04-12T05:55:21.328 INFO:tasks.workunit.client.0.smithi008.stdout:6/15: write f1 [1563574,21138] 0 2024-04-12T05:55:21.334 INFO:tasks.workunit.client.0.smithi008.stdout:2/91: dread f0 [0,4194304] 0 2024-04-12T05:55:21.334 INFO:tasks.workunit.client.0.smithi008.stdout:7/83: dwrite d1/d5/da/fc [8388608,4194304] 0 2024-04-12T05:55:21.335 INFO:tasks.workunit.client.0.smithi008.stdout:3/46: dwrite f1 [0,4194304] 0 2024-04-12T05:55:21.335 INFO:tasks.workunit.client.0.smithi008.stdout:2/92: chown l3 3377260 1 2024-04-12T05:55:21.337 INFO:tasks.workunit.client.0.smithi008.stdout:7/84: write d1/f3 [695286,113355] 0 2024-04-12T05:55:21.338 INFO:tasks.workunit.client.0.smithi008.stdout:2/93: write d6/ff [2182331,58153] 0 2024-04-12T05:55:21.340 INFO:tasks.workunit.client.0.smithi008.stdout:7/85: chown d1/d15/f19 7 1 2024-04-12T05:55:21.340 INFO:tasks.workunit.client.0.smithi008.stdout:0/39: sync 2024-04-12T05:55:21.340 INFO:tasks.workunit.client.0.smithi008.stdout:7/86: stat d1/l9 0 2024-04-12T05:55:21.341 INFO:tasks.workunit.client.0.smithi008.stdout:7/87: read d1/d5/da/fc [543187,60895] 0 2024-04-12T05:55:21.350 INFO:tasks.workunit.client.0.smithi008.stdout:0/40: dwrite d2/fc [0,4194304] 0 2024-04-12T05:55:21.352 INFO:tasks.workunit.client.0.smithi008.stdout:0/41: write d2/fc [2136321,114897] 0 2024-04-12T05:55:21.366 INFO:tasks.workunit.client.0.smithi008.stdout:0/42: dwrite f1 [0,4194304] 0 2024-04-12T05:55:21.399 INFO:tasks.workunit.client.0.smithi008.stdout:4/13: sync 2024-04-12T05:55:21.419 INFO:tasks.workunit.client.0.smithi008.stdout:8/31: creat fb x:0 0 0 2024-04-12T05:55:21.419 INFO:tasks.workunit.client.0.smithi008.stdout:8/32: rmdir - no directory 2024-04-12T05:55:21.419 INFO:tasks.workunit.client.0.smithi008.stdout:8/33: chown f9 432 1 2024-04-12T05:55:21.419 INFO:tasks.workunit.client.0.smithi008.stdout:8/34: dread - fb zero size 2024-04-12T05:55:21.425 INFO:tasks.workunit.client.0.smithi008.stdout:5/138: read d0/dc/d1d/f2b [1343752,115324] 0 2024-04-12T05:55:21.427 INFO:tasks.workunit.client.0.smithi008.stdout:5/139: write d0/dc/d13/f15 [493994,2582] 0 2024-04-12T05:55:21.428 INFO:tasks.workunit.client.0.smithi008.stdout:6/16: creat f4 x:0 0 0 2024-04-12T05:55:21.430 INFO:tasks.workunit.client.0.smithi008.stdout:6/17: chown f4 1291979 1 2024-04-12T05:55:21.431 INFO:tasks.workunit.client.0.smithi008.stdout:7/88: symlink d1/d5/l1a 0 2024-04-12T05:55:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:5/140: dread d0/dd/f1e [0,4194304] 0 2024-04-12T05:55:21.438 INFO:tasks.workunit.client.0.smithi008.stdout:0/43: creat d2/fd x:0 0 0 2024-04-12T05:55:21.439 INFO:tasks.workunit.client.0.smithi008.stdout:4/14: mkdir d1/d3 0 2024-04-12T05:55:21.443 INFO:tasks.workunit.client.0.smithi008.stdout:1/60: mkdir d4/dd 0 2024-04-12T05:55:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:1/61: write d4/d7/fc [437288,104811] 0 2024-04-12T05:55:21.448 INFO:tasks.workunit.client.0.smithi008.stdout:7/89: dwrite d1/fb [0,4194304] 0 2024-04-12T05:55:21.451 INFO:tasks.workunit.client.0.smithi008.stdout:6/18: rename f1 to f5 0 2024-04-12T05:55:21.455 INFO:tasks.workunit.client.0.smithi008.stdout:0/44: dwrite f0 [4194304,4194304] 0 2024-04-12T05:55:21.457 INFO:tasks.workunit.client.0.smithi008.stdout:8/35: rename la to lc 0 2024-04-12T05:55:21.459 INFO:tasks.workunit.client.0.smithi008.stdout:5/141: creat d0/d2/f31 x:0 0 0 2024-04-12T05:55:21.468 INFO:tasks.workunit.client.0.smithi008.stdout:0/45: rename f0 to d2/fe 0 2024-04-12T05:55:21.472 INFO:tasks.workunit.client.0.smithi008.stdout:1/62: dwrite d4/d7/fc [0,4194304] 0 2024-04-12T05:55:21.472 INFO:tasks.workunit.client.0.smithi008.stdout:0/46: fsync f1 0 2024-04-12T05:55:21.478 INFO:tasks.workunit.client.0.smithi008.stdout:4/15: rmdir d1/d3 0 2024-04-12T05:55:21.478 INFO:tasks.workunit.client.0.smithi008.stdout:1/63: chown d4/d6/f8 11441 1 2024-04-12T05:55:21.478 INFO:tasks.workunit.client.0.smithi008.stdout:7/90: symlink d1/d5/da/d11/d17/d18/l1b 0 2024-04-12T05:55:21.481 INFO:tasks.workunit.client.0.smithi008.stdout:8/36: dwrite f7 [0,4194304] 0 2024-04-12T05:55:21.482 INFO:tasks.workunit.client.0.smithi008.stdout:0/47: creat d2/ff x:0 0 0 2024-04-12T05:55:21.485 INFO:tasks.workunit.client.0.smithi008.stdout:7/91: symlink d1/l1c 0 2024-04-12T05:55:21.487 INFO:tasks.workunit.client.0.smithi008.stdout:5/142: dwrite d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:21.489 INFO:tasks.workunit.client.0.smithi008.stdout:0/48: dread - d2/fd zero size 2024-04-12T05:55:21.490 INFO:tasks.workunit.client.0.smithi008.stdout:5/143: chown d0/dc/l10 744 1 2024-04-12T05:55:21.491 INFO:tasks.workunit.client.0.smithi008.stdout:1/64: rename d4/d6/fb to d4/d7/fe 0 2024-04-12T05:55:21.494 INFO:tasks.workunit.client.0.smithi008.stdout:1/65: truncate d4/d7/fc 4851273 0 2024-04-12T05:55:21.496 INFO:tasks.workunit.client.0.smithi008.stdout:0/49: fdatasync d2/f5 0 2024-04-12T05:55:21.503 INFO:tasks.workunit.client.0.smithi008.stdout:8/37: dread f7 [0,4194304] 0 2024-04-12T05:55:21.506 INFO:tasks.workunit.client.0.smithi008.stdout:0/50: symlink d2/l10 0 2024-04-12T05:55:21.510 INFO:tasks.workunit.client.0.smithi008.stdout:5/144: link d0/dc/f2a d0/dc/d26/f32 0 2024-04-12T05:55:21.557 INFO:tasks.workunit.client.0.smithi008.stdout:9/38: truncate d3/f5 322613 0 2024-04-12T05:55:21.559 INFO:tasks.workunit.client.0.smithi008.stdout:9/39: creat d3/df/f11 x:0 0 0 2024-04-12T05:55:21.560 INFO:tasks.workunit.client.0.smithi008.stdout:9/40: stat d3/df/f11 0 2024-04-12T05:55:21.563 INFO:tasks.workunit.client.0.smithi008.stdout:9/41: mkdir d3/df/d10/d12 0 2024-04-12T05:55:21.568 INFO:tasks.workunit.client.0.smithi008.stdout:4/16: rmdir d1 39 2024-04-12T05:55:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:21 smithi194 ceph-mon[49771]: pgmap v6: 65 pgs: 65 active+clean; 147 MiB data, 712 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 6.8 MiB/s wr, 59 op/s 2024-04-12T05:55:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:21 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:21 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:21 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:21 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:21.571 INFO:tasks.workunit.client.0.smithi008.stdout:4/17: unlink d1/f2 0 2024-04-12T05:55:21.571 INFO:tasks.workunit.client.0.smithi008.stdout:4/18: stat d1 0 2024-04-12T05:55:21.572 INFO:tasks.workunit.client.0.smithi008.stdout:4/19: fsync - no filename 2024-04-12T05:55:21.577 INFO:tasks.workunit.client.0.smithi008.stdout:4/20: creat d1/f4 x:0 0 0 2024-04-12T05:55:21.577 INFO:tasks.workunit.client.0.smithi008.stdout:7/92: truncate d1/f6 821354 0 2024-04-12T05:55:21.580 INFO:tasks.workunit.client.0.smithi008.stdout:0/51: rmdir d2 39 2024-04-12T05:55:21.580 INFO:tasks.workunit.client.0.smithi008.stdout:5/145: write d0/dc/f2a [2718902,22638] 0 2024-04-12T05:55:21.588 INFO:tasks.workunit.client.0.smithi008.stdout:0/52: unlink d2/fd 0 2024-04-12T05:55:21.589 INFO:tasks.workunit.client.0.smithi008.stdout:7/93: dwrite d1/d15/f19 [0,4194304] 0 2024-04-12T05:55:21.593 INFO:tasks.workunit.client.0.smithi008.stdout:7/94: readlink d1/l8 0 2024-04-12T05:55:21.593 INFO:tasks.workunit.client.0.smithi008.stdout:4/21: rmdir d1 39 2024-04-12T05:55:21.593 INFO:tasks.workunit.client.0.smithi008.stdout:0/53: write d2/fc [1115918,90432] 0 2024-04-12T05:55:21.594 INFO:tasks.workunit.client.0.smithi008.stdout:5/146: write d0/d2/f5 [4644639,123002] 0 2024-04-12T05:55:21.596 INFO:tasks.workunit.client.0.smithi008.stdout:4/22: read - d1/f4 zero size 2024-04-12T05:55:21.598 INFO:tasks.workunit.client.0.smithi008.stdout:0/54: stat d2 0 2024-04-12T05:55:21.601 INFO:tasks.workunit.client.0.smithi008.stdout:4/23: rename d1 to d1/d5 22 2024-04-12T05:55:21.604 INFO:tasks.workunit.client.0.smithi008.stdout:4/24: read - d1/f4 zero size 2024-04-12T05:55:21.606 INFO:tasks.workunit.client.0.smithi008.stdout:9/42: dwrite d3/f5 [0,4194304] 0 2024-04-12T05:55:21.610 INFO:tasks.workunit.client.0.smithi008.stdout:7/95: rename d1/d5/da/d11/d17 to d1/d5/da/d1d 0 2024-04-12T05:55:21.613 INFO:tasks.workunit.client.0.smithi008.stdout:5/147: truncate d0/dd/f1e 2872719 0 2024-04-12T05:55:21.615 INFO:tasks.workunit.client.0.smithi008.stdout:4/25: rmdir d1 39 2024-04-12T05:55:21.615 INFO:tasks.workunit.client.0.smithi008.stdout:9/43: mkdir d3/df/d13 0 2024-04-12T05:55:21.619 INFO:tasks.workunit.client.0.smithi008.stdout:0/55: dwrite d2/ff [0,4194304] 0 2024-04-12T05:55:21.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/26: dread - d1/f4 zero size 2024-04-12T05:55:21.622 INFO:tasks.workunit.client.0.smithi008.stdout:9/44: mkdir d3/df/d13/d14 0 2024-04-12T05:55:21.626 INFO:tasks.workunit.client.0.smithi008.stdout:7/96: dwrite d1/f3 [0,4194304] 0 2024-04-12T05:55:21.626 INFO:tasks.workunit.client.0.smithi008.stdout:7/97: chown d1/d5/da 127422428 1 2024-04-12T05:55:21.628 INFO:tasks.workunit.client.0.smithi008.stdout:0/56: write d2/fe [186216,128224] 0 2024-04-12T05:55:21.630 INFO:tasks.workunit.client.0.smithi008.stdout:7/98: fdatasync d1/d15/f19 0 2024-04-12T05:55:21.631 INFO:tasks.workunit.client.0.smithi008.stdout:4/27: symlink d1/l6 0 2024-04-12T05:55:21.634 INFO:tasks.workunit.client.0.smithi008.stdout:0/57: mkdir d2/d11 0 2024-04-12T05:55:21.637 INFO:tasks.workunit.client.0.smithi008.stdout:4/28: symlink d1/l7 0 2024-04-12T05:55:21.648 INFO:tasks.workunit.client.0.smithi008.stdout:9/45: dwrite d3/fb [0,4194304] 0 2024-04-12T05:55:21.652 INFO:tasks.workunit.client.0.smithi008.stdout:4/29: getdents d1 0 2024-04-12T05:55:21.653 INFO:tasks.workunit.client.0.smithi008.stdout:9/46: creat d3/df/d10/f15 x:0 0 0 2024-04-12T05:55:21.655 INFO:tasks.workunit.client.0.smithi008.stdout:0/58: dread d2/fe [0,4194304] 0 2024-04-12T05:55:21.656 INFO:tasks.workunit.client.0.smithi008.stdout:9/47: fdatasync d3/fa 0 2024-04-12T05:55:21.658 INFO:tasks.workunit.client.0.smithi008.stdout:4/30: getdents d1 0 2024-04-12T05:55:21.658 INFO:tasks.workunit.client.0.smithi008.stdout:0/59: read d2/fe [1136195,91692] 0 2024-04-12T05:55:21.660 INFO:tasks.workunit.client.0.smithi008.stdout:4/31: rmdir d1 39 2024-04-12T05:55:21.660 INFO:tasks.workunit.client.0.smithi008.stdout:7/99: dwrite d1/d5/da/fc [0,4194304] 0 2024-04-12T05:55:21.661 INFO:tasks.workunit.client.0.smithi008.stdout:9/48: dread - d3/df/d10/f15 zero size 2024-04-12T05:55:21.663 INFO:tasks.workunit.client.0.smithi008.stdout:4/32: dread - d1/f4 zero size 2024-04-12T05:55:21.665 INFO:tasks.workunit.client.0.smithi008.stdout:7/100: stat d1/f6 0 2024-04-12T05:55:21.665 INFO:tasks.workunit.client.0.smithi008.stdout:0/60: write d2/fa [791047,101812] 0 2024-04-12T05:55:21.669 INFO:tasks.workunit.client.0.smithi008.stdout:4/33: mkdir d1/d8 0 2024-04-12T05:55:21.669 INFO:tasks.workunit.client.0.smithi008.stdout:9/49: mknod d3/df/d13/c16 0 2024-04-12T05:55:21.669 INFO:tasks.workunit.client.0.smithi008.stdout:0/61: mknod d2/c12 0 2024-04-12T05:55:21.672 INFO:tasks.workunit.client.0.smithi008.stdout:7/101: truncate d1/d15/f19 3864151 0 2024-04-12T05:55:21.674 INFO:tasks.workunit.client.0.smithi008.stdout:0/62: creat d2/f13 x:0 0 0 2024-04-12T05:55:21.677 INFO:tasks.workunit.client.0.smithi008.stdout:0/63: write d2/fc [4726086,64953] 0 2024-04-12T05:55:21.678 INFO:tasks.workunit.client.0.smithi008.stdout:4/34: link d1/l7 d1/l9 0 2024-04-12T05:55:21.686 INFO:tasks.workunit.client.0.smithi008.stdout:9/50: dwrite d3/f9 [0,4194304] 0 2024-04-12T05:55:21.687 INFO:tasks.workunit.client.0.smithi008.stdout:4/35: creat d1/d8/fa x:0 0 0 2024-04-12T05:55:21.695 INFO:tasks.workunit.client.0.smithi008.stdout:9/51: link d3/fb d3/df/d10/f17 0 2024-04-12T05:55:21.695 INFO:tasks.workunit.client.0.smithi008.stdout:7/102: truncate d1/d15/f19 4236301 0 2024-04-12T05:55:21.696 INFO:tasks.workunit.client.0.smithi008.stdout:0/64: truncate d2/fe 3930058 0 2024-04-12T05:55:21.699 INFO:tasks.workunit.client.0.smithi008.stdout:7/103: symlink d1/d15/l1e 0 2024-04-12T05:55:21.701 INFO:tasks.workunit.client.0.smithi008.stdout:0/65: symlink d2/d11/l14 0 2024-04-12T05:55:21.702 INFO:tasks.workunit.client.0.smithi008.stdout:4/36: dwrite d1/f4 [0,4194304] 0 2024-04-12T05:55:21.703 INFO:tasks.workunit.client.0.smithi008.stdout:4/37: readlink d1/l6 0 2024-04-12T05:55:21.703 INFO:tasks.workunit.client.0.smithi008.stdout:0/66: chown d2/f13 24 1 2024-04-12T05:55:21.704 INFO:tasks.workunit.client.0.smithi008.stdout:0/67: readlink d2/l6 0 2024-04-12T05:55:21.704 INFO:tasks.workunit.client.0.smithi008.stdout:9/52: link d3/f7 d3/df/d13/d14/f18 0 2024-04-12T05:55:21.707 INFO:tasks.workunit.client.0.smithi008.stdout:9/53: write d3/f5 [894690,38781] 0 2024-04-12T05:55:21.708 INFO:tasks.workunit.client.0.smithi008.stdout:7/104: rename d1/d5/da/d1d/d18 to d1/d5/da/d1f 0 2024-04-12T05:55:21.709 INFO:tasks.workunit.client.0.smithi008.stdout:4/38: chown d1/l9 5028 1 2024-04-12T05:55:21.709 INFO:tasks.workunit.client.0.smithi008.stdout:7/105: write d1/f3 [739703,46039] 0 2024-04-12T05:55:21.712 INFO:tasks.workunit.client.0.smithi008.stdout:0/68: mknod d2/c15 0 2024-04-12T05:55:21.714 INFO:tasks.workunit.client.0.smithi008.stdout:4/39: dread d1/f4 [0,4194304] 0 2024-04-12T05:55:21.715 INFO:tasks.workunit.client.0.smithi008.stdout:9/54: rmdir d3/df/d13/d14 39 2024-04-12T05:55:21.717 INFO:tasks.workunit.client.0.smithi008.stdout:9/55: dread - d3/df/f11 zero size 2024-04-12T05:55:21.717 INFO:tasks.workunit.client.0.smithi008.stdout:4/40: stat d1/d8/fa 0 2024-04-12T05:55:21.719 INFO:tasks.workunit.client.0.smithi008.stdout:4/41: dread - d1/d8/fa zero size 2024-04-12T05:55:21.719 INFO:tasks.workunit.client.0.smithi008.stdout:9/56: write d3/f5 [4461521,19406] 0 2024-04-12T05:55:21.720 INFO:tasks.workunit.client.0.smithi008.stdout:9/57: truncate d3/fd 4889380 0 2024-04-12T05:55:21.722 INFO:tasks.workunit.client.0.smithi008.stdout:7/106: link d1/d5/da/fc d1/d15/f20 0 2024-04-12T05:55:21.725 INFO:tasks.workunit.client.0.smithi008.stdout:0/69: link d2/ff d2/d11/f16 0 2024-04-12T05:55:21.727 INFO:tasks.workunit.client.0.smithi008.stdout:4/42: write d1/f4 [5062728,117939] 0 2024-04-12T05:55:21.728 INFO:tasks.workunit.client.0.smithi008.stdout:7/107: mknod d1/d5/da/c21 0 2024-04-12T05:55:21.730 INFO:tasks.workunit.client.0.smithi008.stdout:4/43: fdatasync d1/d8/fa 0 2024-04-12T05:55:21.731 INFO:tasks.workunit.client.0.smithi008.stdout:4/44: dread - d1/d8/fa zero size 2024-04-12T05:55:21.732 INFO:tasks.workunit.client.0.smithi008.stdout:0/70: write d2/ff [4184855,115555] 0 2024-04-12T05:55:21.734 INFO:tasks.workunit.client.0.smithi008.stdout:4/45: read - d1/d8/fa zero size 2024-04-12T05:55:21.735 INFO:tasks.workunit.client.0.smithi008.stdout:4/46: fsync d1/d8/fa 0 2024-04-12T05:55:21.738 INFO:tasks.workunit.client.0.smithi008.stdout:0/71: link d2/l6 d2/l17 0 2024-04-12T05:55:21.740 INFO:tasks.workunit.client.0.smithi008.stdout:4/47: mkdir d1/db 0 2024-04-12T05:55:21.743 INFO:tasks.workunit.client.0.smithi008.stdout:0/72: creat d2/d11/f18 x:0 0 0 2024-04-12T05:55:21.744 INFO:tasks.workunit.client.0.smithi008.stdout:0/73: chown d2/fc 31293843 1 2024-04-12T05:55:21.745 INFO:tasks.workunit.client.0.smithi008.stdout:0/74: fdatasync d2/f13 0 2024-04-12T05:55:21.745 INFO:tasks.workunit.client.0.smithi008.stdout:4/48: unlink d1/l6 0 2024-04-12T05:55:21.745 INFO:tasks.workunit.client.0.smithi008.stdout:4/49: chown d1/d8 23407023 1 2024-04-12T05:55:21.745 INFO:tasks.workunit.client.0.smithi008.stdout:0/75: chown d2/f5 1420 1 2024-04-12T05:55:21.750 INFO:tasks.workunit.client.0.smithi008.stdout:4/50: link d1/d8/fa d1/db/fc 0 2024-04-12T05:55:21.750 INFO:tasks.workunit.client.0.smithi008.stdout:4/51: chown d1/d8 26191 1 2024-04-12T05:55:21.753 INFO:tasks.workunit.client.0.smithi008.stdout:0/76: dread d2/fc [0,4194304] 0 2024-04-12T05:55:21.756 INFO:tasks.workunit.client.0.smithi008.stdout:4/52: write d1/d8/fa [939663,126373] 0 2024-04-12T05:55:21.758 INFO:tasks.workunit.client.0.smithi008.stdout:7/108: truncate d1/f3 1871753 0 2024-04-12T05:55:21.763 INFO:tasks.workunit.client.0.smithi008.stdout:0/77: rmdir d2/d11 39 2024-04-12T05:55:21.763 INFO:tasks.workunit.client.0.smithi008.stdout:0/78: readlink d2/l10 0 2024-04-12T05:55:21.767 INFO:tasks.workunit.client.0.smithi008.stdout:0/79: symlink d2/d11/l19 0 2024-04-12T05:55:21.768 INFO:tasks.workunit.client.0.smithi008.stdout:7/109: dread d1/f3 [0,4194304] 0 2024-04-12T05:55:21.772 INFO:tasks.workunit.client.0.smithi008.stdout:0/80: creat d2/d11/f1a x:0 0 0 2024-04-12T05:55:21.773 INFO:tasks.workunit.client.0.smithi008.stdout:7/110: chown d1/d5/c10 1867568 1 2024-04-12T05:55:21.779 INFO:tasks.workunit.client.0.smithi008.stdout:7/111: link d1/d5/da/c21 d1/d15/c22 0 2024-04-12T05:55:21.780 INFO:tasks.workunit.client.0.smithi008.stdout:0/81: truncate d2/d11/f16 4298794 0 2024-04-12T05:55:21.789 INFO:tasks.workunit.client.0.smithi008.stdout:0/82: getdents d2/d11 0 2024-04-12T05:55:21.794 INFO:tasks.workunit.client.0.smithi008.stdout:0/83: mkdir d2/d1b 0 2024-04-12T05:55:21.796 INFO:tasks.workunit.client.0.smithi008.stdout:7/112: dwrite d1/d15/f20 [4194304,4194304] 0 2024-04-12T05:55:21.797 INFO:tasks.workunit.client.0.smithi008.stdout:0/84: rename d2/d11/l19 to d2/d1b/l1c 0 2024-04-12T05:55:21.799 INFO:tasks.workunit.client.0.smithi008.stdout:7/113: rmdir d1/d5/da 39 2024-04-12T05:55:21.801 INFO:tasks.workunit.client.0.smithi008.stdout:0/85: chown d2/l17 7 1 2024-04-12T05:55:21.804 INFO:tasks.workunit.client.0.smithi008.stdout:7/114: fsync d1/d15/f19 0 2024-04-12T05:55:21.809 INFO:tasks.workunit.client.0.smithi008.stdout:7/115: dread d1/d15/f19 [0,4194304] 0 2024-04-12T05:55:21.810 INFO:tasks.workunit.client.0.smithi008.stdout:0/86: fdatasync d2/ff 0 2024-04-12T05:55:21.813 INFO:tasks.workunit.client.0.smithi008.stdout:0/87: creat d2/d1b/f1d x:0 0 0 2024-04-12T05:55:21.814 INFO:tasks.workunit.client.0.smithi008.stdout:7/116: creat d1/d5/da/d11/f23 x:0 0 0 2024-04-12T05:55:21.814 INFO:tasks.workunit.client.0.smithi008.stdout:0/88: symlink d2/d11/l1e 0 2024-04-12T05:55:21.815 INFO:tasks.workunit.client.0.smithi008.stdout:7/117: read d1/f12 [9624994,83526] 0 2024-04-12T05:55:21.816 INFO:tasks.workunit.client.0.smithi008.stdout:7/118: chown d1/d5/da/d1d 2783396 1 2024-04-12T05:55:21.816 INFO:tasks.workunit.client.0.smithi008.stdout:7/119: stat d1/l9 0 2024-04-12T05:55:21.818 INFO:tasks.workunit.client.0.smithi008.stdout:7/120: write d1/d5/da/d11/f23 [196731,18525] 0 2024-04-12T05:55:21.820 INFO:tasks.workunit.client.0.smithi008.stdout:0/89: getdents d2/d1b 0 2024-04-12T05:55:21.821 INFO:tasks.workunit.client.0.smithi008.stdout:0/90: dread - d2/f13 zero size 2024-04-12T05:55:21.825 INFO:tasks.workunit.client.0.smithi008.stdout:0/91: symlink d2/d11/l1f 0 2024-04-12T05:55:21.827 INFO:tasks.workunit.client.0.smithi008.stdout:0/92: mknod d2/d11/c20 0 2024-04-12T05:55:21.828 INFO:tasks.workunit.client.0.smithi008.stdout:7/121: truncate d1/d15/f20 8697013 0 2024-04-12T05:55:21.830 INFO:tasks.workunit.client.0.smithi008.stdout:0/93: rename d2/d11/f16 to d2/d1b/f21 0 2024-04-12T05:55:21.830 INFO:tasks.workunit.client.0.smithi008.stdout:7/122: rmdir d1/d5/da 39 2024-04-12T05:55:21.831 INFO:tasks.workunit.client.0.smithi008.stdout:7/123: fdatasync d1/f3 0 2024-04-12T05:55:21.836 INFO:tasks.workunit.client.0.smithi008.stdout:7/124: symlink d1/d5/da/d1d/l24 0 2024-04-12T05:55:21.837 INFO:tasks.workunit.client.0.smithi008.stdout:0/94: dwrite d2/d11/f1a [0,4194304] 0 2024-04-12T05:55:21.840 INFO:tasks.workunit.client.0.smithi008.stdout:7/125: fsync d1/f6 0 2024-04-12T05:55:21.844 INFO:tasks.workunit.client.0.smithi008.stdout:7/126: creat d1/d5/da/d11/f25 x:0 0 0 2024-04-12T05:55:21.847 INFO:tasks.workunit.client.0.smithi008.stdout:7/127: creat d1/d15/f26 x:0 0 0 2024-04-12T05:55:21.850 INFO:tasks.workunit.client.0.smithi008.stdout:0/95: write d2/ff [1355006,96803] 0 2024-04-12T05:55:21.853 INFO:tasks.workunit.client.0.smithi008.stdout:0/96: symlink d2/d11/l22 0 2024-04-12T05:55:21.855 INFO:tasks.workunit.client.0.smithi008.stdout:0/97: write d2/fa [1668101,95653] 0 2024-04-12T05:55:21.857 INFO:tasks.workunit.client.0.smithi008.stdout:7/128: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:21.860 INFO:tasks.workunit.client.0.smithi008.stdout:7/129: dread d1/f6 [0,4194304] 0 2024-04-12T05:55:21.865 INFO:tasks.workunit.client.0.smithi008.stdout:0/98: dwrite d2/d1b/f1d [0,4194304] 0 2024-04-12T05:55:21.866 INFO:tasks.workunit.client.0.smithi008.stdout:0/99: fdatasync d2/f5 0 2024-04-12T05:55:21.873 INFO:tasks.workunit.client.0.smithi008.stdout:0/100: creat d2/d11/f23 x:0 0 0 2024-04-12T05:55:21.873 INFO:tasks.workunit.client.0.smithi008.stdout:7/130: rmdir d1/d15 39 2024-04-12T05:55:21.876 INFO:tasks.workunit.client.0.smithi008.stdout:0/101: mkdir d2/d24 0 2024-04-12T05:55:21.877 INFO:tasks.workunit.client.0.smithi008.stdout:7/131: symlink d1/l27 0 2024-04-12T05:55:21.878 INFO:tasks.workunit.client.0.smithi008.stdout:0/102: write d2/d11/f18 [333006,65220] 0 2024-04-12T05:55:21.884 INFO:tasks.workunit.client.0.smithi008.stdout:7/132: link d1/l27 d1/l28 0 2024-04-12T05:55:21.884 INFO:tasks.workunit.client.0.smithi008.stdout:0/103: creat d2/d11/f25 x:0 0 0 2024-04-12T05:55:21.890 INFO:tasks.workunit.client.0.smithi008.stdout:7/133: creat d1/d5/f29 x:0 0 0 2024-04-12T05:55:21.897 INFO:tasks.workunit.client.0.smithi008.stdout:0/104: dwrite d2/f5 [0,4194304] 0 2024-04-12T05:55:21.899 INFO:tasks.workunit.client.0.smithi008.stdout:0/105: write d2/ff [1955319,3601] 0 2024-04-12T05:55:21.902 INFO:tasks.workunit.client.0.smithi008.stdout:7/134: dread d1/d5/da/fc [4194304,4194304] 0 2024-04-12T05:55:21.906 INFO:tasks.workunit.client.0.smithi008.stdout:7/135: creat d1/d5/da/f2a x:0 0 0 2024-04-12T05:55:21.910 INFO:tasks.workunit.client.0.smithi008.stdout:0/106: dwrite f1 [0,4194304] 0 2024-04-12T05:55:21.913 INFO:tasks.workunit.client.0.smithi008.stdout:0/107: rmdir d2/d11 39 2024-04-12T05:55:21.916 INFO:tasks.workunit.client.0.smithi008.stdout:0/108: mkdir d2/d11/d26 0 2024-04-12T05:55:21.919 INFO:tasks.workunit.client.0.smithi008.stdout:0/109: rmdir d2/d11/d26 0 2024-04-12T05:55:21.921 INFO:tasks.workunit.client.0.smithi008.stdout:7/136: truncate d1/fb 3854815 0 2024-04-12T05:55:21.923 INFO:tasks.workunit.client.0.smithi008.stdout:0/110: symlink d2/d11/l27 0 2024-04-12T05:55:21.924 INFO:tasks.workunit.client.0.smithi008.stdout:7/137: symlink d1/d5/da/d1d/l2b 0 2024-04-12T05:55:21.927 INFO:tasks.workunit.client.0.smithi008.stdout:0/111: rename d2/d11/f1a to d2/d24/f28 0 2024-04-12T05:55:21.929 INFO:tasks.workunit.client.0.smithi008.stdout:7/138: mkdir d1/d5/da/d1d/d2c 0 2024-04-12T05:55:21.933 INFO:tasks.workunit.client.0.smithi008.stdout:0/112: rename d2/l6 to d2/l29 0 2024-04-12T05:55:21.933 INFO:tasks.workunit.client.0.smithi008.stdout:0/113: dread - d2/f13 zero size 2024-04-12T05:55:21.935 INFO:tasks.workunit.client.0.smithi008.stdout:7/139: creat d1/f2d x:0 0 0 2024-04-12T05:55:21.937 INFO:tasks.workunit.client.0.smithi008.stdout:0/114: mkdir d2/d11/d2a 0 2024-04-12T05:55:21.938 INFO:tasks.workunit.client.0.smithi008.stdout:7/140: fdatasync d1/f12 0 2024-04-12T05:55:21.939 INFO:tasks.workunit.client.0.smithi008.stdout:0/115: mknod d2/d11/c2b 0 2024-04-12T05:55:21.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/116: creat d2/d1b/f2c x:0 0 0 2024-04-12T05:55:21.945 INFO:tasks.workunit.client.0.smithi008.stdout:0/117: creat d2/d11/d2a/f2d x:0 0 0 2024-04-12T05:55:21.946 INFO:tasks.workunit.client.0.smithi008.stdout:0/118: fsync d2/d1b/f21 0 2024-04-12T05:55:21.948 INFO:tasks.workunit.client.0.smithi008.stdout:0/119: creat d2/d11/f2e x:0 0 0 2024-04-12T05:55:21.949 INFO:tasks.workunit.client.0.smithi008.stdout:0/120: chown d2/l17 369643 1 2024-04-12T05:55:21.951 INFO:tasks.workunit.client.0.smithi008.stdout:0/121: write d2/d24/f28 [3683780,107453] 0 2024-04-12T05:55:21.952 INFO:tasks.workunit.client.0.smithi008.stdout:0/122: readlink d2/d11/l27 0 2024-04-12T05:55:21.958 INFO:tasks.workunit.client.0.smithi008.stdout:0/123: dwrite d2/d24/f28 [0,4194304] 0 2024-04-12T05:55:21.959 INFO:tasks.workunit.client.0.smithi008.stdout:0/124: dread - d2/d11/f25 zero size 2024-04-12T05:55:21.960 INFO:tasks.workunit.client.0.smithi008.stdout:0/125: write d2/d11/f23 [317330,124008] 0 2024-04-12T05:55:21.961 INFO:tasks.workunit.client.0.smithi008.stdout:0/126: write d2/ff [164469,28479] 0 2024-04-12T05:55:21.965 INFO:tasks.workunit.client.0.smithi008.stdout:0/127: rename d2/l17 to d2/d24/l2f 0 2024-04-12T05:55:21.965 INFO:tasks.workunit.client.0.smithi008.stdout:0/128: fsync d2/ff 0 2024-04-12T05:55:21.966 INFO:tasks.workunit.client.0.smithi008.stdout:0/129: dread - d2/d11/d2a/f2d zero size 2024-04-12T05:55:21.967 INFO:tasks.workunit.client.0.smithi008.stdout:0/130: write d2/d1b/f21 [2393097,126215] 0 2024-04-12T05:55:21.971 INFO:tasks.workunit.client.0.smithi008.stdout:0/131: fsync d2/fe 0 2024-04-12T05:55:21.976 INFO:tasks.workunit.client.0.smithi008.stdout:0/132: dread d2/d1b/f21 [0,4194304] 0 2024-04-12T05:55:21.980 INFO:tasks.workunit.client.0.smithi008.stdout:0/133: dread d2/fe [0,4194304] 0 2024-04-12T05:55:21.982 INFO:tasks.workunit.client.0.smithi008.stdout:0/134: mknod d2/d24/c30 0 2024-04-12T05:55:21.988 INFO:tasks.workunit.client.0.smithi008.stdout:0/135: dwrite d2/d1b/f2c [0,4194304] 0 2024-04-12T05:55:21.991 INFO:tasks.workunit.client.0.smithi008.stdout:0/136: rename f1 to d2/d24/f31 0 2024-04-12T05:55:21.994 INFO:tasks.workunit.client.0.smithi008.stdout:3/47: sync 2024-04-12T05:55:21.996 INFO:tasks.workunit.client.0.smithi008.stdout:0/137: creat d2/d24/f32 x:0 0 0 2024-04-12T05:55:21.997 INFO:tasks.workunit.client.0.smithi008.stdout:3/48: mkdir d9/d11 0 2024-04-12T05:55:22.001 INFO:tasks.workunit.client.0.smithi008.stdout:3/49: rename c3 to d9/db/c12 0 2024-04-12T05:55:22.005 INFO:tasks.workunit.client.0.smithi008.stdout:0/138: rmdir d2 39 2024-04-12T05:55:22.011 INFO:tasks.workunit.client.0.smithi008.stdout:3/50: dwrite d9/db/fc [0,4194304] 0 2024-04-12T05:55:22.017 INFO:tasks.workunit.client.0.smithi008.stdout:0/139: dwrite d2/f13 [0,4194304] 0 2024-04-12T05:55:22.023 INFO:tasks.workunit.client.0.smithi008.stdout:3/51: dwrite f1 [0,4194304] 0 2024-04-12T05:55:22.024 INFO:tasks.workunit.client.0.smithi008.stdout:0/140: unlink d2/cb 0 2024-04-12T05:55:22.029 INFO:tasks.workunit.client.0.smithi008.stdout:0/141: rename d2/d1b to d2/d11/d33 0 2024-04-12T05:55:22.031 INFO:tasks.workunit.client.0.smithi008.stdout:0/142: dread - d2/d11/f2e zero size 2024-04-12T05:55:22.033 INFO:tasks.workunit.client.0.smithi008.stdout:3/52: dwrite f7 [0,4194304] 0 2024-04-12T05:55:22.034 INFO:tasks.workunit.client.0.smithi008.stdout:3/53: read f6 [2579115,46260] 0 2024-04-12T05:55:22.039 INFO:tasks.workunit.client.0.smithi008.stdout:3/54: getdents d9 0 2024-04-12T05:55:22.041 INFO:tasks.workunit.client.0.smithi008.stdout:3/55: symlink d9/d10/l13 0 2024-04-12T05:55:22.049 INFO:tasks.workunit.client.0.smithi008.stdout:3/56: dwrite d9/db/fc [0,4194304] 0 2024-04-12T05:55:22.055 INFO:tasks.workunit.client.0.smithi008.stdout:3/57: dread d9/db/fc [0,4194304] 0 2024-04-12T05:55:22.056 INFO:tasks.workunit.client.0.smithi008.stdout:3/58: fdatasync f7 0 2024-04-12T05:55:22.058 INFO:tasks.workunit.client.0.smithi008.stdout:3/59: unlink c8 0 2024-04-12T05:55:22.058 INFO:tasks.workunit.client.0.smithi008.stdout:3/60: chown f7 143315 1 2024-04-12T05:55:22.063 INFO:tasks.workunit.client.0.smithi008.stdout:3/61: link d9/db/c12 d9/d11/c14 0 2024-04-12T05:55:22.066 INFO:tasks.workunit.client.0.smithi008.stdout:3/62: symlink d9/dd/l15 0 2024-04-12T05:55:22.067 INFO:tasks.workunit.client.0.smithi008.stdout:3/63: write f6 [490264,107153] 0 2024-04-12T05:55:22.071 INFO:tasks.workunit.client.0.smithi008.stdout:3/64: rename d9/la to d9/dd/l16 0 2024-04-12T05:55:22.081 INFO:tasks.workunit.client.0.smithi008.stdout:3/65: link d9/d11/c14 d9/dd/c17 0 2024-04-12T05:55:22.091 INFO:tasks.workunit.client.0.smithi008.stdout:3/66: dwrite d9/db/fc [0,4194304] 0 2024-04-12T05:55:22.097 INFO:tasks.workunit.client.0.smithi008.stdout:3/67: truncate d9/db/fc 3077266 0 2024-04-12T05:55:22.098 INFO:tasks.workunit.client.0.smithi008.stdout:3/68: truncate f6 4306962 0 2024-04-12T05:55:22.101 INFO:tasks.workunit.client.0.smithi008.stdout:3/69: rename c0 to d9/dd/c18 0 2024-04-12T05:55:22.108 INFO:tasks.workunit.client.0.smithi008.stdout:3/70: dwrite f1 [0,4194304] 0 2024-04-12T05:55:22.109 INFO:tasks.workunit.client.0.smithi008.stdout:3/71: truncate f6 4539105 0 2024-04-12T05:55:22.116 INFO:tasks.workunit.client.0.smithi008.stdout:3/72: dwrite f6 [0,4194304] 0 2024-04-12T05:55:22.120 INFO:tasks.workunit.client.0.smithi008.stdout:3/73: mkdir d9/d10/d19 0 2024-04-12T05:55:22.120 INFO:tasks.workunit.client.0.smithi008.stdout:3/74: stat c2 0 2024-04-12T05:55:22.121 INFO:tasks.workunit.client.0.smithi008.stdout:3/75: chown d9/db 1 1 2024-04-12T05:55:22.138 INFO:tasks.workunit.client.0.smithi008.stdout:3/76: truncate d9/db/fc 741527 0 2024-04-12T05:55:22.144 INFO:tasks.workunit.client.0.smithi008.stdout:3/77: dwrite f1 [4194304,4194304] 0 2024-04-12T05:55:22.209 INFO:tasks.workunit.client.0.smithi008.stdout:3/78: write d9/db/fc [872753,45600] 0 2024-04-12T05:55:22.210 INFO:tasks.workunit.client.0.smithi008.stdout:2/94: sync 2024-04-12T05:55:22.211 INFO:tasks.workunit.client.0.smithi008.stdout:6/19: sync 2024-04-12T05:55:22.212 INFO:tasks.workunit.client.0.smithi008.stdout:6/20: symlink l6 0 2024-04-12T05:55:22.213 INFO:tasks.workunit.client.0.smithi008.stdout:2/95: symlink d6/db/d10/d13/l19 0 2024-04-12T05:55:22.214 INFO:tasks.workunit.client.0.smithi008.stdout:2/96: chown d6/l8 93632598 1 2024-04-12T05:55:22.215 INFO:tasks.workunit.client.0.smithi008.stdout:6/21: rename l6 to l7 0 2024-04-12T05:55:22.216 INFO:tasks.workunit.client.0.smithi008.stdout:3/79: truncate f6 1636514 0 2024-04-12T05:55:22.217 INFO:tasks.workunit.client.0.smithi008.stdout:3/80: write f1 [245456,38822] 0 2024-04-12T05:55:22.223 INFO:tasks.workunit.client.0.smithi008.stdout:3/81: chown f6 183295089 1 2024-04-12T05:55:22.226 INFO:tasks.workunit.client.0.smithi008.stdout:6/22: dwrite f3 [0,4194304] 0 2024-04-12T05:55:22.228 INFO:tasks.workunit.client.0.smithi008.stdout:3/82: symlink d9/d11/l1a 0 2024-04-12T05:55:22.230 INFO:tasks.workunit.client.0.smithi008.stdout:6/23: readlink l7 0 2024-04-12T05:55:22.233 INFO:tasks.workunit.client.0.smithi008.stdout:2/97: dwrite f0 [0,4194304] 0 2024-04-12T05:55:22.233 INFO:tasks.workunit.client.0.smithi008.stdout:6/24: creat f8 x:0 0 0 2024-04-12T05:55:22.234 INFO:tasks.workunit.client.0.smithi008.stdout:2/98: write f0 [4265632,22879] 0 2024-04-12T05:55:22.235 INFO:tasks.workunit.client.0.smithi008.stdout:2/99: chown d6/l11 19 1 2024-04-12T05:55:22.236 INFO:tasks.workunit.client.0.smithi008.stdout:6/25: creat f9 x:0 0 0 2024-04-12T05:55:22.237 INFO:tasks.workunit.client.0.smithi008.stdout:3/83: link d9/db/cf d9/d10/d19/c1b 0 2024-04-12T05:55:22.238 INFO:tasks.workunit.client.0.smithi008.stdout:3/84: chown c4 1 1 2024-04-12T05:55:22.242 INFO:tasks.workunit.client.0.smithi008.stdout:2/100: dread f0 [0,4194304] 0 2024-04-12T05:55:22.242 INFO:tasks.workunit.client.0.smithi008.stdout:3/85: getdents d9/d11 0 2024-04-12T05:55:22.243 INFO:tasks.workunit.client.0.smithi008.stdout:3/86: fdatasync f1 0 2024-04-12T05:55:22.244 INFO:tasks.workunit.client.0.smithi008.stdout:2/101: write d6/db/fd [569916,77268] 0 2024-04-12T05:55:22.245 INFO:tasks.workunit.client.0.smithi008.stdout:2/102: chown d6/l8 36429 1 2024-04-12T05:55:22.247 INFO:tasks.workunit.client.0.smithi008.stdout:3/87: creat d9/f1c x:0 0 0 2024-04-12T05:55:22.256 INFO:tasks.workunit.client.0.smithi008.stdout:2/103: write d6/f16 [63312,11144] 0 2024-04-12T05:55:22.260 INFO:tasks.workunit.client.0.smithi008.stdout:3/88: dwrite d9/f1c [0,4194304] 0 2024-04-12T05:55:22.265 INFO:tasks.workunit.client.0.smithi008.stdout:3/89: write d9/db/fc [1962448,40829] 0 2024-04-12T05:55:22.266 INFO:tasks.workunit.client.0.smithi008.stdout:2/104: dwrite d6/f9 [0,4194304] 0 2024-04-12T05:55:22.272 INFO:tasks.workunit.client.0.smithi008.stdout:3/90: symlink d9/dd/l1d 0 2024-04-12T05:55:22.275 INFO:tasks.workunit.client.0.smithi008.stdout:3/91: truncate f7 3093237 0 2024-04-12T05:55:22.275 INFO:tasks.workunit.client.0.smithi008.stdout:2/105: read d6/db/fd [546315,25181] 0 2024-04-12T05:55:22.277 INFO:tasks.workunit.client.0.smithi008.stdout:3/92: mkdir d9/d1e 0 2024-04-12T05:55:22.279 INFO:tasks.workunit.client.0.smithi008.stdout:2/106: rename d6/l11 to d6/l1a 0 2024-04-12T05:55:22.286 INFO:tasks.workunit.client.0.smithi008.stdout:3/93: link d9/db/fc d9/db/f1f 0 2024-04-12T05:55:22.287 INFO:tasks.workunit.client.0.smithi008.stdout:2/107: rename d6/db/d10/l18 to d6/db/d10/l1b 0 2024-04-12T05:55:22.288 INFO:tasks.workunit.client.0.smithi008.stdout:3/94: chown d9/dd/c17 743400633 1 2024-04-12T05:55:22.290 INFO:tasks.workunit.client.0.smithi008.stdout:2/108: write d6/db/fd [1289167,5044] 0 2024-04-12T05:55:22.292 INFO:tasks.workunit.client.0.smithi008.stdout:2/109: write d6/ff [2295887,37083] 0 2024-04-12T05:55:22.293 INFO:tasks.workunit.client.0.smithi008.stdout:3/95: mknod d9/d10/d19/c20 0 2024-04-12T05:55:22.297 INFO:tasks.workunit.client.0.smithi008.stdout:2/110: write d6/f9 [2443801,31355] 0 2024-04-12T05:55:22.297 INFO:tasks.workunit.client.0.smithi008.stdout:3/96: read d9/db/fc [420202,89206] 0 2024-04-12T05:55:22.300 INFO:tasks.workunit.client.0.smithi008.stdout:2/111: symlink d6/db/d10/d13/l1c 0 2024-04-12T05:55:22.304 INFO:tasks.workunit.client.0.smithi008.stdout:3/97: fsync d9/db/f1f 0 2024-04-12T05:55:22.309 INFO:tasks.workunit.client.0.smithi008.stdout:3/98: rename d9/d10/d19/c20 to d9/d1e/c21 0 2024-04-12T05:55:22.309 INFO:tasks.workunit.client.0.smithi008.stdout:2/112: link l4 d6/db/d10/d13/l1d 0 2024-04-12T05:55:22.316 INFO:tasks.workunit.client.0.smithi008.stdout:2/113: dwrite d6/db/f14 [0,4194304] 0 2024-04-12T05:55:22.323 INFO:tasks.workunit.client.0.smithi008.stdout:2/114: link d6/db/f14 d6/f1e 0 2024-04-12T05:55:22.324 INFO:tasks.workunit.client.0.smithi008.stdout:2/115: dread d6/f16 [0,4194304] 0 2024-04-12T05:55:22.326 INFO:tasks.workunit.client.0.smithi008.stdout:2/116: chown d6/l1a 8518 1 2024-04-12T05:55:22.333 INFO:tasks.workunit.client.0.smithi008.stdout:2/117: dwrite d6/f15 [0,4194304] 0 2024-04-12T05:55:22.340 INFO:tasks.workunit.client.0.smithi008.stdout:2/118: rename d6/db/d10/l1b to d6/db/d10/l1f 0 2024-04-12T05:55:22.349 INFO:tasks.workunit.client.0.smithi008.stdout:2/119: getdents d6/db/d10 0 2024-04-12T05:55:22.350 INFO:tasks.workunit.client.0.smithi008.stdout:2/120: fsync d6/db/fd 0 2024-04-12T05:55:22.355 INFO:tasks.workunit.client.0.smithi008.stdout:2/121: dread d6/ff [0,4194304] 0 2024-04-12T05:55:22.357 INFO:tasks.workunit.client.0.smithi008.stdout:2/122: write d6/f16 [2292654,36612] 0 2024-04-12T05:55:22.358 INFO:tasks.workunit.client.0.smithi008.stdout:2/123: chown d6/l12 1578801 1 2024-04-12T05:55:22.361 INFO:tasks.workunit.client.0.smithi008.stdout:2/124: rename c2 to d6/db/d10/c20 0 2024-04-12T05:55:22.363 INFO:tasks.workunit.client.0.smithi008.stdout:1/66: sync 2024-04-12T05:55:22.363 INFO:tasks.workunit.client.0.smithi008.stdout:8/38: sync 2024-04-12T05:55:22.363 INFO:tasks.workunit.client.0.smithi008.stdout:5/148: sync 2024-04-12T05:55:22.363 INFO:tasks.workunit.client.0.smithi008.stdout:2/125: creat d6/f21 x:0 0 0 2024-04-12T05:55:22.364 INFO:tasks.workunit.client.0.smithi008.stdout:8/39: mkdir dd 0 2024-04-12T05:55:22.365 INFO:tasks.workunit.client.0.smithi008.stdout:1/67: creat d4/d7/ff x:0 0 0 2024-04-12T05:55:22.367 INFO:tasks.workunit.client.0.smithi008.stdout:2/126: write d6/f1e [458665,115053] 0 2024-04-12T05:55:22.369 INFO:tasks.workunit.client.0.smithi008.stdout:8/40: mkdir dd/de 0 2024-04-12T05:55:22.369 INFO:tasks.workunit.client.0.smithi008.stdout:2/127: write d6/f21 [1003273,15164] 0 2024-04-12T05:55:22.370 INFO:tasks.workunit.client.0.smithi008.stdout:8/41: write fb [23943,87467] 0 2024-04-12T05:55:22.374 INFO:tasks.workunit.client.0.smithi008.stdout:2/128: fsync f0 0 2024-04-12T05:55:22.375 INFO:tasks.workunit.client.0.smithi008.stdout:8/42: creat dd/ff x:0 0 0 2024-04-12T05:55:22.377 INFO:tasks.workunit.client.0.smithi008.stdout:8/43: write fb [1130279,88540] 0 2024-04-12T05:55:22.378 INFO:tasks.workunit.client.0.smithi008.stdout:2/129: write d6/db/f14 [2632008,51559] 0 2024-04-12T05:55:22.381 INFO:tasks.workunit.client.0.smithi008.stdout:5/149: dwrite d0/dd/d11/f19 [0,4194304] 0 2024-04-12T05:55:22.381 INFO:tasks.workunit.client.0.smithi008.stdout:8/44: mknod dd/de/c10 0 2024-04-12T05:55:22.382 INFO:tasks.workunit.client.0.smithi008.stdout:8/45: fsync fb 0 2024-04-12T05:55:22.391 INFO:tasks.workunit.client.0.smithi008.stdout:2/130: rename d6/f1e to d6/db/f22 0 2024-04-12T05:55:22.396 INFO:tasks.workunit.client.0.smithi008.stdout:5/150: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:22.397 INFO:tasks.workunit.client.0.smithi008.stdout:2/131: dread d6/db/f14 [0,4194304] 0 2024-04-12T05:55:22.398 INFO:tasks.workunit.client.0.smithi008.stdout:8/46: getdents dd 0 2024-04-12T05:55:22.400 INFO:tasks.workunit.client.0.smithi008.stdout:5/151: mknod d0/dc/d1d/c33 0 2024-04-12T05:55:22.402 INFO:tasks.workunit.client.0.smithi008.stdout:2/132: mkdir d6/db/d10/d23 0 2024-04-12T05:55:22.405 INFO:tasks.workunit.client.0.smithi008.stdout:5/152: unlink d0/d2/f17 0 2024-04-12T05:55:22.408 INFO:tasks.workunit.client.0.smithi008.stdout:5/153: write d0/dc/ff [1972864,46650] 0 2024-04-12T05:55:22.412 INFO:tasks.workunit.client.0.smithi008.stdout:8/47: dwrite fb [0,4194304] 0 2024-04-12T05:55:22.421 INFO:tasks.workunit.client.0.smithi008.stdout:2/133: dwrite d6/db/f14 [0,4194304] 0 2024-04-12T05:55:22.421 INFO:tasks.workunit.client.0.smithi008.stdout:8/48: dwrite fb [0,4194304] 0 2024-04-12T05:55:22.424 INFO:tasks.workunit.client.0.smithi008.stdout:8/49: mknod dd/de/c11 0 2024-04-12T05:55:22.425 INFO:tasks.workunit.client.0.smithi008.stdout:8/50: write dd/ff [822721,74936] 0 2024-04-12T05:55:22.430 INFO:tasks.workunit.client.0.smithi008.stdout:8/51: dread fb [0,4194304] 0 2024-04-12T05:55:22.433 INFO:tasks.workunit.client.0.smithi008.stdout:8/52: truncate f9 625554 0 2024-04-12T05:55:22.434 INFO:tasks.workunit.client.0.smithi008.stdout:8/53: chown c8 2 1 2024-04-12T05:55:22.437 INFO:tasks.workunit.client.0.smithi008.stdout:8/54: creat dd/de/f12 x:0 0 0 2024-04-12T05:55:22.438 INFO:tasks.workunit.client.0.smithi008.stdout:8/55: stat l6 0 2024-04-12T05:55:22.438 INFO:tasks.workunit.client.0.smithi008.stdout:8/56: write dd/de/f12 [250754,38019] 0 2024-04-12T05:55:22.439 INFO:tasks.workunit.client.0.smithi008.stdout:8/57: chown fb 65874946 1 2024-04-12T05:55:22.446 INFO:tasks.workunit.client.0.smithi008.stdout:8/58: fsync dd/ff 0 2024-04-12T05:55:22.459 INFO:tasks.workunit.client.0.smithi008.stdout:5/154: getdents d0/dc/d1d 0 2024-04-12T05:55:22.465 INFO:tasks.workunit.client.0.smithi008.stdout:8/59: truncate fb 4071300 0 2024-04-12T05:55:22.468 INFO:tasks.workunit.client.0.smithi008.stdout:8/60: symlink dd/l13 0 2024-04-12T05:55:22.471 INFO:tasks.workunit.client.0.smithi008.stdout:2/134: fdatasync d6/db/f14 0 2024-04-12T05:55:22.473 INFO:tasks.workunit.client.0.smithi008.stdout:2/135: readlink d6/db/d10/l1f 0 2024-04-12T05:55:22.474 INFO:tasks.workunit.client.0.smithi008.stdout:5/155: dwrite d0/dd/d11/f19 [0,4194304] 0 2024-04-12T05:55:22.478 INFO:tasks.workunit.client.0.smithi008.stdout:2/136: creat d6/f24 x:0 0 0 2024-04-12T05:55:22.486 INFO:tasks.workunit.client.0.smithi008.stdout:5/156: creat d0/dd/f34 x:0 0 0 2024-04-12T05:55:22.487 INFO:tasks.workunit.client.0.smithi008.stdout:8/61: dread fb [0,4194304] 0 2024-04-12T05:55:22.490 INFO:tasks.workunit.client.0.smithi008.stdout:2/137: dwrite d6/f16 [0,4194304] 0 2024-04-12T05:55:22.490 INFO:tasks.workunit.client.0.smithi008.stdout:8/62: symlink dd/l14 0 2024-04-12T05:55:22.492 INFO:tasks.workunit.client.0.smithi008.stdout:5/157: mkdir d0/dc/d13/d1a/d35 0 2024-04-12T05:55:22.497 INFO:tasks.workunit.client.0.smithi008.stdout:8/63: write fb [4955580,63235] 0 2024-04-12T05:55:22.497 INFO:tasks.workunit.client.0.smithi008.stdout:5/158: getdents d0/dc/d13/d1a/d35 0 2024-04-12T05:55:22.500 INFO:tasks.workunit.client.0.smithi008.stdout:5/159: readlink d0/dc/d26/l25 0 2024-04-12T05:55:22.500 INFO:tasks.workunit.client.0.smithi008.stdout:8/64: fdatasync dd/ff 0 2024-04-12T05:55:22.502 INFO:tasks.workunit.client.0.smithi008.stdout:5/160: dread - d0/dc/d26/f2f zero size 2024-04-12T05:55:22.503 INFO:tasks.workunit.client.0.smithi008.stdout:2/138: dwrite d6/f24 [0,4194304] 0 2024-04-12T05:55:22.508 INFO:tasks.workunit.client.0.smithi008.stdout:5/161: chown d0/dc/d13/d1a/f22 42 1 2024-04-12T05:55:22.508 INFO:tasks.workunit.client.0.smithi008.stdout:2/139: creat d6/db/d10/d13/f25 x:0 0 0 2024-04-12T05:55:22.508 INFO:tasks.workunit.client.0.smithi008.stdout:5/162: chown d0/dc/d26/l2e 55 1 2024-04-12T05:55:22.509 INFO:tasks.workunit.client.0.smithi008.stdout:5/163: dread - d0/dc/d26/f2f zero size 2024-04-12T05:55:22.509 INFO:tasks.workunit.client.0.smithi008.stdout:2/140: dread - d6/db/d10/d13/f25 zero size 2024-04-12T05:55:22.512 INFO:tasks.workunit.client.0.smithi008.stdout:5/164: symlink d0/dc/d26/l36 0 2024-04-12T05:55:22.514 INFO:tasks.workunit.client.0.smithi008.stdout:2/141: mknod d6/db/d10/d23/c26 0 2024-04-12T05:55:22.516 INFO:tasks.workunit.client.0.smithi008.stdout:5/165: unlink d0/dc/d26/l2e 0 2024-04-12T05:55:22.517 INFO:tasks.workunit.client.0.smithi008.stdout:5/166: fdatasync d0/d2/f4 0 2024-04-12T05:55:22.520 INFO:tasks.workunit.client.0.smithi008.stdout:2/142: creat d6/db/d10/d13/f27 x:0 0 0 2024-04-12T05:55:22.522 INFO:tasks.workunit.client.0.smithi008.stdout:5/167: dread d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:22.524 INFO:tasks.workunit.client.0.smithi008.stdout:2/143: rename d6/l12 to d6/db/l28 0 2024-04-12T05:55:22.525 INFO:tasks.workunit.client.0.smithi008.stdout:5/168: mkdir d0/d2/d37 0 2024-04-12T05:55:22.527 INFO:tasks.workunit.client.0.smithi008.stdout:2/144: mkdir d6/db/d29 0 2024-04-12T05:55:22.528 INFO:tasks.workunit.client.0.smithi008.stdout:5/169: rename d0/dc/d1d to d0/dd/d11/d38 0 2024-04-12T05:55:22.538 INFO:tasks.workunit.client.0.smithi008.stdout:2/145: truncate d6/f15 3505097 0 2024-04-12T05:55:22.539 INFO:tasks.workunit.client.0.smithi008.stdout:5/170: dwrite d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:22.546 INFO:tasks.workunit.client.0.smithi008.stdout:5/171: symlink d0/dc/d13/d1a/l39 0 2024-04-12T05:55:22.552 INFO:tasks.workunit.client.0.smithi008.stdout:2/146: truncate d6/f21 59379 0 2024-04-12T05:55:22.553 INFO:tasks.workunit.client.0.smithi008.stdout:5/172: mkdir d0/dc/d13/d1a/d3a 0 2024-04-12T05:55:22.555 INFO:tasks.workunit.client.0.smithi008.stdout:2/147: mknod d6/db/d29/c2a 0 2024-04-12T05:55:22.557 INFO:tasks.workunit.client.0.smithi008.stdout:5/173: unlink d0/dc/d13/f2d 0 2024-04-12T05:55:22.561 INFO:tasks.workunit.client.0.smithi008.stdout:5/174: rename d0/dc/d13/l16 to d0/dc/d13/l3b 0 2024-04-12T05:55:22.563 INFO:tasks.workunit.client.0.smithi008.stdout:5/175: write d0/d2/f2c [917247,16582] 0 2024-04-12T05:55:22.569 INFO:tasks.workunit.client.0.smithi008.stdout:5/176: dread d0/d2/f4 [4194304,4194304] 0 2024-04-12T05:55:22.570 INFO:tasks.workunit.client.0.smithi008.stdout:5/177: write d0/dd/d11/f19 [3713359,3054] 0 2024-04-12T05:55:22.581 INFO:tasks.workunit.client.0.smithi008.stdout:5/178: dwrite d0/dc/ff [0,4194304] 0 2024-04-12T05:55:22.582 INFO:tasks.workunit.client.0.smithi008.stdout:2/148: dwrite d6/f15 [0,4194304] 0 2024-04-12T05:55:22.582 INFO:tasks.workunit.client.0.smithi008.stdout:5/179: stat d0/d2/c7 0 2024-04-12T05:55:22.588 INFO:tasks.workunit.client.0.smithi008.stdout:2/149: creat d6/db/d10/f2b x:0 0 0 2024-04-12T05:55:22.591 INFO:tasks.workunit.client.0.smithi008.stdout:5/180: link d0/dd/f1e d0/dc/d13/f3c 0 2024-04-12T05:55:22.592 INFO:tasks.workunit.client.0.smithi008.stdout:2/150: creat d6/db/f2c x:0 0 0 2024-04-12T05:55:22.593 INFO:tasks.workunit.client.0.smithi008.stdout:2/151: write d6/f24 [4526534,56714] 0 2024-04-12T05:55:22.596 INFO:tasks.workunit.client.0.smithi008.stdout:2/152: chown d6/f9 19796475 1 2024-04-12T05:55:22.598 INFO:tasks.workunit.client.0.smithi008.stdout:2/153: creat d6/db/f2d x:0 0 0 2024-04-12T05:55:22.599 INFO:tasks.workunit.client.0.smithi008.stdout:5/181: dread d0/dd/d11/f19 [0,4194304] 0 2024-04-12T05:55:22.599 INFO:tasks.workunit.client.0.smithi008.stdout:5/182: stat d0 0 2024-04-12T05:55:22.603 INFO:tasks.workunit.client.0.smithi008.stdout:5/183: mkdir d0/dd/d3d 0 2024-04-12T05:55:22.608 INFO:tasks.workunit.client.0.smithi008.stdout:2/154: truncate d6/f24 1955485 0 2024-04-12T05:55:22.609 INFO:tasks.workunit.client.0.smithi008.stdout:5/184: write d0/dd/d11/d38/f2b [2919338,4844] 0 2024-04-12T05:55:22.610 INFO:tasks.workunit.client.0.smithi008.stdout:2/155: rmdir d6/db/d29 39 2024-04-12T05:55:22.611 INFO:tasks.workunit.client.0.smithi008.stdout:2/156: chown d6/db/d10/d23 2897317 1 2024-04-12T05:55:22.611 INFO:tasks.workunit.client.0.smithi008.stdout:5/185: fdatasync d0/dc/d13/d1a/f22 0 2024-04-12T05:55:22.613 INFO:tasks.workunit.client.0.smithi008.stdout:2/157: mkdir d6/db/d10/d23/d2e 0 2024-04-12T05:55:22.615 INFO:tasks.workunit.client.0.smithi008.stdout:5/186: link d0/dc/d13/f3c d0/d2/d37/f3e 0 2024-04-12T05:55:22.617 INFO:tasks.workunit.client.0.smithi008.stdout:2/158: rename d6/db/l28 to d6/db/d10/d13/l2f 0 2024-04-12T05:55:22.618 INFO:tasks.workunit.client.0.smithi008.stdout:2/159: chown d6/db/d10/d23/c26 1280 1 2024-04-12T05:55:22.619 INFO:tasks.workunit.client.0.smithi008.stdout:9/58: sync 2024-04-12T05:55:22.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/53: sync 2024-04-12T05:55:22.619 INFO:tasks.workunit.client.0.smithi008.stdout:0/143: sync 2024-04-12T05:55:22.619 INFO:tasks.workunit.client.0.smithi008.stdout:7/141: sync 2024-04-12T05:55:22.620 INFO:tasks.workunit.client.0.smithi008.stdout:2/160: mknod d6/db/d10/d13/c30 0 2024-04-12T05:55:22.621 INFO:tasks.workunit.client.0.smithi008.stdout:4/54: rename d1/db to d1/db/dd 22 2024-04-12T05:55:22.621 INFO:tasks.workunit.client.0.smithi008.stdout:0/144: write d2/d24/f32 [729176,52409] 0 2024-04-12T05:55:22.623 INFO:tasks.workunit.client.0.smithi008.stdout:9/59: symlink d3/df/d13/d14/l19 0 2024-04-12T05:55:22.624 INFO:tasks.workunit.client.0.smithi008.stdout:2/161: write d6/db/f2c [907734,15884] 0 2024-04-12T05:55:22.626 INFO:tasks.workunit.client.0.smithi008.stdout:2/162: fdatasync f0 0 2024-04-12T05:55:22.628 INFO:tasks.workunit.client.0.smithi008.stdout:0/145: symlink d2/d11/d33/l34 0 2024-04-12T05:55:22.630 INFO:tasks.workunit.client.0.smithi008.stdout:9/60: rmdir d3/df/d13/d14 39 2024-04-12T05:55:22.631 INFO:tasks.workunit.client.0.smithi008.stdout:2/163: write d6/db/f22 [1506685,13767] 0 2024-04-12T05:55:22.631 INFO:tasks.workunit.client.0.smithi008.stdout:9/61: stat d3/ce 0 2024-04-12T05:55:22.634 INFO:tasks.workunit.client.0.smithi008.stdout:0/146: readlink d2/d11/d33/l1c 0 2024-04-12T05:55:22.635 INFO:tasks.workunit.client.0.smithi008.stdout:7/142: dwrite d1/d5/da/d11/f25 [0,4194304] 0 2024-04-12T05:55:22.637 INFO:tasks.workunit.client.0.smithi008.stdout:4/55: truncate d1/f4 1290410 0 2024-04-12T05:55:22.638 INFO:tasks.workunit.client.0.smithi008.stdout:2/164: mkdir d6/db/d10/d31 0 2024-04-12T05:55:22.645 INFO:tasks.workunit.client.0.smithi008.stdout:4/56: creat d1/fe x:0 0 0 2024-04-12T05:55:22.646 INFO:tasks.workunit.client.0.smithi008.stdout:2/165: truncate d6/db/d10/d13/f25 448059 0 2024-04-12T05:55:22.650 INFO:tasks.workunit.client.0.smithi008.stdout:7/143: rename d1/d15/f19 to d1/d5/f2e 0 2024-04-12T05:55:22.650 INFO:tasks.workunit.client.0.smithi008.stdout:0/147: dwrite d2/d11/f2e [0,4194304] 0 2024-04-12T05:55:22.656 INFO:tasks.workunit.client.0.smithi008.stdout:0/148: mknod d2/d11/d33/c35 0 2024-04-12T05:55:22.657 INFO:tasks.workunit.client.0.smithi008.stdout:7/144: unlink d1/d5/da/c21 0 2024-04-12T05:55:22.663 INFO:tasks.workunit.client.0.smithi008.stdout:7/145: mkdir d1/d5/d2f 0 2024-04-12T05:55:22.665 INFO:tasks.workunit.client.0.smithi008.stdout:4/57: dwrite d1/db/fc [0,4194304] 0 2024-04-12T05:55:22.669 INFO:tasks.workunit.client.0.smithi008.stdout:4/58: chown d1/l9 46184437 1 2024-04-12T05:55:22.670 INFO:tasks.workunit.client.0.smithi008.stdout:2/166: dwrite d6/f24 [0,4194304] 0 2024-04-12T05:55:22.679 INFO:tasks.workunit.client.0.smithi008.stdout:2/167: read d6/db/f2c [443552,50824] 0 2024-04-12T05:55:22.682 INFO:tasks.workunit.client.0.smithi008.stdout:7/146: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:22.682 INFO:tasks.workunit.client.0.smithi008.stdout:0/149: getdents d2/d11/d33 0 2024-04-12T05:55:22.684 INFO:tasks.workunit.client.0.smithi008.stdout:0/150: chown d2/fe 1268260761 1 2024-04-12T05:55:22.685 INFO:tasks.workunit.client.0.smithi008.stdout:2/168: creat d6/f32 x:0 0 0 2024-04-12T05:55:22.685 INFO:tasks.workunit.client.0.smithi008.stdout:7/147: fdatasync d1/d5/da/d11/f25 0 2024-04-12T05:55:22.685 INFO:tasks.workunit.client.0.smithi008.stdout:4/59: dread d1/db/fc [0,4194304] 0 2024-04-12T05:55:22.686 INFO:tasks.workunit.client.0.smithi008.stdout:0/151: read - d2/d11/f25 zero size 2024-04-12T05:55:22.689 INFO:tasks.workunit.client.0.smithi008.stdout:0/152: mkdir d2/d11/d36 0 2024-04-12T05:55:22.691 INFO:tasks.workunit.client.0.smithi008.stdout:0/153: chown d2/c9 637 1 2024-04-12T05:55:22.695 INFO:tasks.workunit.client.0.smithi008.stdout:2/169: dread d6/f21 [0,4194304] 0 2024-04-12T05:55:22.695 INFO:tasks.workunit.client.0.smithi008.stdout:4/60: rmdir d1 39 2024-04-12T05:55:22.697 INFO:tasks.workunit.client.0.smithi008.stdout:0/154: creat d2/d11/d2a/f37 x:0 0 0 2024-04-12T05:55:22.699 INFO:tasks.workunit.client.0.smithi008.stdout:4/61: fdatasync d1/fe 0 2024-04-12T05:55:22.700 INFO:tasks.workunit.client.0.smithi008.stdout:2/170: write d6/db/f2c [902127,73877] 0 2024-04-12T05:55:22.703 INFO:tasks.workunit.client.0.smithi008.stdout:0/155: write d2/d24/f31 [4874315,101761] 0 2024-04-12T05:55:22.704 INFO:tasks.workunit.client.0.smithi008.stdout:7/148: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:22.707 INFO:tasks.workunit.client.0.smithi008.stdout:2/171: creat d6/f33 x:0 0 0 2024-04-12T05:55:22.708 INFO:tasks.workunit.client.0.smithi008.stdout:4/62: symlink d1/db/lf 0 2024-04-12T05:55:22.710 INFO:tasks.workunit.client.0.smithi008.stdout:7/149: truncate d1/f6 966865 0 2024-04-12T05:55:22.711 INFO:tasks.workunit.client.0.smithi008.stdout:2/172: unlink d6/f32 0 2024-04-12T05:55:22.712 INFO:tasks.workunit.client.0.smithi008.stdout:0/156: getdents d2/d11/d36 0 2024-04-12T05:55:22.714 INFO:tasks.workunit.client.0.smithi008.stdout:4/63: rename d1/l9 to d1/d8/l10 0 2024-04-12T05:55:22.716 INFO:tasks.workunit.client.0.smithi008.stdout:0/157: write d2/d11/d2a/f2d [177710,43057] 0 2024-04-12T05:55:22.717 INFO:tasks.workunit.client.0.smithi008.stdout:2/173: dread d6/db/d10/d13/f25 [0,4194304] 0 2024-04-12T05:55:22.718 INFO:tasks.workunit.client.0.smithi008.stdout:0/158: chown d2/d11 632997970 1 2024-04-12T05:55:22.718 INFO:tasks.workunit.client.0.smithi008.stdout:4/64: symlink d1/d8/l11 0 2024-04-12T05:55:22.719 INFO:tasks.workunit.client.0.smithi008.stdout:7/150: dread d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:22.723 INFO:tasks.workunit.client.0.smithi008.stdout:0/159: rename d2/c4 to d2/d11/c38 0 2024-04-12T05:55:22.736 INFO:tasks.workunit.client.0.smithi008.stdout:2/174: getdents d6/db 0 2024-04-12T05:55:22.736 INFO:tasks.workunit.client.0.smithi008.stdout:4/65: getdents d1/db 0 2024-04-12T05:55:22.741 INFO:tasks.workunit.client.0.smithi008.stdout:4/66: rename d1/f4 to d1/f12 0 2024-04-12T05:55:22.742 INFO:tasks.workunit.client.0.smithi008.stdout:4/67: write d1/fe [683359,56691] 0 2024-04-12T05:55:22.743 INFO:tasks.workunit.client.0.smithi008.stdout:2/175: rename d6/db/d10/d13/l1c to d6/db/d10/d31/l34 0 2024-04-12T05:55:22.743 INFO:tasks.workunit.client.0.smithi008.stdout:7/151: dwrite d1/d5/da/d11/f23 [4194304,4194304] 0 2024-04-12T05:55:22.743 INFO:tasks.workunit.client.0.smithi008.stdout:0/160: dwrite d2/d11/d2a/f37 [0,4194304] 0 2024-04-12T05:55:22.746 INFO:tasks.workunit.client.0.smithi008.stdout:2/176: write d6/db/f2d [757253,77335] 0 2024-04-12T05:55:22.747 INFO:tasks.workunit.client.0.smithi008.stdout:0/161: fsync d2/f13 0 2024-04-12T05:55:22.755 INFO:tasks.workunit.client.0.smithi008.stdout:4/68: rename d1/d8/fa to d1/f13 0 2024-04-12T05:55:22.755 INFO:tasks.workunit.client.0.smithi008.stdout:0/162: read d2/d11/d33/f1d [2626502,3765] 0 2024-04-12T05:55:22.755 INFO:tasks.workunit.client.0.smithi008.stdout:7/152: symlink d1/l30 0 2024-04-12T05:55:22.757 INFO:tasks.workunit.client.0.smithi008.stdout:0/163: stat d2/d11/l22 0 2024-04-12T05:55:22.757 INFO:tasks.workunit.client.0.smithi008.stdout:7/153: dread - d1/f2d zero size 2024-04-12T05:55:22.760 INFO:tasks.workunit.client.0.smithi008.stdout:2/177: dwrite d6/f16 [0,4194304] 0 2024-04-12T05:55:22.760 INFO:tasks.workunit.client.0.smithi008.stdout:0/164: mkdir d2/d24/d39 0 2024-04-12T05:55:22.761 INFO:tasks.workunit.client.0.smithi008.stdout:7/154: mknod d1/d5/da/d1f/c31 0 2024-04-12T05:55:22.763 INFO:tasks.workunit.client.0.smithi008.stdout:2/178: mkdir d6/db/d10/d23/d35 0 2024-04-12T05:55:22.763 INFO:tasks.workunit.client.0.smithi008.stdout:0/165: write d2/d11/d33/f2c [4085250,115328] 0 2024-04-12T05:55:22.766 INFO:tasks.workunit.client.0.smithi008.stdout:2/179: fdatasync d6/f21 0 2024-04-12T05:55:22.770 INFO:tasks.workunit.client.0.smithi008.stdout:2/180: mknod d6/db/d10/c36 0 2024-04-12T05:55:22.770 INFO:tasks.workunit.client.0.smithi008.stdout:2/181: stat l1 0 2024-04-12T05:55:22.771 INFO:tasks.workunit.client.0.smithi008.stdout:0/166: creat d2/d24/d39/f3a x:0 0 0 2024-04-12T05:55:22.776 INFO:tasks.workunit.client.0.smithi008.stdout:7/155: creat d1/d5/f32 x:0 0 0 2024-04-12T05:55:22.777 INFO:tasks.workunit.client.0.smithi008.stdout:7/156: read d1/f12 [2832745,38786] 0 2024-04-12T05:55:22.778 INFO:tasks.workunit.client.0.smithi008.stdout:2/182: rename d6/db/d10/l17 to d6/db/d10/l37 0 2024-04-12T05:55:22.780 INFO:tasks.workunit.client.0.smithi008.stdout:7/157: fdatasync d1/f12 0 2024-04-12T05:55:22.780 INFO:tasks.workunit.client.0.smithi008.stdout:0/167: dread d2/fc [0,4194304] 0 2024-04-12T05:55:22.782 INFO:tasks.workunit.client.0.smithi008.stdout:4/69: truncate d1/f13 1549461 0 2024-04-12T05:55:22.782 INFO:tasks.workunit.client.0.smithi008.stdout:0/168: chown d2/d11/l27 1828 1 2024-04-12T05:55:22.783 INFO:tasks.workunit.client.0.smithi008.stdout:0/169: chown d2/d11/d33/l1c 21 1 2024-04-12T05:55:22.785 INFO:tasks.workunit.client.0.smithi008.stdout:0/170: write d2/d11/f23 [1317778,85285] 0 2024-04-12T05:55:22.785 INFO:tasks.workunit.client.0.smithi008.stdout:2/183: rename d6/db/d10/d13 to d6/db/d38 0 2024-04-12T05:55:22.793 INFO:tasks.workunit.client.0.smithi008.stdout:0/171: dwrite d2/f13 [0,4194304] 0 2024-04-12T05:55:22.794 INFO:tasks.workunit.client.0.smithi008.stdout:7/158: creat d1/d5/f33 x:0 0 0 2024-04-12T05:55:22.798 INFO:tasks.workunit.client.0.smithi008.stdout:7/159: mknod d1/d5/da/d1d/d2c/c34 0 2024-04-12T05:55:22.800 INFO:tasks.workunit.client.0.smithi008.stdout:4/70: dread d1/f13 [0,4194304] 0 2024-04-12T05:55:22.801 INFO:tasks.workunit.client.0.smithi008.stdout:0/172: link d2/d11/d33/l34 d2/d11/d33/l3b 0 2024-04-12T05:55:22.804 INFO:tasks.workunit.client.0.smithi008.stdout:4/71: symlink d1/d8/l14 0 2024-04-12T05:55:22.807 INFO:tasks.workunit.client.0.smithi008.stdout:0/173: rmdir d2/d11/d2a 39 2024-04-12T05:55:22.808 INFO:tasks.workunit.client.0.smithi008.stdout:0/174: chown d2/fc 230014196 1 2024-04-12T05:55:22.810 INFO:tasks.workunit.client.0.smithi008.stdout:4/72: creat d1/d8/f15 x:0 0 0 2024-04-12T05:55:22.811 INFO:tasks.workunit.client.0.smithi008.stdout:4/73: chown d1/db/lf 108824105 1 2024-04-12T05:55:22.813 INFO:tasks.workunit.client.0.smithi008.stdout:0/175: symlink d2/l3c 0 2024-04-12T05:55:22.816 INFO:tasks.workunit.client.0.smithi008.stdout:2/184: symlink d6/db/d10/d23/d35/l39 0 2024-04-12T05:55:22.819 INFO:tasks.workunit.client.0.smithi008.stdout:7/160: write d1/d5/f2e [3241954,64783] 0 2024-04-12T05:55:22.826 INFO:tasks.workunit.client.0.smithi008.stdout:4/74: dwrite d1/f13 [0,4194304] 0 2024-04-12T05:55:22.828 INFO:tasks.workunit.client.0.smithi008.stdout:0/176: dread d2/d24/f31 [0,4194304] 0 2024-04-12T05:55:22.829 INFO:tasks.workunit.client.0.smithi008.stdout:4/75: truncate d1/d8/f15 980575 0 2024-04-12T05:55:22.830 INFO:tasks.workunit.client.0.smithi008.stdout:2/185: rename d6/db/f2d to d6/db/d10/d31/f3a 0 2024-04-12T05:55:22.830 INFO:tasks.workunit.client.0.smithi008.stdout:4/76: write d1/d8/f15 [12734,111979] 0 2024-04-12T05:55:22.832 INFO:tasks.workunit.client.0.smithi008.stdout:2/186: write d6/f15 [2483662,57355] 0 2024-04-12T05:55:22.832 INFO:tasks.workunit.client.0.smithi008.stdout:7/161: write d1/f12 [9607737,23927] 0 2024-04-12T05:55:22.834 INFO:tasks.workunit.client.0.smithi008.stdout:0/177: rmdir d2/d11/d2a 39 2024-04-12T05:55:22.836 INFO:tasks.workunit.client.0.smithi008.stdout:4/77: creat d1/db/f16 x:0 0 0 2024-04-12T05:55:22.837 INFO:tasks.workunit.client.0.smithi008.stdout:7/162: write d1/d5/f2e [3712109,75940] 0 2024-04-12T05:55:22.838 INFO:tasks.workunit.client.0.smithi008.stdout:6/26: sync 2024-04-12T05:55:22.838 INFO:tasks.workunit.client.0.smithi008.stdout:6/27: rmdir - no directory 2024-04-12T05:55:22.838 INFO:tasks.workunit.client.0.smithi008.stdout:3/99: sync 2024-04-12T05:55:22.838 INFO:tasks.workunit.client.0.smithi008.stdout:1/68: sync 2024-04-12T05:55:22.842 INFO:tasks.workunit.client.0.smithi008.stdout:6/28: read - f9 zero size 2024-04-12T05:55:22.842 INFO:tasks.workunit.client.0.smithi008.stdout:1/69: chown d4/d6/f8 29201104 1 2024-04-12T05:55:22.843 INFO:tasks.workunit.client.0.smithi008.stdout:6/29: fsync f4 0 2024-04-12T05:55:22.848 INFO:tasks.workunit.client.0.smithi008.stdout:3/100: rename d9/db to d9/d10/d19/d22 0 2024-04-12T05:55:22.851 INFO:tasks.workunit.client.0.smithi008.stdout:2/187: fsync d6/db/f2c 0 2024-04-12T05:55:22.855 INFO:tasks.workunit.client.0.smithi008.stdout:1/70: creat d4/d6/f10 x:0 0 0 2024-04-12T05:55:22.857 INFO:tasks.workunit.client.0.smithi008.stdout:0/178: link d2/c15 d2/c3d 0 2024-04-12T05:55:22.858 INFO:tasks.workunit.client.0.smithi008.stdout:7/163: rename d1/d5/da/fc to d1/d5/da/d11/f35 0 2024-04-12T05:55:22.858 INFO:tasks.workunit.client.0.smithi008.stdout:4/78: dwrite d1/db/f16 [0,4194304] 0 2024-04-12T05:55:22.860 INFO:tasks.workunit.client.0.smithi008.stdout:0/179: read d2/d11/f2e [2118679,58875] 0 2024-04-12T05:55:22.860 INFO:tasks.workunit.client.0.smithi008.stdout:2/188: stat d6/db/d10/d23/d2e 0 2024-04-12T05:55:22.860 INFO:tasks.workunit.client.0.smithi008.stdout:3/101: dread d9/d10/d19/d22/fc [0,4194304] 0 2024-04-12T05:55:22.861 INFO:tasks.workunit.client.0.smithi008.stdout:1/71: mkdir d4/d6/d11 0 2024-04-12T05:55:22.861 INFO:tasks.workunit.client.0.smithi008.stdout:4/79: chown d1/fe 0 1 2024-04-12T05:55:22.863 INFO:tasks.workunit.client.0.smithi008.stdout:6/30: getdents . 0 2024-04-12T05:55:22.866 INFO:tasks.workunit.client.0.smithi008.stdout:0/180: dread d2/fe [0,4194304] 0 2024-04-12T05:55:22.866 INFO:tasks.workunit.client.0.smithi008.stdout:7/164: symlink d1/d5/da/d11/l36 0 2024-04-12T05:55:22.868 INFO:tasks.workunit.client.0.smithi008.stdout:2/189: chown d6/db/d38/c30 71 1 2024-04-12T05:55:22.871 INFO:tasks.workunit.client.0.smithi008.stdout:6/31: mknod ca 0 2024-04-12T05:55:22.873 INFO:tasks.workunit.client.0.smithi008.stdout:6/32: read f3 [1181018,96014] 0 2024-04-12T05:55:22.878 INFO:tasks.workunit.client.0.smithi008.stdout:0/181: creat d2/d11/d2a/f3e x:0 0 0 2024-04-12T05:55:22.879 INFO:tasks.workunit.client.0.smithi008.stdout:2/190: dwrite d6/f33 [0,4194304] 0 2024-04-12T05:55:22.879 INFO:tasks.workunit.client.0.smithi008.stdout:7/165: chown d1/d15/c22 1 1 2024-04-12T05:55:22.881 INFO:tasks.workunit.client.0.smithi008.stdout:4/80: write d1/f12 [1621057,54603] 0 2024-04-12T05:55:22.883 INFO:tasks.workunit.client.0.smithi008.stdout:3/102: link d9/dd/c18 d9/d10/c23 0 2024-04-12T05:55:22.886 INFO:tasks.workunit.client.0.smithi008.stdout:2/191: creat d6/f3b x:0 0 0 2024-04-12T05:55:22.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/182: write d2/ff [1759956,75078] 0 2024-04-12T05:55:22.888 INFO:tasks.workunit.client.0.smithi008.stdout:6/33: rename f2 to fb 0 2024-04-12T05:55:22.892 INFO:tasks.workunit.client.0.smithi008.stdout:0/183: unlink d2/d11/l22 0 2024-04-12T05:55:22.893 INFO:tasks.workunit.client.0.smithi008.stdout:2/192: fdatasync d6/db/fd 0 2024-04-12T05:55:22.895 INFO:tasks.workunit.client.0.smithi008.stdout:3/103: dwrite d9/f1c [0,4194304] 0 2024-04-12T05:55:22.900 INFO:tasks.workunit.client.0.smithi008.stdout:2/193: mkdir d6/db/d10/d23/d35/d3c 0 2024-04-12T05:55:22.900 INFO:tasks.workunit.client.0.smithi008.stdout:3/104: dread d9/d10/d19/d22/fc [0,4194304] 0 2024-04-12T05:55:22.900 INFO:tasks.workunit.client.0.smithi008.stdout:6/34: write f5 [4733234,17490] 0 2024-04-12T05:55:22.903 INFO:tasks.workunit.client.0.smithi008.stdout:2/194: read d6/db/d10/d31/f3a [363034,49470] 0 2024-04-12T05:55:22.904 INFO:tasks.workunit.client.0.smithi008.stdout:3/105: mknod d9/d10/d19/d22/c24 0 2024-04-12T05:55:22.904 INFO:tasks.workunit.client.0.smithi008.stdout:2/195: mknod d6/db/d10/d23/c3d 0 2024-04-12T05:55:22.905 INFO:tasks.workunit.client.0.smithi008.stdout:0/184: link d2/d11/c38 d2/c3f 0 2024-04-12T05:55:22.909 INFO:tasks.workunit.client.0.smithi008.stdout:2/196: rmdir d6/db/d29 39 2024-04-12T05:55:22.914 INFO:tasks.workunit.client.0.smithi008.stdout:0/185: dwrite d2/f13 [0,4194304] 0 2024-04-12T05:55:22.915 INFO:tasks.workunit.client.0.smithi008.stdout:6/35: write fb [3193318,25245] 0 2024-04-12T05:55:22.915 INFO:tasks.workunit.client.0.smithi008.stdout:3/106: write d9/d10/d19/d22/f1f [1490376,84085] 0 2024-04-12T05:55:22.920 INFO:tasks.workunit.client.0.smithi008.stdout:0/186: truncate d2/fe 895449 0 2024-04-12T05:55:22.922 INFO:tasks.workunit.client.0.smithi008.stdout:2/197: dwrite d6/f3b [0,4194304] 0 2024-04-12T05:55:22.923 INFO:tasks.workunit.client.0.smithi008.stdout:2/198: chown d6 41301526 1 2024-04-12T05:55:22.930 INFO:tasks.workunit.client.0.smithi008.stdout:0/187: chown d2/d11/d33/l3b 117 1 2024-04-12T05:55:22.933 INFO:tasks.workunit.client.0.smithi008.stdout:3/107: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:22.934 INFO:tasks.workunit.client.0.smithi008.stdout:6/36: dwrite f8 [0,4194304] 0 2024-04-12T05:55:22.936 INFO:tasks.workunit.client.0.smithi008.stdout:3/108: chown d9/d10 1188 1 2024-04-12T05:55:22.939 INFO:tasks.workunit.client.0.smithi008.stdout:6/37: mknod cc 0 2024-04-12T05:55:22.940 INFO:tasks.workunit.client.0.smithi008.stdout:6/38: write f3 [2319066,98859] 0 2024-04-12T05:55:22.943 INFO:tasks.workunit.client.0.smithi008.stdout:2/199: rename d6/db/d38/c30 to d6/db/d10/d23/d35/d3c/c3e 0 2024-04-12T05:55:22.951 INFO:tasks.workunit.client.0.smithi008.stdout:2/200: symlink d6/db/d10/d23/d2e/l3f 0 2024-04-12T05:55:22.954 INFO:tasks.workunit.client.0.smithi008.stdout:6/39: dwrite f3 [4194304,4194304] 0 2024-04-12T05:55:22.954 INFO:tasks.workunit.client.0.smithi008.stdout:2/201: mknod d6/db/d10/d23/d35/d3c/c40 0 2024-04-12T05:55:22.961 INFO:tasks.workunit.client.0.smithi008.stdout:2/202: unlink d6/db/d10/l37 0 2024-04-12T05:55:22.965 INFO:tasks.workunit.client.0.smithi008.stdout:2/203: truncate d6/db/d38/f25 1158535 0 2024-04-12T05:55:22.966 INFO:tasks.workunit.client.0.smithi008.stdout:6/40: dwrite f5 [4194304,4194304] 0 2024-04-12T05:55:22.968 INFO:tasks.workunit.client.0.smithi008.stdout:6/41: truncate f4 70310 0 2024-04-12T05:55:22.968 INFO:tasks.workunit.client.0.smithi008.stdout:6/42: write fb [1239626,44073] 0 2024-04-12T05:55:22.976 INFO:tasks.workunit.client.0.smithi008.stdout:3/109: write d9/d10/d19/d22/fc [4592557,95481] 0 2024-04-12T05:55:22.978 INFO:tasks.workunit.client.0.smithi008.stdout:9/62: sync 2024-04-12T05:55:22.979 INFO:tasks.workunit.client.0.smithi008.stdout:8/65: sync 2024-04-12T05:55:22.979 INFO:tasks.workunit.client.0.smithi008.stdout:5/187: sync 2024-04-12T05:55:22.979 INFO:tasks.workunit.client.0.smithi008.stdout:9/63: chown d3/df 70984 1 2024-04-12T05:55:22.980 INFO:tasks.workunit.client.0.smithi008.stdout:0/188: dread d2/d11/f23 [0,4194304] 0 2024-04-12T05:55:22.981 INFO:tasks.workunit.client.0.smithi008.stdout:6/43: dwrite f8 [0,4194304] 0 2024-04-12T05:55:22.983 INFO:tasks.workunit.client.0.smithi008.stdout:0/189: fsync d2/d24/d39/f3a 0 2024-04-12T05:55:22.983 INFO:tasks.workunit.client.0.smithi008.stdout:9/64: mknod d3/df/d10/c1a 0 2024-04-12T05:55:22.987 INFO:tasks.workunit.client.0.smithi008.stdout:6/44: creat fd x:0 0 0 2024-04-12T05:55:22.988 INFO:tasks.workunit.client.0.smithi008.stdout:6/45: stat cc 0 2024-04-12T05:55:22.989 INFO:tasks.workunit.client.0.smithi008.stdout:9/65: symlink d3/df/d10/d12/l1b 0 2024-04-12T05:55:22.991 INFO:tasks.workunit.client.0.smithi008.stdout:9/66: chown d3 13 1 2024-04-12T05:55:22.993 INFO:tasks.workunit.client.0.smithi008.stdout:0/190: rename d2/d24/f31 to d2/d11/f40 0 2024-04-12T05:55:22.993 INFO:tasks.workunit.client.0.smithi008.stdout:3/110: dwrite f1 [4194304,4194304] 0 2024-04-12T05:55:22.997 INFO:tasks.workunit.client.0.smithi008.stdout:9/67: symlink d3/df/d10/l1c 0 2024-04-12T05:55:22.998 INFO:tasks.workunit.client.0.smithi008.stdout:3/111: write d9/d10/d19/d22/f1f [4260138,112062] 0 2024-04-12T05:55:23.000 INFO:tasks.workunit.client.0.smithi008.stdout:0/191: mknod d2/d11/d33/c41 0 2024-04-12T05:55:23.008 INFO:tasks.workunit.client.0.smithi008.stdout:3/112: creat d9/f25 x:0 0 0 2024-04-12T05:55:23.009 INFO:tasks.workunit.client.0.smithi008.stdout:2/204: rmdir d6 39 2024-04-12T05:55:23.012 INFO:tasks.workunit.client.0.smithi008.stdout:5/188: write d0/d2/f2c [2834693,67881] 0 2024-04-12T05:55:23.012 INFO:tasks.workunit.client.0.smithi008.stdout:0/192: dwrite d2/d24/d39/f3a [0,4194304] 0 2024-04-12T05:55:23.013 INFO:tasks.workunit.client.0.smithi008.stdout:6/46: getdents . 0 2024-04-12T05:55:23.021 INFO:tasks.workunit.client.0.smithi008.stdout:5/189: unlink d0/dc/ff 0 2024-04-12T05:55:23.022 INFO:tasks.workunit.client.0.smithi008.stdout:3/113: symlink d9/d10/d19/d22/l26 0 2024-04-12T05:55:23.023 INFO:tasks.workunit.client.0.smithi008.stdout:6/47: creat fe x:0 0 0 2024-04-12T05:55:23.023 INFO:tasks.workunit.client.0.smithi008.stdout:2/205: rename d6/db/d10/d31 to d6/db/d38/d41 0 2024-04-12T05:55:23.026 INFO:tasks.workunit.client.0.smithi008.stdout:0/193: mkdir d2/d11/d36/d42 0 2024-04-12T05:55:23.027 INFO:tasks.workunit.client.0.smithi008.stdout:2/206: chown d6/f24 242360 1 2024-04-12T05:55:23.027 INFO:tasks.workunit.client.0.smithi008.stdout:6/48: write f9 [282555,49986] 0 2024-04-12T05:55:23.027 INFO:tasks.workunit.client.0.smithi008.stdout:3/114: chown d9/dd/l1d 5794871 1 2024-04-12T05:55:23.032 INFO:tasks.workunit.client.0.smithi008.stdout:3/115: write d9/f25 [588745,105893] 0 2024-04-12T05:55:23.033 INFO:tasks.workunit.client.0.smithi008.stdout:2/207: fsync d6/f21 0 2024-04-12T05:55:23.034 INFO:tasks.workunit.client.0.smithi008.stdout:3/116: fdatasync d9/d10/d19/d22/fc 0 2024-04-12T05:55:23.039 INFO:tasks.workunit.client.0.smithi008.stdout:3/117: chown d9/d1e 2273925 1 2024-04-12T05:55:23.040 INFO:tasks.workunit.client.0.smithi008.stdout:0/194: rename d2/d11/d33/l34 to d2/d11/d2a/l43 0 2024-04-12T05:55:23.041 INFO:tasks.workunit.client.0.smithi008.stdout:2/208: read d6/f16 [4062358,127783] 0 2024-04-12T05:55:23.046 INFO:tasks.workunit.client.0.smithi008.stdout:9/68: dwrite d3/df/d13/d14/f18 [4194304,4194304] 0 2024-04-12T05:55:23.046 INFO:tasks.workunit.client.0.smithi008.stdout:0/195: mknod d2/c44 0 2024-04-12T05:55:23.049 INFO:tasks.workunit.client.0.smithi008.stdout:5/190: dwrite d0/dc/d13/f3c [0,4194304] 0 2024-04-12T05:55:23.049 INFO:tasks.workunit.client.0.smithi008.stdout:6/49: dwrite fb [4194304,4194304] 0 2024-04-12T05:55:23.060 INFO:tasks.workunit.client.0.smithi008.stdout:3/118: dread f7 [0,4194304] 0 2024-04-12T05:55:23.064 INFO:tasks.workunit.client.0.smithi008.stdout:5/191: creat d0/dc/d13/d1a/f3f x:0 0 0 2024-04-12T05:55:23.066 INFO:tasks.workunit.client.0.smithi008.stdout:2/209: link d6/db/d10/f2b d6/db/d10/d23/d2e/f42 0 2024-04-12T05:55:23.068 INFO:tasks.workunit.client.0.smithi008.stdout:9/69: getdents d3/df 0 2024-04-12T05:55:23.068 INFO:tasks.workunit.client.0.smithi008.stdout:0/196: getdents d2/d24/d39 0 2024-04-12T05:55:23.070 INFO:tasks.workunit.client.0.smithi008.stdout:3/119: write d9/d10/d19/d22/fc [2302115,111735] 0 2024-04-12T05:55:23.073 INFO:tasks.workunit.client.0.smithi008.stdout:6/50: dwrite fe [0,4194304] 0 2024-04-12T05:55:23.078 INFO:tasks.workunit.client.0.smithi008.stdout:9/70: creat d3/df/d10/d12/f1d x:0 0 0 2024-04-12T05:55:23.078 INFO:tasks.workunit.client.0.smithi008.stdout:2/210: fsync d6/db/d38/d41/f3a 0 2024-04-12T05:55:23.082 INFO:tasks.workunit.client.0.smithi008.stdout:5/192: dwrite d0/d2/d37/f3e [0,4194304] 0 2024-04-12T05:55:23.084 INFO:tasks.workunit.client.0.smithi008.stdout:9/71: write d3/df/d10/f15 [258700,130552] 0 2024-04-12T05:55:23.092 INFO:tasks.workunit.client.0.smithi008.stdout:5/193: write d0/d2/f31 [6783,22474] 0 2024-04-12T05:55:23.092 INFO:tasks.workunit.client.0.smithi008.stdout:6/51: dwrite f8 [0,4194304] 0 2024-04-12T05:55:23.098 INFO:tasks.workunit.client.0.smithi008.stdout:0/197: dwrite d2/d11/d33/f1d [0,4194304] 0 2024-04-12T05:55:23.099 INFO:tasks.workunit.client.0.smithi008.stdout:2/211: dread d6/f15 [0,4194304] 0 2024-04-12T05:55:23.099 INFO:tasks.workunit.client.0.smithi008.stdout:6/52: read f9 [162181,71007] 0 2024-04-12T05:55:23.100 INFO:tasks.workunit.client.0.smithi008.stdout:5/194: truncate d0/dd/d11/d38/f2b 3071565 0 2024-04-12T05:55:23.102 INFO:tasks.workunit.client.0.smithi008.stdout:6/53: write fe [2686929,52054] 0 2024-04-12T05:55:23.103 INFO:tasks.workunit.client.0.smithi008.stdout:6/54: fsync f5 0 2024-04-12T05:55:23.104 INFO:tasks.workunit.client.0.smithi008.stdout:5/195: readlink d0/dc/d13/d1a/l39 0 2024-04-12T05:55:23.106 INFO:tasks.workunit.client.0.smithi008.stdout:9/72: dwrite d3/df/d10/f15 [0,4194304] 0 2024-04-12T05:55:23.110 INFO:tasks.workunit.client.0.smithi008.stdout:6/55: mkdir df 0 2024-04-12T05:55:23.113 INFO:tasks.workunit.client.0.smithi008.stdout:0/198: symlink d2/d11/d36/d42/l45 0 2024-04-12T05:55:23.115 INFO:tasks.workunit.client.0.smithi008.stdout:9/73: rename d3/f9 to d3/df/d10/d12/f1e 0 2024-04-12T05:55:23.116 INFO:tasks.workunit.client.0.smithi008.stdout:9/74: readlink d3/df/d10/l1c 0 2024-04-12T05:55:23.117 INFO:tasks.workunit.client.0.smithi008.stdout:5/196: dwrite d0/dc/d26/f32 [0,4194304] 0 2024-04-12T05:55:23.118 INFO:tasks.workunit.client.0.smithi008.stdout:6/56: mkdir df/d10 0 2024-04-12T05:55:23.121 INFO:tasks.workunit.client.0.smithi008.stdout:2/212: getdents d6/db/d10 0 2024-04-12T05:55:23.123 INFO:tasks.workunit.client.0.smithi008.stdout:5/197: truncate d0/dc/d13/d1a/f3f 629792 0 2024-04-12T05:55:23.131 INFO:tasks.workunit.client.0.smithi008.stdout:2/213: creat d6/f43 x:0 0 0 2024-04-12T05:55:23.141 INFO:tasks.workunit.client.0.smithi008.stdout:6/57: dwrite f5 [0,4194304] 0 2024-04-12T05:55:23.141 INFO:tasks.workunit.client.0.smithi008.stdout:0/199: dwrite d2/fc [4194304,4194304] 0 2024-04-12T05:55:23.142 INFO:tasks.workunit.client.0.smithi008.stdout:9/75: dwrite d3/fd [0,4194304] 0 2024-04-12T05:55:23.143 INFO:tasks.workunit.client.0.smithi008.stdout:0/200: chown d2/d11/d2a 2621772 1 2024-04-12T05:55:23.143 INFO:tasks.workunit.client.0.smithi008.stdout:6/58: fdatasync fd 0 2024-04-12T05:55:23.145 INFO:tasks.workunit.client.0.smithi008.stdout:0/201: stat d2/d11/d33/c41 0 2024-04-12T05:55:23.147 INFO:tasks.workunit.client.0.smithi008.stdout:2/214: rename d6/db/d10/d23/d2e/l3f to d6/l44 0 2024-04-12T05:55:23.150 INFO:tasks.workunit.client.0.smithi008.stdout:0/202: write d2/d24/f28 [4928418,30534] 0 2024-04-12T05:55:23.151 INFO:tasks.workunit.client.0.smithi008.stdout:6/59: mkdir df/d11 0 2024-04-12T05:55:23.152 INFO:tasks.workunit.client.0.smithi008.stdout:5/198: dwrite d0/dd/d11/d38/f2b [4194304,4194304] 0 2024-04-12T05:55:23.153 INFO:tasks.workunit.client.0.smithi008.stdout:0/203: read - d2/d11/f25 zero size 2024-04-12T05:55:23.154 INFO:tasks.workunit.client.0.smithi008.stdout:9/76: link d3/df/d10/d12/f1e d3/f1f 0 2024-04-12T05:55:23.155 INFO:tasks.workunit.client.0.smithi008.stdout:0/204: fsync d2/d11/d2a/f2d 0 2024-04-12T05:55:23.156 INFO:tasks.workunit.client.0.smithi008.stdout:6/60: creat df/f12 x:0 0 0 2024-04-12T05:55:23.158 INFO:tasks.workunit.client.0.smithi008.stdout:9/77: creat d3/df/f20 x:0 0 0 2024-04-12T05:55:23.160 INFO:tasks.workunit.client.0.smithi008.stdout:6/61: write fb [1777608,41404] 0 2024-04-12T05:55:23.161 INFO:tasks.workunit.client.0.smithi008.stdout:0/205: write d2/d11/f23 [518221,85871] 0 2024-04-12T05:55:23.165 INFO:tasks.workunit.client.0.smithi008.stdout:9/78: mkdir d3/df/d13/d21 0 2024-04-12T05:55:23.167 INFO:tasks.workunit.client.0.smithi008.stdout:6/62: dread f3 [0,4194304] 0 2024-04-12T05:55:23.168 INFO:tasks.workunit.client.0.smithi008.stdout:0/206: link d2/d24/f32 d2/d11/d36/f46 0 2024-04-12T05:55:23.170 INFO:tasks.workunit.client.0.smithi008.stdout:0/207: write d2/d11/d33/f21 [4898416,71748] 0 2024-04-12T05:55:23.178 INFO:tasks.workunit.client.0.smithi008.stdout:0/208: rename d2/d11/d33 to d2/d11/d36/d47 0 2024-04-12T05:55:23.179 INFO:tasks.workunit.client.0.smithi008.stdout:6/63: getdents df 0 2024-04-12T05:55:23.180 INFO:tasks.workunit.client.0.smithi008.stdout:6/64: fsync f5 0 2024-04-12T05:55:23.181 INFO:tasks.workunit.client.0.smithi008.stdout:0/209: rmdir d2/d24/d39 39 2024-04-12T05:55:23.183 INFO:tasks.workunit.client.0.smithi008.stdout:6/65: rmdir df 39 2024-04-12T05:55:23.186 INFO:tasks.workunit.client.0.smithi008.stdout:6/66: dread f4 [0,4194304] 0 2024-04-12T05:55:23.187 INFO:tasks.workunit.client.0.smithi008.stdout:2/215: dwrite d6/db/d10/d23/d2e/f42 [0,4194304] 0 2024-04-12T05:55:23.191 INFO:tasks.workunit.client.0.smithi008.stdout:2/216: symlink d6/db/d10/d23/d2e/l45 0 2024-04-12T05:55:23.196 INFO:tasks.workunit.client.0.smithi008.stdout:6/67: link df/f12 df/d11/f13 0 2024-04-12T05:55:23.197 INFO:tasks.workunit.client.0.smithi008.stdout:6/68: stat df/d11 0 2024-04-12T05:55:23.197 INFO:tasks.workunit.client.0.smithi008.stdout:2/217: write d6/db/f2c [1665311,90773] 0 2024-04-12T05:55:23.204 INFO:tasks.workunit.client.0.smithi008.stdout:2/218: dread d6/db/d38/d41/f3a [0,4194304] 0 2024-04-12T05:55:23.206 INFO:tasks.workunit.client.0.smithi008.stdout:6/69: dwrite fd [0,4194304] 0 2024-04-12T05:55:23.208 INFO:tasks.workunit.client.0.smithi008.stdout:2/219: write d6/db/d10/f2b [2738376,118817] 0 2024-04-12T05:55:23.213 INFO:tasks.workunit.client.0.smithi008.stdout:2/220: rename d6/f24 to d6/db/d38/f46 0 2024-04-12T05:55:23.214 INFO:tasks.workunit.client.0.smithi008.stdout:6/70: dread f9 [0,4194304] 0 2024-04-12T05:55:23.215 INFO:tasks.workunit.client.0.smithi008.stdout:6/71: stat df/d11/f13 0 2024-04-12T05:55:23.217 INFO:tasks.workunit.client.0.smithi008.stdout:6/72: mkdir df/d14 0 2024-04-12T05:55:23.217 INFO:tasks.workunit.client.0.smithi008.stdout:6/73: chown fd 0 1 2024-04-12T05:55:23.218 INFO:tasks.workunit.client.0.smithi008.stdout:2/221: symlink d6/db/d10/d23/d35/l47 0 2024-04-12T05:55:23.221 INFO:tasks.workunit.client.0.smithi008.stdout:6/74: rename df/d11 to df/d10/d15 0 2024-04-12T05:55:23.223 INFO:tasks.workunit.client.0.smithi008.stdout:6/75: write fb [5273371,71017] 0 2024-04-12T05:55:23.227 INFO:tasks.workunit.client.0.smithi008.stdout:6/76: mknod df/d10/d15/c16 0 2024-04-12T05:55:23.227 INFO:tasks.workunit.client.0.smithi008.stdout:2/222: dwrite d6/db/d38/f27 [0,4194304] 0 2024-04-12T05:55:23.230 INFO:tasks.workunit.client.0.smithi008.stdout:2/223: unlink d6/f3b 0 2024-04-12T05:55:23.239 INFO:tasks.workunit.client.0.smithi008.stdout:2/224: truncate d6/db/fd 1120543 0 2024-04-12T05:55:23.240 INFO:tasks.workunit.client.0.smithi008.stdout:2/225: read d6/f21 [20394,14600] 0 2024-04-12T05:55:23.248 INFO:tasks.workunit.client.0.smithi008.stdout:2/226: link d6/db/f22 d6/db/d29/f48 0 2024-04-12T05:55:23.249 INFO:tasks.workunit.client.0.smithi008.stdout:6/77: dwrite df/d10/d15/f13 [0,4194304] 0 2024-04-12T05:55:23.252 INFO:tasks.workunit.client.0.smithi008.stdout:6/78: dread f9 [0,4194304] 0 2024-04-12T05:55:23.261 INFO:tasks.workunit.client.0.smithi008.stdout:6/79: dwrite f5 [4194304,4194304] 0 2024-04-12T05:55:23.261 INFO:tasks.workunit.client.0.smithi008.stdout:2/227: dwrite d6/f16 [0,4194304] 0 2024-04-12T05:55:23.263 INFO:tasks.workunit.client.0.smithi008.stdout:6/80: truncate f3 8723777 0 2024-04-12T05:55:23.267 INFO:tasks.workunit.client.0.smithi008.stdout:6/81: write f8 [1729444,108941] 0 2024-04-12T05:55:23.271 INFO:tasks.workunit.client.0.smithi008.stdout:2/228: link d6/db/d38/f46 d6/db/d10/d23/d35/f49 0 2024-04-12T05:55:23.273 INFO:tasks.workunit.client.0.smithi008.stdout:6/82: symlink df/d10/l17 0 2024-04-12T05:55:23.274 INFO:tasks.workunit.client.0.smithi008.stdout:2/229: mknod d6/db/d38/d41/c4a 0 2024-04-12T05:55:23.275 INFO:tasks.workunit.client.0.smithi008.stdout:6/83: write f9 [85720,51396] 0 2024-04-12T05:55:23.276 INFO:tasks.workunit.client.0.smithi008.stdout:6/84: stat fe 0 2024-04-12T05:55:23.276 INFO:tasks.workunit.client.0.smithi008.stdout:6/85: write f8 [3114425,97943] 0 2024-04-12T05:55:23.278 INFO:tasks.workunit.client.0.smithi008.stdout:6/86: write f9 [501327,85940] 0 2024-04-12T05:55:23.278 INFO:tasks.workunit.client.0.smithi008.stdout:2/230: unlink d6/db/d38/f46 0 2024-04-12T05:55:23.280 INFO:tasks.workunit.client.0.smithi008.stdout:6/87: mknod df/c18 0 2024-04-12T05:55:23.282 INFO:tasks.workunit.client.0.smithi008.stdout:2/231: creat d6/db/f4b x:0 0 0 2024-04-12T05:55:23.284 INFO:tasks.workunit.client.0.smithi008.stdout:6/88: dread df/d10/d15/f13 [0,4194304] 0 2024-04-12T05:55:23.289 INFO:tasks.workunit.client.0.smithi008.stdout:6/89: mknod df/c19 0 2024-04-12T05:55:23.291 INFO:tasks.workunit.client.0.smithi008.stdout:6/90: symlink df/d10/l1a 0 2024-04-12T05:55:23.291 INFO:tasks.workunit.client.0.smithi008.stdout:6/91: readlink l7 0 2024-04-12T05:55:23.297 INFO:tasks.workunit.client.0.smithi008.stdout:6/92: dwrite fb [4194304,4194304] 0 2024-04-12T05:55:23.299 INFO:tasks.workunit.client.0.smithi008.stdout:6/93: getdents df/d14 0 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: pgmap v7: 65 pgs: 65 active+clean; 235 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 29 MiB/s wr, 323 op/s 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: Standby manager daemon smithi008.ixmnlp started 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/crt"}]: dispatch 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/key"}]: dispatch 2024-04-12T05:55:23.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:23 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:55:23.411 INFO:tasks.workunit.client.0.smithi008.stdout:1/72: sync 2024-04-12T05:55:23.424 INFO:tasks.workunit.client.0.smithi008.stdout:1/73: truncate d4/d6/f8 1515283 0 2024-04-12T05:55:23.436 INFO:tasks.workunit.client.0.smithi008.stdout:1/74: dwrite d4/d7/fe [0,4194304] 0 2024-04-12T05:55:23.437 INFO:tasks.workunit.client.0.smithi008.stdout:1/75: chown d4/d7/fe 175 1 2024-04-12T05:55:23.439 INFO:tasks.workunit.client.0.smithi008.stdout:1/76: creat d4/d6/d11/f12 x:0 0 0 2024-04-12T05:55:23.445 INFO:tasks.workunit.client.0.smithi008.stdout:1/77: dwrite d4/d6/f10 [0,4194304] 0 2024-04-12T05:55:23.459 INFO:tasks.workunit.client.0.smithi008.stdout:1/78: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:23.461 INFO:tasks.workunit.client.0.smithi008.stdout:1/79: symlink d4/d7/l13 0 2024-04-12T05:55:23.463 INFO:tasks.workunit.client.0.smithi008.stdout:1/80: symlink d4/l14 0 2024-04-12T05:55:23.469 INFO:tasks.workunit.client.0.smithi008.stdout:1/81: dwrite d4/d7/ff [0,4194304] 0 2024-04-12T05:55:23.472 INFO:tasks.workunit.client.0.smithi008.stdout:1/82: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:23.474 INFO:tasks.workunit.client.0.smithi008.stdout:1/83: rmdir d4 39 2024-04-12T05:55:23.477 INFO:tasks.workunit.client.0.smithi008.stdout:1/84: mknod d4/d6/c15 0 2024-04-12T05:55:23.478 INFO:tasks.workunit.client.0.smithi008.stdout:1/85: truncate d4/d6/f10 4800378 0 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: pgmap v7: 65 pgs: 65 active+clean; 235 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 29 MiB/s wr, 323 op/s 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: Standby manager daemon smithi008.ixmnlp started 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/crt"}]: dispatch 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/key"}]: dispatch 2024-04-12T05:55:23.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:23 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2323762263' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:55:23.591 INFO:tasks.workunit.client.0.smithi008.stdout:4/81: sync 2024-04-12T05:55:23.591 INFO:tasks.workunit.client.0.smithi008.stdout:7/166: sync 2024-04-12T05:55:23.593 INFO:tasks.workunit.client.0.smithi008.stdout:7/167: rmdir d1/d5/da/d1f 39 2024-04-12T05:55:23.594 INFO:tasks.workunit.client.0.smithi008.stdout:4/82: dread d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:23.596 INFO:tasks.workunit.client.0.smithi008.stdout:7/168: rmdir d1/d5/da 39 2024-04-12T05:55:23.597 INFO:tasks.workunit.client.0.smithi008.stdout:7/169: fsync d1/d15/f26 0 2024-04-12T05:55:23.600 INFO:tasks.workunit.client.0.smithi008.stdout:4/83: getdents d1/d8 0 2024-04-12T05:55:23.603 INFO:tasks.workunit.client.0.smithi008.stdout:7/170: link d1/d15/c22 d1/c37 0 2024-04-12T05:55:23.604 INFO:tasks.workunit.client.0.smithi008.stdout:7/171: fdatasync d1/d5/f29 0 2024-04-12T05:55:23.605 INFO:tasks.workunit.client.0.smithi008.stdout:4/84: dread d1/db/f16 [0,4194304] 0 2024-04-12T05:55:23.610 INFO:tasks.workunit.client.0.smithi008.stdout:7/172: dwrite d1/f3 [0,4194304] 0 2024-04-12T05:55:23.612 INFO:tasks.workunit.client.0.smithi008.stdout:7/173: symlink d1/d5/l38 0 2024-04-12T05:55:23.614 INFO:tasks.workunit.client.0.smithi008.stdout:7/174: fsync d1/d15/f20 0 2024-04-12T05:55:23.617 INFO:tasks.workunit.client.0.smithi008.stdout:7/175: read d1/d5/da/d11/f35 [7072624,103459] 0 2024-04-12T05:55:23.617 INFO:tasks.workunit.client.0.smithi008.stdout:7/176: fsync d1/d5/f2e 0 2024-04-12T05:55:23.618 INFO:tasks.workunit.client.0.smithi008.stdout:7/177: dread - d1/d5/f33 zero size 2024-04-12T05:55:23.620 INFO:tasks.workunit.client.0.smithi008.stdout:7/178: mknod d1/d5/da/d1d/c39 0 2024-04-12T05:55:23.620 INFO:tasks.workunit.client.0.smithi008.stdout:7/179: fdatasync d1/d5/da/f2a 0 2024-04-12T05:55:23.621 INFO:tasks.workunit.client.0.smithi008.stdout:7/180: read d1/d5/da/d11/f25 [614535,14916] 0 2024-04-12T05:55:23.624 INFO:tasks.workunit.client.0.smithi008.stdout:7/181: creat d1/d5/da/d1f/f3a x:0 0 0 2024-04-12T05:55:23.633 INFO:tasks.workunit.client.0.smithi008.stdout:7/182: dwrite d1/f3 [0,4194304] 0 2024-04-12T05:55:23.635 INFO:tasks.workunit.client.0.smithi008.stdout:7/183: creat d1/d5/da/d1f/f3b x:0 0 0 2024-04-12T05:55:23.637 INFO:tasks.workunit.client.0.smithi008.stdout:7/184: write d1/d5/da/f2a [257451,112210] 0 2024-04-12T05:55:23.643 INFO:tasks.workunit.client.0.smithi008.stdout:7/185: rename d1/d5/da/d1d/d2c to d1/d5/da/d3c 0 2024-04-12T05:55:23.643 INFO:tasks.workunit.client.0.smithi008.stdout:7/186: chown d1/d5/da/d3c 1902503267 1 2024-04-12T05:55:23.644 INFO:tasks.workunit.client.0.smithi008.stdout:7/187: fdatasync d1/d5/da/d11/f23 0 2024-04-12T05:55:23.723 INFO:tasks.workunit.client.0.smithi008.stdout:3/120: sync 2024-04-12T05:55:23.723 INFO:tasks.workunit.client.0.smithi008.stdout:8/66: sync 2024-04-12T05:55:23.724 INFO:tasks.workunit.client.0.smithi008.stdout:8/67: write dd/ff [970515,12299] 0 2024-04-12T05:55:23.728 INFO:tasks.workunit.client.0.smithi008.stdout:8/68: link dd/de/c11 dd/de/c15 0 2024-04-12T05:55:23.731 INFO:tasks.workunit.client.0.smithi008.stdout:8/69: dread dd/ff [0,4194304] 0 2024-04-12T05:55:23.738 INFO:tasks.workunit.client.0.smithi008.stdout:8/70: dwrite dd/ff [0,4194304] 0 2024-04-12T05:55:23.751 INFO:tasks.workunit.client.0.smithi008.stdout:8/71: fsync dd/ff 0 2024-04-12T05:55:23.754 INFO:tasks.workunit.client.0.smithi008.stdout:8/72: getdents dd/de 0 2024-04-12T05:55:23.758 INFO:tasks.workunit.client.0.smithi008.stdout:8/73: dread f9 [0,4194304] 0 2024-04-12T05:55:23.764 INFO:tasks.workunit.client.0.smithi008.stdout:8/74: dwrite f7 [0,4194304] 0 2024-04-12T05:55:23.766 INFO:tasks.workunit.client.0.smithi008.stdout:8/75: mkdir dd/de/d16 0 2024-04-12T05:55:23.766 INFO:tasks.workunit.client.0.smithi008.stdout:8/76: readlink l1 0 2024-04-12T05:55:23.772 INFO:tasks.workunit.client.0.smithi008.stdout:8/77: dwrite fb [0,4194304] 0 2024-04-12T05:55:23.773 INFO:tasks.workunit.client.0.smithi008.stdout:8/78: stat dd/l14 0 2024-04-12T05:55:23.776 INFO:tasks.workunit.client.0.smithi008.stdout:8/79: creat dd/de/d16/f17 x:0 0 0 2024-04-12T05:55:23.776 INFO:tasks.workunit.client.0.smithi008.stdout:8/80: readlink lc 0 2024-04-12T05:55:23.777 INFO:tasks.workunit.client.0.smithi008.stdout:8/81: write dd/de/f12 [1119219,105421] 0 2024-04-12T05:55:23.782 INFO:tasks.workunit.client.0.smithi008.stdout:8/82: rename dd/de/c11 to dd/de/c18 0 2024-04-12T05:55:23.804 INFO:tasks.workunit.client.0.smithi008.stdout:2/232: sync 2024-04-12T05:55:23.805 INFO:tasks.workunit.client.0.smithi008.stdout:9/79: sync 2024-04-12T05:55:23.805 INFO:tasks.workunit.client.0.smithi008.stdout:5/199: sync 2024-04-12T05:55:23.805 INFO:tasks.workunit.client.0.smithi008.stdout:6/94: sync 2024-04-12T05:55:23.805 INFO:tasks.workunit.client.0.smithi008.stdout:1/86: sync 2024-04-12T05:55:23.807 INFO:tasks.workunit.client.0.smithi008.stdout:0/210: sync 2024-04-12T05:55:23.808 INFO:tasks.workunit.client.0.smithi008.stdout:9/80: fsync d3/f7 0 2024-04-12T05:55:23.808 INFO:tasks.workunit.client.0.smithi008.stdout:2/233: write f0 [3097366,113566] 0 2024-04-12T05:55:23.812 INFO:tasks.workunit.client.0.smithi008.stdout:8/83: getdents dd/de 0 2024-04-12T05:55:23.813 INFO:tasks.workunit.client.0.smithi008.stdout:9/81: dread - d3/df/f20 zero size 2024-04-12T05:55:23.814 INFO:tasks.workunit.client.0.smithi008.stdout:6/95: creat df/d10/d15/f1b x:0 0 0 2024-04-12T05:55:23.816 INFO:tasks.workunit.client.0.smithi008.stdout:6/96: readlink l7 0 2024-04-12T05:55:23.817 INFO:tasks.workunit.client.0.smithi008.stdout:9/82: readlink d3/df/d10/l1c 0 2024-04-12T05:55:23.821 INFO:tasks.workunit.client.0.smithi008.stdout:5/200: readlink d0/dc/d13/l3b 0 2024-04-12T05:55:23.823 INFO:tasks.workunit.client.0.smithi008.stdout:0/211: dwrite d2/d11/d36/d47/f21 [4194304,4194304] 0 2024-04-12T05:55:23.824 INFO:tasks.workunit.client.0.smithi008.stdout:6/97: creat df/d10/d15/f1c x:0 0 0 2024-04-12T05:55:23.825 INFO:tasks.workunit.client.0.smithi008.stdout:9/83: mkdir d3/d22 0 2024-04-12T05:55:23.830 INFO:tasks.workunit.client.0.smithi008.stdout:2/234: dwrite d6/f43 [0,4194304] 0 2024-04-12T05:55:23.830 INFO:tasks.workunit.client.0.smithi008.stdout:6/98: read - df/d10/d15/f1c zero size 2024-04-12T05:55:23.835 INFO:tasks.workunit.client.0.smithi008.stdout:8/84: dwrite fb [0,4194304] 0 2024-04-12T05:55:23.838 INFO:tasks.workunit.client.0.smithi008.stdout:0/212: creat d2/d11/d36/f48 x:0 0 0 2024-04-12T05:55:23.838 INFO:tasks.workunit.client.0.smithi008.stdout:8/85: write dd/ff [719531,124649] 0 2024-04-12T05:55:23.839 INFO:tasks.workunit.client.0.smithi008.stdout:2/235: mkdir d6/db/d10/d4c 0 2024-04-12T05:55:23.840 INFO:tasks.workunit.client.0.smithi008.stdout:2/236: readlink d6/db/d38/l19 0 2024-04-12T05:55:23.846 INFO:tasks.workunit.client.0.smithi008.stdout:8/86: mknod dd/de/d16/c19 0 2024-04-12T05:55:23.849 INFO:tasks.workunit.client.0.smithi008.stdout:6/99: rmdir df/d14 0 2024-04-12T05:55:23.849 INFO:tasks.workunit.client.0.smithi008.stdout:6/100: fdatasync f5 0 2024-04-12T05:55:23.853 INFO:tasks.workunit.client.0.smithi008.stdout:8/87: creat dd/f1a x:0 0 0 2024-04-12T05:55:23.859 INFO:tasks.workunit.client.0.smithi008.stdout:2/237: link d6/db/d29/f48 d6/db/d10/d23/f4d 0 2024-04-12T05:55:23.859 INFO:tasks.workunit.client.0.smithi008.stdout:2/238: chown l3 228887174 1 2024-04-12T05:55:23.861 INFO:tasks.workunit.client.0.smithi008.stdout:9/84: getdents d3/df/d10/d12 0 2024-04-12T05:55:23.864 INFO:tasks.workunit.client.0.smithi008.stdout:8/88: rename dd/de/c18 to dd/c1b 0 2024-04-12T05:55:23.865 INFO:tasks.workunit.client.0.smithi008.stdout:6/101: link df/d10/d15/c16 df/d10/d15/c1d 0 2024-04-12T05:55:23.867 INFO:tasks.workunit.client.0.smithi008.stdout:9/85: creat d3/df/d10/d12/f23 x:0 0 0 2024-04-12T05:55:23.869 INFO:tasks.workunit.client.0.smithi008.stdout:6/102: symlink df/d10/d15/l1e 0 2024-04-12T05:55:23.871 INFO:tasks.workunit.client.0.smithi008.stdout:9/86: symlink d3/d22/l24 0 2024-04-12T05:55:23.873 INFO:tasks.workunit.client.0.smithi008.stdout:9/87: fdatasync d3/df/d10/f17 0 2024-04-12T05:55:23.876 INFO:tasks.workunit.client.0.smithi008.stdout:0/213: rmdir d2/d11/d36 39 2024-04-12T05:55:23.877 INFO:tasks.workunit.client.0.smithi008.stdout:0/214: chown d2/d11 12814186 1 2024-04-12T05:55:23.880 INFO:tasks.workunit.client.0.smithi008.stdout:6/103: dwrite df/d10/d15/f13 [0,4194304] 0 2024-04-12T05:55:23.880 INFO:tasks.workunit.client.0.smithi008.stdout:2/239: rename d6/l7 to d6/db/d10/d23/d2e/l4e 0 2024-04-12T05:55:23.883 INFO:tasks.workunit.client.0.smithi008.stdout:5/201: unlink d0/dc/d13/f3c 0 2024-04-12T05:55:23.885 INFO:tasks.workunit.client.0.smithi008.stdout:6/104: creat df/f1f x:0 0 0 2024-04-12T05:55:23.886 INFO:tasks.workunit.client.0.smithi008.stdout:2/240: mkdir d6/db/d10/d23/d2e/d4f 0 2024-04-12T05:55:23.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/215: link d2/fc d2/d11/d2a/f49 0 2024-04-12T05:55:23.889 INFO:tasks.workunit.client.0.smithi008.stdout:0/216: write d2/d11/f25 [7584,44686] 0 2024-04-12T05:55:23.890 INFO:tasks.workunit.client.0.smithi008.stdout:5/202: mkdir d0/d40 0 2024-04-12T05:55:23.890 INFO:tasks.workunit.client.0.smithi008.stdout:0/217: chown d2/d11/c20 1 1 2024-04-12T05:55:23.891 INFO:tasks.workunit.client.0.smithi008.stdout:0/218: fdatasync d2/f5 0 2024-04-12T05:55:23.892 INFO:tasks.workunit.client.0.smithi008.stdout:6/105: mkdir df/d10/d20 0 2024-04-12T05:55:23.896 INFO:tasks.workunit.client.0.smithi008.stdout:5/203: creat d0/dd/d3d/f41 x:0 0 0 2024-04-12T05:55:23.897 INFO:tasks.workunit.client.0.smithi008.stdout:6/106: mkdir df/d21 0 2024-04-12T05:55:23.898 INFO:tasks.workunit.client.0.smithi008.stdout:6/107: truncate df/f1f 933559 0 2024-04-12T05:55:23.899 INFO:tasks.workunit.client.0.smithi008.stdout:0/219: creat d2/d11/d2a/f4a x:0 0 0 2024-04-12T05:55:23.900 INFO:tasks.workunit.client.0.smithi008.stdout:6/108: write df/d10/d15/f1b [972080,51740] 0 2024-04-12T05:55:23.900 INFO:tasks.workunit.client.0.smithi008.stdout:0/220: write d2/d24/f28 [2392560,45700] 0 2024-04-12T05:55:23.900 INFO:tasks.workunit.client.0.smithi008.stdout:5/204: unlink d0/dd/d3d/f41 0 2024-04-12T05:55:23.903 INFO:tasks.workunit.client.0.smithi008.stdout:9/88: write d3/f1f [2659693,120221] 0 2024-04-12T05:55:23.905 INFO:tasks.workunit.client.0.smithi008.stdout:0/221: write d2/d11/d36/d47/f1d [1947884,39139] 0 2024-04-12T05:55:23.906 INFO:tasks.workunit.client.0.smithi008.stdout:9/89: chown d3/df/d13/d14/l19 21854 1 2024-04-12T05:55:23.907 INFO:tasks.workunit.client.0.smithi008.stdout:9/90: stat d3/fb 0 2024-04-12T05:55:23.908 INFO:tasks.workunit.client.0.smithi008.stdout:5/205: dread d0/dd/d11/d38/f2b [4194304,4194304] 0 2024-04-12T05:55:23.908 INFO:tasks.workunit.client.0.smithi008.stdout:6/109: creat df/d21/f22 x:0 0 0 2024-04-12T05:55:23.909 INFO:tasks.workunit.client.0.smithi008.stdout:9/91: fdatasync d3/df/d10/d12/f1d 0 2024-04-12T05:55:23.910 INFO:tasks.workunit.client.0.smithi008.stdout:6/110: chown fb 2 1 2024-04-12T05:55:23.911 INFO:tasks.workunit.client.0.smithi008.stdout:9/92: write d3/f6 [4868335,108045] 0 2024-04-12T05:55:23.912 INFO:tasks.workunit.client.0.smithi008.stdout:6/111: creat df/f23 x:0 0 0 2024-04-12T05:55:23.914 INFO:tasks.workunit.client.0.smithi008.stdout:0/222: dread d2/d11/f2e [0,4194304] 0 2024-04-12T05:55:23.914 INFO:tasks.workunit.client.0.smithi008.stdout:9/93: creat d3/f25 x:0 0 0 2024-04-12T05:55:23.916 INFO:tasks.workunit.client.0.smithi008.stdout:9/94: fsync d3/fa 0 2024-04-12T05:55:23.916 INFO:tasks.workunit.client.0.smithi008.stdout:0/223: chown d2/f5 803188216 1 2024-04-12T05:55:23.917 INFO:tasks.workunit.client.0.smithi008.stdout:9/95: stat d3/df/d10/d12/f1e 0 2024-04-12T05:55:23.917 INFO:tasks.workunit.client.0.smithi008.stdout:9/96: write d3/fa [883734,67859] 0 2024-04-12T05:55:23.918 INFO:tasks.workunit.client.0.smithi008.stdout:0/224: mkdir d2/d4b 0 2024-04-12T05:55:23.923 INFO:tasks.workunit.client.0.smithi008.stdout:9/97: rename d3/df/d10/d12/l1b to d3/df/d13/d21/l26 0 2024-04-12T05:55:23.924 INFO:tasks.workunit.client.0.smithi008.stdout:6/112: write f4 [138432,128015] 0 2024-04-12T05:55:23.926 INFO:tasks.workunit.client.0.smithi008.stdout:6/113: write fb [649135,72961] 0 2024-04-12T05:55:23.927 INFO:tasks.workunit.client.0.smithi008.stdout:0/225: dwrite d2/f13 [4194304,4194304] 0 2024-04-12T05:55:23.930 INFO:tasks.workunit.client.0.smithi008.stdout:9/98: rename d3/df/d10/f15 to d3/df/d10/d12/f27 0 2024-04-12T05:55:23.931 INFO:tasks.workunit.client.0.smithi008.stdout:9/99: fsync d3/fd 0 2024-04-12T05:55:23.931 INFO:tasks.workunit.client.0.smithi008.stdout:6/114: creat df/d10/d15/f24 x:0 0 0 2024-04-12T05:55:23.933 INFO:tasks.workunit.client.0.smithi008.stdout:6/115: chown df/f12 3852 1 2024-04-12T05:55:23.939 INFO:tasks.workunit.client.0.smithi008.stdout:6/116: truncate f5 4042029 0 2024-04-12T05:55:23.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/226: dread d2/fa [0,4194304] 0 2024-04-12T05:55:23.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/227: stat d2/l10 0 2024-04-12T05:55:23.944 INFO:tasks.workunit.client.0.smithi008.stdout:0/228: mknod d2/d11/d2a/c4c 0 2024-04-12T05:55:23.946 INFO:tasks.workunit.client.0.smithi008.stdout:0/229: write d2/d11/d36/d47/f21 [8095341,112560] 0 2024-04-12T05:55:23.947 INFO:tasks.workunit.client.0.smithi008.stdout:9/100: dread d3/f6 [0,4194304] 0 2024-04-12T05:55:23.950 INFO:tasks.workunit.client.0.smithi008.stdout:9/101: fsync d3/f6 0 2024-04-12T05:55:23.951 INFO:tasks.workunit.client.0.smithi008.stdout:0/230: symlink d2/l4d 0 2024-04-12T05:55:23.953 INFO:tasks.workunit.client.0.smithi008.stdout:9/102: creat d3/df/f28 x:0 0 0 2024-04-12T05:55:23.953 INFO:tasks.workunit.client.0.smithi008.stdout:6/117: write f5 [1231058,27166] 0 2024-04-12T05:55:23.957 INFO:tasks.workunit.client.0.smithi008.stdout:6/118: symlink df/d10/d15/l25 0 2024-04-12T05:55:23.960 INFO:tasks.workunit.client.0.smithi008.stdout:6/119: write df/d21/f22 [930784,9075] 0 2024-04-12T05:55:23.960 INFO:tasks.workunit.client.0.smithi008.stdout:0/231: dwrite d2/f5 [0,4194304] 0 2024-04-12T05:55:23.964 INFO:tasks.workunit.client.0.smithi008.stdout:0/232: read d2/d11/f18 [239242,70392] 0 2024-04-12T05:55:23.965 INFO:tasks.workunit.client.0.smithi008.stdout:9/103: dwrite d3/df/d10/d12/f1d [0,4194304] 0 2024-04-12T05:55:23.967 INFO:tasks.workunit.client.0.smithi008.stdout:9/104: rmdir d3/df/d13/d14 39 2024-04-12T05:55:23.967 INFO:tasks.workunit.client.0.smithi008.stdout:3/121: sync 2024-04-12T05:55:23.967 INFO:tasks.workunit.client.0.smithi008.stdout:4/85: sync 2024-04-12T05:55:23.967 INFO:tasks.workunit.client.0.smithi008.stdout:7/188: sync 2024-04-12T05:55:23.968 INFO:tasks.workunit.client.0.smithi008.stdout:6/120: chown df/d10/d15/c16 6016 1 2024-04-12T05:55:23.968 INFO:tasks.workunit.client.0.smithi008.stdout:9/105: stat d3/l8 0 2024-04-12T05:55:23.971 INFO:tasks.workunit.client.0.smithi008.stdout:3/122: symlink d9/d11/l27 0 2024-04-12T05:55:24.009 INFO:tasks.workunit.client.0.smithi008.stdout:9/106: mknod d3/df/d13/c29 0 2024-04-12T05:55:24.015 INFO:tasks.workunit.client.0.smithi008.stdout:6/121: dread fd [0,4194304] 0 2024-04-12T05:55:24.016 INFO:tasks.workunit.client.0.smithi008.stdout:7/189: dwrite d1/d5/da/d1f/f3a [0,4194304] 0 2024-04-12T05:55:24.017 INFO:tasks.workunit.client.0.smithi008.stdout:4/86: dwrite d1/db/f16 [0,4194304] 0 2024-04-12T05:55:24.021 INFO:tasks.workunit.client.0.smithi008.stdout:9/107: link d3/df/d13/c16 d3/df/d13/d21/c2a 0 2024-04-12T05:55:24.022 INFO:tasks.workunit.client.0.smithi008.stdout:6/122: readlink df/d10/d15/l1e 0 2024-04-12T05:55:24.022 INFO:tasks.workunit.client.0.smithi008.stdout:7/190: rmdir d1/d5/da 39 2024-04-12T05:55:24.026 INFO:tasks.workunit.client.0.smithi008.stdout:3/123: dwrite f6 [0,4194304] 0 2024-04-12T05:55:24.026 INFO:tasks.workunit.client.0.smithi008.stdout:6/123: mknod df/c26 0 2024-04-12T05:55:24.027 INFO:tasks.workunit.client.0.smithi008.stdout:7/191: chown d1/d5/f29 222911 1 2024-04-12T05:55:24.027 INFO:tasks.workunit.client.0.smithi008.stdout:3/124: chown d9/d1e 127 1 2024-04-12T05:55:24.030 INFO:tasks.workunit.client.0.smithi008.stdout:6/124: creat df/d21/f27 x:0 0 0 2024-04-12T05:55:24.033 INFO:tasks.workunit.client.0.smithi008.stdout:4/87: dwrite d1/f12 [0,4194304] 0 2024-04-12T05:55:24.038 INFO:tasks.workunit.client.0.smithi008.stdout:3/125: creat d9/d10/d19/d22/f28 x:0 0 0 2024-04-12T05:55:24.040 INFO:tasks.workunit.client.0.smithi008.stdout:7/192: rename d1/d5/da/d1f to d1/d5/da/d3c/d3d 0 2024-04-12T05:55:24.042 INFO:tasks.workunit.client.0.smithi008.stdout:7/193: dread - d1/d5/f29 zero size 2024-04-12T05:55:24.042 INFO:tasks.workunit.client.0.smithi008.stdout:4/88: mknod d1/db/c17 0 2024-04-12T05:55:24.044 INFO:tasks.workunit.client.0.smithi008.stdout:6/125: mknod df/d10/d20/c28 0 2024-04-12T05:55:24.044 INFO:tasks.workunit.client.0.smithi008.stdout:9/108: truncate d3/df/d10/d12/f1e 511322 0 2024-04-12T05:55:24.045 INFO:tasks.workunit.client.0.smithi008.stdout:6/126: readlink l7 0 2024-04-12T05:55:24.046 INFO:tasks.workunit.client.0.smithi008.stdout:7/194: readlink d1/d5/da/d3c/d3d/l1b 0 2024-04-12T05:55:24.047 INFO:tasks.workunit.client.0.smithi008.stdout:4/89: creat d1/db/f18 x:0 0 0 2024-04-12T05:55:24.048 INFO:tasks.workunit.client.0.smithi008.stdout:9/109: mkdir d3/d22/d2b 0 2024-04-12T05:55:24.050 INFO:tasks.workunit.client.0.smithi008.stdout:6/127: write df/f1f [1254898,117132] 0 2024-04-12T05:55:24.050 INFO:tasks.workunit.client.0.smithi008.stdout:3/126: dwrite d9/d10/d19/d22/f28 [0,4194304] 0 2024-04-12T05:55:24.054 INFO:tasks.workunit.client.0.smithi008.stdout:7/195: mknod d1/c3e 0 2024-04-12T05:55:24.055 INFO:tasks.workunit.client.0.smithi008.stdout:6/128: dread - df/d10/d15/f24 zero size 2024-04-12T05:55:24.056 INFO:tasks.workunit.client.0.smithi008.stdout:7/196: fsync d1/d5/f2e 0 2024-04-12T05:55:24.058 INFO:tasks.workunit.client.0.smithi008.stdout:6/129: chown df/c26 14473767 1 2024-04-12T05:55:24.059 INFO:tasks.workunit.client.0.smithi008.stdout:6/130: chown df/f23 503 1 2024-04-12T05:55:24.061 INFO:tasks.workunit.client.0.smithi008.stdout:4/90: dwrite d1/db/fc [0,4194304] 0 2024-04-12T05:55:24.061 INFO:tasks.workunit.client.0.smithi008.stdout:9/110: rename d3/df/d13/d21/l26 to d3/df/d13/d14/l2c 0 2024-04-12T05:55:24.062 INFO:tasks.workunit.client.0.smithi008.stdout:6/131: truncate df/d21/f22 1467191 0 2024-04-12T05:55:24.062 INFO:tasks.workunit.client.0.smithi008.stdout:7/197: mkdir d1/d5/da/d3f 0 2024-04-12T05:55:24.065 INFO:tasks.workunit.client.0.smithi008.stdout:9/111: chown d3/fa 33490 1 2024-04-12T05:55:24.071 INFO:tasks.workunit.client.0.smithi008.stdout:6/132: dread fe [0,4194304] 0 2024-04-12T05:55:24.074 INFO:tasks.workunit.client.0.smithi008.stdout:7/198: dwrite d1/d15/f20 [0,4194304] 0 2024-04-12T05:55:24.075 INFO:tasks.workunit.client.0.smithi008.stdout:6/133: write f4 [506789,84766] 0 2024-04-12T05:55:24.075 INFO:tasks.workunit.client.0.smithi008.stdout:6/134: read fb [1784258,39436] 0 2024-04-12T05:55:24.077 INFO:tasks.workunit.client.0.smithi008.stdout:6/135: chown fd 5 1 2024-04-12T05:55:24.078 INFO:tasks.workunit.client.0.smithi008.stdout:6/136: read fe [2215436,6311] 0 2024-04-12T05:55:24.078 INFO:tasks.workunit.client.0.smithi008.stdout:4/91: getdents d1/d8 0 2024-04-12T05:55:24.081 INFO:tasks.workunit.client.0.smithi008.stdout:3/127: truncate f6 2489199 0 2024-04-12T05:55:24.083 INFO:tasks.workunit.client.0.smithi008.stdout:6/137: symlink df/l29 0 2024-04-12T05:55:24.083 INFO:tasks.workunit.client.0.smithi008.stdout:4/92: write d1/d8/f15 [300580,51416] 0 2024-04-12T05:55:24.084 INFO:tasks.workunit.client.0.smithi008.stdout:6/138: readlink df/d10/l1a 0 2024-04-12T05:55:24.088 INFO:tasks.workunit.client.0.smithi008.stdout:3/128: rename d9/d11/l1a to d9/d11/l29 0 2024-04-12T05:55:24.090 INFO:tasks.workunit.client.0.smithi008.stdout:7/199: creat d1/d5/d2f/f40 x:0 0 0 2024-04-12T05:55:24.092 INFO:tasks.workunit.client.0.smithi008.stdout:6/139: rmdir df/d10/d15 39 2024-04-12T05:55:24.093 INFO:tasks.workunit.client.0.smithi008.stdout:6/140: chown f9 1824 1 2024-04-12T05:55:24.093 INFO:tasks.workunit.client.0.smithi008.stdout:4/93: dwrite d1/db/f16 [0,4194304] 0 2024-04-12T05:55:24.094 INFO:tasks.workunit.client.0.smithi008.stdout:4/94: stat d1/f13 0 2024-04-12T05:55:24.098 INFO:tasks.workunit.client.0.smithi008.stdout:4/95: mkdir d1/db/d19 0 2024-04-12T05:55:24.099 INFO:tasks.workunit.client.0.smithi008.stdout:4/96: write d1/fe [1177670,38293] 0 2024-04-12T05:55:24.100 INFO:tasks.workunit.client.0.smithi008.stdout:9/112: truncate d3/df/d10/d12/f27 1416253 0 2024-04-12T05:55:24.103 INFO:tasks.workunit.client.0.smithi008.stdout:6/141: creat df/d10/d15/f2a x:0 0 0 2024-04-12T05:55:24.104 INFO:tasks.workunit.client.0.smithi008.stdout:6/142: read f5 [882920,95199] 0 2024-04-12T05:55:24.105 INFO:tasks.workunit.client.0.smithi008.stdout:6/143: write f9 [1106451,29622] 0 2024-04-12T05:55:24.105 INFO:tasks.workunit.client.0.smithi008.stdout:7/200: rmdir d1/d5/da/d3f 0 2024-04-12T05:55:24.107 INFO:tasks.workunit.client.0.smithi008.stdout:4/97: creat d1/db/f1a x:0 0 0 2024-04-12T05:55:24.108 INFO:tasks.workunit.client.0.smithi008.stdout:4/98: readlink d1/d8/l11 0 2024-04-12T05:55:24.108 INFO:tasks.workunit.client.0.smithi008.stdout:3/129: getdents d9/d1e 0 2024-04-12T05:55:24.109 INFO:tasks.workunit.client.0.smithi008.stdout:9/113: mkdir d3/df/d13/d21/d2d 0 2024-04-12T05:55:24.110 INFO:tasks.workunit.client.0.smithi008.stdout:9/114: chown d3/df/d10/f17 451214 1 2024-04-12T05:55:24.111 INFO:tasks.workunit.client.0.smithi008.stdout:3/130: write d9/d10/d19/d22/f28 [4364601,28462] 0 2024-04-12T05:55:24.114 INFO:tasks.workunit.client.0.smithi008.stdout:7/201: mknod d1/d15/c41 0 2024-04-12T05:55:24.115 INFO:tasks.workunit.client.0.smithi008.stdout:3/131: symlink d9/d11/l2a 0 2024-04-12T05:55:24.116 INFO:tasks.workunit.client.0.smithi008.stdout:3/132: stat d9/d10 0 2024-04-12T05:55:24.119 INFO:tasks.workunit.client.0.smithi008.stdout:3/133: chown d9/d10/d19/d22/f28 0 1 2024-04-12T05:55:24.119 INFO:tasks.workunit.client.0.smithi008.stdout:6/144: rename df/c19 to df/d10/c2b 0 2024-04-12T05:55:24.122 INFO:tasks.workunit.client.0.smithi008.stdout:4/99: link d1/d8/l14 d1/db/l1b 0 2024-04-12T05:55:24.122 INFO:tasks.workunit.client.0.smithi008.stdout:6/145: write f4 [1230935,75391] 0 2024-04-12T05:55:24.126 INFO:tasks.workunit.client.0.smithi008.stdout:9/115: dwrite d3/f25 [0,4194304] 0 2024-04-12T05:55:24.130 INFO:tasks.workunit.client.0.smithi008.stdout:3/134: unlink d9/d10/d19/d22/c12 0 2024-04-12T05:55:24.132 INFO:tasks.workunit.client.0.smithi008.stdout:4/100: symlink d1/d8/l1c 0 2024-04-12T05:55:24.132 INFO:tasks.workunit.client.0.smithi008.stdout:3/135: rename d9/d10 to d9/d10/d2b 22 2024-04-12T05:55:24.134 INFO:tasks.workunit.client.0.smithi008.stdout:3/136: chown f7 19475 1 2024-04-12T05:55:24.134 INFO:tasks.workunit.client.0.smithi008.stdout:9/116: creat d3/df/d10/d12/f2e x:0 0 0 2024-04-12T05:55:24.139 INFO:tasks.workunit.client.0.smithi008.stdout:7/202: creat d1/d5/f42 x:0 0 0 2024-04-12T05:55:24.139 INFO:tasks.workunit.client.0.smithi008.stdout:6/146: dwrite df/f1f [0,4194304] 0 2024-04-12T05:55:24.141 INFO:tasks.workunit.client.0.smithi008.stdout:6/147: chown df/d10/d15/f13 202864078 1 2024-04-12T05:55:24.142 INFO:tasks.workunit.client.0.smithi008.stdout:3/137: mkdir d9/d10/d19/d22/d2c 0 2024-04-12T05:55:24.146 INFO:tasks.workunit.client.0.smithi008.stdout:3/138: fsync d9/d10/d19/d22/f1f 0 2024-04-12T05:55:24.146 INFO:tasks.workunit.client.0.smithi008.stdout:4/101: dread d1/db/f16 [0,4194304] 0 2024-04-12T05:55:24.148 INFO:tasks.workunit.client.0.smithi008.stdout:4/102: stat d1/fe 0 2024-04-12T05:55:24.150 INFO:tasks.workunit.client.0.smithi008.stdout:7/203: dwrite d1/d5/da/d11/f35 [0,4194304] 0 2024-04-12T05:55:24.151 INFO:tasks.workunit.client.0.smithi008.stdout:7/204: readlink d1/l9 0 2024-04-12T05:55:24.154 INFO:tasks.workunit.client.0.smithi008.stdout:9/117: write d3/fb [4049909,121205] 0 2024-04-12T05:55:24.156 INFO:tasks.workunit.client.0.smithi008.stdout:3/139: mknod d9/d1e/c2d 0 2024-04-12T05:55:24.157 INFO:tasks.workunit.client.0.smithi008.stdout:4/103: mknod d1/d8/c1d 0 2024-04-12T05:55:24.158 INFO:tasks.workunit.client.0.smithi008.stdout:3/140: chown f6 1357 1 2024-04-12T05:55:24.158 INFO:tasks.workunit.client.0.smithi008.stdout:6/148: rename df/d10/d15/f1b to df/d10/f2c 0 2024-04-12T05:55:24.160 INFO:tasks.workunit.client.0.smithi008.stdout:4/104: fsync d1/db/fc 0 2024-04-12T05:55:24.162 INFO:tasks.workunit.client.0.smithi008.stdout:4/105: chown d1/d8/l1c 1 1 2024-04-12T05:55:24.163 INFO:tasks.workunit.client.0.smithi008.stdout:6/149: write df/d10/f2c [971168,81071] 0 2024-04-12T05:55:24.165 INFO:tasks.workunit.client.0.smithi008.stdout:4/106: write d1/fe [1671662,110707] 0 2024-04-12T05:55:24.166 INFO:tasks.workunit.client.0.smithi008.stdout:6/150: read f4 [878334,40239] 0 2024-04-12T05:55:24.169 INFO:tasks.workunit.client.0.smithi008.stdout:6/151: fdatasync df/d10/d15/f2a 0 2024-04-12T05:55:24.169 INFO:tasks.workunit.client.0.smithi008.stdout:9/118: dwrite d3/df/d13/d14/f18 [0,4194304] 0 2024-04-12T05:55:24.171 INFO:tasks.workunit.client.0.smithi008.stdout:3/141: unlink d9/d10/d19/d22/cf 0 2024-04-12T05:55:24.171 INFO:tasks.workunit.client.0.smithi008.stdout:6/152: write df/d10/d15/f13 [2163010,12168] 0 2024-04-12T05:55:24.175 INFO:tasks.workunit.client.0.smithi008.stdout:7/205: write d1/fb [39016,42879] 0 2024-04-12T05:55:24.178 INFO:tasks.workunit.client.0.smithi008.stdout:7/206: fdatasync d1/d15/f20 0 2024-04-12T05:55:24.181 INFO:tasks.workunit.client.0.smithi008.stdout:9/119: mknod d3/df/c2f 0 2024-04-12T05:55:24.184 INFO:tasks.workunit.client.0.smithi008.stdout:3/142: dread d9/d10/d19/d22/fc [0,4194304] 0 2024-04-12T05:55:24.186 INFO:tasks.workunit.client.0.smithi008.stdout:6/153: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:24.193 INFO:tasks.workunit.client.0.smithi008.stdout:4/107: write d1/f12 [4954883,129632] 0 2024-04-12T05:55:24.193 INFO:tasks.workunit.client.0.smithi008.stdout:3/143: write d9/f25 [328970,68747] 0 2024-04-12T05:55:24.193 INFO:tasks.workunit.client.0.smithi008.stdout:7/207: mkdir d1/d15/d43 0 2024-04-12T05:55:24.194 INFO:tasks.workunit.client.0.smithi008.stdout:3/144: write d9/f1c [26383,79991] 0 2024-04-12T05:55:24.196 INFO:tasks.workunit.client.0.smithi008.stdout:4/108: mknod d1/c1e 0 2024-04-12T05:55:24.199 INFO:tasks.workunit.client.0.smithi008.stdout:7/208: fsync d1/f6 0 2024-04-12T05:55:24.201 INFO:tasks.workunit.client.0.smithi008.stdout:4/109: unlink d1/d8/l1c 0 2024-04-12T05:55:24.201 INFO:tasks.workunit.client.0.smithi008.stdout:9/120: dread d3/f1f [0,4194304] 0 2024-04-12T05:55:24.204 INFO:tasks.workunit.client.0.smithi008.stdout:6/154: dwrite fd [0,4194304] 0 2024-04-12T05:55:24.205 INFO:tasks.workunit.client.0.smithi008.stdout:6/155: stat df 0 2024-04-12T05:55:24.208 INFO:tasks.workunit.client.0.smithi008.stdout:3/145: read d9/d10/d19/d22/fc [3199007,5761] 0 2024-04-12T05:55:24.213 INFO:tasks.workunit.client.0.smithi008.stdout:9/121: mkdir d3/df/d13/d21/d2d/d30 0 2024-04-12T05:55:24.213 INFO:tasks.workunit.client.0.smithi008.stdout:6/156: symlink df/l2d 0 2024-04-12T05:55:24.215 INFO:tasks.workunit.client.0.smithi008.stdout:9/122: read d3/f6 [2282894,27655] 0 2024-04-12T05:55:24.215 INFO:tasks.workunit.client.0.smithi008.stdout:3/146: mkdir d9/d2e 0 2024-04-12T05:55:24.216 INFO:tasks.workunit.client.0.smithi008.stdout:7/209: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:24.216 INFO:tasks.workunit.client.0.smithi008.stdout:6/157: creat df/f2e x:0 0 0 2024-04-12T05:55:24.219 INFO:tasks.workunit.client.0.smithi008.stdout:9/123: write d3/df/f28 [475186,78651] 0 2024-04-12T05:55:24.220 INFO:tasks.workunit.client.0.smithi008.stdout:7/210: unlink d1/l30 0 2024-04-12T05:55:24.224 INFO:tasks.workunit.client.0.smithi008.stdout:7/211: mkdir d1/d5/da/d3c/d44 0 2024-04-12T05:55:24.225 INFO:tasks.workunit.client.0.smithi008.stdout:6/158: dread df/f1f [0,4194304] 0 2024-04-12T05:55:24.227 INFO:tasks.workunit.client.0.smithi008.stdout:3/147: creat d9/dd/f2f x:0 0 0 2024-04-12T05:55:24.230 INFO:tasks.workunit.client.0.smithi008.stdout:7/212: mknod d1/d5/c45 0 2024-04-12T05:55:24.232 INFO:tasks.workunit.client.0.smithi008.stdout:9/124: dwrite d3/df/f11 [0,4194304] 0 2024-04-12T05:55:24.234 INFO:tasks.workunit.client.0.smithi008.stdout:3/148: symlink d9/d1e/l30 0 2024-04-12T05:55:24.234 INFO:tasks.workunit.client.0.smithi008.stdout:9/125: stat d3/df/d13 0 2024-04-12T05:55:24.238 INFO:tasks.workunit.client.0.smithi008.stdout:7/213: symlink d1/d5/da/d3c/d44/l46 0 2024-04-12T05:55:24.240 INFO:tasks.workunit.client.0.smithi008.stdout:7/214: rmdir d1/d5/da/d3c/d3d 39 2024-04-12T05:55:24.241 INFO:tasks.workunit.client.0.smithi008.stdout:7/215: stat d1/d5/da/d3c 0 2024-04-12T05:55:24.242 INFO:tasks.workunit.client.0.smithi008.stdout:7/216: truncate d1/d5/d2f/f40 443581 0 2024-04-12T05:55:24.243 INFO:tasks.workunit.client.0.smithi008.stdout:3/149: mknod d9/dd/c31 0 2024-04-12T05:55:24.244 INFO:tasks.workunit.client.0.smithi008.stdout:7/217: dread - d1/d5/f29 zero size 2024-04-12T05:55:24.246 INFO:tasks.workunit.client.0.smithi008.stdout:7/218: mkdir d1/d5/da/d11/d47 0 2024-04-12T05:55:24.248 INFO:tasks.workunit.client.0.smithi008.stdout:3/150: creat d9/d10/d19/d22/f32 x:0 0 0 2024-04-12T05:55:24.252 INFO:tasks.workunit.client.0.smithi008.stdout:9/126: dwrite d3/f1f [0,4194304] 0 2024-04-12T05:55:24.253 INFO:tasks.workunit.client.0.smithi008.stdout:3/151: chown d9/d11/c14 5661506 1 2024-04-12T05:55:24.253 INFO:tasks.workunit.client.0.smithi008.stdout:7/219: truncate d1/d5/da/d3c/d3d/f3b 364150 0 2024-04-12T05:55:24.253 INFO:tasks.workunit.client.0.smithi008.stdout:3/152: chown d9/d1e/l30 952382 1 2024-04-12T05:55:24.255 INFO:tasks.workunit.client.0.smithi008.stdout:9/127: rmdir d3/df/d13/d14 39 2024-04-12T05:55:24.256 INFO:tasks.workunit.client.0.smithi008.stdout:3/153: creat d9/d10/f33 x:0 0 0 2024-04-12T05:55:24.260 INFO:tasks.workunit.client.0.smithi008.stdout:3/154: write d9/d10/f33 [252101,9018] 0 2024-04-12T05:55:24.262 INFO:tasks.workunit.client.0.smithi008.stdout:3/155: readlink d9/d11/l27 0 2024-04-12T05:55:24.265 INFO:tasks.workunit.client.0.smithi008.stdout:3/156: symlink d9/d10/l34 0 2024-04-12T05:55:24.265 INFO:tasks.workunit.client.0.smithi008.stdout:7/220: dwrite d1/f12 [4194304,4194304] 0 2024-04-12T05:55:24.267 INFO:tasks.workunit.client.0.smithi008.stdout:3/157: dread - d9/dd/f2f zero size 2024-04-12T05:55:24.271 INFO:tasks.workunit.client.0.smithi008.stdout:9/128: dwrite d3/df/f11 [4194304,4194304] 0 2024-04-12T05:55:24.271 INFO:tasks.workunit.client.0.smithi008.stdout:3/158: creat d9/d1e/f35 x:0 0 0 2024-04-12T05:55:24.272 INFO:tasks.workunit.client.0.smithi008.stdout:7/221: mknod d1/d5/da/d11/d47/c48 0 2024-04-12T05:55:24.276 INFO:tasks.workunit.client.0.smithi008.stdout:3/159: chown d9/dd/c18 0 1 2024-04-12T05:55:24.277 INFO:tasks.workunit.client.0.smithi008.stdout:9/129: truncate d3/df/d10/d12/f23 282486 0 2024-04-12T05:55:24.282 INFO:tasks.workunit.client.0.smithi008.stdout:3/160: link f6 d9/dd/f36 0 2024-04-12T05:55:24.284 INFO:tasks.workunit.client.0.smithi008.stdout:7/222: write d1/f6 [648985,86694] 0 2024-04-12T05:55:24.286 INFO:tasks.workunit.client.0.smithi008.stdout:3/161: mkdir d9/d37 0 2024-04-12T05:55:24.289 INFO:tasks.workunit.client.0.smithi008.stdout:3/162: symlink d9/d37/l38 0 2024-04-12T05:55:24.295 INFO:tasks.workunit.client.0.smithi008.stdout:3/163: write f6 [186695,105705] 0 2024-04-12T05:55:24.297 INFO:tasks.workunit.client.0.smithi008.stdout:7/223: dwrite d1/d5/da/d3c/d3d/f3b [0,4194304] 0 2024-04-12T05:55:24.299 INFO:tasks.workunit.client.0.smithi008.stdout:9/130: truncate d3/f7 7666514 0 2024-04-12T05:55:24.300 INFO:tasks.workunit.client.0.smithi008.stdout:7/224: write d1/f12 [2401972,2538] 0 2024-04-12T05:55:24.301 INFO:tasks.workunit.client.0.smithi008.stdout:3/164: symlink d9/d2e/l39 0 2024-04-12T05:55:24.303 INFO:tasks.workunit.client.0.smithi008.stdout:9/131: unlink d3/df/d10/l1c 0 2024-04-12T05:55:24.304 INFO:tasks.workunit.client.0.smithi008.stdout:9/132: fsync d3/fa 0 2024-04-12T05:55:24.307 INFO:tasks.workunit.client.0.smithi008.stdout:3/165: mkdir d9/d10/d19/d3a 0 2024-04-12T05:55:24.308 INFO:tasks.workunit.client.0.smithi008.stdout:9/133: creat d3/df/d13/d21/f31 x:0 0 0 2024-04-12T05:55:24.310 INFO:tasks.workunit.client.0.smithi008.stdout:7/225: rename d1/d5/d2f to d1/d49 0 2024-04-12T05:55:24.312 INFO:tasks.workunit.client.0.smithi008.stdout:9/134: mkdir d3/df/d13/d21/d2d/d32 0 2024-04-12T05:55:24.313 INFO:tasks.workunit.client.0.smithi008.stdout:7/226: write d1/d5/da/d11/f35 [5777166,114055] 0 2024-04-12T05:55:24.313 INFO:tasks.workunit.client.0.smithi008.stdout:9/135: truncate d3/df/f20 175926 0 2024-04-12T05:55:24.316 INFO:tasks.workunit.client.0.smithi008.stdout:9/136: creat d3/d22/d2b/f33 x:0 0 0 2024-04-12T05:55:24.317 INFO:tasks.workunit.client.0.smithi008.stdout:7/227: creat d1/d15/f4a x:0 0 0 2024-04-12T05:55:24.318 INFO:tasks.workunit.client.0.smithi008.stdout:9/137: write d3/df/d10/f17 [3111381,9247] 0 2024-04-12T05:55:24.319 INFO:tasks.workunit.client.0.smithi008.stdout:9/138: chown d3/df/d13/d21 85557249 1 2024-04-12T05:55:24.322 INFO:tasks.workunit.client.0.smithi008.stdout:7/228: creat d1/d49/f4b x:0 0 0 2024-04-12T05:55:24.326 INFO:tasks.workunit.client.0.smithi008.stdout:9/139: creat d3/df/d13/d21/d2d/d32/f34 x:0 0 0 2024-04-12T05:55:24.330 INFO:tasks.workunit.client.0.smithi008.stdout:3/166: link d9/dd/l1d d9/d10/d19/l3b 0 2024-04-12T05:55:24.338 INFO:tasks.workunit.client.0.smithi008.stdout:3/167: dwrite f1 [4194304,4194304] 0 2024-04-12T05:55:24.338 INFO:tasks.workunit.client.0.smithi008.stdout:3/168: stat d9/f1c 0 2024-04-12T05:55:24.341 INFO:tasks.workunit.client.0.smithi008.stdout:3/169: symlink d9/d2e/l3c 0 2024-04-12T05:55:24.342 INFO:tasks.workunit.client.0.smithi008.stdout:3/170: write d9/d10/d19/d22/f1f [4817127,104264] 0 2024-04-12T05:55:24.347 INFO:tasks.workunit.client.0.smithi008.stdout:9/140: dwrite d3/f7 [0,4194304] 0 2024-04-12T05:55:24.353 INFO:tasks.workunit.client.0.smithi008.stdout:9/141: unlink d3/df/d13/d14/l19 0 2024-04-12T05:55:24.354 INFO:tasks.workunit.client.0.smithi008.stdout:3/171: dwrite d9/d10/d19/d22/f28 [4194304,4194304] 0 2024-04-12T05:55:24.356 INFO:tasks.workunit.client.0.smithi008.stdout:3/172: fsync d9/d10/d19/d22/fc 0 2024-04-12T05:55:24.356 INFO:tasks.workunit.client.0.smithi008.stdout:9/142: creat d3/f35 x:0 0 0 2024-04-12T05:55:24.362 INFO:tasks.workunit.client.0.smithi008.stdout:9/143: mkdir d3/df/d10/d12/d36 0 2024-04-12T05:55:24.368 INFO:tasks.workunit.client.0.smithi008.stdout:3/173: getdents d9/d1e 0 2024-04-12T05:55:24.369 INFO:tasks.workunit.client.0.smithi008.stdout:9/144: unlink d3/df/d13/d14/f18 0 2024-04-12T05:55:24.369 INFO:tasks.workunit.client.0.smithi008.stdout:9/145: chown d3/df/c2f 788 1 2024-04-12T05:55:24.371 INFO:tasks.workunit.client.0.smithi008.stdout:3/174: mkdir d9/d2e/d3d 0 2024-04-12T05:55:24.378 INFO:tasks.workunit.client.0.smithi008.stdout:9/146: dwrite d3/df/d13/d21/d2d/d32/f34 [0,4194304] 0 2024-04-12T05:55:24.381 INFO:tasks.workunit.client.0.smithi008.stdout:3/175: dwrite d9/d10/d19/d22/fc [4194304,4194304] 0 2024-04-12T05:55:24.383 INFO:tasks.workunit.client.0.smithi008.stdout:9/147: dread d3/df/f20 [0,4194304] 0 2024-04-12T05:55:24.387 INFO:tasks.workunit.client.0.smithi008.stdout:9/148: mknod d3/d22/c37 0 2024-04-12T05:55:24.387 INFO:tasks.workunit.client.0.smithi008.stdout:3/176: mknod d9/d10/d19/d22/c3e 0 2024-04-12T05:55:24.388 INFO:tasks.workunit.client.0.smithi008.stdout:3/177: fsync d9/d10/d19/d22/f28 0 2024-04-12T05:55:24.391 INFO:tasks.workunit.client.0.smithi008.stdout:9/149: rmdir d3/df/d10 39 2024-04-12T05:55:24.392 INFO:tasks.workunit.client.0.smithi008.stdout:9/150: write d3/f35 [296115,69272] 0 2024-04-12T05:55:24.394 INFO:tasks.workunit.client.0.smithi008.stdout:9/151: stat d3/fd 0 2024-04-12T05:55:24.395 INFO:tasks.workunit.client.0.smithi008.stdout:9/152: readlink d3/d22/l24 0 2024-04-12T05:55:24.395 INFO:tasks.workunit.client.0.smithi008.stdout:9/153: fdatasync d3/f5 0 2024-04-12T05:55:24.395 INFO:tasks.workunit.client.0.smithi008.stdout:3/178: dread d9/d10/d19/d22/f1f [4194304,4194304] 0 2024-04-12T05:55:24.396 INFO:tasks.workunit.client.0.smithi008.stdout:9/154: readlink d3/d22/l24 0 2024-04-12T05:55:24.396 INFO:tasks.workunit.client.0.smithi008.stdout:9/155: readlink d3/d22/l24 0 2024-04-12T05:55:24.399 INFO:tasks.workunit.client.0.smithi008.stdout:3/179: getdents d9/d2e 0 2024-04-12T05:55:24.400 INFO:tasks.workunit.client.0.smithi008.stdout:9/156: mkdir d3/df/d10/d12/d38 0 2024-04-12T05:55:24.406 INFO:tasks.workunit.client.0.smithi008.stdout:3/180: dwrite d9/dd/f36 [0,4194304] 0 2024-04-12T05:55:24.407 INFO:tasks.workunit.client.0.smithi008.stdout:9/157: dwrite d3/df/f28 [0,4194304] 0 2024-04-12T05:55:24.410 INFO:tasks.workunit.client.0.smithi008.stdout:3/181: creat d9/d2e/f3f x:0 0 0 2024-04-12T05:55:24.415 INFO:tasks.workunit.client.0.smithi008.stdout:9/158: rename d3/f7 to d3/df/f39 0 2024-04-12T05:55:24.418 INFO:tasks.workunit.client.0.smithi008.stdout:3/182: dwrite d9/d10/d19/d22/f28 [0,4194304] 0 2024-04-12T05:55:24.420 INFO:tasks.workunit.client.0.smithi008.stdout:9/159: rename d3/df/c2f to d3/d22/d2b/c3a 0 2024-04-12T05:55:24.422 INFO:tasks.workunit.client.0.smithi008.stdout:9/160: mknod d3/df/c3b 0 2024-04-12T05:55:24.424 INFO:tasks.workunit.client.0.smithi008.stdout:9/161: write d3/df/d10/d12/f1e [2269903,13733] 0 2024-04-12T05:55:24.426 INFO:tasks.workunit.client.0.smithi008.stdout:9/162: write d3/df/d10/d12/f1d [2612055,19635] 0 2024-04-12T05:55:24.429 INFO:tasks.workunit.client.0.smithi008.stdout:9/163: creat d3/df/d10/d12/d36/f3c x:0 0 0 2024-04-12T05:55:24.431 INFO:tasks.workunit.client.0.smithi008.stdout:9/164: mknod d3/df/d13/d21/c3d 0 2024-04-12T05:55:24.440 INFO:tasks.workunit.client.0.smithi008.stdout:9/165: getdents d3/df/d10/d12/d36 0 2024-04-12T05:55:24.441 INFO:tasks.workunit.client.0.smithi008.stdout:9/166: creat d3/d22/f3e x:0 0 0 2024-04-12T05:55:24.442 INFO:tasks.workunit.client.0.smithi008.stdout:9/167: fsync d3/df/d10/d12/f23 0 2024-04-12T05:55:24.442 INFO:tasks.workunit.client.0.smithi008.stdout:9/168: readlink d3/l8 0 2024-04-12T05:55:24.443 INFO:tasks.workunit.client.0.smithi008.stdout:9/169: stat d3/df/d10/d12/f1e 0 2024-04-12T05:55:24.444 INFO:tasks.workunit.client.0.smithi008.stdout:9/170: creat d3/df/d10/d12/f3f x:0 0 0 2024-04-12T05:55:24.445 INFO:tasks.workunit.client.0.smithi008.stdout:9/171: rename d3 to d3/df/d13/d21/d2d/d32/d40 22 2024-04-12T05:55:24.446 INFO:tasks.workunit.client.0.smithi008.stdout:9/172: write d3/df/d10/d12/f2e [739997,120440] 0 2024-04-12T05:55:24.447 INFO:tasks.workunit.client.0.smithi008.stdout:9/173: write d3/df/d10/f17 [4945626,119855] 0 2024-04-12T05:55:24.449 INFO:tasks.workunit.client.0.smithi008.stdout:9/174: creat d3/d22/d2b/f41 x:0 0 0 2024-04-12T05:55:24.453 INFO:tasks.workunit.client.0.smithi008.stdout:9/175: dread d3/df/d10/d12/f2e [0,4194304] 0 2024-04-12T05:55:24.454 INFO:tasks.workunit.client.0.smithi008.stdout:9/176: write d3/df/d10/d12/f3f [143955,31087] 0 2024-04-12T05:55:24.455 INFO:tasks.workunit.client.0.smithi008.stdout:9/177: dread - d3/df/d13/d21/f31 zero size 2024-04-12T05:55:24.456 INFO:tasks.workunit.client.0.smithi008.stdout:9/178: read d3/f35 [214282,122951] 0 2024-04-12T05:55:24.457 INFO:tasks.workunit.client.0.smithi008.stdout:9/179: stat d3/df/d13/c16 0 2024-04-12T05:55:24.459 INFO:tasks.workunit.client.0.smithi008.stdout:9/180: mknod d3/df/d13/c42 0 2024-04-12T05:55:24.466 INFO:tasks.workunit.client.0.smithi008.stdout:9/181: dwrite d3/df/d10/d12/d36/f3c [0,4194304] 0 2024-04-12T05:55:24.467 INFO:tasks.workunit.client.0.smithi008.stdout:8/89: sync 2024-04-12T05:55:24.467 INFO:tasks.workunit.client.0.smithi008.stdout:1/87: sync 2024-04-12T05:55:24.474 INFO:tasks.workunit.client.0.smithi008.stdout:8/90: creat dd/de/f1c x:0 0 0 2024-04-12T05:55:24.474 INFO:tasks.workunit.client.0.smithi008.stdout:1/88: dread d4/d7/ff [0,4194304] 0 2024-04-12T05:55:24.476 INFO:tasks.workunit.client.0.smithi008.stdout:1/89: stat d4/d6 0 2024-04-12T05:55:24.476 INFO:tasks.workunit.client.0.smithi008.stdout:8/91: write dd/ff [1102682,39881] 0 2024-04-12T05:55:24.477 INFO:tasks.workunit.client.0.smithi008.stdout:8/92: chown dd/l13 483 1 2024-04-12T05:55:24.478 INFO:tasks.workunit.client.0.smithi008.stdout:1/90: symlink d4/l16 0 2024-04-12T05:55:24.480 INFO:tasks.workunit.client.0.smithi008.stdout:8/93: creat dd/de/f1d x:0 0 0 2024-04-12T05:55:24.481 INFO:tasks.workunit.client.0.smithi008.stdout:8/94: write f7 [2022762,43209] 0 2024-04-12T05:55:24.481 INFO:tasks.workunit.client.0.smithi008.stdout:1/91: mkdir d4/dd/d17 0 2024-04-12T05:55:24.483 INFO:tasks.workunit.client.0.smithi008.stdout:1/92: chown d4/d6/f10 7 1 2024-04-12T05:55:24.484 INFO:tasks.workunit.client.0.smithi008.stdout:8/95: write dd/de/f12 [880076,13386] 0 2024-04-12T05:55:24.486 INFO:tasks.workunit.client.0.smithi008.stdout:1/93: mkdir d4/d7/d18 0 2024-04-12T05:55:24.487 INFO:tasks.workunit.client.0.smithi008.stdout:9/182: dwrite d3/df/d10/d12/f27 [0,4194304] 0 2024-04-12T05:55:24.488 INFO:tasks.workunit.client.0.smithi008.stdout:8/96: creat dd/f1e x:0 0 0 2024-04-12T05:55:24.493 INFO:tasks.workunit.client.0.smithi008.stdout:1/94: dread d4/d7/fe [0,4194304] 0 2024-04-12T05:55:24.493 INFO:tasks.workunit.client.0.smithi008.stdout:8/97: creat dd/f1f x:0 0 0 2024-04-12T05:55:24.495 INFO:tasks.workunit.client.0.smithi008.stdout:1/95: fdatasync d4/d7/fc 0 2024-04-12T05:55:24.495 INFO:tasks.workunit.client.0.smithi008.stdout:9/183: dread d3/df/d10/d12/f3f [0,4194304] 0 2024-04-12T05:55:24.495 INFO:tasks.workunit.client.0.smithi008.stdout:1/96: readlink d4/l14 0 2024-04-12T05:55:24.496 INFO:tasks.workunit.client.0.smithi008.stdout:1/97: write f3 [735513,130319] 0 2024-04-12T05:55:24.497 INFO:tasks.workunit.client.0.smithi008.stdout:8/98: symlink dd/de/l20 0 2024-04-12T05:55:24.498 INFO:tasks.workunit.client.0.smithi008.stdout:1/98: write d4/d7/fc [840065,31687] 0 2024-04-12T05:55:24.499 INFO:tasks.workunit.client.0.smithi008.stdout:9/184: rmdir d3/df/d10/d12/d36 39 2024-04-12T05:55:24.500 INFO:tasks.workunit.client.0.smithi008.stdout:8/99: symlink dd/de/d16/l21 0 2024-04-12T05:55:24.501 INFO:tasks.workunit.client.0.smithi008.stdout:8/100: stat dd/de/l20 0 2024-04-12T05:55:24.502 INFO:tasks.workunit.client.0.smithi008.stdout:1/99: link f3 d4/dd/d17/f19 0 2024-04-12T05:55:24.504 INFO:tasks.workunit.client.0.smithi008.stdout:1/100: chown d4/d6/c15 544260 1 2024-04-12T05:55:24.507 INFO:tasks.workunit.client.0.smithi008.stdout:1/101: write d4/dd/d17/f19 [2093911,89982] 0 2024-04-12T05:55:24.510 INFO:tasks.workunit.client.0.smithi008.stdout:8/101: truncate fb 3082983 0 2024-04-12T05:55:24.512 INFO:tasks.workunit.client.0.smithi008.stdout:8/102: write dd/f1f [397406,30950] 0 2024-04-12T05:55:24.512 INFO:tasks.workunit.client.0.smithi008.stdout:9/185: truncate d3/df/d10/d12/d36/f3c 759262 0 2024-04-12T05:55:24.513 INFO:tasks.workunit.client.0.smithi008.stdout:1/102: getdents d4/d6 0 2024-04-12T05:55:24.514 INFO:tasks.workunit.client.0.smithi008.stdout:8/103: readlink dd/de/d16/l21 0 2024-04-12T05:55:24.514 INFO:tasks.workunit.client.0.smithi008.stdout:9/186: fsync d3/d22/d2b/f41 0 2024-04-12T05:55:24.515 INFO:tasks.workunit.client.0.smithi008.stdout:9/187: chown d3/df/d13 96827279 1 2024-04-12T05:55:24.518 INFO:tasks.workunit.client.0.smithi008.stdout:8/104: link f7 dd/f22 0 2024-04-12T05:55:24.520 INFO:tasks.workunit.client.0.smithi008.stdout:9/188: unlink d3/df/d13/d21/c2a 0 2024-04-12T05:55:24.521 INFO:tasks.workunit.client.0.smithi008.stdout:1/103: dwrite d4/d6/f10 [0,4194304] 0 2024-04-12T05:55:24.524 INFO:tasks.workunit.client.0.smithi008.stdout:8/105: fdatasync f7 0 2024-04-12T05:55:24.525 INFO:tasks.workunit.client.0.smithi008.stdout:9/189: write d3/f25 [2758205,28315] 0 2024-04-12T05:55:24.527 INFO:tasks.workunit.client.0.smithi008.stdout:1/104: mkdir d4/d1a 0 2024-04-12T05:55:24.529 INFO:tasks.workunit.client.0.smithi008.stdout:1/105: write d4/d7/fc [4556050,48310] 0 2024-04-12T05:55:24.531 INFO:tasks.workunit.client.0.smithi008.stdout:1/106: mkdir d4/dd/d1b 0 2024-04-12T05:55:24.532 INFO:tasks.workunit.client.0.smithi008.stdout:8/106: link dd/c1b dd/de/c23 0 2024-04-12T05:55:24.532 INFO:tasks.workunit.client.0.smithi008.stdout:1/107: write d4/d7/fc [200140,9185] 0 2024-04-12T05:55:24.535 INFO:tasks.workunit.client.0.smithi008.stdout:1/108: unlink d4/d6/d11/f12 0 2024-04-12T05:55:24.537 INFO:tasks.workunit.client.0.smithi008.stdout:1/109: read d4/d7/ff [2695968,72321] 0 2024-04-12T05:55:24.539 INFO:tasks.workunit.client.0.smithi008.stdout:1/110: mkdir d4/d1c 0 2024-04-12T05:55:24.540 INFO:tasks.workunit.client.0.smithi008.stdout:8/107: dwrite dd/de/f1d [0,4194304] 0 2024-04-12T05:55:24.542 INFO:tasks.workunit.client.0.smithi008.stdout:1/111: mkdir d4/d1a/d1d 0 2024-04-12T05:55:24.543 INFO:tasks.workunit.client.0.smithi008.stdout:1/112: chown d4/d7/ff 6836163 1 2024-04-12T05:55:24.544 INFO:tasks.workunit.client.0.smithi008.stdout:1/113: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:24.548 INFO:tasks.workunit.client.0.smithi008.stdout:1/114: dread d4/d7/ff [0,4194304] 0 2024-04-12T05:55:24.557 INFO:tasks.workunit.client.0.smithi008.stdout:1/115: dread d4/dd/d17/f19 [0,4194304] 0 2024-04-12T05:55:24.559 INFO:tasks.workunit.client.0.smithi008.stdout:1/116: fdatasync d4/d6/f8 0 2024-04-12T05:55:24.563 INFO:tasks.workunit.client.0.smithi008.stdout:1/117: dread d4/d7/ff [0,4194304] 0 2024-04-12T05:55:24.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:24 smithi194 ceph-mon[49771]: mgrmap e25: smithi194.xbiyaj(active, since 9s), standbys: smithi008.ixmnlp 2024-04-12T05:55:24.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:24 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:55:24.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:24 smithi194 ceph-mon[49771]: pgmap v8: 65 pgs: 65 active+clean; 235 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 824 KiB/s rd, 23 MiB/s wr, 251 op/s 2024-04-12T05:55:24.581 INFO:tasks.workunit.client.0.smithi008.stdout:1/118: dread d4/d6/f10 [4194304,4194304] 0 2024-04-12T05:55:24.588 INFO:tasks.workunit.client.0.smithi008.stdout:1/119: dwrite d4/d7/ff [0,4194304] 0 2024-04-12T05:55:24.591 INFO:tasks.workunit.client.0.smithi008.stdout:1/120: symlink d4/dd/d17/l1e 0 2024-04-12T05:55:24.593 INFO:tasks.workunit.client.0.smithi008.stdout:1/121: creat d4/dd/d17/f1f x:0 0 0 2024-04-12T05:55:24.594 INFO:tasks.workunit.client.0.smithi008.stdout:1/122: stat d4/d1c 0 2024-04-12T05:55:24.595 INFO:tasks.workunit.client.0.smithi008.stdout:1/123: read d4/d7/fe [1355388,74472] 0 2024-04-12T05:55:24.602 INFO:tasks.workunit.client.0.smithi008.stdout:1/124: dwrite d4/d7/fc [4194304,4194304] 0 2024-04-12T05:55:24.603 INFO:tasks.workunit.client.0.smithi008.stdout:1/125: chown d4/d7 28 1 2024-04-12T05:55:24.605 INFO:tasks.workunit.client.0.smithi008.stdout:1/126: mknod d4/dd/d17/c20 0 2024-04-12T05:55:24.633 INFO:tasks.workunit.client.0.smithi008.stdout:1/127: dread d4/d7/fc [0,4194304] 0 2024-04-12T05:55:24.634 INFO:tasks.workunit.client.0.smithi008.stdout:1/128: mkdir d4/dd/d21 0 2024-04-12T05:55:24.635 INFO:tasks.workunit.client.0.smithi008.stdout:1/129: dread - d4/dd/d17/f1f zero size 2024-04-12T05:55:24.637 INFO:tasks.workunit.client.0.smithi008.stdout:1/130: link d4/d7/ff d4/d6/d11/f22 0 2024-04-12T05:55:24.638 INFO:tasks.workunit.client.0.smithi008.stdout:1/131: creat d4/d6/d11/f23 x:0 0 0 2024-04-12T05:55:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:24 smithi008 ceph-mon[39722]: mgrmap e25: smithi194.xbiyaj(active, since 9s), standbys: smithi008.ixmnlp 2024-04-12T05:55:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:24 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:55:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:24 smithi008 ceph-mon[39722]: pgmap v8: 65 pgs: 65 active+clean; 235 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 824 KiB/s rd, 23 MiB/s wr, 251 op/s 2024-04-12T05:55:24.671 INFO:tasks.workunit.client.0.smithi008.stdout:2/241: sync 2024-04-12T05:55:24.672 INFO:tasks.workunit.client.0.smithi008.stdout:5/206: sync 2024-04-12T05:55:24.673 INFO:tasks.workunit.client.0.smithi008.stdout:1/132: mkdir d4/d24 0 2024-04-12T05:55:24.674 INFO:tasks.workunit.client.0.smithi008.stdout:1/133: chown d4/d6/d11/f23 0 1 2024-04-12T05:55:24.675 INFO:tasks.workunit.client.0.smithi008.stdout:1/134: dread - d4/d6/d11/f23 zero size 2024-04-12T05:55:24.678 INFO:tasks.workunit.client.0.smithi008.stdout:1/135: symlink d4/dd/d17/l25 0 2024-04-12T05:55:24.679 INFO:tasks.workunit.client.0.smithi008.stdout:2/242: dread d6/db/f2c [0,4194304] 0 2024-04-12T05:55:24.682 INFO:tasks.workunit.client.0.smithi008.stdout:2/243: unlink d6/db/d38/f27 0 2024-04-12T05:55:24.684 INFO:tasks.workunit.client.0.smithi008.stdout:5/207: dwrite d0/d2/f1b [0,4194304] 0 2024-04-12T05:55:24.689 INFO:tasks.workunit.client.0.smithi008.stdout:5/208: link d0/d2/f5 d0/dc/d13/d1a/d35/f42 0 2024-04-12T05:55:24.690 INFO:tasks.workunit.client.0.smithi008.stdout:2/244: rename d6/db/d10/d23/d35/l39 to d6/db/d10/l50 0 2024-04-12T05:55:24.691 INFO:tasks.workunit.client.0.smithi008.stdout:2/245: fsync d6/db/f4b 0 2024-04-12T05:55:24.692 INFO:tasks.workunit.client.0.smithi008.stdout:2/246: read d6/f16 [2138560,120917] 0 2024-04-12T05:55:24.786 INFO:tasks.workunit.client.0.smithi008.stdout:5/209: dwrite d0/dd/d11/d38/f2b [4194304,4194304] 0 2024-04-12T05:55:24.792 INFO:tasks.workunit.client.0.smithi008.stdout:2/247: write d6/db/f14 [2917109,53635] 0 2024-04-12T05:55:24.816 INFO:tasks.workunit.client.0.smithi008.stdout:0/233: sync 2024-04-12T05:55:24.816 INFO:tasks.workunit.client.0.smithi008.stdout:4/110: sync 2024-04-12T05:55:24.817 INFO:tasks.workunit.client.0.smithi008.stdout:2/248: mknod d6/db/d38/d41/c51 0 2024-04-12T05:55:24.817 INFO:tasks.workunit.client.0.smithi008.stdout:5/210: dwrite d0/dc/d26/f2f [0,4194304] 0 2024-04-12T05:55:24.819 INFO:tasks.workunit.client.0.smithi008.stdout:0/234: write d2/d11/f25 [225083,44424] 0 2024-04-12T05:55:24.822 INFO:tasks.workunit.client.0.smithi008.stdout:0/235: chown d2/d24/c30 1340 1 2024-04-12T05:55:24.826 INFO:tasks.workunit.client.0.smithi008.stdout:0/236: chown d2/d11/f18 9 1 2024-04-12T05:55:24.830 INFO:tasks.workunit.client.0.smithi008.stdout:0/237: stat d2/d11/d2a/f2d 0 2024-04-12T05:55:24.831 INFO:tasks.workunit.client.0.smithi008.stdout:5/211: dwrite d0/dd/f34 [0,4194304] 0 2024-04-12T05:55:24.831 INFO:tasks.workunit.client.0.smithi008.stdout:4/111: dwrite d1/db/f18 [0,4194304] 0 2024-04-12T05:55:24.832 INFO:tasks.workunit.client.0.smithi008.stdout:4/112: readlink d1/db/lf 0 2024-04-12T05:55:24.836 INFO:tasks.workunit.client.0.smithi008.stdout:4/113: write d1/fe [1866029,46371] 0 2024-04-12T05:55:24.836 INFO:tasks.workunit.client.0.smithi008.stdout:2/249: dread d6/ff [0,4194304] 0 2024-04-12T05:55:24.838 INFO:tasks.workunit.client.0.smithi008.stdout:0/238: write d2/d11/d2a/f49 [7558771,25937] 0 2024-04-12T05:55:24.840 INFO:tasks.workunit.client.0.smithi008.stdout:0/239: readlink d2/l4d 0 2024-04-12T05:55:24.840 INFO:tasks.workunit.client.0.smithi008.stdout:4/114: mknod d1/d8/c1f 0 2024-04-12T05:55:24.841 INFO:tasks.workunit.client.0.smithi008.stdout:2/250: mknod d6/db/d10/d4c/c52 0 2024-04-12T05:55:24.842 INFO:tasks.workunit.client.0.smithi008.stdout:5/212: dread d0/dd/d11/d38/f2b [4194304,4194304] 0 2024-04-12T05:55:24.842 INFO:tasks.workunit.client.0.smithi008.stdout:0/240: readlink d2/d11/d36/d47/l3b 0 2024-04-12T05:55:24.848 INFO:tasks.workunit.client.0.smithi008.stdout:5/213: getdents d0/dd/d3d 0 2024-04-12T05:55:24.849 INFO:tasks.workunit.client.0.smithi008.stdout:0/241: dread d2/fa [0,4194304] 0 2024-04-12T05:55:24.852 INFO:tasks.workunit.client.0.smithi008.stdout:4/115: dwrite d1/f12 [4194304,4194304] 0 2024-04-12T05:55:24.855 INFO:tasks.workunit.client.0.smithi008.stdout:4/116: write d1/f13 [605300,30455] 0 2024-04-12T05:55:24.855 INFO:tasks.workunit.client.0.smithi008.stdout:5/214: creat d0/dd/d11/d38/f43 x:0 0 0 2024-04-12T05:55:24.856 INFO:tasks.workunit.client.0.smithi008.stdout:2/251: dwrite d6/db/d29/f48 [0,4194304] 0 2024-04-12T05:55:24.859 INFO:tasks.workunit.client.0.smithi008.stdout:0/242: unlink d2/d11/d36/d47/l3b 0 2024-04-12T05:55:24.860 INFO:tasks.workunit.client.0.smithi008.stdout:2/252: fsync d6/f16 0 2024-04-12T05:55:24.860 INFO:tasks.workunit.client.0.smithi008.stdout:5/215: dread d0/dc/d26/f32 [4194304,4194304] 0 2024-04-12T05:55:24.862 INFO:tasks.workunit.client.0.smithi008.stdout:5/216: fsync d0/d2/f4 0 2024-04-12T05:55:24.863 INFO:tasks.workunit.client.0.smithi008.stdout:2/253: stat d6/db/fd 0 2024-04-12T05:55:24.865 INFO:tasks.workunit.client.0.smithi008.stdout:2/254: chown d6 29114730 1 2024-04-12T05:55:24.865 INFO:tasks.workunit.client.0.smithi008.stdout:0/243: rmdir d2/d11/d2a 39 2024-04-12T05:55:24.871 INFO:tasks.workunit.client.0.smithi008.stdout:0/244: dread d2/d11/f2e [0,4194304] 0 2024-04-12T05:55:24.871 INFO:tasks.workunit.client.0.smithi008.stdout:0/245: chown d2/ff 52448 1 2024-04-12T05:55:24.872 INFO:tasks.workunit.client.0.smithi008.stdout:0/246: fsync d2/d24/f28 0 2024-04-12T05:55:24.878 INFO:tasks.workunit.client.0.smithi008.stdout:5/217: dread d0/d2/f5 [0,4194304] 0 2024-04-12T05:55:24.878 INFO:tasks.workunit.client.0.smithi008.stdout:5/218: stat d0/dc/d13/d1a/f22 0 2024-04-12T05:55:24.879 INFO:tasks.workunit.client.0.smithi008.stdout:5/219: stat d0/d2/cb 0 2024-04-12T05:55:24.879 INFO:tasks.workunit.client.0.smithi008.stdout:5/220: dread - d0/dd/d11/d38/f43 zero size 2024-04-12T05:55:24.881 INFO:tasks.workunit.client.0.smithi008.stdout:5/221: read - d0/d2/f21 zero size 2024-04-12T05:55:24.881 INFO:tasks.workunit.client.0.smithi008.stdout:5/222: fsync d0/d2/f4 0 2024-04-12T05:55:24.883 INFO:tasks.workunit.client.0.smithi008.stdout:5/223: fsync d0/dd/d11/f19 0 2024-04-12T05:55:24.884 INFO:tasks.workunit.client.0.smithi008.stdout:0/247: dread d2/d24/f32 [0,4194304] 0 2024-04-12T05:55:24.885 INFO:tasks.workunit.client.0.smithi008.stdout:5/224: chown d0/dd/d11/d38/c33 8601 1 2024-04-12T05:55:24.887 INFO:tasks.workunit.client.0.smithi008.stdout:5/225: rmdir d0/dd/d11 39 2024-04-12T05:55:24.889 INFO:tasks.workunit.client.0.smithi008.stdout:0/248: symlink d2/l4e 0 2024-04-12T05:55:24.889 INFO:tasks.workunit.client.0.smithi008.stdout:5/226: read d0/dc/d13/f15 [3636725,64948] 0 2024-04-12T05:55:24.891 INFO:tasks.workunit.client.0.smithi008.stdout:0/249: creat d2/d11/d36/d47/f4f x:0 0 0 2024-04-12T05:55:24.896 INFO:tasks.workunit.client.0.smithi008.stdout:5/227: truncate d0/f12 1563844 0 2024-04-12T05:55:24.897 INFO:tasks.workunit.client.0.smithi008.stdout:5/228: readlink d0/dc/d26/l25 0 2024-04-12T05:55:24.897 INFO:tasks.workunit.client.0.smithi008.stdout:0/250: dwrite d2/d11/d36/d47/f21 [0,4194304] 0 2024-04-12T05:55:24.898 INFO:tasks.workunit.client.0.smithi008.stdout:5/229: dread - d0/d2/f21 zero size 2024-04-12T05:55:24.902 INFO:tasks.workunit.client.0.smithi008.stdout:0/251: mknod d2/d4b/c50 0 2024-04-12T05:55:24.902 INFO:tasks.workunit.client.0.smithi008.stdout:0/252: chown d2/l10 39 1 2024-04-12T05:55:24.903 INFO:tasks.workunit.client.0.smithi008.stdout:5/230: mkdir d0/d29/d44 0 2024-04-12T05:55:24.925 INFO:tasks.workunit.client.0.smithi008.stdout:5/231: dread d0/d2/f4 [4194304,4194304] 0 2024-04-12T05:55:24.938 INFO:tasks.workunit.client.0.smithi008.stdout:5/232: dwrite d0/d2/f21 [0,4194304] 0 2024-04-12T05:55:24.939 INFO:tasks.workunit.client.0.smithi008.stdout:5/233: chown d0/f12 407386 1 2024-04-12T05:55:24.944 INFO:tasks.workunit.client.0.smithi008.stdout:3/183: sync 2024-04-12T05:55:24.944 INFO:tasks.workunit.client.0.smithi008.stdout:7/229: sync 2024-04-12T05:55:24.944 INFO:tasks.workunit.client.0.smithi008.stdout:6/159: sync 2024-04-12T05:55:24.944 INFO:tasks.workunit.client.0.smithi008.stdout:6/160: stat fe 0 2024-04-12T05:55:24.945 INFO:tasks.workunit.client.0.smithi008.stdout:5/234: link d0/dc/d13/d1a/f22 d0/d2/d37/f45 0 2024-04-12T05:55:24.945 INFO:tasks.workunit.client.0.smithi008.stdout:7/230: chown d1/d5/f2e 1928 1 2024-04-12T05:55:24.946 INFO:tasks.workunit.client.0.smithi008.stdout:3/184: unlink d9/dd/c18 0 2024-04-12T05:55:24.947 INFO:tasks.workunit.client.0.smithi008.stdout:6/161: symlink df/l2f 0 2024-04-12T05:55:24.948 INFO:tasks.workunit.client.0.smithi008.stdout:5/235: symlink d0/d29/l46 0 2024-04-12T05:55:24.949 INFO:tasks.workunit.client.0.smithi008.stdout:6/162: symlink df/d10/d15/l30 0 2024-04-12T05:55:24.950 INFO:tasks.workunit.client.0.smithi008.stdout:7/231: creat d1/d15/d43/f4c x:0 0 0 2024-04-12T05:55:24.952 INFO:tasks.workunit.client.0.smithi008.stdout:5/236: creat d0/d29/d44/f47 x:0 0 0 2024-04-12T05:55:24.954 INFO:tasks.workunit.client.0.smithi008.stdout:6/163: link df/f12 df/f31 0 2024-04-12T05:55:24.956 INFO:tasks.workunit.client.0.smithi008.stdout:7/232: creat d1/d5/f4d x:0 0 0 2024-04-12T05:55:24.957 INFO:tasks.workunit.client.0.smithi008.stdout:6/164: symlink df/d21/l32 0 2024-04-12T05:55:24.957 INFO:tasks.workunit.client.0.smithi008.stdout:5/237: dread d0/d2/f21 [0,4194304] 0 2024-04-12T05:55:24.961 INFO:tasks.workunit.client.0.smithi008.stdout:5/238: creat d0/dc/d13/f48 x:0 0 0 2024-04-12T05:55:24.969 INFO:tasks.workunit.client.0.smithi008.stdout:6/165: getdents df 0 2024-04-12T05:55:24.970 INFO:tasks.workunit.client.0.smithi008.stdout:6/166: chown df/d21 10 1 2024-04-12T05:55:24.970 INFO:tasks.workunit.client.0.smithi008.stdout:5/239: truncate d0/dc/d26/f32 6919391 0 2024-04-12T05:55:24.972 INFO:tasks.workunit.client.0.smithi008.stdout:7/233: getdents d1/d5/da/d11/d47 0 2024-04-12T05:55:24.973 INFO:tasks.workunit.client.0.smithi008.stdout:7/234: dread - d1/d5/f33 zero size 2024-04-12T05:55:24.975 INFO:tasks.workunit.client.0.smithi008.stdout:5/240: dread d0/d2/f1b [0,4194304] 0 2024-04-12T05:55:24.977 INFO:tasks.workunit.client.0.smithi008.stdout:5/241: chown d0/dc/d26/l25 156809 1 2024-04-12T05:55:24.978 INFO:tasks.workunit.client.0.smithi008.stdout:6/167: write df/d10/d15/f13 [3216101,76318] 0 2024-04-12T05:55:24.981 INFO:tasks.workunit.client.0.smithi008.stdout:7/235: link d1/d5/f42 d1/d5/f4e 0 2024-04-12T05:55:24.981 INFO:tasks.workunit.client.0.smithi008.stdout:7/236: fsync d1/d5/f2e 0 2024-04-12T05:55:24.982 INFO:tasks.workunit.client.0.smithi008.stdout:5/242: mknod d0/dd/c49 0 2024-04-12T05:55:24.983 INFO:tasks.workunit.client.0.smithi008.stdout:7/237: dread - d1/d15/d43/f4c zero size 2024-04-12T05:55:24.983 INFO:tasks.workunit.client.0.smithi008.stdout:5/243: dread - d0/d29/d44/f47 zero size 2024-04-12T05:55:24.984 INFO:tasks.workunit.client.0.smithi008.stdout:5/244: write d0/dd/f34 [857462,28301] 0 2024-04-12T05:55:24.988 INFO:tasks.workunit.client.0.smithi008.stdout:7/238: link d1/d5/f32 d1/f4f 0 2024-04-12T05:55:24.992 INFO:tasks.workunit.client.0.smithi008.stdout:6/168: truncate df/f1f 1107880 0 2024-04-12T05:55:24.996 INFO:tasks.workunit.client.0.smithi008.stdout:6/169: read f4 [460107,51037] 0 2024-04-12T05:55:24.999 INFO:tasks.workunit.client.0.smithi008.stdout:7/239: dwrite d1/d5/f29 [0,4194304] 0 2024-04-12T05:55:25.003 INFO:tasks.workunit.client.0.smithi008.stdout:5/245: dwrite d0/dc/d13/d1a/f22 [0,4194304] 0 2024-04-12T05:55:25.008 INFO:tasks.workunit.client.0.smithi008.stdout:7/240: rename d1/d15/c41 to d1/d5/da/d11/d47/c50 0 2024-04-12T05:55:25.008 INFO:tasks.workunit.client.0.smithi008.stdout:5/246: truncate d0/dc/d13/d1a/f24 1318708 0 2024-04-12T05:55:25.012 INFO:tasks.workunit.client.0.smithi008.stdout:6/170: dread df/d21/f22 [0,4194304] 0 2024-04-12T05:55:25.014 INFO:tasks.workunit.client.0.smithi008.stdout:7/241: link d1/d5/da/d11/f35 d1/d5/da/d3c/d44/f51 0 2024-04-12T05:55:25.015 INFO:tasks.workunit.client.0.smithi008.stdout:5/247: link d0/dd/c49 d0/d40/c4a 0 2024-04-12T05:55:25.016 INFO:tasks.workunit.client.0.smithi008.stdout:7/242: write d1/d15/f26 [611313,3318] 0 2024-04-12T05:55:25.016 INFO:tasks.workunit.client.0.smithi008.stdout:6/171: symlink df/l33 0 2024-04-12T05:55:25.020 INFO:tasks.workunit.client.0.smithi008.stdout:7/243: creat d1/d5/da/d3c/d3d/f52 x:0 0 0 2024-04-12T05:55:25.022 INFO:tasks.workunit.client.0.smithi008.stdout:6/172: dread fd [0,4194304] 0 2024-04-12T05:55:25.022 INFO:tasks.workunit.client.0.smithi008.stdout:5/248: dread d0/d2/f1b [0,4194304] 0 2024-04-12T05:55:25.023 INFO:tasks.workunit.client.0.smithi008.stdout:6/173: write df/f12 [2791850,66164] 0 2024-04-12T05:55:25.027 INFO:tasks.workunit.client.0.smithi008.stdout:6/174: mknod df/d10/d20/c34 0 2024-04-12T05:55:25.030 INFO:tasks.workunit.client.0.smithi008.stdout:6/175: write df/f23 [332065,77065] 0 2024-04-12T05:55:25.034 INFO:tasks.workunit.client.0.smithi008.stdout:6/176: write df/d10/f2c [1769200,113999] 0 2024-04-12T05:55:25.035 INFO:tasks.workunit.client.0.smithi008.stdout:5/249: dwrite d0/d2/f4 [8388608,4194304] 0 2024-04-12T05:55:25.038 INFO:tasks.workunit.client.0.smithi008.stdout:6/177: symlink df/d10/l35 0 2024-04-12T05:55:25.039 INFO:tasks.workunit.client.0.smithi008.stdout:7/244: dwrite d1/d5/da/d3c/d44/f51 [0,4194304] 0 2024-04-12T05:55:25.040 INFO:tasks.workunit.client.0.smithi008.stdout:6/178: chown df/d10/d20/c28 0 1 2024-04-12T05:55:25.043 INFO:tasks.workunit.client.0.smithi008.stdout:7/245: creat d1/d49/f53 x:0 0 0 2024-04-12T05:55:25.047 INFO:tasks.workunit.client.0.smithi008.stdout:6/179: rename df/l29 to df/d21/l36 0 2024-04-12T05:55:25.053 INFO:tasks.workunit.client.0.smithi008.stdout:6/180: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:25.062 INFO:tasks.workunit.client.0.smithi008.stdout:7/246: write d1/f4f [335510,7623] 0 2024-04-12T05:55:25.069 INFO:tasks.workunit.client.0.smithi008.stdout:7/247: dwrite d1/d15/f20 [8388608,4194304] 0 2024-04-12T05:55:25.070 INFO:tasks.workunit.client.0.smithi008.stdout:5/250: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:25.070 INFO:tasks.workunit.client.0.smithi008.stdout:7/248: chown d1/l2 971193 1 2024-04-12T05:55:25.072 INFO:tasks.workunit.client.0.smithi008.stdout:7/249: write d1/d5/da/d3c/d3d/f3a [585651,36121] 0 2024-04-12T05:55:25.072 INFO:tasks.workunit.client.0.smithi008.stdout:9/190: sync 2024-04-12T05:55:25.072 INFO:tasks.workunit.client.0.smithi008.stdout:8/108: sync 2024-04-12T05:55:25.073 INFO:tasks.workunit.client.0.smithi008.stdout:1/136: sync 2024-04-12T05:55:25.074 INFO:tasks.workunit.client.0.smithi008.stdout:9/191: stat d3/fd 0 2024-04-12T05:55:25.075 INFO:tasks.workunit.client.0.smithi008.stdout:9/192: stat d3/df/d10/f17 0 2024-04-12T05:55:25.077 INFO:tasks.workunit.client.0.smithi008.stdout:8/109: mknod dd/de/c24 0 2024-04-12T05:55:25.078 INFO:tasks.workunit.client.0.smithi008.stdout:1/137: mkdir d4/dd/d17/d26 0 2024-04-12T05:55:25.088 INFO:tasks.workunit.client.0.smithi008.stdout:1/138: creat d4/f27 x:0 0 0 2024-04-12T05:55:25.088 INFO:tasks.workunit.client.0.smithi008.stdout:9/193: dread d3/df/d13/d21/d2d/d32/f34 [0,4194304] 0 2024-04-12T05:55:25.089 INFO:tasks.workunit.client.0.smithi008.stdout:9/194: chown d3 5515363 1 2024-04-12T05:55:25.092 INFO:tasks.workunit.client.0.smithi008.stdout:9/195: creat d3/f43 x:0 0 0 2024-04-12T05:55:25.093 INFO:tasks.workunit.client.0.smithi008.stdout:1/139: rename d4/d1a/d1d to d4/dd/d17/d28 0 2024-04-12T05:55:25.096 INFO:tasks.workunit.client.0.smithi008.stdout:7/250: dread d1/d5/f2e [0,4194304] 0 2024-04-12T05:55:25.097 INFO:tasks.workunit.client.0.smithi008.stdout:5/251: dread d0/dc/d13/d1a/f24 [0,4194304] 0 2024-04-12T05:55:25.097 INFO:tasks.workunit.client.0.smithi008.stdout:9/196: rename d3/df/d10/d12/f23 to d3/df/d13/d21/f44 0 2024-04-12T05:55:25.098 INFO:tasks.workunit.client.0.smithi008.stdout:1/140: mknod d4/d7/d18/c29 0 2024-04-12T05:55:25.100 INFO:tasks.workunit.client.0.smithi008.stdout:7/251: dread d1/f4f [0,4194304] 0 2024-04-12T05:55:25.101 INFO:tasks.workunit.client.0.smithi008.stdout:5/252: getdents d0/dd/d3d 0 2024-04-12T05:55:25.102 INFO:tasks.workunit.client.0.smithi008.stdout:9/197: getdents d3/df/d10/d12/d38 0 2024-04-12T05:55:25.103 INFO:tasks.workunit.client.0.smithi008.stdout:5/253: chown d0/dc/d26/f2f 29497133 1 2024-04-12T05:55:25.103 INFO:tasks.workunit.client.0.smithi008.stdout:7/252: mkdir d1/d15/d43/d54 0 2024-04-12T05:55:25.104 INFO:tasks.workunit.client.0.smithi008.stdout:5/254: readlink d0/dc/d26/l25 0 2024-04-12T05:55:25.104 INFO:tasks.workunit.client.0.smithi008.stdout:8/110: dread f7 [0,4194304] 0 2024-04-12T05:55:25.106 INFO:tasks.workunit.client.0.smithi008.stdout:9/198: mkdir d3/df/d10/d12/d45 0 2024-04-12T05:55:25.108 INFO:tasks.workunit.client.0.smithi008.stdout:5/255: creat d0/dd/d3d/f4b x:0 0 0 2024-04-12T05:55:25.109 INFO:tasks.workunit.client.0.smithi008.stdout:8/111: mknod dd/de/d16/c25 0 2024-04-12T05:55:25.111 INFO:tasks.workunit.client.0.smithi008.stdout:8/112: chown dd/de/l20 823835 1 2024-04-12T05:55:25.112 INFO:tasks.workunit.client.0.smithi008.stdout:5/256: creat d0/dc/d13/d1a/d35/f4c x:0 0 0 2024-04-12T05:55:25.114 INFO:tasks.workunit.client.0.smithi008.stdout:9/199: dwrite d3/f43 [0,4194304] 0 2024-04-12T05:55:25.116 INFO:tasks.workunit.client.0.smithi008.stdout:8/113: creat dd/f26 x:0 0 0 2024-04-12T05:55:25.116 INFO:tasks.workunit.client.0.smithi008.stdout:1/141: truncate d4/d6/f10 2490549 0 2024-04-12T05:55:25.118 INFO:tasks.workunit.client.0.smithi008.stdout:5/257: write d0/dc/d26/f32 [2914936,53884] 0 2024-04-12T05:55:25.121 INFO:tasks.workunit.client.0.smithi008.stdout:1/142: dread - d4/d6/d11/f23 zero size 2024-04-12T05:55:25.126 INFO:tasks.workunit.client.0.smithi008.stdout:5/258: rename d0/dc/l10 to d0/d40/l4d 0 2024-04-12T05:55:25.126 INFO:tasks.workunit.client.0.smithi008.stdout:9/200: rename d3/df/d13/d21/d2d/d32 to d3/df/d13/d21/d2d/d32/d46 22 2024-04-12T05:55:25.127 INFO:tasks.workunit.client.0.smithi008.stdout:1/143: mknod d4/dd/d1b/c2a 0 2024-04-12T05:55:25.127 INFO:tasks.workunit.client.0.smithi008.stdout:8/114: dwrite dd/f1f [0,4194304] 0 2024-04-12T05:55:25.131 INFO:tasks.workunit.client.0.smithi008.stdout:5/259: fsync d0/dd/d11/f19 0 2024-04-12T05:55:25.134 INFO:tasks.workunit.client.0.smithi008.stdout:5/260: write d0/dc/d13/d1a/f3f [647449,45768] 0 2024-04-12T05:55:25.134 INFO:tasks.workunit.client.0.smithi008.stdout:8/115: creat dd/de/f27 x:0 0 0 2024-04-12T05:55:25.142 INFO:tasks.workunit.client.0.smithi008.stdout:9/201: getdents d3/df 0 2024-04-12T05:55:25.143 INFO:tasks.workunit.client.0.smithi008.stdout:9/202: read - d3/d22/d2b/f33 zero size 2024-04-12T05:55:25.144 INFO:tasks.workunit.client.0.smithi008.stdout:8/116: creat dd/de/d16/f28 x:0 0 0 2024-04-12T05:55:25.144 INFO:tasks.workunit.client.0.smithi008.stdout:8/117: readlink dd/de/l20 0 2024-04-12T05:55:25.146 INFO:tasks.workunit.client.0.smithi008.stdout:5/261: mknod d0/dc/c4e 0 2024-04-12T05:55:25.147 INFO:tasks.workunit.client.0.smithi008.stdout:9/203: creat d3/df/d13/d21/d2d/f47 x:0 0 0 2024-04-12T05:55:25.152 INFO:tasks.workunit.client.0.smithi008.stdout:9/204: truncate d3/fa 1021118 0 2024-04-12T05:55:25.152 INFO:tasks.workunit.client.0.smithi008.stdout:5/262: dread d0/d2/f31 [0,4194304] 0 2024-04-12T05:55:25.156 INFO:tasks.workunit.client.0.smithi008.stdout:9/205: rmdir d3/df/d10/d12 39 2024-04-12T05:55:25.158 INFO:tasks.workunit.client.0.smithi008.stdout:1/144: dwrite d4/d7/ff [0,4194304] 0 2024-04-12T05:55:25.160 INFO:tasks.workunit.client.0.smithi008.stdout:9/206: fdatasync d3/d22/d2b/f33 0 2024-04-12T05:55:25.160 INFO:tasks.workunit.client.0.smithi008.stdout:8/118: dwrite dd/de/d16/f28 [0,4194304] 0 2024-04-12T05:55:25.161 INFO:tasks.workunit.client.0.smithi008.stdout:9/207: fsync d3/df/d13/d21/f31 0 2024-04-12T05:55:25.164 INFO:tasks.workunit.client.0.smithi008.stdout:5/263: creat d0/dc/f4f x:0 0 0 2024-04-12T05:55:25.164 INFO:tasks.workunit.client.0.smithi008.stdout:1/145: unlink d4/dd/d17/l25 0 2024-04-12T05:55:25.166 INFO:tasks.workunit.client.0.smithi008.stdout:8/119: creat dd/de/f29 x:0 0 0 2024-04-12T05:55:25.166 INFO:tasks.workunit.client.0.smithi008.stdout:8/120: chown dd/de/d16 253263 1 2024-04-12T05:55:25.168 INFO:tasks.workunit.client.0.smithi008.stdout:1/146: write d4/dd/d17/f19 [3922174,102728] 0 2024-04-12T05:55:25.170 INFO:tasks.workunit.client.0.smithi008.stdout:8/121: unlink dd/f26 0 2024-04-12T05:55:25.171 INFO:tasks.workunit.client.0.smithi008.stdout:9/208: dread d3/df/d10/d12/f1e [0,4194304] 0 2024-04-12T05:55:25.174 INFO:tasks.workunit.client.0.smithi008.stdout:1/147: creat d4/d1c/f2b x:0 0 0 2024-04-12T05:55:25.176 INFO:tasks.workunit.client.0.smithi008.stdout:5/264: link d0/d29/l46 d0/dd/l50 0 2024-04-12T05:55:25.178 INFO:tasks.workunit.client.0.smithi008.stdout:9/209: dwrite d3/f25 [0,4194304] 0 2024-04-12T05:55:25.180 INFO:tasks.workunit.client.0.smithi008.stdout:1/148: getdents d4/d7/d18 0 2024-04-12T05:55:25.234 INFO:tasks.workunit.client.0.smithi008.stdout:9/210: creat d3/df/d10/d12/f48 x:0 0 0 2024-04-12T05:55:25.235 INFO:tasks.workunit.client.0.smithi008.stdout:9/211: chown d3/f5 10 1 2024-04-12T05:55:25.238 INFO:tasks.workunit.client.0.smithi008.stdout:1/149: dwrite d4/d1c/f2b [0,4194304] 0 2024-04-12T05:55:25.241 INFO:tasks.workunit.client.0.smithi008.stdout:9/212: creat d3/df/d10/d12/d36/f49 x:0 0 0 2024-04-12T05:55:25.241 INFO:tasks.workunit.client.0.smithi008.stdout:1/150: mkdir d4/d6/d2c 0 2024-04-12T05:55:25.244 INFO:tasks.workunit.client.0.smithi008.stdout:5/265: dwrite d0/dd/f1e [4194304,4194304] 0 2024-04-12T05:55:25.248 INFO:tasks.workunit.client.0.smithi008.stdout:5/266: mknod d0/d29/d44/c51 0 2024-04-12T05:55:25.251 INFO:tasks.workunit.client.0.smithi008.stdout:9/213: chown d3/df/d13/c16 35179794 1 2024-04-12T05:55:25.251 INFO:tasks.workunit.client.0.smithi008.stdout:1/151: link d4/d6/c15 d4/dd/d17/c2d 0 2024-04-12T05:55:25.254 INFO:tasks.workunit.client.0.smithi008.stdout:5/267: fsync d0/dc/d26/f32 0 2024-04-12T05:55:25.255 INFO:tasks.workunit.client.0.smithi008.stdout:1/152: read d4/d7/fe [2853359,7508] 0 2024-04-12T05:55:25.255 INFO:tasks.workunit.client.0.smithi008.stdout:5/268: chown d0/dc/d13/d1a/f3f 6557 1 2024-04-12T05:55:25.255 INFO:tasks.workunit.client.0.smithi008.stdout:1/153: dread - d4/d6/d11/f23 zero size 2024-04-12T05:55:25.255 INFO:tasks.workunit.client.0.smithi008.stdout:1/154: chown d4/dd/d1b 27716 1 2024-04-12T05:55:25.256 INFO:tasks.workunit.client.0.smithi008.stdout:9/214: rmdir d3/df/d13/d21/d2d/d32 39 2024-04-12T05:55:25.258 INFO:tasks.workunit.client.0.smithi008.stdout:5/269: write d0/dc/d13/d1a/d35/f42 [3778974,77437] 0 2024-04-12T05:55:25.261 INFO:tasks.workunit.client.0.smithi008.stdout:1/155: unlink d4/d6/c15 0 2024-04-12T05:55:25.263 INFO:tasks.workunit.client.0.smithi008.stdout:5/270: fdatasync d0/d2/f1b 0 2024-04-12T05:55:25.264 INFO:tasks.workunit.client.0.smithi008.stdout:5/271: chown d0/d2/d37 0 1 2024-04-12T05:55:25.269 INFO:tasks.workunit.client.0.smithi008.stdout:1/156: creat d4/dd/d17/d26/f2e x:0 0 0 2024-04-12T05:55:25.269 INFO:tasks.workunit.client.0.smithi008.stdout:1/157: rename d4/d7 to d4/d7/d2f 22 2024-04-12T05:55:25.273 INFO:tasks.workunit.client.0.smithi008.stdout:1/158: write d4/d7/fe [351787,52986] 0 2024-04-12T05:55:25.274 INFO:tasks.workunit.client.0.smithi008.stdout:5/272: link d0/dc/d13/d1a/f3f d0/dc/f52 0 2024-04-12T05:55:25.281 INFO:tasks.workunit.client.0.smithi008.stdout:5/273: dwrite d0/dc/d13/d1a/d35/f4c [0,4194304] 0 2024-04-12T05:55:25.283 INFO:tasks.workunit.client.0.smithi008.stdout:5/274: truncate d0/dc/d26/f2f 4219063 0 2024-04-12T05:55:25.292 INFO:tasks.workunit.client.0.smithi008.stdout:1/159: mkdir d4/dd/d17/d28/d30 0 2024-04-12T05:55:25.294 INFO:tasks.workunit.client.0.smithi008.stdout:5/275: link d0/d2/f31 d0/dd/d3d/f53 0 2024-04-12T05:55:25.295 INFO:tasks.workunit.client.0.smithi008.stdout:9/215: dread d3/df/f11 [0,4194304] 0 2024-04-12T05:55:25.297 INFO:tasks.workunit.client.0.smithi008.stdout:9/216: mkdir d3/d22/d2b/d4a 0 2024-04-12T05:55:25.299 INFO:tasks.workunit.client.0.smithi008.stdout:9/217: write d3/df/d10/d12/f3f [867070,14217] 0 2024-04-12T05:55:25.300 INFO:tasks.workunit.client.0.smithi008.stdout:1/160: creat d4/dd/d17/f31 x:0 0 0 2024-04-12T05:55:25.301 INFO:tasks.workunit.client.0.smithi008.stdout:9/218: creat d3/f4b x:0 0 0 2024-04-12T05:55:25.301 INFO:tasks.workunit.client.0.smithi008.stdout:4/117: sync 2024-04-12T05:55:25.301 INFO:tasks.workunit.client.0.smithi008.stdout:2/255: sync 2024-04-12T05:55:25.305 INFO:tasks.workunit.client.0.smithi008.stdout:4/118: chown d1/l7 3085156 1 2024-04-12T05:55:25.307 INFO:tasks.workunit.client.0.smithi008.stdout:1/161: link d4/d1c/f2b d4/d24/f32 0 2024-04-12T05:55:25.308 INFO:tasks.workunit.client.0.smithi008.stdout:5/276: dwrite d0/dc/f52 [0,4194304] 0 2024-04-12T05:55:25.308 INFO:tasks.workunit.client.0.smithi008.stdout:1/162: readlink d4/l14 0 2024-04-12T05:55:25.309 INFO:tasks.workunit.client.0.smithi008.stdout:1/163: stat d4/d6/f8 0 2024-04-12T05:55:25.309 INFO:tasks.workunit.client.0.smithi008.stdout:2/256: mknod d6/db/d10/d23/d2e/c53 0 2024-04-12T05:55:25.314 INFO:tasks.workunit.client.0.smithi008.stdout:1/164: mkdir d4/d33 0 2024-04-12T05:55:25.317 INFO:tasks.workunit.client.0.smithi008.stdout:5/277: link d0/d2/f5 d0/dc/d13/f54 0 2024-04-12T05:55:25.318 INFO:tasks.workunit.client.0.smithi008.stdout:2/257: mknod d6/db/d10/d23/d35/d3c/c54 0 2024-04-12T05:55:25.319 INFO:tasks.workunit.client.0.smithi008.stdout:2/258: write d6/db/f4b [563405,125065] 0 2024-04-12T05:55:25.321 INFO:tasks.workunit.client.0.smithi008.stdout:5/278: mkdir d0/dd/d11/d38/d55 0 2024-04-12T05:55:25.323 INFO:tasks.workunit.client.0.smithi008.stdout:9/219: truncate d3/df/f39 252812 0 2024-04-12T05:55:25.325 INFO:tasks.workunit.client.0.smithi008.stdout:2/259: rename d6/db/d10/f2b to d6/db/d10/f55 0 2024-04-12T05:55:25.326 INFO:tasks.workunit.client.0.smithi008.stdout:5/279: mkdir d0/d40/d56 0 2024-04-12T05:55:25.327 INFO:tasks.workunit.client.0.smithi008.stdout:5/280: readlink d0/dc/d13/d1a/l39 0 2024-04-12T05:55:25.330 INFO:tasks.workunit.client.0.smithi008.stdout:1/165: truncate d4/d7/fe 502832 0 2024-04-12T05:55:25.332 INFO:tasks.workunit.client.0.smithi008.stdout:9/220: link d3/df/d10/f17 d3/df/d10/d12/d38/f4c 0 2024-04-12T05:55:25.334 INFO:tasks.workunit.client.0.smithi008.stdout:1/166: symlink d4/d7/d18/l34 0 2024-04-12T05:55:25.334 INFO:tasks.workunit.client.0.smithi008.stdout:9/221: write d3/d22/f3e [460862,80994] 0 2024-04-12T05:55:25.335 INFO:tasks.workunit.client.0.smithi008.stdout:2/260: creat d6/db/f56 x:0 0 0 2024-04-12T05:55:25.337 INFO:tasks.workunit.client.0.smithi008.stdout:5/281: rename d0/dc/d26 to d0/dc/d57 0 2024-04-12T05:55:25.340 INFO:tasks.workunit.client.0.smithi008.stdout:1/167: fdatasync d4/f9 0 2024-04-12T05:55:25.340 INFO:tasks.workunit.client.0.smithi008.stdout:2/261: mkdir d6/db/d10/d23/d2e/d57 0 2024-04-12T05:55:25.340 INFO:tasks.workunit.client.0.smithi008.stdout:1/168: stat f3 0 2024-04-12T05:55:25.341 INFO:tasks.workunit.client.0.smithi008.stdout:5/282: mknod d0/d40/c58 0 2024-04-12T05:55:25.345 INFO:tasks.workunit.client.0.smithi008.stdout:2/262: unlink d6/l8 0 2024-04-12T05:55:25.346 INFO:tasks.workunit.client.0.smithi008.stdout:9/222: dwrite d3/f43 [0,4194304] 0 2024-04-12T05:55:25.349 INFO:tasks.workunit.client.0.smithi008.stdout:5/283: creat d0/dd/d11/d38/f59 x:0 0 0 2024-04-12T05:55:25.356 INFO:tasks.workunit.client.0.smithi008.stdout:5/284: creat d0/dd/d11/f5a x:0 0 0 2024-04-12T05:55:25.358 INFO:tasks.workunit.client.0.smithi008.stdout:1/169: rmdir d4/d6/d2c 0 2024-04-12T05:55:25.359 INFO:tasks.workunit.client.0.smithi008.stdout:5/285: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:25.362 INFO:tasks.workunit.client.0.smithi008.stdout:1/170: write d4/d7/fc [4316881,57709] 0 2024-04-12T05:55:25.363 INFO:tasks.workunit.client.0.smithi008.stdout:9/223: getdents d3/d22/d2b 0 2024-04-12T05:55:25.366 INFO:tasks.workunit.client.0.smithi008.stdout:2/263: dread d6/db/d10/d23/d2e/f42 [0,4194304] 0 2024-04-12T05:55:25.370 INFO:tasks.workunit.client.0.smithi008.stdout:2/264: readlink d6/db/d38/l1d 0 2024-04-12T05:55:25.371 INFO:tasks.workunit.client.0.smithi008.stdout:9/224: creat d3/df/d13/d21/d2d/d30/f4d x:0 0 0 2024-04-12T05:55:25.372 INFO:tasks.workunit.client.0.smithi008.stdout:9/225: chown d3/df/d10/f17 8554 1 2024-04-12T05:55:25.375 INFO:tasks.workunit.client.0.smithi008.stdout:1/171: creat d4/dd/d17/d26/f35 x:0 0 0 2024-04-12T05:55:25.376 INFO:tasks.workunit.client.0.smithi008.stdout:5/286: getdents d0/dd/d11 0 2024-04-12T05:55:25.377 INFO:tasks.workunit.client.0.smithi008.stdout:1/172: chown d4/d7/ff 4999916 1 2024-04-12T05:55:25.377 INFO:tasks.workunit.client.0.smithi008.stdout:9/226: mkdir d3/d22/d2b/d4e 0 2024-04-12T05:55:25.377 INFO:tasks.workunit.client.0.smithi008.stdout:2/265: dwrite d6/f16 [0,4194304] 0 2024-04-12T05:55:25.380 INFO:tasks.workunit.client.0.smithi008.stdout:5/287: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:25.381 INFO:tasks.workunit.client.0.smithi008.stdout:1/173: truncate d4/dd/d17/d26/f35 796716 0 2024-04-12T05:55:25.384 INFO:tasks.workunit.client.0.smithi008.stdout:2/266: mknod d6/db/d29/c58 0 2024-04-12T05:55:25.385 INFO:tasks.workunit.client.0.smithi008.stdout:1/174: creat d4/dd/d17/d26/f36 x:0 0 0 2024-04-12T05:55:25.387 INFO:tasks.workunit.client.0.smithi008.stdout:5/288: creat d0/dd/d11/d38/d55/f5b x:0 0 0 2024-04-12T05:55:25.388 INFO:tasks.workunit.client.0.smithi008.stdout:2/267: creat d6/db/d10/d23/d2e/f59 x:0 0 0 2024-04-12T05:55:25.388 INFO:tasks.workunit.client.0.smithi008.stdout:5/289: dread d0/d2/f31 [0,4194304] 0 2024-04-12T05:55:25.389 INFO:tasks.workunit.client.0.smithi008.stdout:2/268: dread - d6/db/f56 zero size 2024-04-12T05:55:25.390 INFO:tasks.workunit.client.0.smithi008.stdout:5/290: dread - d0/dd/d3d/f4b zero size 2024-04-12T05:55:25.391 INFO:tasks.workunit.client.0.smithi008.stdout:1/175: mkdir d4/d33/d37 0 2024-04-12T05:55:25.391 INFO:tasks.workunit.client.0.smithi008.stdout:9/227: write d3/df/d10/d12/d38/f4c [1892464,46304] 0 2024-04-12T05:55:25.397 INFO:tasks.workunit.client.0.smithi008.stdout:2/269: fsync d6/db/d38/f25 0 2024-04-12T05:55:25.398 INFO:tasks.workunit.client.0.smithi008.stdout:1/176: write d4/dd/d17/f19 [3507555,45742] 0 2024-04-12T05:55:25.401 INFO:tasks.workunit.client.0.smithi008.stdout:5/291: dwrite d0/dc/d13/d1a/d35/f4c [0,4194304] 0 2024-04-12T05:55:25.404 INFO:tasks.workunit.client.0.smithi008.stdout:9/228: link d3/d22/d2b/c3a d3/c4f 0 2024-04-12T05:55:25.405 INFO:tasks.workunit.client.0.smithi008.stdout:2/270: dread d6/db/d38/f25 [0,4194304] 0 2024-04-12T05:55:25.410 INFO:tasks.workunit.client.0.smithi008.stdout:2/271: creat d6/db/d10/d4c/f5a x:0 0 0 2024-04-12T05:55:25.411 INFO:tasks.workunit.client.0.smithi008.stdout:1/177: write d4/d1c/f2b [3388837,7526] 0 2024-04-12T05:55:25.414 INFO:tasks.workunit.client.0.smithi008.stdout:9/229: rename d3/d22/c37 to d3/df/d10/c50 0 2024-04-12T05:55:25.415 INFO:tasks.workunit.client.0.smithi008.stdout:2/272: creat d6/db/d38/f5b x:0 0 0 2024-04-12T05:55:25.416 INFO:tasks.workunit.client.0.smithi008.stdout:5/292: getdents d0/dc/d13/d1a/d35 0 2024-04-12T05:55:25.418 INFO:tasks.workunit.client.0.smithi008.stdout:1/178: dread f3 [0,4194304] 0 2024-04-12T05:55:25.421 INFO:tasks.workunit.client.0.smithi008.stdout:5/293: mkdir d0/dd/d5c 0 2024-04-12T05:55:25.423 INFO:tasks.workunit.client.0.smithi008.stdout:1/179: truncate d4/dd/d17/f1f 579542 0 2024-04-12T05:55:25.423 INFO:tasks.workunit.client.0.smithi008.stdout:5/294: mkdir d0/dd/d3d/d5d 0 2024-04-12T05:55:25.424 INFO:tasks.workunit.client.0.smithi008.stdout:5/295: chown d0/dc/f52 644 1 2024-04-12T05:55:25.425 INFO:tasks.workunit.client.0.smithi008.stdout:9/230: rename d3/df/f39 to d3/df/d13/f51 0 2024-04-12T05:55:25.425 INFO:tasks.workunit.client.0.smithi008.stdout:9/231: stat d3/d22/d2b 0 2024-04-12T05:55:25.425 INFO:tasks.workunit.client.0.smithi008.stdout:1/180: symlink d4/dd/l38 0 2024-04-12T05:55:25.427 INFO:tasks.workunit.client.0.smithi008.stdout:5/296: rename d0/dc/d57/f32 to d0/d40/f5e 0 2024-04-12T05:55:25.428 INFO:tasks.workunit.client.0.smithi008.stdout:1/181: creat d4/dd/f39 x:0 0 0 2024-04-12T05:55:25.429 INFO:tasks.workunit.client.0.smithi008.stdout:2/273: truncate d6/ff 1579267 0 2024-04-12T05:55:25.527 INFO:tasks.workunit.client.0.smithi008.stdout:0/253: sync 2024-04-12T05:55:25.527 INFO:tasks.workunit.client.0.smithi008.stdout:3/185: sync 2024-04-12T05:55:25.527 INFO:tasks.workunit.client.0.smithi008.stdout:6/181: sync 2024-04-12T05:55:25.528 INFO:tasks.workunit.client.0.smithi008.stdout:5/297: unlink d0/dc/c4e 0 2024-04-12T05:55:25.528 INFO:tasks.workunit.client.0.smithi008.stdout:1/182: fsync f3 0 2024-04-12T05:55:25.529 INFO:tasks.workunit.client.0.smithi008.stdout:3/186: dread - d9/d2e/f3f zero size 2024-04-12T05:55:25.530 INFO:tasks.workunit.client.0.smithi008.stdout:6/182: write df/f23 [4392247,89276] 0 2024-04-12T05:55:25.533 INFO:tasks.workunit.client.0.smithi008.stdout:1/183: symlink d4/d24/l3a 0 2024-04-12T05:55:25.533 INFO:tasks.workunit.client.0.smithi008.stdout:0/254: dread d2/d11/f18 [0,4194304] 0 2024-04-12T05:55:25.535 INFO:tasks.workunit.client.0.smithi008.stdout:6/183: write df/d10/d15/f1c [1036715,28904] 0 2024-04-12T05:55:25.535 INFO:tasks.workunit.client.0.smithi008.stdout:2/274: truncate d6/db/d10/d23/d35/f49 20279 0 2024-04-12T05:55:25.536 INFO:tasks.workunit.client.0.smithi008.stdout:1/184: write d4/f27 [37022,41244] 0 2024-04-12T05:55:25.539 INFO:tasks.workunit.client.0.smithi008.stdout:6/184: symlink df/d10/d20/l37 0 2024-04-12T05:55:25.540 INFO:tasks.workunit.client.0.smithi008.stdout:1/185: symlink d4/dd/d17/d26/l3b 0 2024-04-12T05:55:25.542 INFO:tasks.workunit.client.0.smithi008.stdout:6/185: fdatasync f4 0 2024-04-12T05:55:25.544 INFO:tasks.workunit.client.0.smithi008.stdout:6/186: fdatasync df/d10/d15/f1c 0 2024-04-12T05:55:25.546 INFO:tasks.workunit.client.0.smithi008.stdout:3/187: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:25.549 INFO:tasks.workunit.client.0.smithi008.stdout:1/186: rename d4/d6/d11/f23 to d4/d24/f3c 0 2024-04-12T05:55:25.556 INFO:tasks.workunit.client.0.smithi008.stdout:5/298: truncate d0/d2/d37/f45 6106284 0 2024-04-12T05:55:25.558 INFO:tasks.workunit.client.0.smithi008.stdout:1/187: mknod d4/dd/d17/d28/d30/c3d 0 2024-04-12T05:55:25.560 INFO:tasks.workunit.client.0.smithi008.stdout:5/299: symlink d0/l5f 0 2024-04-12T05:55:25.561 INFO:tasks.workunit.client.0.smithi008.stdout:1/188: dread d4/dd/d17/d26/f35 [0,4194304] 0 2024-04-12T05:55:25.569 INFO:tasks.workunit.client.0.smithi008.stdout:5/300: dwrite d0/d2/d37/f3e [0,4194304] 0 2024-04-12T05:55:25.569 INFO:tasks.workunit.client.0.smithi008.stdout:3/188: dread d9/f1c [0,4194304] 0 2024-04-12T05:55:25.570 INFO:tasks.workunit.client.0.smithi008.stdout:3/189: fdatasync f6 0 2024-04-12T05:55:25.580 INFO:tasks.workunit.client.0.smithi008.stdout:5/301: dwrite d0/d2/f4 [4194304,4194304] 0 2024-04-12T05:55:25.581 INFO:tasks.workunit.client.0.smithi008.stdout:1/189: write d4/d6/f10 [1068718,58375] 0 2024-04-12T05:55:25.583 INFO:tasks.workunit.client.0.smithi008.stdout:3/190: mknod d9/c40 0 2024-04-12T05:55:25.585 INFO:tasks.workunit.client.0.smithi008.stdout:5/302: mknod d0/dd/d11/c60 0 2024-04-12T05:55:25.587 INFO:tasks.workunit.client.0.smithi008.stdout:6/187: dread fb [0,4194304] 0 2024-04-12T05:55:25.588 INFO:tasks.workunit.client.0.smithi008.stdout:1/190: creat d4/f3e x:0 0 0 2024-04-12T05:55:25.592 INFO:tasks.workunit.client.0.smithi008.stdout:6/188: stat df/d10/d15/f24 0 2024-04-12T05:55:25.592 INFO:tasks.workunit.client.0.smithi008.stdout:1/191: stat d4/dd/d17/d26/f35 0 2024-04-12T05:55:25.593 INFO:tasks.workunit.client.0.smithi008.stdout:5/303: rename d0/d29/d44/c51 to d0/d2/d37/c61 0 2024-04-12T05:55:25.594 INFO:tasks.workunit.client.0.smithi008.stdout:6/189: chown df/d10/d20 16935681 1 2024-04-12T05:55:25.595 INFO:tasks.workunit.client.0.smithi008.stdout:6/190: chown f5 38860885 1 2024-04-12T05:55:25.598 INFO:tasks.workunit.client.0.smithi008.stdout:3/191: dwrite d9/dd/f2f [0,4194304] 0 2024-04-12T05:55:25.598 INFO:tasks.workunit.client.0.smithi008.stdout:6/191: write df/f2e [742414,25563] 0 2024-04-12T05:55:25.604 INFO:tasks.workunit.client.0.smithi008.stdout:5/304: fsync d0/dc/d13/d1a/f24 0 2024-04-12T05:55:25.605 INFO:tasks.workunit.client.0.smithi008.stdout:3/192: symlink d9/d11/l41 0 2024-04-12T05:55:25.607 INFO:tasks.workunit.client.0.smithi008.stdout:5/305: stat d0/d2/d37 0 2024-04-12T05:55:25.610 INFO:tasks.workunit.client.0.smithi008.stdout:3/193: read - d9/d10/d19/d22/f32 zero size 2024-04-12T05:55:25.610 INFO:tasks.workunit.client.0.smithi008.stdout:1/192: dwrite d4/dd/d17/d26/f2e [0,4194304] 0 2024-04-12T05:55:25.610 INFO:tasks.workunit.client.0.smithi008.stdout:5/306: stat d0/d29/d44 0 2024-04-12T05:55:25.613 INFO:tasks.workunit.client.0.smithi008.stdout:3/194: write f6 [3557393,4385] 0 2024-04-12T05:55:25.615 INFO:tasks.workunit.client.0.smithi008.stdout:5/307: symlink d0/dd/d11/d38/l62 0 2024-04-12T05:55:25.616 INFO:tasks.workunit.client.0.smithi008.stdout:6/192: dwrite fd [0,4194304] 0 2024-04-12T05:55:25.617 INFO:tasks.workunit.client.0.smithi008.stdout:1/193: mknod d4/dd/d21/c3f 0 2024-04-12T05:55:25.619 INFO:tasks.workunit.client.0.smithi008.stdout:1/194: write d4/d6/f10 [1527942,88467] 0 2024-04-12T05:55:25.621 INFO:tasks.workunit.client.0.smithi008.stdout:5/308: read - d0/dd/d11/d38/d55/f5b zero size 2024-04-12T05:55:25.621 INFO:tasks.workunit.client.0.smithi008.stdout:3/195: fsync d9/dd/f36 0 2024-04-12T05:55:25.621 INFO:tasks.workunit.client.0.smithi008.stdout:6/193: dread df/d21/f22 [0,4194304] 0 2024-04-12T05:55:25.622 INFO:tasks.workunit.client.0.smithi008.stdout:5/309: chown d0/dd/d11 0 1 2024-04-12T05:55:25.623 INFO:tasks.workunit.client.0.smithi008.stdout:3/196: write f6 [424248,77501] 0 2024-04-12T05:55:25.624 INFO:tasks.workunit.client.0.smithi008.stdout:5/310: readlink d0/d40/l4d 0 2024-04-12T05:55:25.625 INFO:tasks.workunit.client.0.smithi008.stdout:3/197: chown d9/d11/l27 5563 1 2024-04-12T05:55:25.625 INFO:tasks.workunit.client.0.smithi008.stdout:5/311: readlink d0/dc/d13/d1a/l39 0 2024-04-12T05:55:25.628 INFO:tasks.workunit.client.0.smithi008.stdout:6/194: rename df/d21/l32 to df/d10/d15/l38 0 2024-04-12T05:55:25.631 INFO:tasks.workunit.client.0.smithi008.stdout:6/195: fsync fb 0 2024-04-12T05:55:25.632 INFO:tasks.workunit.client.0.smithi008.stdout:1/195: getdents d4/d7/d18 0 2024-04-12T05:55:25.633 INFO:tasks.workunit.client.0.smithi008.stdout:5/312: creat d0/dc/d13/d1a/d3a/f63 x:0 0 0 2024-04-12T05:55:25.634 INFO:tasks.workunit.client.0.smithi008.stdout:3/198: dread d9/f25 [0,4194304] 0 2024-04-12T05:55:25.638 INFO:tasks.workunit.client.0.smithi008.stdout:1/196: fdatasync d4/dd/d17/d26/f35 0 2024-04-12T05:55:25.639 INFO:tasks.workunit.client.0.smithi008.stdout:1/197: dread - d4/d24/f3c zero size 2024-04-12T05:55:25.641 INFO:tasks.workunit.client.0.smithi008.stdout:6/196: link df/f23 df/d10/f39 0 2024-04-12T05:55:25.642 INFO:tasks.workunit.client.0.smithi008.stdout:1/198: write d4/d24/f32 [1779283,93029] 0 2024-04-12T05:55:25.643 INFO:tasks.workunit.client.0.smithi008.stdout:5/313: creat d0/dc/d13/d1a/d35/f64 x:0 0 0 2024-04-12T05:55:25.645 INFO:tasks.workunit.client.0.smithi008.stdout:1/199: chown d4/d7/d18/l34 1171870421 1 2024-04-12T05:55:25.647 INFO:tasks.workunit.client.0.smithi008.stdout:3/199: dwrite d9/dd/f2f [0,4194304] 0 2024-04-12T05:55:25.650 INFO:tasks.workunit.client.0.smithi008.stdout:3/200: dread - d9/d10/d19/d22/f32 zero size 2024-04-12T05:55:25.651 INFO:tasks.workunit.client.0.smithi008.stdout:1/200: dread - d4/dd/d17/d26/f36 zero size 2024-04-12T05:55:25.651 INFO:tasks.workunit.client.0.smithi008.stdout:8/122: sync 2024-04-12T05:55:25.652 INFO:tasks.workunit.client.0.smithi008.stdout:4/119: sync 2024-04-12T05:55:25.652 INFO:tasks.workunit.client.0.smithi008.stdout:7/253: sync 2024-04-12T05:55:25.652 INFO:tasks.workunit.client.0.smithi008.stdout:6/197: mkdir df/d10/d3a 0 2024-04-12T05:55:25.652 INFO:tasks.workunit.client.0.smithi008.stdout:3/201: fdatasync f6 0 2024-04-12T05:55:25.660 INFO:tasks.workunit.client.0.smithi008.stdout:4/120: stat d1/d8/l14 0 2024-04-12T05:55:25.660 INFO:tasks.workunit.client.0.smithi008.stdout:6/198: symlink df/d21/l3b 0 2024-04-12T05:55:25.660 INFO:tasks.workunit.client.0.smithi008.stdout:4/121: chown d1/d8/l14 2033962 1 2024-04-12T05:55:25.660 INFO:tasks.workunit.client.0.smithi008.stdout:5/314: dwrite d0/dc/d13/d1a/f3f [0,4194304] 0 2024-04-12T05:55:25.661 INFO:tasks.workunit.client.0.smithi008.stdout:3/202: creat d9/d1e/f42 x:0 0 0 2024-04-12T05:55:25.661 INFO:tasks.workunit.client.0.smithi008.stdout:4/122: fdatasync d1/db/fc 0 2024-04-12T05:55:25.662 INFO:tasks.workunit.client.0.smithi008.stdout:3/203: chown d9/d10/d19 22 1 2024-04-12T05:55:25.663 INFO:tasks.workunit.client.0.smithi008.stdout:8/123: truncate fb 855480 0 2024-04-12T05:55:25.663 INFO:tasks.workunit.client.0.smithi008.stdout:5/315: chown d0/dc/d13/d1a 77 1 2024-04-12T05:55:25.667 INFO:tasks.workunit.client.0.smithi008.stdout:7/254: creat d1/f55 x:0 0 0 2024-04-12T05:55:25.667 INFO:tasks.workunit.client.0.smithi008.stdout:6/199: rename l7 to df/d10/d3a/l3c 0 2024-04-12T05:55:25.671 INFO:tasks.workunit.client.0.smithi008.stdout:1/201: dwrite d4/dd/d17/f31 [0,4194304] 0 2024-04-12T05:55:25.672 INFO:tasks.workunit.client.0.smithi008.stdout:8/124: unlink dd/l13 0 2024-04-12T05:55:25.673 INFO:tasks.workunit.client.0.smithi008.stdout:3/204: creat d9/d10/d19/d22/f43 x:0 0 0 2024-04-12T05:55:25.674 INFO:tasks.workunit.client.0.smithi008.stdout:8/125: chown f7 3315996 1 2024-04-12T05:55:25.675 INFO:tasks.workunit.client.0.smithi008.stdout:8/126: read dd/f1f [3945774,92676] 0 2024-04-12T05:55:25.679 INFO:tasks.workunit.client.0.smithi008.stdout:7/255: mkdir d1/d5/da/d11/d56 0 2024-04-12T05:55:25.680 INFO:tasks.workunit.client.0.smithi008.stdout:1/202: creat d4/d6/f40 x:0 0 0 2024-04-12T05:55:25.682 INFO:tasks.workunit.client.0.smithi008.stdout:1/203: readlink d4/d7/d18/l34 0 2024-04-12T05:55:25.684 INFO:tasks.workunit.client.0.smithi008.stdout:1/204: fdatasync d4/d7/ff 0 2024-04-12T05:55:25.685 INFO:tasks.workunit.client.0.smithi008.stdout:5/316: getdents d0/d2 0 2024-04-12T05:55:25.687 INFO:tasks.workunit.client.0.smithi008.stdout:1/205: dread - d4/f3e zero size 2024-04-12T05:55:25.692 INFO:tasks.workunit.client.0.smithi008.stdout:4/123: dread d1/f12 [0,4194304] 0 2024-04-12T05:55:25.693 INFO:tasks.workunit.client.0.smithi008.stdout:1/206: symlink d4/d33/d37/l41 0 2024-04-12T05:55:25.694 INFO:tasks.workunit.client.0.smithi008.stdout:3/205: dwrite d9/d10/d19/d22/f28 [4194304,4194304] 0 2024-04-12T05:55:25.696 INFO:tasks.workunit.client.0.smithi008.stdout:4/124: write d1/db/fc [2792719,74107] 0 2024-04-12T05:55:25.698 INFO:tasks.workunit.client.0.smithi008.stdout:1/207: truncate d4/d6/d11/f22 4845457 0 2024-04-12T05:55:25.704 INFO:tasks.workunit.client.0.smithi008.stdout:5/317: rename d0/d29/d44/f47 to d0/dc/f65 0 2024-04-12T05:55:25.704 INFO:tasks.workunit.client.0.smithi008.stdout:6/200: dwrite f5 [0,4194304] 0 2024-04-12T05:55:25.705 INFO:tasks.workunit.client.0.smithi008.stdout:1/208: creat d4/d6/d11/f42 x:0 0 0 2024-04-12T05:55:25.708 INFO:tasks.workunit.client.0.smithi008.stdout:3/206: mkdir d9/d44 0 2024-04-12T05:55:25.711 INFO:tasks.workunit.client.0.smithi008.stdout:5/318: creat d0/dc/d57/f66 x:0 0 0 2024-04-12T05:55:25.712 INFO:tasks.workunit.client.0.smithi008.stdout:3/207: symlink d9/d1e/l45 0 2024-04-12T05:55:25.713 INFO:tasks.workunit.client.0.smithi008.stdout:4/125: dwrite d1/db/fc [0,4194304] 0 2024-04-12T05:55:25.715 INFO:tasks.workunit.client.0.smithi008.stdout:3/208: write d9/d1e/f35 [518723,4969] 0 2024-04-12T05:55:25.716 INFO:tasks.workunit.client.0.smithi008.stdout:1/209: rename d4/d24/f3c to d4/dd/d17/d28/f43 0 2024-04-12T05:55:25.717 INFO:tasks.workunit.client.0.smithi008.stdout:1/210: stat d4/dd/d17/d26/l3b 0 2024-04-12T05:55:25.719 INFO:tasks.workunit.client.0.smithi008.stdout:5/319: dread d0/d2/f21 [0,4194304] 0 2024-04-12T05:55:25.719 INFO:tasks.workunit.client.0.smithi008.stdout:5/320: fdatasync d0/dd/d3d/f4b 0 2024-04-12T05:55:25.720 INFO:tasks.workunit.client.0.smithi008.stdout:4/126: creat d1/f20 x:0 0 0 2024-04-12T05:55:25.724 INFO:tasks.workunit.client.0.smithi008.stdout:1/211: dread d4/d7/fe [0,4194304] 0 2024-04-12T05:55:25.728 INFO:tasks.workunit.client.0.smithi008.stdout:1/212: mknod d4/d7/c44 0 2024-04-12T05:55:25.730 INFO:tasks.workunit.client.0.smithi008.stdout:1/213: rmdir d4/d24 39 2024-04-12T05:55:25.732 INFO:tasks.workunit.client.0.smithi008.stdout:5/321: write d0/d2/f5 [4248821,6431] 0 2024-04-12T05:55:25.734 INFO:tasks.workunit.client.0.smithi008.stdout:4/127: truncate d1/db/f16 3106887 0 2024-04-12T05:55:25.734 INFO:tasks.workunit.client.0.smithi008.stdout:1/214: write d4/d6/f40 [406237,38115] 0 2024-04-12T05:55:25.735 INFO:tasks.workunit.client.0.smithi008.stdout:6/201: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:25.739 INFO:tasks.workunit.client.0.smithi008.stdout:6/202: chown df/f23 7362566 1 2024-04-12T05:55:25.741 INFO:tasks.workunit.client.0.smithi008.stdout:1/215: write d4/f3e [822545,89771] 0 2024-04-12T05:55:25.741 INFO:tasks.workunit.client.0.smithi008.stdout:5/322: creat d0/d40/d56/f67 x:0 0 0 2024-04-12T05:55:25.746 INFO:tasks.workunit.client.0.smithi008.stdout:3/209: dwrite f7 [0,4194304] 0 2024-04-12T05:55:25.746 INFO:tasks.workunit.client.0.smithi008.stdout:6/203: write df/d21/f27 [1016073,109084] 0 2024-04-12T05:55:25.751 INFO:tasks.workunit.client.0.smithi008.stdout:1/216: creat d4/d24/f45 x:0 0 0 2024-04-12T05:55:25.752 INFO:tasks.workunit.client.0.smithi008.stdout:3/210: chown d9/d10/d19/d22/fc 187719 1 2024-04-12T05:55:25.753 INFO:tasks.workunit.client.0.smithi008.stdout:6/204: rmdir df/d10/d20 39 2024-04-12T05:55:25.754 INFO:tasks.workunit.client.0.smithi008.stdout:1/217: write d4/dd/f39 [1034100,56230] 0 2024-04-12T05:55:25.756 INFO:tasks.workunit.client.0.smithi008.stdout:5/323: write d0/dc/d13/d1a/f24 [2332091,35062] 0 2024-04-12T05:55:25.757 INFO:tasks.workunit.client.0.smithi008.stdout:5/324: stat d0/dc/d13 0 2024-04-12T05:55:25.759 INFO:tasks.workunit.client.0.smithi008.stdout:6/205: symlink df/d10/d3a/l3d 0 2024-04-12T05:55:25.761 INFO:tasks.workunit.client.0.smithi008.stdout:1/218: mkdir d4/dd/d17/d28/d46 0 2024-04-12T05:55:25.761 INFO:tasks.workunit.client.0.smithi008.stdout:3/211: rename d9/d11/l27 to d9/dd/l46 0 2024-04-12T05:55:25.765 INFO:tasks.workunit.client.0.smithi008.stdout:4/128: dwrite d1/db/f16 [0,4194304] 0 2024-04-12T05:55:25.765 INFO:tasks.workunit.client.0.smithi008.stdout:3/212: creat d9/d11/f47 x:0 0 0 2024-04-12T05:55:25.766 INFO:tasks.workunit.client.0.smithi008.stdout:5/325: creat d0/dc/f68 x:0 0 0 2024-04-12T05:55:25.769 INFO:tasks.workunit.client.0.smithi008.stdout:3/213: creat d9/d11/f48 x:0 0 0 2024-04-12T05:55:25.769 INFO:tasks.workunit.client.0.smithi008.stdout:5/326: chown d0/dd/d11/c60 17353435 1 2024-04-12T05:55:25.770 INFO:tasks.workunit.client.0.smithi008.stdout:6/206: dread f3 [0,4194304] 0 2024-04-12T05:55:25.772 INFO:tasks.workunit.client.0.smithi008.stdout:5/327: dread - d0/dd/d11/d38/f59 zero size 2024-04-12T05:55:25.772 INFO:tasks.workunit.client.0.smithi008.stdout:3/214: creat d9/d11/f49 x:0 0 0 2024-04-12T05:55:25.774 INFO:tasks.workunit.client.0.smithi008.stdout:6/207: fdatasync f4 0 2024-04-12T05:55:25.775 INFO:tasks.workunit.client.0.smithi008.stdout:3/215: mknod d9/d1e/c4a 0 2024-04-12T05:55:25.776 INFO:tasks.workunit.client.0.smithi008.stdout:5/328: read d0/dc/d13/f54 [2712138,102402] 0 2024-04-12T05:55:25.776 INFO:tasks.workunit.client.0.smithi008.stdout:1/219: dwrite d4/d6/d11/f42 [0,4194304] 0 2024-04-12T05:55:25.779 INFO:tasks.workunit.client.0.smithi008.stdout:5/329: write d0/dc/d13/d1a/f3f [3203579,87993] 0 2024-04-12T05:55:25.783 INFO:tasks.workunit.client.0.smithi008.stdout:6/208: mkdir df/d10/d20/d3e 0 2024-04-12T05:55:25.783 INFO:tasks.workunit.client.0.smithi008.stdout:3/216: stat d9/dd/l16 0 2024-04-12T05:55:25.784 INFO:tasks.workunit.client.0.smithi008.stdout:6/209: write df/d10/d15/f2a [697798,24426] 0 2024-04-12T05:55:25.794 INFO:tasks.workunit.client.0.smithi008.stdout:5/330: dwrite d0/dc/f65 [0,4194304] 0 2024-04-12T05:55:25.797 INFO:tasks.workunit.client.0.smithi008.stdout:6/210: dwrite df/f23 [4194304,4194304] 0 2024-04-12T05:55:25.798 INFO:tasks.workunit.client.0.smithi008.stdout:3/217: dwrite d9/f1c [0,4194304] 0 2024-04-12T05:55:25.802 INFO:tasks.workunit.client.0.smithi008.stdout:5/331: fdatasync d0/dd/d3d/f53 0 2024-04-12T05:55:25.804 INFO:tasks.workunit.client.0.smithi008.stdout:5/332: read - d0/dd/d11/f5a zero size 2024-04-12T05:55:25.805 INFO:tasks.workunit.client.0.smithi008.stdout:3/218: mkdir d9/d10/d4b 0 2024-04-12T05:55:25.807 INFO:tasks.workunit.client.0.smithi008.stdout:3/219: write d9/d10/d19/d22/f28 [1521257,97527] 0 2024-04-12T05:55:25.814 INFO:tasks.workunit.client.0.smithi008.stdout:3/220: dwrite d9/d11/f49 [0,4194304] 0 2024-04-12T05:55:25.817 INFO:tasks.workunit.client.0.smithi008.stdout:3/221: rename d9/d10/d19/d22/f43 to d9/d2e/d3d/f4c 0 2024-04-12T05:55:25.817 INFO:tasks.workunit.client.0.smithi008.stdout:3/222: stat c4 0 2024-04-12T05:55:25.821 INFO:tasks.workunit.client.0.smithi008.stdout:4/129: read d1/d8/f15 [719594,30038] 0 2024-04-12T05:55:25.827 INFO:tasks.workunit.client.0.smithi008.stdout:3/223: dwrite d9/d2e/d3d/f4c [0,4194304] 0 2024-04-12T05:55:25.829 INFO:tasks.workunit.client.0.smithi008.stdout:3/224: read d9/f25 [105002,5435] 0 2024-04-12T05:55:25.829 INFO:tasks.workunit.client.0.smithi008.stdout:4/130: dread d1/f12 [4194304,4194304] 0 2024-04-12T05:55:25.830 INFO:tasks.workunit.client.0.smithi008.stdout:3/225: chown d9/d44 26059 1 2024-04-12T05:55:25.832 INFO:tasks.workunit.client.0.smithi008.stdout:4/131: creat d1/d8/f21 x:0 0 0 2024-04-12T05:55:25.833 INFO:tasks.workunit.client.0.smithi008.stdout:3/226: creat d9/d10/d19/d22/f4d x:0 0 0 2024-04-12T05:55:25.834 INFO:tasks.workunit.client.0.smithi008.stdout:4/132: read d1/db/f18 [733551,65177] 0 2024-04-12T05:55:25.837 INFO:tasks.workunit.client.0.smithi008.stdout:3/227: mknod d9/d10/d19/d3a/c4e 0 2024-04-12T05:55:25.838 INFO:tasks.workunit.client.0.smithi008.stdout:4/133: mkdir d1/d22 0 2024-04-12T05:55:25.838 INFO:tasks.workunit.client.0.smithi008.stdout:3/228: fsync d9/f25 0 2024-04-12T05:55:25.840 INFO:tasks.workunit.client.0.smithi008.stdout:4/134: creat d1/d8/f23 x:0 0 0 2024-04-12T05:55:25.841 INFO:tasks.workunit.client.0.smithi008.stdout:3/229: write d9/d10/f33 [852231,44671] 0 2024-04-12T05:55:25.841 INFO:tasks.workunit.client.0.smithi008.stdout:4/135: read d1/f12 [119513,40265] 0 2024-04-12T05:55:25.842 INFO:tasks.workunit.client.0.smithi008.stdout:3/230: read d9/f25 [488494,9848] 0 2024-04-12T05:55:25.844 INFO:tasks.workunit.client.0.smithi008.stdout:6/211: dread f9 [0,4194304] 0 2024-04-12T05:55:25.845 INFO:tasks.workunit.client.0.smithi008.stdout:4/136: creat d1/db/d19/f24 x:0 0 0 2024-04-12T05:55:25.845 INFO:tasks.workunit.client.0.smithi008.stdout:3/231: mknod d9/d44/c4f 0 2024-04-12T05:55:25.856 INFO:tasks.workunit.client.0.smithi008.stdout:4/137: write d1/db/d19/f24 [188892,121406] 0 2024-04-12T05:55:25.856 INFO:tasks.workunit.client.0.smithi008.stdout:3/232: dread d9/f25 [0,4194304] 0 2024-04-12T05:55:25.857 INFO:tasks.workunit.client.0.smithi008.stdout:6/212: getdents df/d21 0 2024-04-12T05:55:25.857 INFO:tasks.workunit.client.0.smithi008.stdout:3/233: creat d9/d10/d19/f50 x:0 0 0 2024-04-12T05:55:25.857 INFO:tasks.workunit.client.0.smithi008.stdout:3/234: unlink d9/d1e/l45 0 2024-04-12T05:55:25.857 INFO:tasks.workunit.client.0.smithi008.stdout:3/235: dread - d9/d10/d19/d22/f32 zero size 2024-04-12T05:55:25.863 INFO:tasks.workunit.client.0.smithi008.stdout:5/333: write d0/dd/d3d/f53 [315486,120838] 0 2024-04-12T05:55:25.864 INFO:tasks.workunit.client.0.smithi008.stdout:3/236: dwrite f7 [0,4194304] 0 2024-04-12T05:55:25.869 INFO:tasks.workunit.client.0.smithi008.stdout:6/213: fsync df/d10/f39 0 2024-04-12T05:55:25.871 INFO:tasks.workunit.client.0.smithi008.stdout:5/334: dread d0/dc/f52 [0,4194304] 0 2024-04-12T05:55:25.871 INFO:tasks.workunit.client.0.smithi008.stdout:3/237: rmdir d9/d2e/d3d 39 2024-04-12T05:55:25.872 INFO:tasks.workunit.client.0.smithi008.stdout:6/214: chown df/d10/d15/f1c 107332 1 2024-04-12T05:55:25.873 INFO:tasks.workunit.client.0.smithi008.stdout:6/215: stat fb 0 2024-04-12T05:55:25.874 INFO:tasks.workunit.client.0.smithi008.stdout:6/216: chown f3 26924027 1 2024-04-12T05:55:25.875 INFO:tasks.workunit.client.0.smithi008.stdout:5/335: symlink d0/d2/d37/l69 0 2024-04-12T05:55:25.878 INFO:tasks.workunit.client.0.smithi008.stdout:3/238: truncate f6 674971 0 2024-04-12T05:55:25.880 INFO:tasks.workunit.client.0.smithi008.stdout:6/217: creat df/d10/d15/f3f x:0 0 0 2024-04-12T05:55:25.887 INFO:tasks.workunit.client.0.smithi008.stdout:3/239: creat d9/d2e/d3d/f51 x:0 0 0 2024-04-12T05:55:25.888 INFO:tasks.workunit.client.0.smithi008.stdout:6/218: dread f5 [0,4194304] 0 2024-04-12T05:55:25.888 INFO:tasks.workunit.client.0.smithi008.stdout:6/219: chown df/l2f 6 1 2024-04-12T05:55:25.888 INFO:tasks.workunit.client.0.smithi008.stdout:3/240: readlink d9/dd/l1d 0 2024-04-12T05:55:25.890 INFO:tasks.workunit.client.0.smithi008.stdout:5/336: truncate d0/dd/d11/f19 4306985 0 2024-04-12T05:55:25.890 INFO:tasks.workunit.client.0.smithi008.stdout:3/241: dread - d9/d11/f48 zero size 2024-04-12T05:55:25.891 INFO:tasks.workunit.client.0.smithi008.stdout:5/337: dread - d0/dd/d3d/f4b zero size 2024-04-12T05:55:25.892 INFO:tasks.workunit.client.0.smithi008.stdout:3/242: write d9/d1e/f35 [8697,65116] 0 2024-04-12T05:55:25.893 INFO:tasks.workunit.client.0.smithi008.stdout:6/220: truncate df/d10/f39 5010924 0 2024-04-12T05:55:25.894 INFO:tasks.workunit.client.0.smithi008.stdout:5/338: write d0/dc/f65 [1044949,108939] 0 2024-04-12T05:55:25.895 INFO:tasks.workunit.client.0.smithi008.stdout:6/221: stat df/d21/l36 0 2024-04-12T05:55:25.897 INFO:tasks.workunit.client.0.smithi008.stdout:6/222: rename df/d10 to df/d10/d20/d3e/d40 22 2024-04-12T05:55:25.897 INFO:tasks.workunit.client.0.smithi008.stdout:9/232: sync 2024-04-12T05:55:25.898 INFO:tasks.workunit.client.0.smithi008.stdout:9/233: chown d3/df/f20 4867 1 2024-04-12T05:55:25.901 INFO:tasks.workunit.client.0.smithi008.stdout:9/234: truncate d3/d22/d2b/f41 215713 0 2024-04-12T05:55:25.902 INFO:tasks.workunit.client.0.smithi008.stdout:3/243: dread d9/d10/d19/d22/fc [0,4194304] 0 2024-04-12T05:55:25.907 INFO:tasks.workunit.client.0.smithi008.stdout:6/223: symlink df/d10/l41 0 2024-04-12T05:55:25.913 INFO:tasks.workunit.client.0.smithi008.stdout:3/244: link d9/d1e/f42 d9/d10/d19/f52 0 2024-04-12T05:55:25.915 INFO:tasks.workunit.client.0.smithi008.stdout:5/339: dwrite d0/dc/d13/d1a/d35/f42 [0,4194304] 0 2024-04-12T05:55:25.915 INFO:tasks.workunit.client.0.smithi008.stdout:3/245: write d9/d11/f48 [636274,40605] 0 2024-04-12T05:55:25.916 INFO:tasks.workunit.client.0.smithi008.stdout:6/224: dread df/f23 [4194304,4194304] 0 2024-04-12T05:55:25.918 INFO:tasks.workunit.client.0.smithi008.stdout:6/225: chown df/d10/d20/c28 4250155 1 2024-04-12T05:55:25.919 INFO:tasks.workunit.client.0.smithi008.stdout:5/340: truncate d0/dd/d11/d38/f59 1031087 0 2024-04-12T05:55:25.920 INFO:tasks.workunit.client.0.smithi008.stdout:3/246: unlink d9/d10/d19/d22/l26 0 2024-04-12T05:55:25.921 INFO:tasks.workunit.client.0.smithi008.stdout:3/247: dread - d9/d10/d19/d22/f32 zero size 2024-04-12T05:55:25.922 INFO:tasks.workunit.client.0.smithi008.stdout:6/226: truncate f4 1955758 0 2024-04-12T05:55:25.923 INFO:tasks.workunit.client.0.smithi008.stdout:3/248: write d9/f1c [4551465,100572] 0 2024-04-12T05:55:25.925 INFO:tasks.workunit.client.0.smithi008.stdout:5/341: creat d0/dc/f6a x:0 0 0 2024-04-12T05:55:25.927 INFO:tasks.workunit.client.0.smithi008.stdout:6/227: rename df/d10/d15/l30 to df/d10/d3a/l42 0 2024-04-12T05:55:25.929 INFO:tasks.workunit.client.0.smithi008.stdout:3/249: creat d9/dd/f53 x:0 0 0 2024-04-12T05:55:25.933 INFO:tasks.workunit.client.0.smithi008.stdout:6/228: link df/f23 df/d21/f43 0 2024-04-12T05:55:25.934 INFO:tasks.workunit.client.0.smithi008.stdout:6/229: chown df/l2f 264355 1 2024-04-12T05:55:25.936 INFO:tasks.workunit.client.0.smithi008.stdout:6/230: chown df/d21/l3b 191792658 1 2024-04-12T05:55:25.937 INFO:tasks.workunit.client.0.smithi008.stdout:3/250: dwrite d9/d10/d19/d22/f32 [0,4194304] 0 2024-04-12T05:55:25.941 INFO:tasks.workunit.client.0.smithi008.stdout:5/342: truncate d0/d2/f5 4325207 0 2024-04-12T05:55:25.946 INFO:tasks.workunit.client.0.smithi008.stdout:5/343: rename d0/d40/c4a to d0/dd/d11/d38/c6b 0 2024-04-12T05:55:25.947 INFO:tasks.workunit.client.0.smithi008.stdout:6/231: getdents df 0 2024-04-12T05:55:25.948 INFO:tasks.workunit.client.0.smithi008.stdout:5/344: chown d0/dc/f6a 7 1 2024-04-12T05:55:25.952 INFO:tasks.workunit.client.0.smithi008.stdout:6/232: dread df/d10/d15/f1c [0,4194304] 0 2024-04-12T05:55:25.957 INFO:tasks.workunit.client.0.smithi008.stdout:6/233: rename df/d10/d3a/l3d to df/d10/d20/l44 0 2024-04-12T05:55:25.959 INFO:tasks.workunit.client.0.smithi008.stdout:5/345: link d0/d2/f5 d0/d40/d56/f6c 0 2024-04-12T05:55:25.960 INFO:tasks.workunit.client.0.smithi008.stdout:6/234: symlink df/d21/l45 0 2024-04-12T05:55:25.961 INFO:tasks.workunit.client.0.smithi008.stdout:5/346: write d0/d40/d56/f6c [1029895,53910] 0 2024-04-12T05:55:25.966 INFO:tasks.workunit.client.0.smithi008.stdout:5/347: symlink d0/d2/l6d 0 2024-04-12T05:55:25.969 INFO:tasks.workunit.client.0.smithi008.stdout:5/348: chown d0/dc/d13/f15 5 1 2024-04-12T05:55:25.981 INFO:tasks.workunit.client.0.smithi008.stdout:6/235: dwrite df/d21/f22 [0,4194304] 0 2024-04-12T05:55:25.983 INFO:tasks.workunit.client.0.smithi008.stdout:5/349: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:25.986 INFO:tasks.workunit.client.0.smithi008.stdout:6/236: stat df/d21/f43 0 2024-04-12T05:55:25.986 INFO:tasks.workunit.client.0.smithi008.stdout:5/350: chown d0/dd/c49 740183 1 2024-04-12T05:55:25.988 INFO:tasks.workunit.client.0.smithi008.stdout:6/237: readlink df/d10/d3a/l3c 0 2024-04-12T05:55:25.989 INFO:tasks.workunit.client.0.smithi008.stdout:5/351: mknod d0/d40/c6e 0 2024-04-12T05:55:25.989 INFO:tasks.workunit.client.0.smithi008.stdout:5/352: chown d0/dc 3468132 1 2024-04-12T05:55:25.996 INFO:tasks.workunit.client.0.smithi008.stdout:6/238: link df/d10/d15/l25 df/d10/d15/l46 0 2024-04-12T05:55:26.003 INFO:tasks.workunit.client.0.smithi008.stdout:6/239: dwrite fe [4194304,4194304] 0 2024-04-12T05:55:26.005 INFO:tasks.workunit.client.0.smithi008.stdout:6/240: rmdir df/d21 39 2024-04-12T05:55:26.006 INFO:tasks.workunit.client.0.smithi008.stdout:6/241: readlink df/d10/d15/l1e 0 2024-04-12T05:55:26.009 INFO:tasks.workunit.client.0.smithi008.stdout:6/242: mknod df/d10/d20/d3e/c47 0 2024-04-12T05:55:26.009 INFO:tasks.workunit.client.0.smithi008.stdout:5/353: write d0/d2/d37/f3e [2175686,8005] 0 2024-04-12T05:55:26.011 INFO:tasks.workunit.client.0.smithi008.stdout:6/243: fdatasync df/d10/d15/f1c 0 2024-04-12T05:55:26.011 INFO:tasks.workunit.client.0.smithi008.stdout:5/354: read - d0/dc/f68 zero size 2024-04-12T05:55:26.014 INFO:tasks.workunit.client.0.smithi008.stdout:5/355: symlink d0/l6f 0 2024-04-12T05:55:26.016 INFO:tasks.workunit.client.0.smithi008.stdout:5/356: mknod d0/dc/d13/d1a/d3a/c70 0 2024-04-12T05:55:26.022 INFO:tasks.workunit.client.0.smithi008.stdout:5/357: rmdir d0/dc/d13/d1a/d3a 39 2024-04-12T05:55:26.026 INFO:tasks.workunit.client.0.smithi008.stdout:5/358: mknod d0/dc/d13/d1a/c71 0 2024-04-12T05:55:26.027 INFO:tasks.workunit.client.0.smithi008.stdout:6/244: dwrite fb [4194304,4194304] 0 2024-04-12T05:55:26.027 INFO:tasks.workunit.client.0.smithi008.stdout:5/359: chown d0/d2/d37/f3e 1 1 2024-04-12T05:55:26.029 INFO:tasks.workunit.client.0.smithi008.stdout:5/360: chown d0/dd/d11/d38/l62 217 1 2024-04-12T05:55:26.030 INFO:tasks.workunit.client.0.smithi008.stdout:5/361: fdatasync d0/d2/f5 0 2024-04-12T05:55:26.032 INFO:tasks.workunit.client.0.smithi008.stdout:5/362: fsync d0/dc/d13/f15 0 2024-04-12T05:55:26.034 INFO:tasks.workunit.client.0.smithi008.stdout:5/363: write d0/d2/f5 [1892472,14193] 0 2024-04-12T05:55:26.036 INFO:tasks.workunit.client.0.smithi008.stdout:5/364: dread - d0/dc/f6a zero size 2024-04-12T05:55:26.036 INFO:tasks.workunit.client.0.smithi008.stdout:6/245: getdents df/d10 0 2024-04-12T05:55:26.039 INFO:tasks.workunit.client.0.smithi008.stdout:6/246: mknod df/d10/d20/d3e/c48 0 2024-04-12T05:55:26.040 INFO:tasks.workunit.client.0.smithi008.stdout:5/365: dread - d0/dc/d13/d1a/d3a/f63 zero size 2024-04-12T05:55:26.041 INFO:tasks.workunit.client.0.smithi008.stdout:6/247: creat df/d10/d20/f49 x:0 0 0 2024-04-12T05:55:26.043 INFO:tasks.workunit.client.0.smithi008.stdout:6/248: dread - df/d10/d15/f24 zero size 2024-04-12T05:55:26.044 INFO:tasks.workunit.client.0.smithi008.stdout:6/249: readlink df/d10/l1a 0 2024-04-12T05:55:26.046 INFO:tasks.workunit.client.0.smithi008.stdout:5/366: rename d0/d40/d56/f6c to d0/dc/d13/f72 0 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.050+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/3615373888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c4100dc0 msgr2=0x7ff9c4101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.050+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/3615373888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c4100dc0 0x7ff9c4101220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009910 tx=0x7ff9c0009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/3615373888 shutdown_connections 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/3615373888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c4100dc0 0x7ff9c4101220 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/3615373888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c40ffbc0 0x7ff9c40fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/3615373888 >> 172.21.15.8:0/3615373888 conn(0x7ff9c40fb160 msgr2=0x7ff9c40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/3615373888 shutdown_connections 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.051+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/3615373888 wait complete. 2024-04-12T05:55:26.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.052+0000 7ff9cc7e9700 1 Processor -- start 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9cc7e9700 1 -- start start 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9cc7e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c40ffbc0 0x7ff9c4069970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9cc7e9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9cc7e9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4197300 con 0x7ff9c40ffbc0 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9cc7e9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c419ddd0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9c9d84700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9c9d84700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:38104/0 (socket says 172.21.15.8:38104) 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9c9d84700 1 -- 172.21.15.8:0/2595261309 learned_addr learned my addr 172.21.15.8:0/2595261309 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9ca585700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c40ffbc0 0x7ff9c4069970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9c9d84700 1 -- 172.21.15.8:0/2595261309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c40ffbc0 msgr2=0x7ff9c4069970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.053+0000 7ff9c9d84700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c40ffbc0 0x7ff9c4069970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9c9d84700 1 -- 172.21.15.8:0/2595261309 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c00095f0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9c9d84700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000bde0 tx=0x7ff9c000be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:26.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c001d070 con 0x7ff9c4100dc0 2024-04-12T05:55:26.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c419e020 con 0x7ff9c4100dc0 2024-04-12T05:55:26.050 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff9c0005dc0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9c0003ab0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.054+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c419e4b0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.055+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c418f3f0 con 0x7ff9c4100dc0 2024-04-12T05:55:26.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.056+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7ff9c0003c10 con 0x7ff9c4100dc0 2024-04-12T05:55:26.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.056+0000 7ff9bb7fe700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 0x7ff9b006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.056+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff9c008d890 con 0x7ff9c4100dc0 2024-04-12T05:55:26.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.057+0000 7ff9ca585700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 0x7ff9b006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:26.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.057+0000 7ff9ca585700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 0x7ff9b006eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4100c20 tx=0x7ff9b4009260 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:26.057 INFO:tasks.workunit.client.0.smithi008.stdout:5/367: dwrite d0/dc/f6a [0,4194304] 0 2024-04-12T05:55:26.057 INFO:tasks.workunit.client.0.smithi008.stdout:5/368: stat d0/c1 0 2024-04-12T05:55:26.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.064+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff9c005b310 con 0x7ff9c4100dc0 2024-04-12T05:55:26.061 INFO:tasks.workunit.client.0.smithi008.stdout:5/369: rename d0/dc/d13/d1a/d3a/c70 to d0/d2/d37/c73 0 2024-04-12T05:55:26.065 INFO:tasks.workunit.client.0.smithi008.stdout:5/370: dread d0/dd/d3d/f53 [0,4194304] 0 2024-04-12T05:55:26.069 INFO:tasks.workunit.client.0.smithi008.stdout:5/371: mknod d0/dc/d57/c74 0 2024-04-12T05:55:26.069 INFO:tasks.workunit.client.0.smithi008.stdout:6/250: dread f8 [0,4194304] 0 2024-04-12T05:55:26.071 INFO:tasks.workunit.client.0.smithi008.stdout:5/372: symlink d0/dc/d13/d1a/d35/l75 0 2024-04-12T05:55:26.072 INFO:tasks.workunit.client.0.smithi008.stdout:6/251: chown df/d21/f27 0 1 2024-04-12T05:55:26.083 INFO:tasks.workunit.client.0.smithi008.stdout:5/373: truncate d0/d2/f4 1837534 0 2024-04-12T05:55:26.119 INFO:tasks.workunit.client.0.smithi008.stdout:0/255: sync 2024-04-12T05:55:26.119 INFO:tasks.workunit.client.0.smithi008.stdout:2/275: sync 2024-04-12T05:55:26.119 INFO:tasks.workunit.client.0.smithi008.stdout:8/127: sync 2024-04-12T05:55:26.121 INFO:tasks.workunit.client.0.smithi008.stdout:2/276: write d6/db/f22 [1392579,31982] 0 2024-04-12T05:55:26.124 INFO:tasks.workunit.client.0.smithi008.stdout:8/128: creat dd/de/f2a x:0 0 0 2024-04-12T05:55:26.126 INFO:tasks.workunit.client.0.smithi008.stdout:2/277: readlink d6/db/d10/l50 0 2024-04-12T05:55:26.128 INFO:tasks.workunit.client.0.smithi008.stdout:0/256: symlink d2/d24/d39/l51 0 2024-04-12T05:55:26.129 INFO:tasks.workunit.client.0.smithi008.stdout:6/252: dread df/f31 [0,4194304] 0 2024-04-12T05:55:26.129 INFO:tasks.workunit.client.0.smithi008.stdout:5/374: write d0/dc/f2a [2801809,116248] 0 2024-04-12T05:55:26.130 INFO:tasks.workunit.client.0.smithi008.stdout:2/278: write d6/f15 [3731793,51812] 0 2024-04-12T05:55:26.134 INFO:tasks.workunit.client.0.smithi008.stdout:0/257: mknod d2/d11/d36/d42/c52 0 2024-04-12T05:55:26.135 INFO:tasks.workunit.client.0.smithi008.stdout:6/253: mknod df/d10/d3a/c4a 0 2024-04-12T05:55:26.136 INFO:tasks.workunit.client.0.smithi008.stdout:5/375: mknod d0/dd/d11/d38/c76 0 2024-04-12T05:55:26.137 INFO:tasks.workunit.client.0.smithi008.stdout:8/129: truncate dd/f22 3296326 0 2024-04-12T05:55:26.138 INFO:tasks.workunit.client.0.smithi008.stdout:8/130: stat dd/de/c10 0 2024-04-12T05:55:26.139 INFO:tasks.workunit.client.0.smithi008.stdout:2/279: creat d6/db/d10/d23/f5c x:0 0 0 2024-04-12T05:55:26.140 INFO:tasks.workunit.client.0.smithi008.stdout:6/254: creat df/d10/d15/f4b x:0 0 0 2024-04-12T05:55:26.142 INFO:tasks.workunit.client.0.smithi008.stdout:5/376: creat d0/d40/f77 x:0 0 0 2024-04-12T05:55:26.143 INFO:tasks.workunit.client.0.smithi008.stdout:6/255: truncate df/d10/d20/f49 227777 0 2024-04-12T05:55:26.144 INFO:tasks.workunit.client.0.smithi008.stdout:8/131: symlink dd/l2b 0 2024-04-12T05:55:26.145 INFO:tasks.workunit.client.0.smithi008.stdout:2/280: mknod d6/db/d29/c5d 0 2024-04-12T05:55:26.146 INFO:tasks.workunit.client.0.smithi008.stdout:2/281: fdatasync d6/db/f56 0 2024-04-12T05:55:26.150 INFO:tasks.workunit.client.0.smithi008.stdout:6/256: mkdir df/d21/d4c 0 2024-04-12T05:55:26.151 INFO:tasks.workunit.client.0.smithi008.stdout:8/132: fdatasync f9 0 2024-04-12T05:55:26.153 INFO:tasks.workunit.client.0.smithi008.stdout:0/258: getdents d2/d11 0 2024-04-12T05:55:26.154 INFO:tasks.workunit.client.0.smithi008.stdout:2/282: symlink d6/db/d38/d41/l5e 0 2024-04-12T05:55:26.156 INFO:tasks.workunit.client.0.smithi008.stdout:6/257: mknod df/d21/c4d 0 2024-04-12T05:55:26.160 INFO:tasks.workunit.client.0.smithi008.stdout:0/259: write d2/d11/f18 [754033,98903] 0 2024-04-12T05:55:26.164 INFO:tasks.workunit.client.0.smithi008.stdout:6/258: mknod df/d21/c4e 0 2024-04-12T05:55:26.164 INFO:tasks.workunit.client.0.smithi008.stdout:5/377: getdents d0/dc/d13/d1a/d35 0 2024-04-12T05:55:26.165 INFO:tasks.workunit.client.0.smithi008.stdout:6/259: fsync fd 0 2024-04-12T05:55:26.166 INFO:tasks.workunit.client.0.smithi008.stdout:8/133: link dd/f1f dd/de/d16/f2c 0 2024-04-12T05:55:26.168 INFO:tasks.workunit.client.0.smithi008.stdout:8/134: readlink dd/de/l20 0 2024-04-12T05:55:26.169 INFO:tasks.workunit.client.0.smithi008.stdout:2/283: rename l1 to d6/db/d10/d23/d35/l5f 0 2024-04-12T05:55:26.171 INFO:tasks.workunit.client.0.smithi008.stdout:0/260: dwrite d2/d11/d2a/f49 [0,4194304] 0 2024-04-12T05:55:26.174 INFO:tasks.workunit.client.0.smithi008.stdout:8/135: creat dd/de/f2d x:0 0 0 2024-04-12T05:55:26.174 INFO:tasks.workunit.client.0.smithi008.stdout:5/378: creat d0/dd/d3d/d5d/f78 x:0 0 0 2024-04-12T05:55:26.175 INFO:tasks.workunit.client.0.smithi008.stdout:2/284: write d6/db/d10/d23/f4d [267605,90933] 0 2024-04-12T05:55:26.176 INFO:tasks.workunit.client.0.smithi008.stdout:0/261: unlink d2/l4d 0 2024-04-12T05:55:26.177 INFO:tasks.workunit.client.0.smithi008.stdout:2/285: chown d6/db/d10/f55 1059694 1 2024-04-12T05:55:26.178 INFO:tasks.workunit.client.0.smithi008.stdout:8/136: creat dd/de/d16/f2e x:0 0 0 2024-04-12T05:55:26.179 INFO:tasks.workunit.client.0.smithi008.stdout:8/137: dread - dd/f1e zero size 2024-04-12T05:55:26.181 INFO:tasks.workunit.client.0.smithi008.stdout:0/262: unlink d2/d24/d39/l51 0 2024-04-12T05:55:26.181 INFO:tasks.workunit.client.0.smithi008.stdout:0/263: chown d2/l4e 6370698 1 2024-04-12T05:55:26.183 INFO:tasks.workunit.client.0.smithi008.stdout:0/264: write d2/d24/d39/f3a [1492029,41456] 0 2024-04-12T05:55:26.184 INFO:tasks.workunit.client.0.smithi008.stdout:5/379: rename d0/d2/l6d to d0/d2/l79 0 2024-04-12T05:55:26.189 INFO:tasks.workunit.client.0.smithi008.stdout:0/265: creat d2/d24/d39/f53 x:0 0 0 2024-04-12T05:55:26.193 INFO:tasks.workunit.client.0.smithi008.stdout:5/380: mknod d0/d2/d37/c7a 0 2024-04-12T05:55:26.197 INFO:tasks.workunit.client.0.smithi008.stdout:2/286: rename d6/db/d10/d23/d2e to d6/db/d60 0 2024-04-12T05:55:26.197 INFO:tasks.workunit.client.0.smithi008.stdout:6/260: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:26.198 INFO:tasks.workunit.client.0.smithi008.stdout:2/287: chown d6/ca 5 1 2024-04-12T05:55:26.204 INFO:tasks.workunit.client.0.smithi008.stdout:6/261: mkdir df/d10/d15/d4f 0 2024-04-12T05:55:26.205 INFO:tasks.workunit.client.0.smithi008.stdout:0/266: rename d2/d11/l1f to d2/d24/l54 0 2024-04-12T05:55:26.206 INFO:tasks.workunit.client.0.smithi008.stdout:8/138: write fb [155647,66769] 0 2024-04-12T05:55:26.209 INFO:tasks.workunit.client.0.smithi008.stdout:2/288: dwrite d6/db/d38/f5b [0,4194304] 0 2024-04-12T05:55:26.210 INFO:tasks.workunit.client.0.smithi008.stdout:0/267: write d2/d24/f28 [3982197,102110] 0 2024-04-12T05:55:26.212 INFO:tasks.workunit.client.0.smithi008.stdout:5/381: rename d0/dd/d11/c60 to d0/d40/c7b 0 2024-04-12T05:55:26.217 INFO:tasks.workunit.client.0.smithi008.stdout:8/139: link dd/de/d16/f2e dd/de/f2f 0 2024-04-12T05:55:26.221 INFO:tasks.workunit.client.0.smithi008.stdout:6/262: mkdir df/d10/d15/d4f/d50 0 2024-04-12T05:55:26.221 INFO:tasks.workunit.client.0.smithi008.stdout:0/268: dwrite d2/d24/f28 [0,4194304] 0 2024-04-12T05:55:26.226 INFO:tasks.workunit.client.0.smithi008.stdout:8/140: mkdir dd/d30 0 2024-04-12T05:55:26.230 INFO:tasks.workunit.client.0.smithi008.stdout:6/263: rename df/d10/f39 to df/d10/d15/d4f/d50/f51 0 2024-04-12T05:55:26.230 INFO:tasks.workunit.client.0.smithi008.stdout:2/289: creat d6/db/d60/d4f/f61 x:0 0 0 2024-04-12T05:55:26.232 INFO:tasks.workunit.client.0.smithi008.stdout:0/269: symlink d2/d11/l55 0 2024-04-12T05:55:26.234 INFO:tasks.workunit.client.0.smithi008.stdout:8/141: write dd/de/d16/f28 [3576949,48111] 0 2024-04-12T05:55:26.237 INFO:tasks.workunit.client.0.smithi008.stdout:5/382: dwrite d0/dc/f52 [0,4194304] 0 2024-04-12T05:55:26.238 INFO:tasks.workunit.client.0.smithi008.stdout:2/290: write d6/db/f56 [617426,49689] 0 2024-04-12T05:55:26.241 INFO:tasks.workunit.client.0.smithi008.stdout:8/142: write dd/de/f2a [528168,46166] 0 2024-04-12T05:55:26.244 INFO:tasks.workunit.client.0.smithi008.stdout:0/270: rename d2/l10 to d2/d24/l56 0 2024-04-12T05:55:26.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/143: getdents dd 0 2024-04-12T05:55:26.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/144: chown dd/d30 0 1 2024-04-12T05:55:26.256 INFO:tasks.workunit.client.0.smithi008.stdout:5/383: dwrite d0/d2/f5 [4194304,4194304] 0 2024-04-12T05:55:26.256 INFO:tasks.workunit.client.0.smithi008.stdout:6/264: truncate df/f23 3598356 0 2024-04-12T05:55:26.257 INFO:tasks.workunit.client.0.smithi008.stdout:8/145: write dd/de/d16/f17 [584172,83311] 0 2024-04-12T05:55:26.261 INFO:tasks.workunit.client.0.smithi008.stdout:5/384: write d0/dc/d13/d1a/f3f [3576159,10802] 0 2024-04-12T05:55:26.262 INFO:tasks.workunit.client.0.smithi008.stdout:8/146: mknod dd/de/c31 0 2024-04-12T05:55:26.266 INFO:tasks.workunit.client.0.smithi008.stdout:6/265: write df/d10/d15/f2a [1573258,21905] 0 2024-04-12T05:55:26.267 INFO:tasks.workunit.client.0.smithi008.stdout:5/385: symlink d0/dc/d13/d1a/d35/l7c 0 2024-04-12T05:55:26.270 INFO:tasks.workunit.client.0.smithi008.stdout:0/271: dwrite d2/d11/d2a/f37 [0,4194304] 0 2024-04-12T05:55:26.272 INFO:tasks.workunit.client.0.smithi008.stdout:8/147: mkdir dd/d30/d32 0 2024-04-12T05:55:26.274 INFO:tasks.workunit.client.0.smithi008.stdout:0/272: unlink d2/f13 0 2024-04-12T05:55:26.277 INFO:tasks.workunit.client.0.smithi008.stdout:5/386: dread d0/dd/d11/f19 [0,4194304] 0 2024-04-12T05:55:26.279 INFO:tasks.workunit.client.0.smithi008.stdout:0/273: write d2/d11/f25 [1076076,127574] 0 2024-04-12T05:55:26.280 INFO:tasks.workunit.client.0.smithi008.stdout:5/387: write d0/dc/d57/f66 [499462,108891] 0 2024-04-12T05:55:26.285 INFO:tasks.workunit.client.0.smithi008.stdout:0/274: creat d2/d11/d36/d42/f57 x:0 0 0 2024-04-12T05:55:26.287 INFO:tasks.workunit.client.0.smithi008.stdout:6/266: truncate fe 4983828 0 2024-04-12T05:55:26.287 INFO:tasks.workunit.client.0.smithi008.stdout:6/267: chown fd 326 1 2024-04-12T05:55:26.288 INFO:tasks.workunit.client.0.smithi008.stdout:5/388: creat d0/dc/d13/d1a/f7d x:0 0 0 2024-04-12T05:55:26.289 INFO:tasks.workunit.client.0.smithi008.stdout:5/389: stat d0/dd/f1e 0 2024-04-12T05:55:26.292 INFO:tasks.workunit.client.0.smithi008.stdout:6/268: creat df/d21/d4c/f52 x:0 0 0 2024-04-12T05:55:26.292 INFO:tasks.workunit.client.0.smithi008.stdout:8/148: dread fb [0,4194304] 0 2024-04-12T05:55:26.297 INFO:tasks.workunit.client.0.smithi008.stdout:0/275: dread d2/fc [0,4194304] 0 2024-04-12T05:55:26.298 INFO:tasks.workunit.client.0.smithi008.stdout:5/390: rename d0/dc/f52 to d0/dd/d3d/d5d/f7e 0 2024-04-12T05:55:26.298 INFO:tasks.workunit.client.0.smithi008.stdout:6/269: write fb [2370011,43393] 0 2024-04-12T05:55:26.301 INFO:tasks.workunit.client.0.smithi008.stdout:0/276: unlink d2/l3c 0 2024-04-12T05:55:26.302 INFO:tasks.workunit.client.0.smithi008.stdout:5/391: symlink d0/dd/d3d/d5d/l7f 0 2024-04-12T05:55:26.302 INFO:tasks.workunit.client.0.smithi008.stdout:6/270: fdatasync df/d10/d15/f13 0 2024-04-12T05:55:26.306 INFO:tasks.workunit.client.0.smithi008.stdout:8/149: getdents dd/de 0 2024-04-12T05:55:26.307 INFO:tasks.workunit.client.0.smithi008.stdout:6/271: rename df/d21/f27 to df/d21/d4c/f53 0 2024-04-12T05:55:26.308 INFO:tasks.workunit.client.0.smithi008.stdout:6/272: fsync df/d10/d15/f2a 0 2024-04-12T05:55:26.312 INFO:tasks.workunit.client.0.smithi008.stdout:5/392: truncate d0/dc/d13/f15 3071212 0 2024-04-12T05:55:26.318 INFO:tasks.workunit.client.0.smithi008.stdout:8/150: rename dd/de/f1d to dd/de/d16/f33 0 2024-04-12T05:55:26.318 INFO:tasks.workunit.client.0.smithi008.stdout:8/151: chown dd/l2b 2354 1 2024-04-12T05:55:26.318 INFO:tasks.workunit.client.0.smithi008.stdout:6/273: symlink df/d10/d15/l54 0 2024-04-12T05:55:26.320 INFO:tasks.workunit.client.0.smithi008.stdout:8/152: truncate dd/de/f1c 844636 0 2024-04-12T05:55:26.322 INFO:tasks.workunit.client.0.smithi008.stdout:5/393: mknod d0/dd/d5c/c80 0 2024-04-12T05:55:26.322 INFO:tasks.workunit.client.0.smithi008.stdout:8/153: dread - dd/de/f2d zero size 2024-04-12T05:55:26.326 INFO:tasks.workunit.client.0.smithi008.stdout:5/394: symlink d0/dd/d11/l81 0 2024-04-12T05:55:26.332 INFO:tasks.workunit.client.0.smithi008.stdout:6/274: truncate df/d10/d15/f2a 702821 0 2024-04-12T05:55:26.332 INFO:tasks.workunit.client.0.smithi008.stdout:0/277: truncate d2/d11/f40 1262140 0 2024-04-12T05:55:26.339 INFO:tasks.workunit.client.0.smithi008.stdout:8/154: write dd/de/d16/f2c [4406204,116107] 0 2024-04-12T05:55:26.341 INFO:tasks.workunit.client.0.smithi008.stdout:8/155: dread - dd/f1a zero size 2024-04-12T05:55:26.344 INFO:tasks.workunit.client.0.smithi008.stdout:8/156: chown dd/de/c31 1 1 2024-04-12T05:55:26.345 INFO:tasks.workunit.client.0.smithi008.stdout:5/395: write d0/dd/f1e [972902,71298] 0 2024-04-12T05:55:26.347 INFO:tasks.workunit.client.0.smithi008.stdout:8/157: dread - dd/f1a zero size 2024-04-12T05:55:26.349 INFO:tasks.workunit.client.0.smithi008.stdout:0/278: dwrite d2/fa [0,4194304] 0 2024-04-12T05:55:26.351 INFO:tasks.workunit.client.0.smithi008.stdout:5/396: write d0/dc/d13/d1a/d3a/f63 [292076,126523] 0 2024-04-12T05:55:26.356 INFO:tasks.workunit.client.0.smithi008.stdout:0/279: dread d2/d24/f28 [0,4194304] 0 2024-04-12T05:55:26.357 INFO:tasks.workunit.client.0.smithi008.stdout:8/158: link dd/de/c23 dd/de/c34 0 2024-04-12T05:55:26.358 INFO:tasks.workunit.client.0.smithi008.stdout:8/159: write dd/f1a [547802,88638] 0 2024-04-12T05:55:26.358 INFO:tasks.workunit.client.0.smithi008.stdout:5/397: link d0/c1 d0/dd/c82 0 2024-04-12T05:55:26.362 INFO:tasks.workunit.client.0.smithi008.stdout:8/160: rmdir dd/de/d16 39 2024-04-12T05:55:26.364 INFO:tasks.workunit.client.0.smithi008.stdout:8/161: write dd/de/f27 [934135,125931] 0 2024-04-12T05:55:26.365 INFO:tasks.workunit.client.0.smithi008.stdout:5/398: creat d0/dc/d13/d1a/f83 x:0 0 0 2024-04-12T05:55:26.368 INFO:tasks.workunit.client.0.smithi008.stdout:8/162: unlink lc 0 2024-04-12T05:55:26.369 INFO:tasks.workunit.client.0.smithi008.stdout:0/280: dwrite d2/ff [0,4194304] 0 2024-04-12T05:55:26.379 INFO:tasks.workunit.client.0.smithi008.stdout:0/281: unlink d2/fc 0 2024-04-12T05:55:26.381 INFO:tasks.workunit.client.0.smithi008.stdout:8/163: getdents dd/d30/d32 0 2024-04-12T05:55:26.383 INFO:tasks.workunit.client.0.smithi008.stdout:0/282: chown d2/d11/d36/d47/l1c 1337 1 2024-04-12T05:55:26.384 INFO:tasks.workunit.client.0.smithi008.stdout:7/256: sync 2024-04-12T05:55:26.387 INFO:tasks.workunit.client.0.smithi008.stdout:7/257: write d1/d15/f4a [1028038,111008] 0 2024-04-12T05:55:26.387 INFO:tasks.workunit.client.0.smithi008.stdout:5/399: dwrite d0/dc/d13/d1a/f24 [0,4194304] 0 2024-04-12T05:55:26.395 INFO:tasks.workunit.client.0.smithi008.stdout:5/400: rmdir d0/dd/d3d/d5d 39 2024-04-12T05:55:26.396 INFO:tasks.workunit.client.0.smithi008.stdout:0/283: dread d2/d11/d36/f46 [0,4194304] 0 2024-04-12T05:55:26.397 INFO:tasks.workunit.client.0.smithi008.stdout:8/164: dread dd/de/f12 [0,4194304] 0 2024-04-12T05:55:26.398 INFO:tasks.workunit.client.0.smithi008.stdout:7/258: creat d1/d5/da/d11/f57 x:0 0 0 2024-04-12T05:55:26.398 INFO:tasks.workunit.client.0.smithi008.stdout:0/284: fdatasync d2/d24/d39/f53 0 2024-04-12T05:55:26.400 INFO:tasks.workunit.client.0.smithi008.stdout:5/401: creat d0/d2/f84 x:0 0 0 2024-04-12T05:55:26.402 INFO:tasks.workunit.client.0.smithi008.stdout:5/402: dread - d0/dc/d13/d1a/d35/f64 zero size 2024-04-12T05:55:26.403 INFO:tasks.workunit.client.0.smithi008.stdout:0/285: symlink d2/d11/d36/l58 0 2024-04-12T05:55:26.403 INFO:tasks.workunit.client.0.smithi008.stdout:7/259: chown d1/l27 276 1 2024-04-12T05:55:26.409 INFO:tasks.workunit.client.0.smithi008.stdout:7/260: mkdir d1/d15/d43/d54/d58 0 2024-04-12T05:55:26.411 INFO:tasks.workunit.client.0.smithi008.stdout:7/261: write d1/f3 [1888752,75915] 0 2024-04-12T05:55:26.417 INFO:tasks.workunit.client.0.smithi008.stdout:8/165: write f7 [419964,79374] 0 2024-04-12T05:55:26.422 INFO:tasks.workunit.client.0.smithi008.stdout:7/262: dwrite d1/d49/f4b [0,4194304] 0 2024-04-12T05:55:26.426 INFO:tasks.workunit.client.0.smithi008.stdout:7/263: write d1/d5/da/d11/f23 [9232064,86184] 0 2024-04-12T05:55:26.426 INFO:tasks.workunit.client.0.smithi008.stdout:5/403: truncate d0/dc/d13/d1a/f24 30980 0 2024-04-12T05:55:26.428 INFO:tasks.workunit.client.0.smithi008.stdout:5/404: read d0/dd/d3d/f53 [57908,85575] 0 2024-04-12T05:55:26.428 INFO:tasks.workunit.client.0.smithi008.stdout:8/166: unlink dd/de/d16/f2c 0 2024-04-12T05:55:26.430 INFO:tasks.workunit.client.0.smithi008.stdout:5/405: read d0/d2/f21 [2233775,96902] 0 2024-04-12T05:55:26.435 INFO:tasks.workunit.client.0.smithi008.stdout:7/264: creat d1/d15/f59 x:0 0 0 2024-04-12T05:55:26.435 INFO:tasks.workunit.client.0.smithi008.stdout:8/167: getdents dd/d30/d32 0 2024-04-12T05:55:26.438 INFO:tasks.workunit.client.0.smithi008.stdout:7/265: write d1/f6 [1003860,71269] 0 2024-04-12T05:55:26.439 INFO:tasks.workunit.client.0.smithi008.stdout:7/266: chown d1/d5/da/d11/f23 115630021 1 2024-04-12T05:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.445+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c402d050 con 0x7ff9b006ca20 2024-04-12T05:55:26.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.446+0000 7ff9bb7fe700 1 -- 172.21.15.8:0/2595261309 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff9c402d050 con 0x7ff9b006ca20 2024-04-12T05:55:26.443 INFO:tasks.workunit.client.0.smithi008.stdout:5/406: getdents d0/dc 0 2024-04-12T05:55:26.444 INFO:tasks.workunit.client.0.smithi008.stdout:8/168: stat dd/c1b 0 2024-04-12T05:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 msgr2=0x7ff9b006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 0x7ff9b006eee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4100c20 tx=0x7ff9b4009260 comp rx=0 tx=0).stop 2024-04-12T05:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 msgr2=0x7ff9c419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000bde0 tx=0x7ff9c000be10 comp rx=0 tx=0).stop 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 shutdown_connections 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7ff9b006ca20 0x7ff9b006eee0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9c40ffbc0 0x7ff9c4069970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 --2- 172.21.15.8:0/2595261309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9c4100dc0 0x7ff9c419d890 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 >> 172.21.15.8:0/2595261309 conn(0x7ff9c40fb160 msgr2=0x7ff9c41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:26.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.449+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 shutdown_connections 2024-04-12T05:55:26.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.450+0000 7ff9cc7e9700 1 -- 172.21.15.8:0/2595261309 wait complete. 2024-04-12T05:55:26.447 INFO:tasks.workunit.client.0.smithi008.stdout:7/267: mkdir d1/d15/d43/d5a 0 2024-04-12T05:55:26.447 INFO:tasks.workunit.client.0.smithi008.stdout:5/407: truncate d0/dc/f4f 517473 0 2024-04-12T05:55:26.448 INFO:tasks.workunit.client.0.smithi008.stdout:7/268: chown d1/d49/f40 2 1 2024-04-12T05:55:26.452 INFO:tasks.workunit.client.0.smithi008.stdout:8/169: rename f9 to dd/d30/f35 0 2024-04-12T05:55:26.453 INFO:tasks.workunit.client.0.smithi008.stdout:5/408: write d0/dd/d3d/d5d/f7e [4345562,58045] 0 2024-04-12T05:55:26.455 INFO:tasks.workunit.client.0.smithi008.stdout:8/170: creat dd/f36 x:0 0 0 2024-04-12T05:55:26.456 INFO:tasks.workunit.client.0.smithi008.stdout:8/171: chown dd/f1a 120106700 1 2024-04-12T05:55:26.457 INFO:tasks.workunit.client.0.smithi008.stdout:7/269: rename d1/d15/l1e to d1/d5/da/d11/l5b 0 2024-04-12T05:55:26.458 INFO:tasks.workunit.client.0.smithi008.stdout:8/172: read dd/de/d16/f28 [2152345,80226] 0 2024-04-12T05:55:26.460 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:55:26.464 INFO:tasks.workunit.client.0.smithi008.stdout:7/270: rename d1/d5/da/d3c/d3d/f52 to d1/d5/da/d1d/f5c 0 2024-04-12T05:55:26.466 INFO:tasks.workunit.client.0.smithi008.stdout:5/409: chown d0/dc/d13/d1a/f24 1 1 2024-04-12T05:55:26.467 INFO:tasks.workunit.client.0.smithi008.stdout:5/410: dread - d0/dc/d13/d1a/d35/f64 zero size 2024-04-12T05:55:26.468 INFO:tasks.workunit.client.0.smithi008.stdout:7/271: mknod d1/d5/da/d11/d47/c5d 0 2024-04-12T05:55:26.470 INFO:tasks.workunit.client.0.smithi008.stdout:5/411: write d0/d2/f84 [792879,93572] 0 2024-04-12T05:55:26.482 INFO:tasks.workunit.client.0.smithi008.stdout:5/412: truncate d0/d2/f2c 1221392 0 2024-04-12T05:55:26.484 INFO:tasks.workunit.client.0.smithi008.stdout:7/272: dwrite d1/d15/f4a [0,4194304] 0 2024-04-12T05:55:26.490 INFO:tasks.workunit.client.0.smithi008.stdout:7/273: unlink d1/d15/f26 0 2024-04-12T05:55:26.498 INFO:tasks.workunit.client.0.smithi008.stdout:7/274: mkdir d1/d15/d43/d54/d58/d5e 0 2024-04-12T05:55:26.499 INFO:tasks.workunit.client.0.smithi008.stdout:7/275: truncate d1/d5/f42 332540 0 2024-04-12T05:55:26.502 INFO:tasks.workunit.client.0.smithi008.stdout:7/276: readlink d1/d5/da/d11/l5b 0 2024-04-12T05:55:26.504 INFO:tasks.workunit.client.0.smithi008.stdout:5/413: getdents d0/dd/d11/d38/d55 0 2024-04-12T05:55:26.506 INFO:tasks.workunit.client.0.smithi008.stdout:7/277: symlink d1/d15/d43/d54/d58/l5f 0 2024-04-12T05:55:26.512 INFO:tasks.workunit.client.0.smithi008.stdout:5/414: read d0/dd/d11/f19 [537819,44115] 0 2024-04-12T05:55:26.515 INFO:tasks.workunit.client.0.smithi008.stdout:5/415: creat d0/dc/d13/d1a/d35/f85 x:0 0 0 2024-04-12T05:55:26.520 INFO:tasks.workunit.client.0.smithi008.stdout:5/416: chown d0/dc/d13/d1a/c71 3 1 2024-04-12T05:55:26.521 INFO:tasks.workunit.client.0.smithi008.stdout:5/417: stat d0/c1 0 2024-04-12T05:55:26.523 INFO:tasks.workunit.client.0.smithi008.stdout:7/278: dread d1/d5/f2e [0,4194304] 0 2024-04-12T05:55:26.524 INFO:tasks.workunit.client.0.smithi008.stdout:5/418: dread d0/d2/f31 [0,4194304] 0 2024-04-12T05:55:26.524 INFO:tasks.workunit.client.0.smithi008.stdout:7/279: fsync d1/d5/f2e 0 2024-04-12T05:55:26.526 INFO:tasks.workunit.client.0.smithi008.stdout:5/419: chown d0/dc/f68 1349015 1 2024-04-12T05:55:26.530 INFO:tasks.workunit.client.0.smithi008.stdout:5/420: symlink d0/dc/d13/d1a/d35/l86 0 2024-04-12T05:55:26.533 INFO:tasks.workunit.client.0.smithi008.stdout:7/280: rename d1/d5/da/d11/d47 to d1/d5/da/d60 0 2024-04-12T05:55:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:7/281: symlink d1/l61 0 2024-04-12T05:55:26.535 INFO:tasks.workunit.client.0.smithi008.stdout:5/421: read d0/dc/f2a [1249663,4305] 0 2024-04-12T05:55:26.537 INFO:tasks.workunit.client.0.smithi008.stdout:5/422: write d0/dc/d57/f2f [4835438,55590] 0 2024-04-12T05:55:26.542 INFO:tasks.workunit.client.0.smithi008.stdout:5/423: mknod d0/dc/c87 0 2024-04-12T05:55:26.544 INFO:tasks.workunit.client.0.smithi008.stdout:7/282: creat d1/d5/da/f62 x:0 0 0 2024-04-12T05:55:26.548 INFO:tasks.workunit.client.0.smithi008.stdout:7/283: mknod d1/d5/da/d1d/c63 0 2024-04-12T05:55:26.550 INFO:tasks.workunit.client.0.smithi008.stdout:7/284: write d1/d5/f33 [743535,68769] 0 2024-04-12T05:55:26.554 INFO:tasks.workunit.client.0.smithi008.stdout:5/424: write d0/dc/d13/d1a/d35/f42 [361147,86376] 0 2024-04-12T05:55:26.557 INFO:tasks.workunit.client.0.smithi008.stdout:7/285: unlink d1/d5/da/d60/c50 0 2024-04-12T05:55:26.558 INFO:tasks.workunit.client.0.smithi008.stdout:7/286: dread - d1/d15/d43/f4c zero size 2024-04-12T05:55:26.558 INFO:tasks.workunit.client.0.smithi008.stdout:5/425: mknod d0/dd/d11/c88 0 2024-04-12T05:55:26.560 INFO:tasks.workunit.client.0.smithi008.stdout:5/426: rmdir d0/d2/d37 39 2024-04-12T05:55:26.563 INFO:tasks.workunit.client.0.smithi008.stdout:5/427: fdatasync d0/dc/d13/d1a/f3f 0 2024-04-12T05:55:26.567 INFO:tasks.workunit.client.0.smithi008.stdout:7/287: dwrite d1/f55 [0,4194304] 0 2024-04-12T05:55:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:5/428: mkdir d0/dd/d11/d38/d55/d89 0 2024-04-12T05:55:26.570 INFO:tasks.workunit.client.0.smithi008.stdout:7/288: chown d1/d15/d43/d54/d58/l5f 218013 1 2024-04-12T05:55:26.578 INFO:tasks.workunit.client.0.smithi008.stdout:7/289: symlink d1/d5/da/d3c/l64 0 2024-04-12T05:55:26.578 INFO:tasks.workunit.client.0.smithi008.stdout:7/290: creat d1/d5/da/d11/f65 x:0 0 0 2024-04-12T05:55:26.583 INFO:tasks.workunit.client.0.smithi008.stdout:7/291: read d1/d5/f4e [141970,88891] 0 2024-04-12T05:55:26.584 INFO:tasks.workunit.client.0.smithi008.stdout:5/429: dwrite d0/dc/f65 [0,4194304] 0 2024-04-12T05:55:26.588 INFO:tasks.workunit.client.0.smithi008.stdout:7/292: creat d1/d15/f66 x:0 0 0 2024-04-12T05:55:26.588 INFO:tasks.workunit.client.0.smithi008.stdout:5/430: chown d0/dd/d11/d38/f43 43270720 1 2024-04-12T05:55:26.588 INFO:tasks.workunit.client.0.smithi008.stdout:7/293: dread - d1/d5/da/d1d/f5c zero size 2024-04-12T05:55:26.589 INFO:tasks.workunit.client.0.smithi008.stdout:7/294: chown d1/c3e 0 1 2024-04-12T05:55:26.596 INFO:tasks.workunit.client.0.smithi008.stdout:5/431: write d0/dc/d13/d1a/f22 [2120953,69470] 0 2024-04-12T05:55:26.596 INFO:tasks.workunit.client.0.smithi008.stdout:7/295: creat d1/d15/d43/d54/d58/f67 x:0 0 0 2024-04-12T05:55:26.599 INFO:tasks.workunit.client.0.smithi008.stdout:7/296: readlink d1/d5/da/d1d/l24 0 2024-04-12T05:55:26.599 INFO:tasks.workunit.client.0.smithi008.stdout:7/297: chown d1/l2 4672650 1 2024-04-12T05:55:26.599 INFO:tasks.workunit.client.0.smithi008.stdout:7/298: chown d1/l28 7642709 1 2024-04-12T05:55:26.601 INFO:tasks.workunit.client.0.smithi008.stdout:7/299: readlink d1/l61 0 2024-04-12T05:55:26.602 INFO:tasks.workunit.client.0.smithi008.stdout:7/300: chown d1/d15/d43/d54/d58/f67 4002786 1 2024-04-12T05:55:26.602 INFO:tasks.workunit.client.0.smithi008.stdout:5/432: dread d0/dc/f65 [0,4194304] 0 2024-04-12T05:55:26.603 INFO:tasks.workunit.client.0.smithi008.stdout:7/301: chown d1/d15/d43/d54/d58 3248975 1 2024-04-12T05:55:26.605 INFO:tasks.workunit.client.0.smithi008.stdout:7/302: mkdir d1/d5/da/d1d/d68 0 2024-04-12T05:55:26.605 INFO:tasks.workunit.client.0.smithi008.stdout:7/303: dread - d1/d49/f53 zero size 2024-04-12T05:55:26.607 INFO:tasks.workunit.client.0.smithi008.stdout:5/433: symlink d0/dc/d13/d1a/l8a 0 2024-04-12T05:55:26.611 INFO:tasks.workunit.client.0.smithi008.stdout:7/304: rmdir d1/d5/da/d60 39 2024-04-12T05:55:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 -- 172.21.15.8:0/1142612348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc680ff540 msgr2=0x7fcc68101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/1142612348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc680ff540 0x7fcc68101970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009910 tx=0x7fcc58009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:26.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 -- 172.21.15.8:0/1142612348 shutdown_connections 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/1142612348 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc68101eb0 0x7fcc68104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/1142612348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc680ff540 0x7fcc68101970 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009910 tx=0x7fcc58009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.616+0000 7fcc6fab3700 1 -- 172.21.15.8:0/1142612348 >> 172.21.15.8:0/1142612348 conn(0x7fcc680fb160 msgr2=0x7fcc680fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.618+0000 7fcc6fab3700 1 -- 172.21.15.8:0/1142612348 shutdown_connections 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.618+0000 7fcc6fab3700 1 -- 172.21.15.8:0/1142612348 wait complete. 2024-04-12T05:55:26.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.619+0000 7fcc6fab3700 1 Processor -- start 2024-04-12T05:55:26.614 INFO:tasks.workunit.client.0.smithi008.stdout:5/434: dread d0/d2/f84 [0,4194304] 0 2024-04-12T05:55:26.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.619+0000 7fcc6fab3700 1 -- start start 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6fab3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6fab3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc681b6c90 0x7fcc681bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6fab3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681b71a0 con 0x7fcc681b6c90 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6fab3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681b7310 con 0x7fcc68101eb0 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:38114/0 (socket says 172.21.15.8:38114) 2024-04-12T05:55:26.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 -- 172.21.15.8:0/3958569969 learned_addr learned my addr 172.21.15.8:0/3958569969 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 -- 172.21.15.8:0/3958569969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc681b6c90 msgr2=0x7fcc681bbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc681b6c90 0x7fcc681bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.620+0000 7fcc6d84f700 1 -- 172.21.15.8:0/3958569969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580095f0 con 0x7fcc68101eb0 2024-04-12T05:55:26.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc6d84f700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc5800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc5801d070 con 0x7fcc68101eb0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fcc5800f460 con 0x7fcc68101eb0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc681bc240 con 0x7fcc68101eb0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc58017630 con 0x7fcc68101eb0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.621+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc681bc730 con 0x7fcc68101eb0 2024-04-12T05:55:26.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.622+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc681b02e0 con 0x7fcc68101eb0 2024-04-12T05:55:26.619 INFO:tasks.workunit.client.0.smithi008.stdout:7/305: rename d1/d15/f4a to d1/d5/da/d11/f69 0 2024-04-12T05:55:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.623+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7fcc58003a60 con 0x7fcc68101eb0 2024-04-12T05:55:26.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.623+0000 7fcc5effd700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 0x7fcc5406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.624+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fcc5808d420 con 0x7fcc68101eb0 2024-04-12T05:55:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.624+0000 7fcc6d04e700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 0x7fcc5406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:26.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.625+0000 7fcc6d04e700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 0x7fcc5406ef20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005950 tx=0x7fcc640058e0 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:26.622 INFO:tasks.workunit.client.0.smithi008.stdout:5/435: mknod d0/d2/d37/c8b 0 2024-04-12T05:55:26.623 INFO:tasks.workunit.client.0.smithi008.stdout:7/306: symlink d1/d15/l6a 0 2024-04-12T05:55:26.624 INFO:tasks.workunit.client.0.smithi008.stdout:5/436: write d0/d40/d56/f67 [365312,17080] 0 2024-04-12T05:55:26.624 INFO:tasks.workunit.client.0.smithi008.stdout:7/307: readlink d1/d5/da/d1d/l2b 0 2024-04-12T05:55:26.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:26.630+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fcc5805af80 con 0x7fcc68101eb0 2024-04-12T05:55:26.627 INFO:tasks.workunit.client.0.smithi008.stdout:5/437: dread d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:26.643 INFO:tasks.workunit.client.0.smithi008.stdout:7/308: dread d1/d15/f20 [4194304,4194304] 0 2024-04-12T05:55:26.644 INFO:tasks.workunit.client.0.smithi008.stdout:7/309: truncate d1/d5/f32 853295 0 2024-04-12T05:55:26.645 INFO:tasks.workunit.client.0.smithi008.stdout:7/310: fdatasync d1/f3 0 2024-04-12T05:55:26.646 INFO:tasks.workunit.client.0.smithi008.stdout:7/311: readlink d1/d15/d43/d54/d58/l5f 0 2024-04-12T05:55:26.657 INFO:tasks.workunit.client.0.smithi008.stdout:7/312: rename d1/l9 to d1/d5/da/d1d/l6b 0 2024-04-12T05:55:26.657 INFO:tasks.workunit.client.0.smithi008.stdout:7/313: chown d1/d5/da/d11/d56 22605 1 2024-04-12T05:55:26.665 INFO:tasks.workunit.client.0.smithi008.stdout:3/251: sync 2024-04-12T05:55:26.665 INFO:tasks.workunit.client.0.smithi008.stdout:9/235: sync 2024-04-12T05:55:26.665 INFO:tasks.workunit.client.0.smithi008.stdout:1/220: sync 2024-04-12T05:55:26.665 INFO:tasks.workunit.client.0.smithi008.stdout:2/291: sync 2024-04-12T05:55:26.668 INFO:tasks.workunit.client.0.smithi008.stdout:4/138: sync 2024-04-12T05:55:26.669 INFO:tasks.workunit.client.0.smithi008.stdout:5/438: dwrite d0/d2/f2c [0,4194304] 0 2024-04-12T05:55:26.672 INFO:tasks.workunit.client.0.smithi008.stdout:4/139: fsync d1/f13 0 2024-04-12T05:55:26.675 INFO:tasks.workunit.client.0.smithi008.stdout:2/292: read d6/db/f14 [2124659,66530] 0 2024-04-12T05:55:26.680 INFO:tasks.workunit.client.0.smithi008.stdout:3/252: symlink d9/d37/l54 0 2024-04-12T05:55:26.686 INFO:tasks.workunit.client.0.smithi008.stdout:2/293: dread - d6/db/d10/d4c/f5a zero size 2024-04-12T05:55:26.690 INFO:tasks.workunit.client.0.smithi008.stdout:4/140: creat d1/d8/f25 x:0 0 0 2024-04-12T05:55:26.692 INFO:tasks.workunit.client.0.smithi008.stdout:1/221: dread d4/dd/d17/f31 [0,4194304] 0 2024-04-12T05:55:26.693 INFO:tasks.workunit.client.0.smithi008.stdout:3/253: unlink d9/d10/d19/f52 0 2024-04-12T05:55:26.695 INFO:tasks.workunit.client.0.smithi008.stdout:1/222: chown d4/dd/d17/d28/d46 2877839 1 2024-04-12T05:55:26.698 INFO:tasks.workunit.client.0.smithi008.stdout:7/314: dwrite d1/d5/f42 [0,4194304] 0 2024-04-12T05:55:26.700 INFO:tasks.workunit.client.0.smithi008.stdout:1/223: write d4/d7/ff [810428,84954] 0 2024-04-12T05:55:26.704 INFO:tasks.workunit.client.0.smithi008.stdout:9/236: mkdir d3/df/d13/d21/d2d/d32/d52 0 2024-04-12T05:55:26.707 INFO:tasks.workunit.client.0.smithi008.stdout:2/294: symlink d6/db/d10/d23/d35/d3c/l62 0 2024-04-12T05:55:26.707 INFO:tasks.workunit.client.0.smithi008.stdout:4/141: fdatasync d1/f12 0 2024-04-12T05:55:26.709 INFO:tasks.workunit.client.0.smithi008.stdout:7/315: write d1/d15/d43/d54/d58/f67 [94362,21142] 0 2024-04-12T05:55:26.717 INFO:tasks.workunit.client.0.smithi008.stdout:5/439: getdents d0/d2/d37 0 2024-04-12T05:55:26.720 INFO:tasks.workunit.client.0.smithi008.stdout:7/316: chown d1/d5/da/d3c/d44/f51 91627 1 2024-04-12T05:55:26.723 INFO:tasks.workunit.client.0.smithi008.stdout:0/286: sync 2024-04-12T05:55:26.724 INFO:tasks.workunit.client.0.smithi008.stdout:6/275: sync 2024-04-12T05:55:26.725 INFO:tasks.workunit.client.0.smithi008.stdout:4/142: dwrite d1/d8/f21 [0,4194304] 0 2024-04-12T05:55:26.730 INFO:tasks.workunit.client.0.smithi008.stdout:0/287: stat d2/d24/f28 0 2024-04-12T05:55:26.732 INFO:tasks.workunit.client.0.smithi008.stdout:2/295: creat d6/db/d38/f63 x:0 0 0 2024-04-12T05:55:26.742 INFO:tasks.workunit.client.0.smithi008.stdout:3/254: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:26.742 INFO:tasks.workunit.client.0.smithi008.stdout:9/237: dwrite d3/df/d13/d21/d2d/f47 [0,4194304] 0 2024-04-12T05:55:26.744 INFO:tasks.workunit.client.0.smithi008.stdout:6/276: write df/d21/d4c/f53 [400403,128757] 0 2024-04-12T05:55:26.748 INFO:tasks.workunit.client.0.smithi008.stdout:1/224: truncate d4/d7/fc 453749 0 2024-04-12T05:55:26.748 INFO:tasks.workunit.client.0.smithi008.stdout:3/255: dread - d9/d2e/d3d/f51 zero size 2024-04-12T05:55:26.748 INFO:tasks.workunit.client.0.smithi008.stdout:6/277: stat f9 0 2024-04-12T05:55:26.750 INFO:tasks.workunit.client.0.smithi008.stdout:6/278: dread - df/d10/d15/f4b zero size 2024-04-12T05:55:26.753 INFO:tasks.workunit.client.0.smithi008.stdout:0/288: creat d2/d11/d36/d47/f59 x:0 0 0 2024-04-12T05:55:26.755 INFO:tasks.workunit.client.0.smithi008.stdout:1/225: chown d4/d7/d18 46057 1 2024-04-12T05:55:26.755 INFO:tasks.workunit.client.0.smithi008.stdout:9/238: mknod d3/d22/d2b/c53 0 2024-04-12T05:55:26.756 INFO:tasks.workunit.client.0.smithi008.stdout:4/143: dwrite d1/fe [0,4194304] 0 2024-04-12T05:55:26.762 INFO:tasks.workunit.client.0.smithi008.stdout:6/279: chown df/d10/d20/d3e 1885766 1 2024-04-12T05:55:26.762 INFO:tasks.workunit.client.0.smithi008.stdout:9/239: dread - d3/df/d10/d12/f48 zero size 2024-04-12T05:55:26.765 INFO:tasks.workunit.client.0.smithi008.stdout:7/317: creat d1/f6c x:0 0 0 2024-04-12T05:55:26.767 INFO:tasks.workunit.client.0.smithi008.stdout:2/296: mknod d6/db/d60/d57/c64 0 2024-04-12T05:55:26.768 INFO:tasks.workunit.client.0.smithi008.stdout:5/440: dwrite d0/dc/d13/d1a/f24 [0,4194304] 0 2024-04-12T05:55:26.772 INFO:tasks.workunit.client.0.smithi008.stdout:3/256: creat d9/d10/f55 x:0 0 0 2024-04-12T05:55:26.777 INFO:tasks.workunit.client.0.smithi008.stdout:0/289: link d2/d24/d39/f53 d2/d24/d39/f5a 0 2024-04-12T05:55:26.779 INFO:tasks.workunit.client.0.smithi008.stdout:5/441: chown d0/dd/d11/d38/c76 299602 1 2024-04-12T05:55:26.783 INFO:tasks.workunit.client.0.smithi008.stdout:6/280: dread fd [0,4194304] 0 2024-04-12T05:55:26.785 INFO:tasks.workunit.client.0.smithi008.stdout:4/144: dwrite d1/fe [0,4194304] 0 2024-04-12T05:55:26.786 INFO:tasks.workunit.client.0.smithi008.stdout:7/318: dread d1/f55 [0,4194304] 0 2024-04-12T05:55:26.787 INFO:tasks.workunit.client.0.smithi008.stdout:4/145: write d1/db/fc [196321,39026] 0 2024-04-12T05:55:26.789 INFO:tasks.workunit.client.0.smithi008.stdout:7/319: write d1/d5/da/d11/f25 [4399572,30595] 0 2024-04-12T05:55:26.789 INFO:tasks.workunit.client.0.smithi008.stdout:3/257: dwrite f1 [4194304,4194304] 0 2024-04-12T05:55:26.797 INFO:tasks.workunit.client.0.smithi008.stdout:4/146: dwrite d1/f13 [0,4194304] 0 2024-04-12T05:55:26.812 INFO:tasks.workunit.client.0.smithi008.stdout:4/147: dwrite d1/d8/f21 [0,4194304] 0 2024-04-12T05:55:26.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:26 smithi194 ceph-mon[49771]: pgmap v9: 65 pgs: 65 active+clean; 527 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 74 MiB/s wr, 399 op/s 2024-04-12T05:55:26.829 INFO:tasks.workunit.client.0.smithi008.stdout:0/290: readlink d2/d11/l14 0 2024-04-12T05:55:26.830 INFO:tasks.workunit.client.0.smithi008.stdout:5/442: rmdir d0/dd/d3d/d5d 39 2024-04-12T05:55:26.832 INFO:tasks.workunit.client.0.smithi008.stdout:6/281: creat df/d21/f55 x:0 0 0 2024-04-12T05:55:26.833 INFO:tasks.workunit.client.0.smithi008.stdout:6/282: write df/d21/d4c/f53 [1303033,2375] 0 2024-04-12T05:55:26.833 INFO:tasks.workunit.client.0.smithi008.stdout:3/258: creat d9/d2e/d3d/f56 x:0 0 0 2024-04-12T05:55:26.834 INFO:tasks.workunit.client.0.smithi008.stdout:3/259: truncate d9/f25 990538 0 2024-04-12T05:55:26.836 INFO:tasks.workunit.client.0.smithi008.stdout:4/148: creat d1/db/d19/f26 x:0 0 0 2024-04-12T05:55:26.837 INFO:tasks.workunit.client.0.smithi008.stdout:4/149: write d1/d8/f21 [1115197,68250] 0 2024-04-12T05:55:26.838 INFO:tasks.workunit.client.0.smithi008.stdout:5/443: mknod d0/d40/c8c 0 2024-04-12T05:55:26.839 INFO:tasks.workunit.client.0.smithi008.stdout:1/226: truncate d4/d1c/f2b 2896584 0 2024-04-12T05:55:26.840 INFO:tasks.workunit.client.0.smithi008.stdout:5/444: fdatasync d0/dd/d11/d38/f43 0 2024-04-12T05:55:26.840 INFO:tasks.workunit.client.0.smithi008.stdout:1/227: chown d4/d24/f45 19 1 2024-04-12T05:55:26.841 INFO:tasks.workunit.client.0.smithi008.stdout:5/445: fdatasync d0/d40/d56/f67 0 2024-04-12T05:55:26.843 INFO:tasks.workunit.client.0.smithi008.stdout:2/297: rename d6/db/d38/d41/l5e to d6/db/d10/d23/d35/l65 0 2024-04-12T05:55:26.844 INFO:tasks.workunit.client.0.smithi008.stdout:2/298: chown d6/db/d38/d41 119405 1 2024-04-12T05:55:26.845 INFO:tasks.workunit.client.0.smithi008.stdout:4/150: rmdir d1/d8 39 2024-04-12T05:55:26.845 INFO:tasks.workunit.client.0.smithi008.stdout:4/151: dread - d1/f20 zero size 2024-04-12T05:55:26.846 INFO:tasks.workunit.client.0.smithi008.stdout:4/152: stat d1 0 2024-04-12T05:55:26.847 INFO:tasks.workunit.client.0.smithi008.stdout:3/260: link d9/d11/f49 d9/d1e/f57 0 2024-04-12T05:55:26.848 INFO:tasks.workunit.client.0.smithi008.stdout:3/261: stat c4 0 2024-04-12T05:55:26.849 INFO:tasks.workunit.client.0.smithi008.stdout:7/320: rename d1/d5/da/d1d/c63 to d1/d49/c6d 0 2024-04-12T05:55:26.850 INFO:tasks.workunit.client.0.smithi008.stdout:7/321: dread - d1/d5/f4d zero size 2024-04-12T05:55:26.851 INFO:tasks.workunit.client.0.smithi008.stdout:6/283: creat df/d10/f56 x:0 0 0 2024-04-12T05:55:26.851 INFO:tasks.workunit.client.0.smithi008.stdout:3/262: dread d9/f25 [0,4194304] 0 2024-04-12T05:55:26.851 INFO:tasks.workunit.client.0.smithi008.stdout:2/299: truncate d6/db/d38/f25 1998257 0 2024-04-12T05:55:26.854 INFO:tasks.workunit.client.0.smithi008.stdout:1/228: creat d4/f47 x:0 0 0 2024-04-12T05:55:26.856 INFO:tasks.workunit.client.0.smithi008.stdout:6/284: chown df/d10/f2c 113 1 2024-04-12T05:55:26.858 INFO:tasks.workunit.client.0.smithi008.stdout:5/446: link d0/dc/d57/l25 d0/d29/l8d 0 2024-04-12T05:55:26.858 INFO:tasks.workunit.client.0.smithi008.stdout:1/229: readlink d4/dd/d17/l1e 0 2024-04-12T05:55:26.859 INFO:tasks.workunit.client.0.smithi008.stdout:7/322: write d1/d5/da/d11/f69 [4600552,6561] 0 2024-04-12T05:55:26.864 INFO:tasks.workunit.client.0.smithi008.stdout:5/447: fsync d0/d2/f31 0 2024-04-12T05:55:26.868 INFO:tasks.workunit.client.0.smithi008.stdout:1/230: rename d4/f3e to d4/d7/f48 0 2024-04-12T05:55:26.869 INFO:tasks.workunit.client.0.smithi008.stdout:2/300: dwrite d6/db/d60/d4f/f61 [0,4194304] 0 2024-04-12T05:55:26.869 INFO:tasks.workunit.client.0.smithi008.stdout:3/263: dwrite d9/d10/f55 [0,4194304] 0 2024-04-12T05:55:26.871 INFO:tasks.workunit.client.0.smithi008.stdout:5/448: write d0/dc/d13/d1a/f22 [311688,31728] 0 2024-04-12T05:55:26.873 INFO:tasks.workunit.client.0.smithi008.stdout:7/323: symlink d1/d15/d43/d5a/l6e 0 2024-04-12T05:55:26.881 INFO:tasks.workunit.client.0.smithi008.stdout:1/231: creat d4/dd/d17/d28/d30/f49 x:0 0 0 2024-04-12T05:55:26.884 INFO:tasks.workunit.client.0.smithi008.stdout:5/449: write d0/dd/d11/d38/d55/f5b [236526,66809] 0 2024-04-12T05:55:26.888 INFO:tasks.workunit.client.0.smithi008.stdout:1/232: symlink d4/d6/d11/l4a 0 2024-04-12T05:55:26.890 INFO:tasks.workunit.client.0.smithi008.stdout:2/301: rename d6/db/d10/d23/d35/l65 to d6/db/d10/d23/d35/d3c/l66 0 2024-04-12T05:55:26.893 INFO:tasks.workunit.client.0.smithi008.stdout:6/285: write df/f1f [1762053,113357] 0 2024-04-12T05:55:26.893 INFO:tasks.workunit.client.0.smithi008.stdout:8/173: sync 2024-04-12T05:55:26.895 INFO:tasks.workunit.client.0.smithi008.stdout:6/286: fdatasync fb 0 2024-04-12T05:55:26.895 INFO:tasks.workunit.client.0.smithi008.stdout:1/233: symlink d4/d6/l4b 0 2024-04-12T05:55:26.897 INFO:tasks.workunit.client.0.smithi008.stdout:7/324: rename d1/d15/d43/f4c to d1/d5/da/d3c/d44/f6f 0 2024-04-12T05:55:26.898 INFO:tasks.workunit.client.0.smithi008.stdout:5/450: dwrite d0/dd/d11/d38/f2b [4194304,4194304] 0 2024-04-12T05:55:26.899 INFO:tasks.workunit.client.0.smithi008.stdout:8/174: readlink l6 0 2024-04-12T05:55:26.900 INFO:tasks.workunit.client.0.smithi008.stdout:8/175: chown dd/f1e 20664 1 2024-04-12T05:55:26.902 INFO:tasks.workunit.client.0.smithi008.stdout:6/287: creat df/d21/f57 x:0 0 0 2024-04-12T05:55:26.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:26 smithi008 ceph-mon[39722]: pgmap v9: 65 pgs: 65 active+clean; 527 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 74 MiB/s wr, 399 op/s 2024-04-12T05:55:26.907 INFO:tasks.workunit.client.0.smithi008.stdout:3/264: write d9/dd/f36 [1150912,22460] 0 2024-04-12T05:55:26.907 INFO:tasks.workunit.client.0.smithi008.stdout:8/176: symlink dd/d30/l37 0 2024-04-12T05:55:26.909 INFO:tasks.workunit.client.0.smithi008.stdout:1/234: read d4/d1c/f2b [1688532,111580] 0 2024-04-12T05:55:26.913 INFO:tasks.workunit.client.0.smithi008.stdout:3/265: mknod d9/d2e/d3d/c58 0 2024-04-12T05:55:26.913 INFO:tasks.workunit.client.0.smithi008.stdout:7/325: mkdir d1/d5/da/d70 0 2024-04-12T05:55:26.914 INFO:tasks.workunit.client.0.smithi008.stdout:7/326: chown d1/d5 483 1 2024-04-12T05:55:26.917 INFO:tasks.workunit.client.0.smithi008.stdout:8/177: dwrite dd/ff [0,4194304] 0 2024-04-12T05:55:26.921 INFO:tasks.workunit.client.0.smithi008.stdout:5/451: creat d0/dd/d3d/f8e x:0 0 0 2024-04-12T05:55:26.921 INFO:tasks.workunit.client.0.smithi008.stdout:2/302: write d6/db/fd [577939,7442] 0 2024-04-12T05:55:26.922 INFO:tasks.workunit.client.0.smithi008.stdout:7/327: symlink d1/d49/l71 0 2024-04-12T05:55:26.924 INFO:tasks.workunit.client.0.smithi008.stdout:5/452: fdatasync d0/dc/d13/f72 0 2024-04-12T05:55:26.925 INFO:tasks.workunit.client.0.smithi008.stdout:7/328: chown d1/l16 2 1 2024-04-12T05:55:26.925 INFO:tasks.workunit.client.0.smithi008.stdout:1/235: write d4/d24/f32 [3147675,100730] 0 2024-04-12T05:55:26.928 INFO:tasks.workunit.client.0.smithi008.stdout:6/288: truncate df/d21/f22 2601571 0 2024-04-12T05:55:26.929 INFO:tasks.workunit.client.0.smithi008.stdout:3/266: creat d9/d2e/f59 x:0 0 0 2024-04-12T05:55:26.930 INFO:tasks.workunit.client.0.smithi008.stdout:2/303: rename d6/db/d38/d41/c51 to d6/db/d29/c67 0 2024-04-12T05:55:26.931 INFO:tasks.workunit.client.0.smithi008.stdout:6/289: dread - df/d21/f57 zero size 2024-04-12T05:55:26.932 INFO:tasks.workunit.client.0.smithi008.stdout:1/236: truncate d4/d6/f40 591863 0 2024-04-12T05:55:26.934 INFO:tasks.workunit.client.0.smithi008.stdout:7/329: mknod d1/d5/da/d3c/d44/c72 0 2024-04-12T05:55:26.935 INFO:tasks.workunit.client.0.smithi008.stdout:3/267: fsync d9/d2e/d3d/f51 0 2024-04-12T05:55:26.941 INFO:tasks.workunit.client.0.smithi008.stdout:6/290: dread - df/d10/f56 zero size 2024-04-12T05:55:26.941 INFO:tasks.workunit.client.0.smithi008.stdout:8/178: dwrite dd/de/f29 [0,4194304] 0 2024-04-12T05:55:26.949 INFO:tasks.workunit.client.0.smithi008.stdout:8/179: write dd/de/f29 [2168815,126942] 0 2024-04-12T05:55:26.949 INFO:tasks.workunit.client.0.smithi008.stdout:2/304: mknod d6/db/d10/d23/d35/d3c/c68 0 2024-04-12T05:55:26.955 INFO:tasks.workunit.client.0.smithi008.stdout:7/330: mknod d1/d5/da/d60/c73 0 2024-04-12T05:55:26.955 INFO:tasks.workunit.client.0.smithi008.stdout:8/180: fdatasync dd/de/d16/f17 0 2024-04-12T05:55:26.955 INFO:tasks.workunit.client.0.smithi008.stdout:3/268: creat d9/d1e/f5a x:0 0 0 2024-04-12T05:55:26.955 INFO:tasks.workunit.client.0.smithi008.stdout:5/453: dwrite d0/dd/d3d/d5d/f78 [0,4194304] 0 2024-04-12T05:55:26.955 INFO:tasks.workunit.client.0.smithi008.stdout:6/291: creat df/d21/d4c/f58 x:0 0 0 2024-04-12T05:55:26.959 INFO:tasks.workunit.client.0.smithi008.stdout:1/237: write d4/d7/fe [1143185,79796] 0 2024-04-12T05:55:26.959 INFO:tasks.workunit.client.0.smithi008.stdout:5/454: write d0/dd/d11/d38/f59 [1420056,70189] 0 2024-04-12T05:55:26.959 INFO:tasks.workunit.client.0.smithi008.stdout:7/331: fdatasync d1/d5/da/f62 0 2024-04-12T05:55:26.964 INFO:tasks.workunit.client.0.smithi008.stdout:7/332: fdatasync d1/d5/f4d 0 2024-04-12T05:55:26.964 INFO:tasks.workunit.client.0.smithi008.stdout:8/181: truncate dd/f1f 2447601 0 2024-04-12T05:55:26.965 INFO:tasks.workunit.client.0.smithi008.stdout:2/305: creat d6/db/f69 x:0 0 0 2024-04-12T05:55:26.966 INFO:tasks.workunit.client.0.smithi008.stdout:6/292: readlink df/d10/d3a/l42 0 2024-04-12T05:55:26.967 INFO:tasks.workunit.client.0.smithi008.stdout:7/333: read d1/d5/f33 [640251,47522] 0 2024-04-12T05:55:26.968 INFO:tasks.workunit.client.0.smithi008.stdout:1/238: rename d4/dd/d1b to d4/dd/d17/d28/d46/d4c 0 2024-04-12T05:55:26.970 INFO:tasks.workunit.client.0.smithi008.stdout:5/455: rmdir d0/d40 39 2024-04-12T05:55:26.972 INFO:tasks.workunit.client.0.smithi008.stdout:5/456: dread - d0/dc/d13/d1a/f83 zero size 2024-04-12T05:55:26.973 INFO:tasks.workunit.client.0.smithi008.stdout:7/334: dread d1/d5/f42 [0,4194304] 0 2024-04-12T05:55:26.976 INFO:tasks.workunit.client.0.smithi008.stdout:1/239: mknod d4/dd/d17/d28/d30/c4d 0 2024-04-12T05:55:26.977 INFO:tasks.workunit.client.0.smithi008.stdout:2/306: mknod d6/db/d60/d4f/c6a 0 2024-04-12T05:55:26.978 INFO:tasks.workunit.client.0.smithi008.stdout:5/457: read d0/d2/f1b [3995981,14632] 0 2024-04-12T05:55:26.981 INFO:tasks.workunit.client.0.smithi008.stdout:7/335: creat d1/d5/da/d70/f74 x:0 0 0 2024-04-12T05:55:26.982 INFO:tasks.workunit.client.0.smithi008.stdout:7/336: read d1/d49/f4b [2300314,13409] 0 2024-04-12T05:55:26.984 INFO:tasks.workunit.client.0.smithi008.stdout:3/269: truncate f7 210364 0 2024-04-12T05:55:26.984 INFO:tasks.workunit.client.0.smithi008.stdout:1/240: creat d4/d7/f4e x:0 0 0 2024-04-12T05:55:26.985 INFO:tasks.workunit.client.0.smithi008.stdout:1/241: chown d4/l16 60881371 1 2024-04-12T05:55:26.987 INFO:tasks.workunit.client.0.smithi008.stdout:2/307: creat d6/db/d38/d41/f6b x:0 0 0 2024-04-12T05:55:26.987 INFO:tasks.workunit.client.0.smithi008.stdout:1/242: write d4/dd/d17/d26/f36 [886646,123840] 0 2024-04-12T05:55:26.989 INFO:tasks.workunit.client.0.smithi008.stdout:1/243: write d4/f27 [702049,73172] 0 2024-04-12T05:55:26.991 INFO:tasks.workunit.client.0.smithi008.stdout:7/337: mknod d1/d5/da/d1d/c75 0 2024-04-12T05:55:26.991 INFO:tasks.workunit.client.0.smithi008.stdout:5/458: creat d0/dd/d11/f8f x:0 0 0 2024-04-12T05:55:26.993 INFO:tasks.workunit.client.0.smithi008.stdout:5/459: chown d0/dc/d13/d1a/d35 53174 1 2024-04-12T05:55:26.994 INFO:tasks.workunit.client.0.smithi008.stdout:3/270: mkdir d9/d11/d5b 0 2024-04-12T05:55:26.995 INFO:tasks.workunit.client.0.smithi008.stdout:6/293: getdents df 0 2024-04-12T05:55:26.996 INFO:tasks.workunit.client.0.smithi008.stdout:2/308: readlink d6/db/d10/d23/d35/l47 0 2024-04-12T05:55:27.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.005+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc680611d0 con 0x7fcc5406ca60 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.007+0000 7fcc5effd700 1 -- 172.21.15.8:0/3958569969 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fcc680611d0 con 0x7fcc5406ca60 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 msgr2=0x7fcc5406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 0x7fcc5406ef20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005950 tx=0x7fcc640058e0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 msgr2=0x7fcc681b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcc58000c00 tx=0x7fcc5800be20 comp rx=0 tx=0).stop 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 shutdown_connections 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fcc5406ca60 0x7fcc5406ef20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc68101eb0 0x7fcc681b6750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 --2- 172.21.15.8:0/3958569969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc681b6c90 0x7fcc681bbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 >> 172.21.15.8:0/3958569969 conn(0x7fcc680fb160 msgr2=0x7fcc680fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 shutdown_connections 2024-04-12T05:55:27.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.010+0000 7fcc6fab3700 1 -- 172.21.15.8:0/3958569969 wait complete. 2024-04-12T05:55:27.008 INFO:tasks.workunit.client.0.smithi008.stdout:1/244: mkdir d4/d1c/d4f 0 2024-04-12T05:55:27.010 INFO:tasks.workunit.client.0.smithi008.stdout:7/338: rmdir d1/d5/da/d3c/d44 39 2024-04-12T05:55:27.010 INFO:tasks.workunit.client.0.smithi008.stdout:5/460: symlink d0/dd/d3d/d5d/l90 0 2024-04-12T05:55:27.011 INFO:tasks.workunit.client.0.smithi008.stdout:1/245: write f3 [322497,115477] 0 2024-04-12T05:55:27.013 INFO:tasks.workunit.client.0.smithi008.stdout:3/271: symlink d9/d10/d19/d3a/l5c 0 2024-04-12T05:55:27.013 INFO:tasks.workunit.client.0.smithi008.stdout:5/461: write d0/dc/d13/d1a/f22 [2155859,39645] 0 2024-04-12T05:55:27.019 INFO:tasks.workunit.client.0.smithi008.stdout:3/272: write d9/d10/d19/d22/f1f [7413713,119678] 0 2024-04-12T05:55:27.020 INFO:tasks.workunit.client.0.smithi008.stdout:1/246: symlink d4/d7/d18/l50 0 2024-04-12T05:55:27.022 INFO:tasks.workunit.client.0.smithi008.stdout:7/339: symlink d1/d5/da/d1d/d68/l76 0 2024-04-12T05:55:27.024 INFO:tasks.workunit.client.0.smithi008.stdout:5/462: write d0/dc/f65 [2579932,39592] 0 2024-04-12T05:55:27.028 INFO:tasks.workunit.client.0.smithi008.stdout:7/340: truncate d1/d15/d43/d54/d58/f67 489075 0 2024-04-12T05:55:27.030 INFO:tasks.workunit.client.0.smithi008.stdout:2/309: getdents d6/db/d10/d23 0 2024-04-12T05:55:27.032 INFO:tasks.workunit.client.0.smithi008.stdout:3/273: dread d9/d10/f55 [0,4194304] 0 2024-04-12T05:55:27.032 INFO:tasks.workunit.client.0.smithi008.stdout:5/463: rmdir d0/d2 39 2024-04-12T05:55:27.033 INFO:tasks.workunit.client.0.smithi008.stdout:7/341: write d1/d5/da/d70/f74 [921586,68093] 0 2024-04-12T05:55:27.033 INFO:tasks.workunit.client.0.smithi008.stdout:6/294: link df/d21/c4e df/d10/c59 0 2024-04-12T05:55:27.034 INFO:tasks.workunit.client.0.smithi008.stdout:2/310: chown d6/db/d10/d23/d35/d3c/c54 457 1 2024-04-12T05:55:27.036 INFO:tasks.workunit.client.0.smithi008.stdout:3/274: read - d9/d10/d19/d22/f4d zero size 2024-04-12T05:55:27.038 INFO:tasks.workunit.client.0.smithi008.stdout:1/247: link d4/d6/f8 d4/d1a/f51 0 2024-04-12T05:55:27.039 INFO:tasks.workunit.client.0.smithi008.stdout:6/295: creat df/d21/f5a x:0 0 0 2024-04-12T05:55:27.040 INFO:tasks.workunit.client.0.smithi008.stdout:7/342: mkdir d1/d15/d43/d54/d77 0 2024-04-12T05:55:27.040 INFO:tasks.workunit.client.0.smithi008.stdout:6/296: chown cc 2 1 2024-04-12T05:55:27.042 INFO:tasks.workunit.client.0.smithi008.stdout:3/275: truncate d9/d2e/d3d/f56 828571 0 2024-04-12T05:55:27.043 INFO:tasks.workunit.client.0.smithi008.stdout:1/248: symlink d4/dd/l52 0 2024-04-12T05:55:27.045 INFO:tasks.workunit.client.0.smithi008.stdout:7/343: write d1/d5/da/d11/f35 [13339177,84184] 0 2024-04-12T05:55:27.048 INFO:tasks.workunit.client.0.smithi008.stdout:7/344: chown d1/d5/da/d11 3771 1 2024-04-12T05:55:27.049 INFO:tasks.workunit.client.0.smithi008.stdout:2/311: dread d6/db/d60/f42 [0,4194304] 0 2024-04-12T05:55:27.050 INFO:tasks.workunit.client.0.smithi008.stdout:5/464: truncate d0/dd/d11/d38/f2b 873861 0 2024-04-12T05:55:27.051 INFO:tasks.workunit.client.0.smithi008.stdout:6/297: getdents df/d10/d20/d3e 0 2024-04-12T05:55:27.052 INFO:tasks.workunit.client.0.smithi008.stdout:6/298: chown df/f31 0 1 2024-04-12T05:55:27.052 INFO:tasks.workunit.client.0.smithi008.stdout:3/276: mknod d9/d10/d19/d22/d2c/c5d 0 2024-04-12T05:55:27.057 INFO:tasks.workunit.client.0.smithi008.stdout:3/277: creat d9/d10/d19/d22/d2c/f5e x:0 0 0 2024-04-12T05:55:27.058 INFO:tasks.workunit.client.0.smithi008.stdout:3/278: truncate d9/d2e/f3f 527879 0 2024-04-12T05:55:27.060 INFO:tasks.workunit.client.0.smithi008.stdout:7/345: rename d1/d5/da/d70 to d1/d5/da/d78 0 2024-04-12T05:55:27.061 INFO:tasks.workunit.client.0.smithi008.stdout:5/465: symlink d0/d2/l91 0 2024-04-12T05:55:27.065 INFO:tasks.workunit.client.0.smithi008.stdout:3/279: unlink d9/dd/l1d 0 2024-04-12T05:55:27.067 INFO:tasks.workunit.client.0.smithi008.stdout:1/249: truncate d4/d6/d11/f22 11053 0 2024-04-12T05:55:27.069 INFO:tasks.workunit.client.0.smithi008.stdout:2/312: getdents d6/db/d38 0 2024-04-12T05:55:27.070 INFO:tasks.workunit.client.0.smithi008.stdout:6/299: getdents df/d10/d3a 0 2024-04-12T05:55:27.071 INFO:tasks.workunit.client.0.smithi008.stdout:7/346: dwrite d1/f2d [0,4194304] 0 2024-04-12T05:55:27.074 INFO:tasks.workunit.client.0.smithi008.stdout:1/250: truncate d4/dd/d17/d28/d30/f49 417837 0 2024-04-12T05:55:27.084 INFO:tasks.workunit.client.0.smithi008.stdout:2/313: truncate d6/f21 873846 0 2024-04-12T05:55:27.086 INFO:tasks.workunit.client.0.smithi008.stdout:2/314: dread - d6/db/d10/d4c/f5a zero size 2024-04-12T05:55:27.087 INFO:tasks.workunit.client.0.smithi008.stdout:5/466: getdents d0/dc/d13/d1a 0 2024-04-12T05:55:27.088 INFO:tasks.workunit.client.0.smithi008.stdout:1/251: creat d4/d6/f53 x:0 0 0 2024-04-12T05:55:27.088 INFO:tasks.workunit.client.0.smithi008.stdout:6/300: creat df/d10/d15/d4f/d50/f5b x:0 0 0 2024-04-12T05:55:27.089 INFO:tasks.workunit.client.0.smithi008.stdout:3/280: write f7 [276112,93692] 0 2024-04-12T05:55:27.091 INFO:tasks.workunit.client.0.smithi008.stdout:5/467: write d0/d2/f2c [4531413,48282] 0 2024-04-12T05:55:27.092 INFO:tasks.workunit.client.0.smithi008.stdout:2/315: readlink d6/l44 0 2024-04-12T05:55:27.093 INFO:tasks.workunit.client.0.smithi008.stdout:2/316: stat d6/db/d60/d4f/f61 0 2024-04-12T05:55:27.094 INFO:tasks.workunit.client.0.smithi008.stdout:1/252: write f3 [1358418,64784] 0 2024-04-12T05:55:27.098 INFO:tasks.workunit.client.0.smithi008.stdout:7/347: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:27.098 INFO:tasks.workunit.client.0.smithi008.stdout:1/253: chown d4/dd/d17/f1f 14 1 2024-04-12T05:55:27.101 INFO:tasks.workunit.client.0.smithi008.stdout:3/281: creat d9/d10/d19/d22/f5f x:0 0 0 2024-04-12T05:55:27.102 INFO:tasks.workunit.client.0.smithi008.stdout:6/301: creat df/d10/d20/d3e/f5c x:0 0 0 2024-04-12T05:55:27.106 INFO:tasks.workunit.client.0.smithi008.stdout:7/348: mknod d1/d15/d43/d54/d77/c79 0 2024-04-12T05:55:27.108 INFO:tasks.workunit.client.0.smithi008.stdout:1/254: symlink d4/dd/d17/l54 0 2024-04-12T05:55:27.114 INFO:tasks.workunit.client.0.smithi008.stdout:7/349: symlink d1/d15/d43/l7a 0 2024-04-12T05:55:27.115 INFO:tasks.workunit.client.0.smithi008.stdout:3/282: getdents d9 0 2024-04-12T05:55:27.118 INFO:tasks.workunit.client.0.smithi008.stdout:6/302: getdents df/d10/d15 0 2024-04-12T05:55:27.123 INFO:tasks.workunit.client.0.smithi008.stdout:1/255: dread d4/d1a/f51 [0,4194304] 0 2024-04-12T05:55:27.125 INFO:tasks.workunit.client.0.smithi008.stdout:5/468: dwrite d0/d2/d37/f45 [0,4194304] 0 2024-04-12T05:55:27.125 INFO:tasks.workunit.client.0.smithi008.stdout:6/303: mknod df/d10/d15/c5d 0 2024-04-12T05:55:27.138 INFO:tasks.workunit.client.0.smithi008.stdout:1/256: dread d4/dd/d17/d26/f36 [0,4194304] 0 2024-04-12T05:55:27.140 INFO:tasks.workunit.client.0.smithi008.stdout:7/350: dwrite d1/f2d [4194304,4194304] 0 2024-04-12T05:55:27.141 INFO:tasks.workunit.client.0.smithi008.stdout:3/283: dwrite d9/d1e/f42 [0,4194304] 0 2024-04-12T05:55:27.144 INFO:tasks.workunit.client.0.smithi008.stdout:7/351: readlink d1/d15/d43/d54/d58/l5f 0 2024-04-12T05:55:27.148 INFO:tasks.workunit.client.0.smithi008.stdout:7/352: chown d1/d5/l38 289094799 1 2024-04-12T05:55:27.151 INFO:tasks.workunit.client.0.smithi008.stdout:6/304: getdents df 0 2024-04-12T05:55:27.155 INFO:tasks.workunit.client.0.smithi008.stdout:3/284: dwrite d9/d10/d19/d22/d2c/f5e [0,4194304] 0 2024-04-12T05:55:27.156 INFO:tasks.workunit.client.0.smithi008.stdout:1/257: creat d4/d33/d37/f55 x:0 0 0 2024-04-12T05:55:27.158 INFO:tasks.workunit.client.0.smithi008.stdout:6/305: unlink fb 0 2024-04-12T05:55:27.159 INFO:tasks.workunit.client.0.smithi008.stdout:1/258: creat d4/d7/f56 x:0 0 0 2024-04-12T05:55:27.160 INFO:tasks.workunit.client.0.smithi008.stdout:6/306: chown df/d10/f56 1471710 1 2024-04-12T05:55:27.162 INFO:tasks.workunit.client.0.smithi008.stdout:1/259: mkdir d4/d1a/d57 0 2024-04-12T05:55:27.165 INFO:tasks.workunit.client.0.smithi008.stdout:6/307: mknod df/d10/d15/c5e 0 2024-04-12T05:55:27.166 INFO:tasks.workunit.client.0.smithi008.stdout:7/353: dread d1/d5/f33 [0,4194304] 0 2024-04-12T05:55:27.167 INFO:tasks.workunit.client.0.smithi008.stdout:1/260: mknod d4/d6/c58 0 2024-04-12T05:55:27.171 INFO:tasks.workunit.client.0.smithi008.stdout:7/354: chown d1/d5/da/d3c/c34 461 1 2024-04-12T05:55:27.173 INFO:tasks.workunit.client.0.smithi008.stdout:3/285: dread d9/d10/d19/d22/fc [0,4194304] 0 2024-04-12T05:55:27.176 INFO:tasks.workunit.client.0.smithi008.stdout:3/286: chown f1 10 1 2024-04-12T05:55:27.177 INFO:tasks.workunit.client.0.smithi008.stdout:3/287: dread - d9/d11/f47 zero size 2024-04-12T05:55:27.181 INFO:tasks.workunit.client.0.smithi008.stdout:1/261: getdents d4/d1a 0 2024-04-12T05:55:27.186 INFO:tasks.workunit.client.0.smithi008.stdout:3/288: creat d9/d11/d5b/f60 x:0 0 0 2024-04-12T05:55:27.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.196+0000 7fd76c524700 1 -- 172.21.15.8:0/57182946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640fef60 msgr2=0x7fd7640ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.196+0000 7fd76c524700 1 --2- 172.21.15.8:0/57182946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640fef60 0x7fd7640ff380 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd760009910 tx=0x7fd760009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:27.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.197+0000 7fd76c524700 1 -- 172.21.15.8:0/57182946 shutdown_connections 2024-04-12T05:55:27.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.197+0000 7fd76c524700 1 --2- 172.21.15.8:0/57182946 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640ffcd0 0x7fd764100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.197+0000 7fd76c524700 1 --2- 172.21.15.8:0/57182946 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640fef60 0x7fd7640ff380 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.197+0000 7fd76c524700 1 -- 172.21.15.8:0/57182946 >> 172.21.15.8:0/57182946 conn(0x7fd7640fa8c0 msgr2=0x7fd7640fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:27.193 INFO:tasks.workunit.client.0.smithi008.stdout:1/262: getdents d4/dd/d17 0 2024-04-12T05:55:27.193 INFO:tasks.workunit.client.0.smithi008.stdout:7/355: dwrite d1/d5/da/d78/f74 [0,4194304] 0 2024-04-12T05:55:27.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.198+0000 7fd76c524700 1 -- 172.21.15.8:0/57182946 shutdown_connections 2024-04-12T05:55:27.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.198+0000 7fd76c524700 1 -- 172.21.15.8:0/57182946 wait complete. 2024-04-12T05:55:27.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.198+0000 7fd76c524700 1 Processor -- start 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76c524700 1 -- start start 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76c524700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 0x7fd76410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76c524700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 0x7fd76410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76c524700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76410e5e0 con 0x7fd7640ffcd0 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76c524700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7641aa850 con 0x7fd7640fef60 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd769abf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 0x7fd76410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd769abf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 0x7fd76410df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56164/0 (socket says 172.21.15.8:56164) 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd769abf700 1 -- 172.21.15.8:0/3919545375 learned_addr learned my addr 172.21.15.8:0/3919545375 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:27.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.199+0000 7fd76a2c0700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 0x7fd76410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.195 INFO:tasks.workunit.client.0.smithi008.stdout:3/289: mknod d9/d10/d19/c61 0 2024-04-12T05:55:27.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.200+0000 7fd76a2c0700 1 -- 172.21.15.8:0/3919545375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 msgr2=0x7fd76410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.200+0000 7fd76a2c0700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 0x7fd76410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.200+0000 7fd76a2c0700 1 -- 172.21.15.8:0/3919545375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600095f0 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.201+0000 7fd76a2c0700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 0x7fd76410da50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd760000c00 tx=0x7fd76000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.201+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd76001d070 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.201+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd76000f460 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.201+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd760017630 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.201+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76406a810 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.202+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76406ad00 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.203+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76410fd60 con 0x7fd7640fef60 2024-04-12T05:55:27.199 INFO:tasks.workunit.client.0.smithi008.stdout:1/263: creat d4/d6/f59 x:0 0 0 2024-04-12T05:55:27.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.205+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7fd76000fa00 con 0x7fd7640fef60 2024-04-12T05:55:27.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.206+0000 7fd75b7fe700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 0x7fd75006ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.206+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd76008d5f0 con 0x7fd7640fef60 2024-04-12T05:55:27.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.207+0000 7fd769abf700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 0x7fd75006ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.208+0000 7fd769abf700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 0x7fd75006ef70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7641a6400 tx=0x7fd754009290 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:27.204 INFO:tasks.workunit.client.0.smithi008.stdout:7/356: mkdir d1/d5/da/d60/d7b 0 2024-04-12T05:55:27.204 INFO:tasks.workunit.client.0.smithi008.stdout:6/308: dwrite df/d10/d15/d4f/d50/f51 [4194304,4194304] 0 2024-04-12T05:55:27.206 INFO:tasks.workunit.client.0.smithi008.stdout:3/290: mknod d9/d10/c62 0 2024-04-12T05:55:27.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.213+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd76005b0b0 con 0x7fd7640fef60 2024-04-12T05:55:27.211 INFO:tasks.workunit.client.0.smithi008.stdout:3/291: creat d9/d44/f63 x:0 0 0 2024-04-12T05:55:27.211 INFO:tasks.workunit.client.0.smithi008.stdout:6/309: rename df/c18 to df/d10/d3a/c5f 0 2024-04-12T05:55:27.213 INFO:tasks.workunit.client.0.smithi008.stdout:6/310: write df/f2e [588337,2020] 0 2024-04-12T05:55:27.215 INFO:tasks.workunit.client.0.smithi008.stdout:7/357: dread d1/f2d [4194304,4194304] 0 2024-04-12T05:55:27.215 INFO:tasks.workunit.client.0.smithi008.stdout:3/292: creat d9/d1e/f64 x:0 0 0 2024-04-12T05:55:27.217 INFO:tasks.workunit.client.0.smithi008.stdout:6/311: unlink df/d10/d15/f1c 0 2024-04-12T05:55:27.221 INFO:tasks.workunit.client.0.smithi008.stdout:6/312: rename df/d10/d15/c1d to df/c60 0 2024-04-12T05:55:27.222 INFO:tasks.workunit.client.0.smithi008.stdout:6/313: chown df/f12 2516 1 2024-04-12T05:55:27.223 INFO:tasks.workunit.client.0.smithi008.stdout:6/314: chown df/f2e 28806 1 2024-04-12T05:55:27.225 INFO:tasks.workunit.client.0.smithi008.stdout:7/358: getdents d1/d15/d43 0 2024-04-12T05:55:27.227 INFO:tasks.workunit.client.0.smithi008.stdout:3/293: chown d9/d10/d19/c1b 20240946 1 2024-04-12T05:55:27.228 INFO:tasks.workunit.client.0.smithi008.stdout:3/294: dread - d9/d1e/f64 zero size 2024-04-12T05:55:27.233 INFO:tasks.workunit.client.0.smithi008.stdout:7/359: rename d1/d15/d43/d5a/l6e to d1/d5/da/d11/d56/l7c 0 2024-04-12T05:55:27.234 INFO:tasks.workunit.client.0.smithi008.stdout:6/315: getdents df 0 2024-04-12T05:55:27.236 INFO:tasks.workunit.client.0.smithi008.stdout:3/295: mkdir d9/d10/d19/d65 0 2024-04-12T05:55:27.236 INFO:tasks.workunit.client.0.smithi008.stdout:6/316: write df/d21/d4c/f53 [660057,27483] 0 2024-04-12T05:55:27.237 INFO:tasks.workunit.client.0.smithi008.stdout:7/360: creat d1/d5/da/d11/f7d x:0 0 0 2024-04-12T05:55:27.244 INFO:tasks.workunit.client.0.smithi008.stdout:3/296: mknod d9/d10/d19/d3a/c66 0 2024-04-12T05:55:27.246 INFO:tasks.workunit.client.0.smithi008.stdout:6/317: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:27.246 INFO:tasks.workunit.client.0.smithi008.stdout:3/297: write d9/d44/f63 [262359,112524] 0 2024-04-12T05:55:27.247 INFO:tasks.workunit.client.0.smithi008.stdout:7/361: rmdir d1 39 2024-04-12T05:55:27.250 INFO:tasks.workunit.client.0.smithi008.stdout:6/318: write df/d10/d20/f49 [554071,25502] 0 2024-04-12T05:55:27.250 INFO:tasks.workunit.client.0.smithi008.stdout:3/298: stat d9/d10/d19/d22 0 2024-04-12T05:55:27.250 INFO:tasks.workunit.client.0.smithi008.stdout:6/319: dread - df/d10/f56 zero size 2024-04-12T05:55:27.252 INFO:tasks.workunit.client.0.smithi008.stdout:3/299: read - d9/d1e/f64 zero size 2024-04-12T05:55:27.255 INFO:tasks.workunit.client.0.smithi008.stdout:6/320: creat df/d10/d20/d3e/f61 x:0 0 0 2024-04-12T05:55:27.255 INFO:tasks.workunit.client.0.smithi008.stdout:3/300: chown d9/d10/d19/d22/f5f 1 1 2024-04-12T05:55:27.257 INFO:tasks.workunit.client.0.smithi008.stdout:3/301: write d9/dd/f2f [5167866,122186] 0 2024-04-12T05:55:27.260 INFO:tasks.workunit.client.0.smithi008.stdout:7/362: dread d1/d5/da/f2a [0,4194304] 0 2024-04-12T05:55:27.264 INFO:tasks.workunit.client.0.smithi008.stdout:3/302: dread d9/d11/f48 [0,4194304] 0 2024-04-12T05:55:27.266 INFO:tasks.workunit.client.0.smithi008.stdout:3/303: dread - d9/d11/d5b/f60 zero size 2024-04-12T05:55:27.266 INFO:tasks.workunit.client.0.smithi008.stdout:6/321: dread df/d21/d4c/f53 [0,4194304] 0 2024-04-12T05:55:27.270 INFO:tasks.workunit.client.0.smithi008.stdout:7/363: dwrite d1/f55 [0,4194304] 0 2024-04-12T05:55:27.276 INFO:tasks.workunit.client.0.smithi008.stdout:7/364: symlink d1/d15/d43/d54/l7e 0 2024-04-12T05:55:27.278 INFO:tasks.workunit.client.0.smithi008.stdout:3/304: dwrite d9/d1e/f64 [0,4194304] 0 2024-04-12T05:55:27.288 INFO:tasks.workunit.client.0.smithi008.stdout:7/365: dread d1/d5/f32 [0,4194304] 0 2024-04-12T05:55:27.293 INFO:tasks.workunit.client.0.smithi008.stdout:3/305: dwrite d9/d10/d19/f50 [0,4194304] 0 2024-04-12T05:55:27.296 INFO:tasks.workunit.client.0.smithi008.stdout:7/366: mkdir d1/d15/d43/d54/d58/d5e/d7f 0 2024-04-12T05:55:27.296 INFO:tasks.workunit.client.0.smithi008.stdout:3/306: stat d9/dd/l16 0 2024-04-12T05:55:27.297 INFO:tasks.workunit.client.0.smithi008.stdout:3/307: write d9/d1e/f5a [862557,93147] 0 2024-04-12T05:55:27.301 INFO:tasks.workunit.client.0.smithi008.stdout:7/367: truncate d1/d5/da/d11/f57 229184 0 2024-04-12T05:55:27.305 INFO:tasks.workunit.client.0.smithi008.stdout:7/368: mknod d1/d5/da/d3c/d44/c80 0 2024-04-12T05:55:27.402 INFO:tasks.workunit.client.0.smithi008.stdout:9/240: sync 2024-04-12T05:55:27.402 INFO:tasks.workunit.client.0.smithi008.stdout:7/369: symlink d1/d15/d43/d54/d58/d5e/l81 0 2024-04-12T05:55:27.402 INFO:tasks.workunit.client.0.smithi008.stdout:3/308: dwrite d9/d10/d19/d22/f32 [0,4194304] 0 2024-04-12T05:55:27.403 INFO:tasks.workunit.client.0.smithi008.stdout:3/309: fsync d9/d11/d5b/f60 0 2024-04-12T05:55:27.403 INFO:tasks.workunit.client.0.smithi008.stdout:9/241: rmdir d3/df/d13/d21/d2d/d32 39 2024-04-12T05:55:27.404 INFO:tasks.workunit.client.0.smithi008.stdout:7/370: mkdir d1/d5/da/d1d/d82 0 2024-04-12T05:55:27.405 INFO:tasks.workunit.client.0.smithi008.stdout:9/242: dread - d3/d22/d2b/f33 zero size 2024-04-12T05:55:27.405 INFO:tasks.workunit.client.0.smithi008.stdout:3/310: mkdir d9/d10/d19/d3a/d67 0 2024-04-12T05:55:27.408 INFO:tasks.workunit.client.0.smithi008.stdout:3/311: creat d9/d11/d5b/f68 x:0 0 0 2024-04-12T05:55:27.410 INFO:tasks.workunit.client.0.smithi008.stdout:9/243: unlink d3/df/d10/d12/d36/f49 0 2024-04-12T05:55:27.412 INFO:tasks.workunit.client.0.smithi008.stdout:7/371: dwrite d1/f55 [0,4194304] 0 2024-04-12T05:55:27.414 INFO:tasks.workunit.client.0.smithi008.stdout:9/244: chown d3/df/d13/c42 893735 1 2024-04-12T05:55:27.419 INFO:tasks.workunit.client.0.smithi008.stdout:9/245: link d3/f25 d3/df/d10/d12/d38/f54 0 2024-04-12T05:55:27.421 INFO:tasks.workunit.client.0.smithi008.stdout:9/246: write d3/df/d10/d12/f1e [2524968,39150] 0 2024-04-12T05:55:27.427 INFO:tasks.workunit.client.0.smithi008.stdout:9/247: link d3/fa d3/df/d10/d12/f55 0 2024-04-12T05:55:27.429 INFO:tasks.workunit.client.0.smithi008.stdout:9/248: rename d3 to d3/df/d13/d21/d2d/d32/d56 22 2024-04-12T05:55:27.432 INFO:tasks.workunit.client.0.smithi008.stdout:9/249: symlink d3/df/d13/d21/d2d/d32/d52/l57 0 2024-04-12T05:55:27.433 INFO:tasks.workunit.client.0.smithi008.stdout:9/250: write d3/df/d13/d21/d2d/d30/f4d [177137,47722] 0 2024-04-12T05:55:27.439 INFO:tasks.workunit.client.0.smithi008.stdout:9/251: dwrite d3/df/d10/f17 [0,4194304] 0 2024-04-12T05:55:27.441 INFO:tasks.workunit.client.0.smithi008.stdout:9/252: read d3/df/d10/d12/f55 [443917,44067] 0 2024-04-12T05:55:27.459 INFO:tasks.workunit.client.0.smithi008.stdout:9/253: rename d3/df/d10/d12 to d3/d22/d2b/d58 0 2024-04-12T05:55:27.460 INFO:tasks.workunit.client.0.smithi008.stdout:9/254: chown d3/df/d13/c16 7616 1 2024-04-12T05:55:27.462 INFO:tasks.workunit.client.0.smithi008.stdout:9/255: mknod d3/d22/d2b/c59 0 2024-04-12T05:55:27.462 INFO:tasks.workunit.client.0.smithi008.stdout:9/256: chown d3/df/f11 268 1 2024-04-12T05:55:27.463 INFO:tasks.workunit.client.0.smithi008.stdout:9/257: chown d3/d22/d2b/d58/f55 28838971 1 2024-04-12T05:55:27.464 INFO:tasks.workunit.client.0.smithi008.stdout:9/258: readlink d3/l8 0 2024-04-12T05:55:27.465 INFO:tasks.workunit.client.0.smithi008.stdout:9/259: fdatasync d3/f25 0 2024-04-12T05:55:27.467 INFO:tasks.workunit.client.0.smithi008.stdout:9/260: readlink d3/df/d13/d14/l2c 0 2024-04-12T05:55:27.479 INFO:tasks.workunit.client.0.smithi008.stdout:9/261: dwrite d3/df/d13/f51 [0,4194304] 0 2024-04-12T05:55:27.479 INFO:tasks.workunit.client.0.smithi008.stdout:9/262: chown d3/df 71370 1 2024-04-12T05:55:27.484 INFO:tasks.workunit.client.0.smithi008.stdout:9/263: unlink d3/df/d13/d21/d2d/f47 0 2024-04-12T05:55:27.487 INFO:tasks.workunit.client.0.smithi008.stdout:9/264: creat d3/df/d13/f5a x:0 0 0 2024-04-12T05:55:27.504 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.509+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd764104620 con 0x7fd75006cab0 2024-04-12T05:55:27.508 INFO:tasks.workunit.client.0.smithi008.stdout:9/265: dread d3/f25 [0,4194304] 0 2024-04-12T05:55:27.510 INFO:tasks.workunit.client.0.smithi008.stdout:9/266: symlink d3/d22/d2b/l5b 0 2024-04-12T05:55:27.512 INFO:tasks.workunit.client.0.smithi008.stdout:9/267: mkdir d3/df/d13/d21/d5c 0 2024-04-12T05:55:27.519 INFO:tasks.workunit.client.0.smithi008.stdout:9/268: truncate d3/d22/d2b/f41 76109 0 2024-04-12T05:55:27.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.526+0000 7fd75b7fe700 1 -- 172.21.15.8:0/3919545375 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd764104620 con 0x7fd75006cab0 2024-04-12T05:55:27.521 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:55:27.521 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (7m) 2m ago 8m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:55:27.521 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (9m) 2m ago 8m 7478k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (7m) 7s ago 7m 7662k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (8m) 2m ago 8m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (7m) 7s ago 7m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (6m) 2m ago 8m 86.1M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (2m) 2m ago 2m 15.0M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (2m) 2m ago 2m 12.8M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (2m) 7s ago 2m 45.9M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (2m) 7s ago 2m 65.7M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:9283,8765,8443 running (10m) 2m ago 10m 489M - 18.2.1 5be31c24972a 545bb7b78dec 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (28s) 7s ago 7m 476M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b06c49f54d22 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (10m) 2m ago 10m 46.9M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (7m) 7s ago 7m 40.3M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:55:27.522 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (8m) 2m ago 8m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (7m) 7s ago 7m 18.2M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (6m) 2m ago 6m 48.1M 3959M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (6m) 2m ago 6m 48.8M 3959M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (5m) 2m ago 5m 45.8M 3959M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (5m) 7s ago 5m 105M 4982M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (4m) 7s ago 4m 103M 4982M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (4m) 7s ago 4m 93.4M 4982M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:55:27.523 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (6m) 2m ago 8m 56.7M - 2.43.0 a07b618ecd1d c0932eae2f67 2024-04-12T05:55:27.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 msgr2=0x7fd75006ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 0x7fd75006ef70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd7641a6400 tx=0x7fd754009290 comp rx=0 tx=0).stop 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 msgr2=0x7fd76410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 0x7fd76410da50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd760000c00 tx=0x7fd76000bd70 comp rx=0 tx=0).stop 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 shutdown_connections 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd75006cab0 0x7fd75006ef70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd7640fef60 0x7fd76410da50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 --2- 172.21.15.8:0/3919545375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd7640ffcd0 0x7fd76410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 >> 172.21.15.8:0/3919545375 conn(0x7fd7640fa8c0 msgr2=0x7fd764102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.530+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 shutdown_connections 2024-04-12T05:55:27.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.531+0000 7fd76c524700 1 -- 172.21.15.8:0/3919545375 wait complete. 2024-04-12T05:55:27.529 INFO:tasks.workunit.client.0.smithi008.stdout:9/269: dwrite d3/df/d13/f51 [0,4194304] 0 2024-04-12T05:55:27.530 INFO:tasks.workunit.client.0.smithi008.stdout:9/270: chown d3/df/d13 439227505 1 2024-04-12T05:55:27.532 INFO:tasks.workunit.client.0.smithi008.stdout:9/271: truncate d3/fd 5123609 0 2024-04-12T05:55:27.535 INFO:tasks.workunit.client.0.smithi008.stdout:9/272: mkdir d3/d22/d2b/d58/d5d 0 2024-04-12T05:55:27.552 INFO:tasks.workunit.client.0.smithi008.stdout:9/273: fdatasync d3/df/d13/f51 0 2024-04-12T05:55:27.606 INFO:tasks.workunit.client.0.smithi008.stdout:0/291: sync 2024-04-12T05:55:27.610 INFO:tasks.workunit.client.0.smithi008.stdout:0/292: fdatasync d2/d11/f40 0 2024-04-12T05:55:27.613 INFO:tasks.workunit.client.0.smithi008.stdout:0/293: rename d2/c9 to d2/d24/d39/c5b 0 2024-04-12T05:55:27.624 INFO:tasks.workunit.client.0.smithi008.stdout:0/294: dwrite d2/d11/f40 [0,4194304] 0 2024-04-12T05:55:27.624 INFO:tasks.workunit.client.0.smithi008.stdout:0/295: fsync d2/ff 0 2024-04-12T05:55:27.627 INFO:tasks.workunit.client.0.smithi008.stdout:0/296: write d2/d11/f18 [100570,120200] 0 2024-04-12T05:55:27.631 INFO:tasks.workunit.client.0.smithi008.stdout:0/297: stat d2/d11/f23 0 2024-04-12T05:55:27.634 INFO:tasks.workunit.client.0.smithi008.stdout:0/298: read d2/d11/f25 [1033666,32355] 0 2024-04-12T05:55:27.635 INFO:tasks.workunit.client.0.smithi008.stdout:0/299: dread - d2/d24/d39/f5a zero size 2024-04-12T05:55:27.643 INFO:tasks.workunit.client.0.smithi008.stdout:0/300: write d2/d11/d2a/f4a [781630,126750] 0 2024-04-12T05:55:27.651 INFO:tasks.workunit.client.0.smithi008.stdout:0/301: dwrite d2/d24/d39/f3a [0,4194304] 0 2024-04-12T05:55:27.657 INFO:tasks.workunit.client.0.smithi008.stdout:0/302: creat d2/d11/d2a/f5c x:0 0 0 2024-04-12T05:55:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 -- 172.21.15.8:0/3476881579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 msgr2=0x7f7bfc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 --2- 172.21.15.8:0/3476881579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc100030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009910 tx=0x7f7bec009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 -- 172.21.15.8:0/3476881579 shutdown_connections 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 --2- 172.21.15.8:0/3476881579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 0x7f7bfc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 --2- 172.21.15.8:0/3476881579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc100030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.669+0000 7f7c016a1700 1 -- 172.21.15.8:0/3476881579 >> 172.21.15.8:0/3476881579 conn(0x7f7bfc0fb190 msgr2=0x7f7bfc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.670+0000 7f7c016a1700 1 -- 172.21.15.8:0/3476881579 shutdown_connections 2024-04-12T05:55:27.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.670+0000 7f7c016a1700 1 -- 172.21.15.8:0/3476881579 wait complete. 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 Processor -- start 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 -- start start 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 0x7f7bfc198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc199160 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.671+0000 7f7c016a1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc1992a0 con 0x7f7bfc100e10 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56180/0 (socket says 172.21.15.8:56180) 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 -- 172.21.15.8:0/859403729 learned_addr learned my addr 172.21.15.8:0/859403729 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:27.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfa7fc700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 0x7f7bfc198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 -- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 msgr2=0x7f7bfc198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 0x7f7bfc198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 -- 172.21.15.8:0/859403729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec0095f0 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.672+0000 7f7bfaffd700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005e00 tx=0x7f7bec0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:27.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.673+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec01d070 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.673+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc19dcf0 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.673+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc19e190 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.674+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f7bec00bd70 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.674+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bec003890 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.675+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be8005320 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.676+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f7bec004200 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.676+0000 7f7be3fff700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 0x7f7be406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.676+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7bec08d2f0 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.677+0000 7f7bfa7fc700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 0x7f7be406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:27.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.677+0000 7f7bfa7fc700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 0x7f7be406eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007be0 tx=0x7f7bf00073d0 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:27.673 INFO:tasks.workunit.client.0.smithi008.stdout:0/303: dwrite d2/d24/f32 [0,4194304] 0 2024-04-12T05:55:27.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:27.683+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f7bec05ad30 con 0x7f7bfc0ffc10 2024-04-12T05:55:27.680 INFO:tasks.workunit.client.0.smithi008.stdout:0/304: link d2/d11/d36/d47/f2c d2/d11/d2a/f5d 0 2024-04-12T05:55:27.680 INFO:tasks.workunit.client.0.smithi008.stdout:0/305: write d2/d11/d36/d47/f21 [3928255,98147] 0 2024-04-12T05:55:27.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:27 smithi194 ceph-mon[49771]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:27.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:27 smithi194 ceph-mon[49771]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:27.830 INFO:tasks.workunit.client.0.smithi008.stdout:4/153: sync 2024-04-12T05:55:27.833 INFO:tasks.workunit.client.0.smithi008.stdout:4/154: mkdir d1/d27 0 2024-04-12T05:55:27.836 INFO:tasks.workunit.client.0.smithi008.stdout:4/155: rename d1/d27 to d1/db/d19/d28 0 2024-04-12T05:55:27.838 INFO:tasks.workunit.client.0.smithi008.stdout:4/156: read d1/d8/f21 [686904,15351] 0 2024-04-12T05:55:27.839 INFO:tasks.workunit.client.0.smithi008.stdout:4/157: dread - d1/db/f1a zero size 2024-04-12T05:55:27.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:27 smithi008 ceph-mon[39722]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:27.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:27 smithi008 ceph-mon[39722]: from='client.24419 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:27.906 INFO:tasks.workunit.client.0.smithi008.stdout:4/158: mkdir d1/d22/d29 0 2024-04-12T05:55:27.907 INFO:tasks.workunit.client.0.smithi008.stdout:4/159: truncate d1/db/f1a 17815 0 2024-04-12T05:55:27.913 INFO:tasks.workunit.client.0.smithi008.stdout:4/160: dread d1/f12 [0,4194304] 0 2024-04-12T05:55:27.917 INFO:tasks.workunit.client.0.smithi008.stdout:4/161: dread d1/db/d19/f24 [0,4194304] 0 2024-04-12T05:55:27.919 INFO:tasks.workunit.client.0.smithi008.stdout:4/162: symlink d1/db/d19/l2a 0 2024-04-12T05:55:27.921 INFO:tasks.workunit.client.0.smithi008.stdout:4/163: write d1/f13 [3047299,81401] 0 2024-04-12T05:55:27.923 INFO:tasks.workunit.client.0.smithi008.stdout:4/164: symlink d1/d22/l2b 0 2024-04-12T05:55:27.923 INFO:tasks.workunit.client.0.smithi008.stdout:4/165: stat d1/db 0 2024-04-12T05:55:27.924 INFO:tasks.workunit.client.0.smithi008.stdout:4/166: chown d1/db/fc 1 1 2024-04-12T05:55:27.925 INFO:tasks.workunit.client.0.smithi008.stdout:4/167: mknod d1/db/d19/d28/c2c 0 2024-04-12T05:55:27.927 INFO:tasks.workunit.client.0.smithi008.stdout:8/182: sync 2024-04-12T05:55:27.929 INFO:tasks.workunit.client.0.smithi008.stdout:8/183: symlink dd/d30/l38 0 2024-04-12T05:55:27.931 INFO:tasks.workunit.client.0.smithi008.stdout:8/184: creat dd/f39 x:0 0 0 2024-04-12T05:55:27.932 INFO:tasks.workunit.client.0.smithi008.stdout:8/185: chown dd/de/d16/f17 527158 1 2024-04-12T05:55:27.933 INFO:tasks.workunit.client.0.smithi008.stdout:4/168: dwrite d1/db/d19/f24 [0,4194304] 0 2024-04-12T05:55:27.935 INFO:tasks.workunit.client.0.smithi008.stdout:4/169: creat d1/d8/f2d x:0 0 0 2024-04-12T05:55:27.939 INFO:tasks.workunit.client.0.smithi008.stdout:4/170: mknod d1/d22/d29/c2e 0 2024-04-12T05:55:27.941 INFO:tasks.workunit.client.0.smithi008.stdout:4/171: chown d1/d22/l2b 104178218 1 2024-04-12T05:55:27.944 INFO:tasks.workunit.client.0.smithi008.stdout:8/186: dwrite fb [0,4194304] 0 2024-04-12T05:55:27.946 INFO:tasks.workunit.client.0.smithi008.stdout:8/187: write dd/de/d16/f33 [2832023,28624] 0 2024-04-12T05:55:27.959 INFO:tasks.workunit.client.0.smithi008.stdout:8/188: write dd/f1f [361516,108788] 0 2024-04-12T05:55:27.961 INFO:tasks.workunit.client.0.smithi008.stdout:8/189: creat dd/d30/d32/f3a x:0 0 0 2024-04-12T05:55:27.962 INFO:tasks.workunit.client.0.smithi008.stdout:8/190: read - dd/d30/d32/f3a zero size 2024-04-12T05:55:27.964 INFO:tasks.workunit.client.0.smithi008.stdout:8/191: mkdir dd/d30/d3b 0 2024-04-12T05:55:27.971 INFO:tasks.workunit.client.0.smithi008.stdout:8/192: dwrite dd/f1a [0,4194304] 0 2024-04-12T05:55:27.972 INFO:tasks.workunit.client.0.smithi008.stdout:8/193: write dd/f22 [3121718,34916] 0 2024-04-12T05:55:27.973 INFO:tasks.workunit.client.0.smithi008.stdout:8/194: write dd/de/d16/f33 [1802364,21321] 0 2024-04-12T05:55:27.979 INFO:tasks.workunit.client.0.smithi008.stdout:8/195: write dd/de/f2f [389604,128760] 0 2024-04-12T05:55:27.983 INFO:tasks.workunit.client.0.smithi008.stdout:8/196: dread fb [0,4194304] 0 2024-04-12T05:55:27.984 INFO:tasks.workunit.client.0.smithi008.stdout:8/197: fdatasync dd/de/d16/f17 0 2024-04-12T05:55:27.991 INFO:tasks.workunit.client.0.smithi008.stdout:8/198: dwrite dd/de/d16/f28 [0,4194304] 0 2024-04-12T05:55:28.000 INFO:tasks.workunit.client.0.smithi008.stdout:8/199: dwrite dd/de/f2a [0,4194304] 0 2024-04-12T05:55:28.008 INFO:tasks.workunit.client.0.smithi008.stdout:8/200: dwrite dd/f39 [0,4194304] 0 2024-04-12T05:55:28.014 INFO:tasks.workunit.client.0.smithi008.stdout:8/201: creat dd/d30/d3b/f3c x:0 0 0 2024-04-12T05:55:28.037 INFO:tasks.workunit.client.0.smithi008.stdout:8/202: dread dd/de/d16/f33 [0,4194304] 0 2024-04-12T05:55:28.038 INFO:tasks.workunit.client.0.smithi008.stdout:8/203: mknod dd/de/d16/c3d 0 2024-04-12T05:55:28.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.047+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7be8005cc0 con 0x7f7bfc0ffc10 2024-04-12T05:55:28.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.047+0000 7f7be3fff700 1 -- 172.21.15.8:0/859403729 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f7bec05a8c0 con 0x7f7bfc0ffc10 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:55:28.043 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:55:28.044 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:55:28.045 INFO:tasks.workunit.client.0.smithi008.stdout:8/204: truncate dd/de/f27 311215 0 2024-04-12T05:55:28.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 msgr2=0x7f7be406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 0x7f7be406eed0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0007be0 tx=0x7f7bf00073d0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 msgr2=0x7f7bfc198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005e00 tx=0x7f7bec0056a0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 shutdown_connections 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f7be406ca10 0x7f7be406eed0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7bfc0ffc10 0x7f7bfc198600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 --2- 172.21.15.8:0/859403729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bfc100e10 0x7f7bfc198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 >> 172.21.15.8:0/859403729 conn(0x7f7bfc0fb190 msgr2=0x7f7bfc068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 shutdown_connections 2024-04-12T05:55:28.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.050+0000 7f7c016a1700 1 -- 172.21.15.8:0/859403729 wait complete. 2024-04-12T05:55:28.050 INFO:tasks.workunit.client.0.smithi008.stdout:8/205: truncate dd/de/d16/f28 3216039 0 2024-04-12T05:55:28.052 INFO:tasks.workunit.client.0.smithi008.stdout:8/206: write fb [4457780,52380] 0 2024-04-12T05:55:28.053 INFO:tasks.workunit.client.0.smithi008.stdout:8/207: write dd/de/f1c [1838456,26589] 0 2024-04-12T05:55:28.054 INFO:tasks.workunit.client.0.smithi008.stdout:8/208: write dd/de/f29 [1903290,71546] 0 2024-04-12T05:55:28.059 INFO:tasks.workunit.client.0.smithi008.stdout:8/209: rename dd/de/d16/f17 to dd/d30/d32/f3e 0 2024-04-12T05:55:28.066 INFO:tasks.workunit.client.0.smithi008.stdout:5/469: sync 2024-04-12T05:55:28.066 INFO:tasks.workunit.client.0.smithi008.stdout:7/372: sync 2024-04-12T05:55:28.066 INFO:tasks.workunit.client.0.smithi008.stdout:3/312: sync 2024-04-12T05:55:28.067 INFO:tasks.workunit.client.0.smithi008.stdout:6/322: sync 2024-04-12T05:55:28.067 INFO:tasks.workunit.client.0.smithi008.stdout:2/317: sync 2024-04-12T05:55:28.067 INFO:tasks.workunit.client.0.smithi008.stdout:1/264: sync 2024-04-12T05:55:28.070 INFO:tasks.workunit.client.0.smithi008.stdout:8/210: dwrite dd/de/f2d [0,4194304] 0 2024-04-12T05:55:28.073 INFO:tasks.workunit.client.0.smithi008.stdout:7/373: write d1/f6c [915986,40451] 0 2024-04-12T05:55:28.075 INFO:tasks.workunit.client.0.smithi008.stdout:3/313: truncate d9/d11/f48 1414560 0 2024-04-12T05:55:28.077 INFO:tasks.workunit.client.0.smithi008.stdout:2/318: creat d6/db/d29/f6c x:0 0 0 2024-04-12T05:55:28.080 INFO:tasks.workunit.client.0.smithi008.stdout:6/323: read f3 [6635933,89831] 0 2024-04-12T05:55:28.080 INFO:tasks.workunit.client.0.smithi008.stdout:2/319: chown d6/db/d38/l1d 25438249 1 2024-04-12T05:55:28.082 INFO:tasks.workunit.client.0.smithi008.stdout:5/470: rename d0/dd/f1e to d0/dc/d13/f92 0 2024-04-12T05:55:28.084 INFO:tasks.workunit.client.0.smithi008.stdout:7/374: mknod d1/d15/d43/d5a/c83 0 2024-04-12T05:55:28.088 INFO:tasks.workunit.client.0.smithi008.stdout:3/314: dread d9/d2e/f3f [0,4194304] 0 2024-04-12T05:55:28.089 INFO:tasks.workunit.client.0.smithi008.stdout:1/265: getdents d4/dd/d17/d28/d30 0 2024-04-12T05:55:28.094 INFO:tasks.workunit.client.0.smithi008.stdout:6/324: creat df/d10/f62 x:0 0 0 2024-04-12T05:55:28.095 INFO:tasks.workunit.client.0.smithi008.stdout:5/471: dwrite d0/dc/d13/d1a/f7d [0,4194304] 0 2024-04-12T05:55:28.096 INFO:tasks.workunit.client.0.smithi008.stdout:3/315: mkdir d9/d37/d69 0 2024-04-12T05:55:28.098 INFO:tasks.workunit.client.0.smithi008.stdout:7/375: mknod d1/d5/da/d60/d7b/c84 0 2024-04-12T05:55:28.101 INFO:tasks.workunit.client.0.smithi008.stdout:7/376: fsync d1/d5/da/d11/f69 0 2024-04-12T05:55:28.101 INFO:tasks.workunit.client.0.smithi008.stdout:8/211: dread dd/f22 [0,4194304] 0 2024-04-12T05:55:28.102 INFO:tasks.workunit.client.0.smithi008.stdout:6/325: creat df/f63 x:0 0 0 2024-04-12T05:55:28.103 INFO:tasks.workunit.client.0.smithi008.stdout:5/472: creat d0/dc/d13/d1a/d3a/f93 x:0 0 0 2024-04-12T05:55:28.107 INFO:tasks.workunit.client.0.smithi008.stdout:8/212: readlink dd/de/d16/l21 0 2024-04-12T05:55:28.110 INFO:tasks.workunit.client.0.smithi008.stdout:6/326: creat df/f64 x:0 0 0 2024-04-12T05:55:28.111 INFO:tasks.workunit.client.0.smithi008.stdout:3/316: dwrite d9/d11/d5b/f68 [0,4194304] 0 2024-04-12T05:55:28.115 INFO:tasks.workunit.client.0.smithi008.stdout:5/473: fsync d0/dc/d13/d1a/f7d 0 2024-04-12T05:55:28.119 INFO:tasks.workunit.client.0.smithi008.stdout:6/327: write df/d10/d20/d3e/f5c [149677,91513] 0 2024-04-12T05:55:28.124 INFO:tasks.workunit.client.0.smithi008.stdout:3/317: creat d9/d10/f6a x:0 0 0 2024-04-12T05:55:28.126 INFO:tasks.workunit.client.0.smithi008.stdout:7/377: dwrite d1/d5/da/d3c/d3d/f3b [0,4194304] 0 2024-04-12T05:55:28.127 INFO:tasks.workunit.client.0.smithi008.stdout:1/266: dwrite d4/d6/f8 [0,4194304] 0 2024-04-12T05:55:28.128 INFO:tasks.workunit.client.0.smithi008.stdout:3/318: chown d9/d10/d4b 237487 1 2024-04-12T05:55:28.129 INFO:tasks.workunit.client.0.smithi008.stdout:7/378: stat d1/d5/da/d60/c5d 0 2024-04-12T05:55:28.132 INFO:tasks.workunit.client.0.smithi008.stdout:7/379: chown d1/d15/d43 2120 1 2024-04-12T05:55:28.133 INFO:tasks.workunit.client.0.smithi008.stdout:7/380: stat d1/d5/f4d 0 2024-04-12T05:55:28.135 INFO:tasks.workunit.client.0.smithi008.stdout:7/381: write d1/d5/da/d11/f57 [289735,84481] 0 2024-04-12T05:55:28.137 INFO:tasks.workunit.client.0.smithi008.stdout:6/328: rename df/f12 to df/d10/d15/d4f/f65 0 2024-04-12T05:55:28.138 INFO:tasks.workunit.client.0.smithi008.stdout:7/382: chown d1/d5/da/d60/c5d 3673683 1 2024-04-12T05:55:28.138 INFO:tasks.workunit.client.0.smithi008.stdout:3/319: creat d9/d2e/d3d/f6b x:0 0 0 2024-04-12T05:55:28.138 INFO:tasks.workunit.client.0.smithi008.stdout:1/267: symlink d4/dd/d17/l5a 0 2024-04-12T05:55:28.141 INFO:tasks.workunit.client.0.smithi008.stdout:5/474: rename d0/dd/d11/f5a to d0/dc/d13/d1a/d35/f94 0 2024-04-12T05:55:28.142 INFO:tasks.workunit.client.0.smithi008.stdout:3/320: chown d9/d2e/d3d/f51 159 1 2024-04-12T05:55:28.142 INFO:tasks.workunit.client.0.smithi008.stdout:6/329: write df/f31 [4448054,124240] 0 2024-04-12T05:55:28.144 INFO:tasks.workunit.client.0.smithi008.stdout:1/268: write d4/d6/f10 [3048546,118269] 0 2024-04-12T05:55:28.147 INFO:tasks.workunit.client.0.smithi008.stdout:6/330: truncate df/f63 11886 0 2024-04-12T05:55:28.148 INFO:tasks.workunit.client.0.smithi008.stdout:7/383: write d1/d5/da/d11/f69 [3665714,78933] 0 2024-04-12T05:55:28.148 INFO:tasks.workunit.client.0.smithi008.stdout:1/269: dread - d4/d7/f4e zero size 2024-04-12T05:55:28.150 INFO:tasks.workunit.client.0.smithi008.stdout:3/321: unlink d9/d10/d19/d22/fc 0 2024-04-12T05:55:28.151 INFO:tasks.workunit.client.0.smithi008.stdout:8/213: dread dd/f1f [0,4194304] 0 2024-04-12T05:55:28.153 INFO:tasks.workunit.client.0.smithi008.stdout:6/331: fdatasync f8 0 2024-04-12T05:55:28.156 INFO:tasks.workunit.client.0.smithi008.stdout:8/214: read dd/ff [1506559,89955] 0 2024-04-12T05:55:28.157 INFO:tasks.workunit.client.0.smithi008.stdout:3/322: truncate d9/d11/f47 927336 0 2024-04-12T05:55:28.158 INFO:tasks.workunit.client.0.smithi008.stdout:6/332: fdatasync df/d21/d4c/f58 0 2024-04-12T05:55:28.160 INFO:tasks.workunit.client.0.smithi008.stdout:1/270: fdatasync d4/dd/d17/f31 0 2024-04-12T05:55:28.160 INFO:tasks.workunit.client.0.smithi008.stdout:6/333: write df/d21/f57 [262947,32710] 0 2024-04-12T05:55:28.164 INFO:tasks.workunit.client.0.smithi008.stdout:3/323: getdents d9/d37/d69 0 2024-04-12T05:55:28.164 INFO:tasks.workunit.client.0.smithi008.stdout:7/384: dwrite d1/f3 [4194304,4194304] 0 2024-04-12T05:55:28.166 INFO:tasks.workunit.client.0.smithi008.stdout:5/475: dread d0/dc/d13/d1a/d35/f4c [0,4194304] 0 2024-04-12T05:55:28.167 INFO:tasks.workunit.client.0.smithi008.stdout:1/271: mknod d4/d33/d37/c5b 0 2024-04-12T05:55:28.170 INFO:tasks.workunit.client.0.smithi008.stdout:1/272: creat d4/dd/d21/f5c x:0 0 0 2024-04-12T05:55:28.173 INFO:tasks.workunit.client.0.smithi008.stdout:5/476: getdents d0/dc/d57 0 2024-04-12T05:55:28.174 INFO:tasks.workunit.client.0.smithi008.stdout:1/273: creat d4/dd/d17/d28/d30/f5d x:0 0 0 2024-04-12T05:55:28.175 INFO:tasks.workunit.client.0.smithi008.stdout:1/274: chown d4/d1a 4086939 1 2024-04-12T05:55:28.181 INFO:tasks.workunit.client.0.smithi008.stdout:1/275: write d4/dd/d17/f31 [1607863,79860] 0 2024-04-12T05:55:28.182 INFO:tasks.workunit.client.0.smithi008.stdout:5/477: dread d0/dd/d3d/d5d/f78 [0,4194304] 0 2024-04-12T05:55:28.182 INFO:tasks.workunit.client.0.smithi008.stdout:3/324: write d9/d10/d19/d22/f4d [722574,12912] 0 2024-04-12T05:55:28.185 INFO:tasks.workunit.client.0.smithi008.stdout:1/276: rmdir d4/d33/d37 39 2024-04-12T05:55:28.185 INFO:tasks.workunit.client.0.smithi008.stdout:5/478: chown d0/d40/l4d 1592 1 2024-04-12T05:55:28.186 INFO:tasks.workunit.client.0.smithi008.stdout:3/325: unlink d9/d11/f47 0 2024-04-12T05:55:28.191 INFO:tasks.workunit.client.0.smithi008.stdout:1/277: mknod d4/d1c/d4f/c5e 0 2024-04-12T05:55:28.194 INFO:tasks.workunit.client.0.smithi008.stdout:1/278: truncate f3 4636697 0 2024-04-12T05:55:28.197 INFO:tasks.workunit.client.0.smithi008.stdout:5/479: dwrite d0/dc/d13/d1a/d35/f64 [0,4194304] 0 2024-04-12T05:55:28.198 INFO:tasks.workunit.client.0.smithi008.stdout:1/279: creat d4/d1a/f5f x:0 0 0 2024-04-12T05:55:28.201 INFO:tasks.workunit.client.0.smithi008.stdout:3/326: dwrite d9/d1e/f57 [0,4194304] 0 2024-04-12T05:55:28.202 INFO:tasks.workunit.client.0.smithi008.stdout:5/480: creat d0/d2/f95 x:0 0 0 2024-04-12T05:55:28.203 INFO:tasks.workunit.client.0.smithi008.stdout:3/327: chown d9/d10/d19 100040694 1 2024-04-12T05:55:28.206 INFO:tasks.workunit.client.0.smithi008.stdout:3/328: dread d9/d1e/f35 [0,4194304] 0 2024-04-12T05:55:28.208 INFO:tasks.workunit.client.0.smithi008.stdout:1/280: getdents d4/d6/d11 0 2024-04-12T05:55:28.208 INFO:tasks.workunit.client.0.smithi008.stdout:3/329: chown d9/d2e/l3c 94603964 1 2024-04-12T05:55:28.210 INFO:tasks.workunit.client.0.smithi008.stdout:1/281: mknod d4/dd/d17/c60 0 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 -- 172.21.15.8:0/1810132154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa8100e10 msgr2=0x7f4fa8101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 --2- 172.21.15.8:0/1810132154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa8100e10 0x7f4fa8101270 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009910 tx=0x7f4f98009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 -- 172.21.15.8:0/1810132154 shutdown_connections 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 --2- 172.21.15.8:0/1810132154 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa8100e10 0x7f4fa8101270 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 --2- 172.21.15.8:0/1810132154 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa80ffc10 0x7f4fa8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 -- 172.21.15.8:0/1810132154 >> 172.21.15.8:0/1810132154 conn(0x7f4fa80fb190 msgr2=0x7f4fa80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:28.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 -- 172.21.15.8:0/1810132154 shutdown_connections 2024-04-12T05:55:28.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.232+0000 7f4faff53700 1 -- 172.21.15.8:0/1810132154 wait complete. 2024-04-12T05:55:28.228 INFO:tasks.workunit.client.0.smithi008.stdout:5/481: rmdir d0/d2 39 2024-04-12T05:55:28.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 Processor -- start 2024-04-12T05:55:28.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 -- start start 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa8100e10 0x7f4fa8196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8196f20 con 0x7f4fa80ffc10 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4faff53700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa8197060 con 0x7f4fa8100e10 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4fadcef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4fadcef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56194/0 (socket says 172.21.15.8:56194) 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.233+0000 7f4fadcef700 1 -- 172.21.15.8:0/4234805083 learned_addr learned my addr 172.21.15.8:0/4234805083 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:28.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4fad4ee700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa8100e10 0x7f4fa8196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4fadcef700 1 -- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa8100e10 msgr2=0x7f4fa8196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4fadcef700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa8100e10 0x7f4fa8196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4fadcef700 1 -- 172.21.15.8:0/4234805083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980095f0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4fadcef700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400d900 tx=0x7f4fa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa400b920 con 0x7f4fa80ffc10 2024-04-12T05:55:28.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f4fa4010460 con 0x7f4fa80ffc10 2024-04-12T05:55:28.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.234+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa400f5d0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.235+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa819bb10 con 0x7f4fa80ffc10 2024-04-12T05:55:28.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.235+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa819c060 con 0x7f4fa80ffc10 2024-04-12T05:55:28.232 INFO:tasks.workunit.client.0.smithi008.stdout:1/282: truncate d4/d24/f32 520635 0 2024-04-12T05:55:28.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.237+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa81085c0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.237+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f4fa40105d0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.238+0000 7f4f9effd700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 0x7f4f9406efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:28.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.238+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f4fa408bfa0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.238+0000 7f4fad4ee700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 0x7f4f9406efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:28.235 INFO:tasks.workunit.client.0.smithi008.stdout:5/482: write d0/dc/d13/d1a/f22 [441069,68196] 0 2024-04-12T05:55:28.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.239+0000 7f4fad4ee700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 0x7f4f9406efe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000c00 tx=0x7f4f98005fb0 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:28.238 INFO:tasks.workunit.client.0.smithi008.stdout:1/283: mkdir d4/dd/d17/d61 0 2024-04-12T05:55:28.239 INFO:tasks.workunit.client.0.smithi008.stdout:3/330: dwrite d9/f25 [0,4194304] 0 2024-04-12T05:55:28.240 INFO:tasks.workunit.client.0.smithi008.stdout:3/331: stat d9/d37 0 2024-04-12T05:55:28.242 INFO:tasks.workunit.client.0.smithi008.stdout:0/306: sync 2024-04-12T05:55:28.242 INFO:tasks.workunit.client.0.smithi008.stdout:9/274: sync 2024-04-12T05:55:28.243 INFO:tasks.workunit.client.0.smithi008.stdout:3/332: truncate d9/d2e/d3d/f56 1793113 0 2024-04-12T05:55:28.244 INFO:tasks.workunit.client.0.smithi008.stdout:9/275: chown d3/df/d13/d21/f44 7377145 1 2024-04-12T05:55:28.245 INFO:tasks.workunit.client.0.smithi008.stdout:3/333: stat d9/d11/l41 0 2024-04-12T05:55:28.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.247+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f4fa4059ae0 con 0x7f4fa80ffc10 2024-04-12T05:55:28.255 INFO:tasks.workunit.client.0.smithi008.stdout:9/276: symlink d3/df/d13/d21/d5c/l5e 0 2024-04-12T05:55:28.258 INFO:tasks.workunit.client.0.smithi008.stdout:0/307: getdents d2/d11 0 2024-04-12T05:55:28.269 INFO:tasks.workunit.client.0.smithi008.stdout:9/277: stat d3/d22/d2b/d58/f55 0 2024-04-12T05:55:28.270 INFO:tasks.workunit.client.0.smithi008.stdout:1/284: getdents d4/dd/d21 0 2024-04-12T05:55:28.270 INFO:tasks.workunit.client.0.smithi008.stdout:0/308: chown d2/d11/d36/d42/c52 39824456 1 2024-04-12T05:55:28.270 INFO:tasks.workunit.client.0.smithi008.stdout:3/334: dwrite d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:28.270 INFO:tasks.workunit.client.0.smithi008.stdout:9/278: write d3/d22/d2b/d58/f2e [783807,125888] 0 2024-04-12T05:55:28.270 INFO:tasks.workunit.client.0.smithi008.stdout:0/309: mknod d2/c5e 0 2024-04-12T05:55:28.276 INFO:tasks.workunit.client.0.smithi008.stdout:1/285: dwrite d4/d6/f59 [0,4194304] 0 2024-04-12T05:55:28.277 INFO:tasks.workunit.client.0.smithi008.stdout:5/483: dread d0/dc/f65 [0,4194304] 0 2024-04-12T05:55:28.277 INFO:tasks.workunit.client.0.smithi008.stdout:0/310: fsync d2/d24/d39/f53 0 2024-04-12T05:55:28.277 INFO:tasks.workunit.client.0.smithi008.stdout:9/279: truncate d3/d22/d2b/d58/f55 1388470 0 2024-04-12T05:55:28.282 INFO:tasks.workunit.client.0.smithi008.stdout:9/280: chown d3/d22/d2b/d58/d36/f3c 15 1 2024-04-12T05:55:28.284 INFO:tasks.workunit.client.0.smithi008.stdout:4/172: sync 2024-04-12T05:55:28.284 INFO:tasks.workunit.client.0.smithi008.stdout:0/311: link d2/d11/d36/d47/f59 d2/d4b/f5f 0 2024-04-12T05:55:28.287 INFO:tasks.workunit.client.0.smithi008.stdout:5/484: getdents d0/dd/d11/d38 0 2024-04-12T05:55:28.287 INFO:tasks.workunit.client.0.smithi008.stdout:0/312: creat d2/d24/d39/f60 x:0 0 0 2024-04-12T05:55:28.288 INFO:tasks.workunit.client.0.smithi008.stdout:4/173: mkdir d1/d8/d2f 0 2024-04-12T05:55:28.290 INFO:tasks.workunit.client.0.smithi008.stdout:5/485: mkdir d0/d29/d96 0 2024-04-12T05:55:28.290 INFO:tasks.workunit.client.0.smithi008.stdout:0/313: truncate d2/d4b/f5f 609936 0 2024-04-12T05:55:28.292 INFO:tasks.workunit.client.0.smithi008.stdout:4/174: mkdir d1/d22/d29/d30 0 2024-04-12T05:55:28.293 INFO:tasks.workunit.client.0.smithi008.stdout:4/175: read - d1/d8/f23 zero size 2024-04-12T05:55:28.294 INFO:tasks.workunit.client.0.smithi008.stdout:0/314: chown d2/d11/d2a/f2d 17 1 2024-04-12T05:55:28.294 INFO:tasks.workunit.client.0.smithi008.stdout:5/486: rmdir d0/dd/d3d/d5d 39 2024-04-12T05:55:28.300 INFO:tasks.workunit.client.0.smithi008.stdout:0/315: write d2/d24/f28 [4290031,87165] 0 2024-04-12T05:55:28.304 INFO:tasks.workunit.client.0.smithi008.stdout:4/176: link d1/db/f1a d1/d22/d29/d30/f31 0 2024-04-12T05:55:28.305 INFO:tasks.workunit.client.0.smithi008.stdout:5/487: creat d0/f97 x:0 0 0 2024-04-12T05:55:28.306 INFO:tasks.workunit.client.0.smithi008.stdout:4/177: fsync d1/f13 0 2024-04-12T05:55:28.308 INFO:tasks.workunit.client.0.smithi008.stdout:1/286: dread d4/dd/d17/d26/f2e [0,4194304] 0 2024-04-12T05:55:28.308 INFO:tasks.workunit.client.0.smithi008.stdout:9/281: dwrite d3/d22/d2b/d58/d36/f3c [0,4194304] 0 2024-04-12T05:55:28.310 INFO:tasks.workunit.client.0.smithi008.stdout:5/488: chown d0/d2/d37 4 1 2024-04-12T05:55:28.319 INFO:tasks.workunit.client.0.smithi008.stdout:1/287: readlink d4/dd/d17/l54 0 2024-04-12T05:55:28.322 INFO:tasks.workunit.client.0.smithi008.stdout:5/489: dwrite d0/dd/d3d/f53 [0,4194304] 0 2024-04-12T05:55:28.324 INFO:tasks.workunit.client.0.smithi008.stdout:5/490: chown d0/dc/d13/d1a 99 1 2024-04-12T05:55:28.324 INFO:tasks.workunit.client.0.smithi008.stdout:4/178: rmdir d1 39 2024-04-12T05:55:28.331 INFO:tasks.workunit.client.0.smithi008.stdout:5/491: unlink d0/dd/d11/f8f 0 2024-04-12T05:55:28.332 INFO:tasks.workunit.client.0.smithi008.stdout:1/288: getdents d4/d7/d18 0 2024-04-12T05:55:28.334 INFO:tasks.workunit.client.0.smithi008.stdout:1/289: stat d4/dd/d17/d28/d30/c3d 0 2024-04-12T05:55:28.336 INFO:tasks.workunit.client.0.smithi008.stdout:0/316: truncate d2/d11/d36/d47/f1d 1902117 0 2024-04-12T05:55:28.337 INFO:tasks.workunit.client.0.smithi008.stdout:5/492: dread d0/d2/f31 [0,4194304] 0 2024-04-12T05:55:28.338 INFO:tasks.workunit.client.0.smithi008.stdout:4/179: symlink d1/d22/d29/d30/l32 0 2024-04-12T05:55:28.338 INFO:tasks.workunit.client.0.smithi008.stdout:4/180: chown d1/d22 776767 1 2024-04-12T05:55:28.340 INFO:tasks.workunit.client.0.smithi008.stdout:9/282: dread d3/df/d13/d21/d2d/d32/f34 [0,4194304] 0 2024-04-12T05:55:28.341 INFO:tasks.workunit.client.0.smithi008.stdout:5/493: chown d0/dd/d3d/d5d/f7e 20 1 2024-04-12T05:55:28.342 INFO:tasks.workunit.client.0.smithi008.stdout:5/494: dread - d0/dc/d13/d1a/f83 zero size 2024-04-12T05:55:28.343 INFO:tasks.workunit.client.0.smithi008.stdout:0/317: mkdir d2/d24/d61 0 2024-04-12T05:55:28.344 INFO:tasks.workunit.client.0.smithi008.stdout:4/181: fdatasync d1/d22/d29/d30/f31 0 2024-04-12T05:55:28.350 INFO:tasks.workunit.client.0.smithi008.stdout:5/495: unlink d0/dd/d11/d38/c23 0 2024-04-12T05:55:28.352 INFO:tasks.workunit.client.0.smithi008.stdout:1/290: write d4/d7/fc [1272960,57940] 0 2024-04-12T05:55:28.352 INFO:tasks.workunit.client.0.smithi008.stdout:4/182: dwrite d1/f20 [0,4194304] 0 2024-04-12T05:55:28.354 INFO:tasks.workunit.client.0.smithi008.stdout:0/318: rename d2/d24/d39/f5a to d2/d11/d36/d42/f62 0 2024-04-12T05:55:28.359 INFO:tasks.workunit.client.0.smithi008.stdout:4/183: symlink d1/d22/d29/l33 0 2024-04-12T05:55:28.362 INFO:tasks.workunit.client.0.smithi008.stdout:9/283: truncate d3/d22/d2b/d58/f1d 2128916 0 2024-04-12T05:55:28.364 INFO:tasks.workunit.client.0.smithi008.stdout:9/284: chown d3/f25 9 1 2024-04-12T05:55:28.367 INFO:tasks.workunit.client.0.smithi008.stdout:9/285: fdatasync d3/df/d13/d21/d2d/d30/f4d 0 2024-04-12T05:55:28.367 INFO:tasks.workunit.client.0.smithi008.stdout:1/291: dwrite d4/f27 [0,4194304] 0 2024-04-12T05:55:28.368 INFO:tasks.workunit.client.0.smithi008.stdout:4/184: creat d1/d22/d29/d30/f34 x:0 0 0 2024-04-12T05:55:28.370 INFO:tasks.workunit.client.0.smithi008.stdout:4/185: read d1/f13 [4021818,64650] 0 2024-04-12T05:55:28.372 INFO:tasks.workunit.client.0.smithi008.stdout:4/186: truncate d1/d8/f2d 965494 0 2024-04-12T05:55:28.372 INFO:tasks.workunit.client.0.smithi008.stdout:4/187: chown d1/db 104144 1 2024-04-12T05:55:28.372 INFO:tasks.workunit.client.0.smithi008.stdout:5/496: link d0/d2/d37/c73 d0/d40/c98 0 2024-04-12T05:55:28.374 INFO:tasks.workunit.client.0.smithi008.stdout:5/497: dread - d0/dc/d13/d1a/f83 zero size 2024-04-12T05:55:28.376 INFO:tasks.workunit.client.0.smithi008.stdout:9/286: chown d3/d22/d2b/c3a 2167 1 2024-04-12T05:55:28.376 INFO:tasks.workunit.client.0.smithi008.stdout:1/292: mkdir d4/d24/d62 0 2024-04-12T05:55:28.377 INFO:tasks.workunit.client.0.smithi008.stdout:4/188: dread d1/d8/f2d [0,4194304] 0 2024-04-12T05:55:28.379 INFO:tasks.workunit.client.0.smithi008.stdout:0/319: dread d2/d24/f28 [0,4194304] 0 2024-04-12T05:55:28.380 INFO:tasks.workunit.client.0.smithi008.stdout:4/189: fsync d1/d8/f23 0 2024-04-12T05:55:28.381 INFO:tasks.workunit.client.0.smithi008.stdout:1/293: getdents d4/d1a 0 2024-04-12T05:55:28.382 INFO:tasks.workunit.client.0.smithi008.stdout:4/190: chown d1/d22/l2b 25750 1 2024-04-12T05:55:28.388 INFO:tasks.workunit.client.0.smithi008.stdout:9/287: dwrite d3/df/d13/d21/f44 [0,4194304] 0 2024-04-12T05:55:28.392 INFO:tasks.workunit.client.0.smithi008.stdout:0/320: getdents d2/d11/d36/d47 0 2024-04-12T05:55:28.399 INFO:tasks.workunit.client.0.smithi008.stdout:9/288: dwrite d3/df/d13/d21/f31 [0,4194304] 0 2024-04-12T05:55:28.401 INFO:tasks.workunit.client.0.smithi008.stdout:9/289: readlink d3/d22/l24 0 2024-04-12T05:55:28.401 INFO:tasks.workunit.client.0.smithi008.stdout:0/321: creat d2/d24/f63 x:0 0 0 2024-04-12T05:55:28.404 INFO:tasks.workunit.client.0.smithi008.stdout:1/294: link d4/d1c/f2b d4/dd/d17/f63 0 2024-04-12T05:55:28.405 INFO:tasks.workunit.client.0.smithi008.stdout:0/322: stat d2/d11/c38 0 2024-04-12T05:55:28.406 INFO:tasks.workunit.client.0.smithi008.stdout:0/323: stat d2/d11/l1e 0 2024-04-12T05:55:28.408 INFO:tasks.workunit.client.0.smithi008.stdout:0/324: fsync d2/d11/d2a/f37 0 2024-04-12T05:55:28.409 INFO:tasks.workunit.client.0.smithi008.stdout:9/290: read d3/d22/d2b/d58/f27 [4001549,105802] 0 2024-04-12T05:55:28.410 INFO:tasks.workunit.client.0.smithi008.stdout:1/295: getdents d4/dd 0 2024-04-12T05:55:28.412 INFO:tasks.workunit.client.0.smithi008.stdout:9/291: write d3/df/d10/f17 [1909149,31205] 0 2024-04-12T05:55:28.413 INFO:tasks.workunit.client.0.smithi008.stdout:4/191: dread d1/db/f16 [0,4194304] 0 2024-04-12T05:55:28.415 INFO:tasks.workunit.client.0.smithi008.stdout:1/296: read d4/dd/d17/f63 [214913,59204] 0 2024-04-12T05:55:28.416 INFO:tasks.workunit.client.0.smithi008.stdout:9/292: rmdir d3/df/d13/d21/d2d/d32/d52 39 2024-04-12T05:55:28.418 INFO:tasks.workunit.client.0.smithi008.stdout:9/293: symlink d3/df/d13/d14/l5f 0 2024-04-12T05:55:28.419 INFO:tasks.workunit.client.0.smithi008.stdout:1/297: creat d4/dd/d17/f64 x:0 0 0 2024-04-12T05:55:28.421 INFO:tasks.workunit.client.0.smithi008.stdout:9/294: mkdir d3/d22/d2b/d4e/d60 0 2024-04-12T05:55:28.431 INFO:tasks.workunit.client.0.smithi008.stdout:0/325: dread d2/fa [0,4194304] 0 2024-04-12T05:55:28.431 INFO:tasks.workunit.client.0.smithi008.stdout:9/295: dwrite d3/df/f20 [0,4194304] 0 2024-04-12T05:55:28.432 INFO:tasks.workunit.client.0.smithi008.stdout:9/296: stat d3/f35 0 2024-04-12T05:55:28.435 INFO:tasks.workunit.client.0.smithi008.stdout:0/326: creat d2/d11/f64 x:0 0 0 2024-04-12T05:55:28.436 INFO:tasks.workunit.client.0.smithi008.stdout:9/297: rename d3/df/d13/d21/f31 to d3/d22/d2b/d4e/f61 0 2024-04-12T05:55:28.436 INFO:tasks.workunit.client.0.smithi008.stdout:9/298: fsync d3/f5 0 2024-04-12T05:55:28.437 INFO:tasks.workunit.client.0.smithi008.stdout:0/327: truncate d2/d11/d2a/f5c 324524 0 2024-04-12T05:55:28.439 INFO:tasks.workunit.client.0.smithi008.stdout:0/328: write d2/d11/d2a/f4a [1872882,56145] 0 2024-04-12T05:55:28.441 INFO:tasks.workunit.client.0.smithi008.stdout:9/299: mkdir d3/d22/d2b/d4a/d62 0 2024-04-12T05:55:28.443 INFO:tasks.workunit.client.0.smithi008.stdout:9/300: creat d3/df/f63 x:0 0 0 2024-04-12T05:55:28.446 INFO:tasks.workunit.client.0.smithi008.stdout:9/301: symlink d3/df/d10/l64 0 2024-04-12T05:55:28.450 INFO:tasks.workunit.client.0.smithi008.stdout:9/302: fsync d3/d22/d2b/f41 0 2024-04-12T05:55:28.452 INFO:tasks.workunit.client.0.smithi008.stdout:9/303: mknod d3/df/d13/c65 0 2024-04-12T05:55:28.461 INFO:tasks.workunit.client.0.smithi008.stdout:9/304: dwrite d3/fa [0,4194304] 0 2024-04-12T05:55:28.484 INFO:tasks.workunit.client.0.smithi008.stdout:9/305: dread d3/d22/d2b/d58/f27 [0,4194304] 0 2024-04-12T05:55:28.487 INFO:tasks.workunit.client.0.smithi008.stdout:9/306: mknod d3/d22/d2b/d4e/d60/c66 0 2024-04-12T05:55:28.490 INFO:tasks.workunit.client.0.smithi008.stdout:9/307: creat d3/d22/d2b/d58/d38/f67 x:0 0 0 2024-04-12T05:55:28.493 INFO:tasks.workunit.client.0.smithi008.stdout:9/308: rmdir d3/df/d13/d21/d2d 39 2024-04-12T05:55:28.508 INFO:tasks.workunit.client.0.smithi008.stdout:9/309: dread d3/f1f [0,4194304] 0 2024-04-12T05:55:28.515 INFO:tasks.workunit.client.0.smithi008.stdout:9/310: dwrite d3/d22/d2b/d58/d38/f67 [0,4194304] 0 2024-04-12T05:55:28.518 INFO:tasks.workunit.client.0.smithi008.stdout:9/311: rename d3/d22/d2b/d58/d45 to d3/d22/d2b/d4e/d60/d68 0 2024-04-12T05:55:28.522 INFO:tasks.workunit.client.0.smithi008.stdout:9/312: dread d3/d22/f3e [0,4194304] 0 2024-04-12T05:55:28.524 INFO:tasks.workunit.client.0.smithi008.stdout:9/313: mknod d3/d22/c69 0 2024-04-12T05:55:28.531 INFO:tasks.workunit.client.0.smithi008.stdout:9/314: dwrite d3/df/f63 [0,4194304] 0 2024-04-12T05:55:28.533 INFO:tasks.workunit.client.0.smithi008.stdout:9/315: mkdir d3/d6a 0 2024-04-12T05:55:28.536 INFO:tasks.workunit.client.0.smithi008.stdout:9/316: creat d3/d22/d2b/d4e/f6b x:0 0 0 2024-04-12T05:55:28.538 INFO:tasks.workunit.client.0.smithi008.stdout:9/317: write d3/d22/d2b/d58/d38/f67 [578928,51179] 0 2024-04-12T05:55:28.539 INFO:tasks.workunit.client.0.smithi008.stdout:9/318: write d3/df/d13/f5a [400039,34160] 0 2024-04-12T05:55:28.541 INFO:tasks.workunit.client.0.smithi008.stdout:9/319: fsync d3/d22/d2b/d58/f3f 0 2024-04-12T05:55:28.542 INFO:tasks.workunit.client.0.smithi008.stdout:9/320: stat d3/df/d13/d14/l2c 0 2024-04-12T05:55:28.544 INFO:tasks.workunit.client.0.smithi008.stdout:9/321: fdatasync d3/df/d13/d21/d2d/d32/f34 0 2024-04-12T05:55:28.545 INFO:tasks.workunit.client.0.smithi008.stdout:9/322: chown d3/d22/d2b/f33 4272426 1 2024-04-12T05:55:28.549 INFO:tasks.workunit.client.0.smithi008.stdout:9/323: getdents d3/df/d13/d21/d2d/d32/d52 0 2024-04-12T05:55:28.550 INFO:tasks.workunit.client.0.smithi008.stdout:9/324: write d3/df/d10/f17 [3085927,118271] 0 2024-04-12T05:55:28.554 INFO:tasks.workunit.client.0.smithi008.stdout:9/325: dread d3/fa [0,4194304] 0 2024-04-12T05:55:28.556 INFO:tasks.workunit.client.0.smithi008.stdout:9/326: chown d3/d22/d2b/d58/f1d 108897397 1 2024-04-12T05:55:28.558 INFO:tasks.workunit.client.0.smithi008.stdout:9/327: mknod d3/df/d13/d21/d2d/d32/c6c 0 2024-04-12T05:55:28.558 INFO:tasks.workunit.client.0.smithi008.stdout:9/328: read d3/d22/f3e [124416,89442] 0 2024-04-12T05:55:28.561 INFO:tasks.workunit.client.0.smithi008.stdout:9/329: creat d3/d22/d2b/d4e/d60/f6d x:0 0 0 2024-04-12T05:55:28.563 INFO:tasks.workunit.client.0.smithi008.stdout:9/330: rename d3/d22/d2b/c53 to d3/d22/d2b/d58/d5d/c6e 0 2024-04-12T05:55:28.564 INFO:tasks.workunit.client.0.smithi008.stdout:9/331: truncate d3/f4b 637324 0 2024-04-12T05:55:28.570 INFO:tasks.workunit.client.0.smithi008.stdout:9/332: dwrite d3/df/d13/f5a [0,4194304] 0 2024-04-12T05:55:28.571 INFO:tasks.workunit.client.0.smithi008.stdout:9/333: truncate d3/d22/f3e 969853 0 2024-04-12T05:55:28.572 INFO:tasks.workunit.client.0.smithi008.stdout:9/334: read d3/df/d13/f51 [2196569,105864] 0 2024-04-12T05:55:28.575 INFO:tasks.workunit.client.0.smithi008.stdout:9/335: write d3/d22/d2b/d58/d36/f3c [737072,89879] 0 2024-04-12T05:55:28.578 INFO:tasks.workunit.client.0.smithi008.stdout:9/336: symlink d3/d22/d2b/d58/l6f 0 2024-04-12T05:55:28.584 INFO:tasks.workunit.client.0.smithi008.stdout:9/337: write d3/d22/d2b/d58/d38/f54 [1109842,8722] 0 2024-04-12T05:55:28.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.591+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fa8068c90 con 0x7f4f9406cb20 2024-04-12T05:55:28.587 INFO:tasks.workunit.client.0.smithi008.stdout:9/338: mknod d3/df/d10/c70 0 2024-04-12T05:55:28.587 INFO:tasks.workunit.client.0.smithi008.stdout:2/320: sync 2024-04-12T05:55:28.587 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.592+0000 7f4f9effd700 1 -- 172.21.15.8:0/4234805083 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4fa8068c90 con 0x7f4f9406cb20 2024-04-12T05:55:28.587 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "1/2 daemons upgraded", 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "", 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:55:28.588 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:55:28.588 INFO:tasks.workunit.client.0.smithi008.stdout:2/321: write d6/f16 [117072,6204] 0 2024-04-12T05:55:28.590 INFO:tasks.workunit.client.0.smithi008.stdout:9/339: symlink d3/df/d13/d21/d2d/d32/l71 0 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 msgr2=0x7f4f9406efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 0x7f4f9406efe0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f98000c00 tx=0x7f4f98005fb0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 msgr2=0x7f4fa81963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400d900 tx=0x7f4fa400dcc0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 shutdown_connections 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f4f9406cb20 0x7f4f9406efe0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4fa80ffc10 0x7f4fa81963c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 --2- 172.21.15.8:0/4234805083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fa8100e10 0x7f4fa8196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:28.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 >> 172.21.15.8:0/4234805083 conn(0x7f4fa80fb190 msgr2=0x7f4fa8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:28.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 shutdown_connections 2024-04-12T05:55:28.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:28.596+0000 7f4faff53700 1 -- 172.21.15.8:0/4234805083 wait complete. 2024-04-12T05:55:28.593 INFO:tasks.workunit.client.0.smithi008.stdout:9/340: creat d3/d22/d2b/f72 x:0 0 0 2024-04-12T05:55:28.597 INFO:tasks.workunit.client.0.smithi008.stdout:2/322: dwrite d6/db/f4b [0,4194304] 0 2024-04-12T05:55:28.599 INFO:tasks.workunit.client.0.smithi008.stdout:2/323: write d6/db/f56 [911105,65619] 0 2024-04-12T05:55:28.599 INFO:tasks.workunit.client.0.smithi008.stdout:9/341: link d3/d22/l24 d3/d22/d2b/d58/l73 0 2024-04-12T05:55:28.606 INFO:tasks.workunit.client.0.smithi008.stdout:2/324: read d6/f43 [2534028,13555] 0 2024-04-12T05:55:28.608 INFO:tasks.workunit.client.0.smithi008.stdout:9/342: dwrite d3/d22/d2b/d58/d38/f67 [0,4194304] 0 2024-04-12T05:55:28.611 INFO:tasks.workunit.client.0.smithi008.stdout:9/343: fdatasync d3/d22/d2b/d58/f1d 0 2024-04-12T05:55:28.613 INFO:tasks.workunit.client.0.smithi008.stdout:2/325: dwrite d6/db/f4b [0,4194304] 0 2024-04-12T05:55:28.616 INFO:tasks.workunit.client.0.smithi008.stdout:9/344: mkdir d3/df/d13/d21/d2d/d30/d74 0 2024-04-12T05:55:28.617 INFO:tasks.workunit.client.0.smithi008.stdout:2/326: write d6/db/d60/f42 [2250170,119620] 0 2024-04-12T05:55:28.617 INFO:tasks.workunit.client.0.smithi008.stdout:9/345: chown d3/d22/f3e 27364 1 2024-04-12T05:55:28.618 INFO:tasks.workunit.client.0.smithi008.stdout:9/346: readlink d3/df/d13/d14/l5f 0 2024-04-12T05:55:28.620 INFO:tasks.workunit.client.0.smithi008.stdout:2/327: rmdir d6/db/d38 39 2024-04-12T05:55:28.620 INFO:tasks.workunit.client.0.smithi008.stdout:9/347: write d3/df/d13/f51 [1621632,20293] 0 2024-04-12T05:55:28.640 INFO:tasks.workunit.client.0.smithi008.stdout:2/328: truncate d6/db/f4b 655130 0 2024-04-12T05:55:28.640 INFO:tasks.workunit.client.0.smithi008.stdout:2/329: readlink l4 0 2024-04-12T05:55:28.640 INFO:tasks.workunit.client.0.smithi008.stdout:9/348: truncate d3/d22/d2b/d58/d36/f3c 462175 0 2024-04-12T05:55:28.641 INFO:tasks.workunit.client.0.smithi008.stdout:2/330: write f0 [4380860,128867] 0 2024-04-12T05:55:28.644 INFO:tasks.workunit.client.0.smithi008.stdout:9/349: mkdir d3/df/d13/d21/d2d/d75 0 2024-04-12T05:55:28.644 INFO:tasks.workunit.client.0.smithi008.stdout:2/331: fsync d6/db/d38/d41/f3a 0 2024-04-12T05:55:28.649 INFO:tasks.workunit.client.0.smithi008.stdout:9/350: rename d3/d22/d2b/d58/f2e to d3/df/d13/d21/f76 0 2024-04-12T05:55:28.650 INFO:tasks.workunit.client.0.smithi008.stdout:9/351: chown d3/d22/d2b/d58/d38 127 1 2024-04-12T05:55:28.653 INFO:tasks.workunit.client.0.smithi008.stdout:2/332: rename d6/db/d38/d41/f3a to d6/f6d 0 2024-04-12T05:55:28.654 INFO:tasks.workunit.client.0.smithi008.stdout:9/352: mkdir d3/d22/d2b/d77 0 2024-04-12T05:55:28.655 INFO:tasks.workunit.client.0.smithi008.stdout:2/333: symlink d6/db/d10/l6e 0 2024-04-12T05:55:28.659 INFO:tasks.workunit.client.0.smithi008.stdout:9/353: dread d3/d22/d2b/d58/d38/f54 [0,4194304] 0 2024-04-12T05:55:28.662 INFO:tasks.workunit.client.0.smithi008.stdout:9/354: creat d3/d22/d2b/d77/f78 x:0 0 0 2024-04-12T05:55:28.664 INFO:tasks.workunit.client.0.smithi008.stdout:9/355: rmdir d3/d22/d2b/d4e 39 2024-04-12T05:55:28.669 INFO:tasks.workunit.client.0.smithi008.stdout:9/356: rename d3/d22/d2b/d58/l73 to d3/df/l79 0 2024-04-12T05:55:28.670 INFO:tasks.workunit.client.0.smithi008.stdout:9/357: readlink d3/df/d13/d21/d2d/d32/l71 0 2024-04-12T05:55:28.670 INFO:tasks.workunit.client.0.smithi008.stdout:9/358: fdatasync d3/df/d13/d21/d2d/d32/f34 0 2024-04-12T05:55:28.671 INFO:tasks.workunit.client.0.smithi008.stdout:9/359: chown d3/df/c3b 0 1 2024-04-12T05:55:28.674 INFO:tasks.workunit.client.0.smithi008.stdout:9/360: unlink d3/d22/d2b/d4e/d60/c66 0 2024-04-12T05:55:28.674 INFO:tasks.workunit.client.0.smithi008.stdout:9/361: chown d3/df/d13/c29 0 1 2024-04-12T05:55:28.676 INFO:tasks.workunit.client.0.smithi008.stdout:9/362: truncate d3/df/d13/d21/d2d/d30/f4d 1090353 0 2024-04-12T05:55:28.678 INFO:tasks.workunit.client.0.smithi008.stdout:9/363: creat d3/df/d13/f7a x:0 0 0 2024-04-12T05:55:28.681 INFO:tasks.workunit.client.0.smithi008.stdout:9/364: rmdir d3/d22/d2b/d4e/d60/d68 0 2024-04-12T05:55:28.682 INFO:tasks.workunit.client.0.smithi008.stdout:9/365: write d3/d22/d2b/f33 [336507,25297] 0 2024-04-12T05:55:28.684 INFO:tasks.workunit.client.0.smithi008.stdout:9/366: mknod d3/df/d13/d21/d2d/d30/d74/c7b 0 2024-04-12T05:55:28.686 INFO:tasks.workunit.client.0.smithi008.stdout:9/367: mkdir d3/df/d13/d21/d2d/d75/d7c 0 2024-04-12T05:55:28.689 INFO:tasks.workunit.client.0.smithi008.stdout:9/368: dread d3/d22/d2b/d58/f27 [0,4194304] 0 2024-04-12T05:55:28.690 INFO:tasks.workunit.client.0.smithi008.stdout:9/369: fsync d3/f35 0 2024-04-12T05:55:28.693 INFO:tasks.workunit.client.0.smithi008.stdout:9/370: creat d3/df/d13/d21/d2d/d32/f7d x:0 0 0 2024-04-12T05:55:28.694 INFO:tasks.workunit.client.0.smithi008.stdout:9/371: chown d3/f1f 1 1 2024-04-12T05:55:28.695 INFO:tasks.workunit.client.0.smithi008.stdout:9/372: write d3/df/d13/f7a [558189,118240] 0 2024-04-12T05:55:28.698 INFO:tasks.workunit.client.0.smithi008.stdout:9/373: link d3/df/d13/c42 d3/d22/d2b/d4a/d62/c7e 0 2024-04-12T05:55:28.699 INFO:tasks.workunit.client.0.smithi008.stdout:9/374: fdatasync d3/d22/d2b/d58/d38/f54 0 2024-04-12T05:55:28.702 INFO:tasks.workunit.client.0.smithi008.stdout:9/375: link d3/f25 d3/df/d13/f7f 0 2024-04-12T05:55:28.708 INFO:tasks.workunit.client.0.smithi008.stdout:9/376: dwrite d3/d22/d2b/d77/f78 [0,4194304] 0 2024-04-12T05:55:28.713 INFO:tasks.workunit.client.0.smithi008.stdout:9/377: mknod d3/df/d13/d21/d2d/d32/c80 0 2024-04-12T05:55:28.716 INFO:tasks.workunit.client.0.smithi008.stdout:9/378: symlink d3/d22/l81 0 2024-04-12T05:55:28.717 INFO:tasks.workunit.client.0.smithi008.stdout:9/379: dread - d3/d22/d2b/d4e/d60/f6d zero size 2024-04-12T05:55:28.720 INFO:tasks.workunit.client.0.smithi008.stdout:9/380: mkdir d3/d22/d2b/d77/d82 0 2024-04-12T05:55:28.722 INFO:tasks.workunit.client.0.smithi008.stdout:9/381: creat d3/df/d13/d21/d2d/d30/d74/f83 x:0 0 0 2024-04-12T05:55:28.723 INFO:tasks.workunit.client.0.smithi008.stdout:9/382: write d3/f5 [3703322,34412] 0 2024-04-12T05:55:28.725 INFO:tasks.workunit.client.0.smithi008.stdout:9/383: creat d3/d22/d2b/d77/f84 x:0 0 0 2024-04-12T05:55:28.728 INFO:tasks.workunit.client.0.smithi008.stdout:9/384: dread d3/f1f [0,4194304] 0 2024-04-12T05:55:28.762 INFO:tasks.workunit.client.0.smithi008.stdout:8/215: sync 2024-04-12T05:55:28.763 INFO:tasks.workunit.client.0.smithi008.stdout:8/216: rename dd/d30 to dd/d30/d3f 22 2024-04-12T05:55:28.763 INFO:tasks.workunit.client.0.smithi008.stdout:8/217: rename dd to dd/de/d16/d40 22 2024-04-12T05:55:28.764 INFO:tasks.workunit.client.0.smithi008.stdout:8/218: write dd/f1e [515986,18648] 0 2024-04-12T05:55:28.767 INFO:tasks.workunit.client.0.smithi008.stdout:8/219: rename dd/d30/d3b/f3c to dd/d30/f41 0 2024-04-12T05:55:28.772 INFO:tasks.workunit.client.0.smithi008.stdout:8/220: dwrite dd/de/f2f [0,4194304] 0 2024-04-12T05:55:28.774 INFO:tasks.workunit.client.0.smithi008.stdout:8/221: chown dd/de/d16/f28 33732592 1 2024-04-12T05:55:28.788 INFO:tasks.workunit.client.0.smithi008.stdout:8/222: dwrite dd/de/d16/f28 [0,4194304] 0 2024-04-12T05:55:28.807 INFO:tasks.workunit.client.0.smithi008.stdout:8/223: mknod dd/d30/d32/c42 0 2024-04-12T05:55:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:28 smithi194 ceph-mon[49771]: pgmap v10: 65 pgs: 65 active+clean; 527 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 68 MiB/s wr, 365 op/s 2024-04-12T05:55:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:28 smithi194 ceph-mon[49771]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:28 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/859403729' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:55:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:28 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:28.879 INFO:tasks.workunit.client.0.smithi008.stdout:8/224: creat dd/de/d16/f43 x:0 0 0 2024-04-12T05:55:28.885 INFO:tasks.workunit.client.0.smithi008.stdout:8/225: dwrite dd/de/f2d [0,4194304] 0 2024-04-12T05:55:28.887 INFO:tasks.workunit.client.0.smithi008.stdout:8/226: mknod dd/c44 0 2024-04-12T05:55:28.887 INFO:tasks.workunit.client.0.smithi008.stdout:8/227: fsync dd/de/d16/f43 0 2024-04-12T05:55:28.889 INFO:tasks.workunit.client.0.smithi008.stdout:8/228: creat dd/de/d16/f45 x:0 0 0 2024-04-12T05:55:28.890 INFO:tasks.workunit.client.0.smithi008.stdout:8/229: fdatasync dd/f22 0 2024-04-12T05:55:28.891 INFO:tasks.workunit.client.0.smithi008.stdout:8/230: fsync dd/de/f1c 0 2024-04-12T05:55:28.892 INFO:tasks.workunit.client.0.smithi008.stdout:8/231: mknod dd/d30/d32/c46 0 2024-04-12T05:55:28.895 INFO:tasks.workunit.client.0.smithi008.stdout:8/232: creat dd/de/f47 x:0 0 0 2024-04-12T05:55:28.896 INFO:tasks.workunit.client.0.smithi008.stdout:8/233: chown dd/d30/f41 2236 1 2024-04-12T05:55:28.898 INFO:tasks.workunit.client.0.smithi008.stdout:8/234: creat dd/f48 x:0 0 0 2024-04-12T05:55:28.899 INFO:tasks.workunit.client.0.smithi008.stdout:8/235: creat dd/f49 x:0 0 0 2024-04-12T05:55:28.902 INFO:tasks.workunit.client.0.smithi008.stdout:8/236: link dd/f36 dd/de/d16/f4a 0 2024-04-12T05:55:28.902 INFO:tasks.workunit.client.0.smithi008.stdout:8/237: fdatasync dd/f48 0 2024-04-12T05:55:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:28 smithi008 ceph-mon[39722]: pgmap v10: 65 pgs: 65 active+clean; 527 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 68 MiB/s wr, 365 op/s 2024-04-12T05:55:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:28 smithi008 ceph-mon[39722]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:28 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/859403729' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:55:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:28 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:28.904 INFO:tasks.workunit.client.0.smithi008.stdout:8/238: symlink dd/d30/d32/l4b 0 2024-04-12T05:55:28.908 INFO:tasks.workunit.client.0.smithi008.stdout:7/385: sync 2024-04-12T05:55:28.908 INFO:tasks.workunit.client.0.smithi008.stdout:6/334: sync 2024-04-12T05:55:28.908 INFO:tasks.workunit.client.0.smithi008.stdout:3/335: sync 2024-04-12T05:55:28.909 INFO:tasks.workunit.client.0.smithi008.stdout:6/335: stat df/d10/d15/f24 0 2024-04-12T05:55:28.911 INFO:tasks.workunit.client.0.smithi008.stdout:8/239: getdents dd 0 2024-04-12T05:55:28.913 INFO:tasks.workunit.client.0.smithi008.stdout:6/336: chown df/d21/f22 44 1 2024-04-12T05:55:28.913 INFO:tasks.workunit.client.0.smithi008.stdout:7/386: mknod d1/d5/da/c85 0 2024-04-12T05:55:28.922 INFO:tasks.workunit.client.0.smithi008.stdout:3/336: write d9/d10/d19/d22/d2c/f5e [2522163,33694] 0 2024-04-12T05:55:28.922 INFO:tasks.workunit.client.0.smithi008.stdout:8/240: mkdir dd/de/d4c 0 2024-04-12T05:55:28.923 INFO:tasks.workunit.client.0.smithi008.stdout:6/337: mknod df/d10/d15/c66 0 2024-04-12T05:55:28.924 INFO:tasks.workunit.client.0.smithi008.stdout:7/387: creat d1/d5/da/d3c/d3d/f86 x:0 0 0 2024-04-12T05:55:28.925 INFO:tasks.workunit.client.0.smithi008.stdout:3/337: mknod d9/d1e/c6c 0 2024-04-12T05:55:28.925 INFO:tasks.workunit.client.0.smithi008.stdout:8/241: mknod dd/de/d16/c4d 0 2024-04-12T05:55:29.035 INFO:tasks.workunit.client.0.smithi008.stdout:6/338: rename df/f64 to df/f67 0 2024-04-12T05:55:29.036 INFO:tasks.workunit.client.0.smithi008.stdout:7/388: mknod d1/d15/d43/c87 0 2024-04-12T05:55:29.038 INFO:tasks.workunit.client.0.smithi008.stdout:8/242: rename dd/de/d16/f43 to dd/f4e 0 2024-04-12T05:55:29.040 INFO:tasks.workunit.client.0.smithi008.stdout:7/389: write d1/f4f [490500,88169] 0 2024-04-12T05:55:29.041 INFO:tasks.workunit.client.0.smithi008.stdout:3/338: mknod d9/d2e/c6d 0 2024-04-12T05:55:29.041 INFO:tasks.workunit.client.0.smithi008.stdout:6/339: mknod df/d10/c68 0 2024-04-12T05:55:29.044 INFO:tasks.workunit.client.0.smithi008.stdout:7/390: unlink d1/d5/da/d11/f7d 0 2024-04-12T05:55:29.047 INFO:tasks.workunit.client.0.smithi008.stdout:7/391: creat d1/d5/da/d78/f88 x:0 0 0 2024-04-12T05:55:29.048 INFO:tasks.workunit.client.0.smithi008.stdout:6/340: dwrite df/d21/d4c/f52 [0,4194304] 0 2024-04-12T05:55:29.050 INFO:tasks.workunit.client.0.smithi008.stdout:7/392: creat d1/d15/d43/d54/d77/f89 x:0 0 0 2024-04-12T05:55:29.056 INFO:tasks.workunit.client.0.smithi008.stdout:7/393: rename d1/d5/da/d3c/d3d to d1/d5/da/d3c/d44/d8a 0 2024-04-12T05:55:29.058 INFO:tasks.workunit.client.0.smithi008.stdout:6/341: getdents df/d10/d15/d4f 0 2024-04-12T05:55:29.059 INFO:tasks.workunit.client.0.smithi008.stdout:6/342: truncate df/d10/d20/f49 1497649 0 2024-04-12T05:55:29.060 INFO:tasks.workunit.client.0.smithi008.stdout:6/343: chown df/f1f 13402 1 2024-04-12T05:55:29.062 INFO:tasks.workunit.client.0.smithi008.stdout:6/344: dread df/f63 [0,4194304] 0 2024-04-12T05:55:29.062 INFO:tasks.workunit.client.0.smithi008.stdout:7/394: link d1/l27 d1/d15/d43/d54/d58/l8b 0 2024-04-12T05:55:29.063 INFO:tasks.workunit.client.0.smithi008.stdout:7/395: chown d1/d49/f53 4674520 1 2024-04-12T05:55:29.065 INFO:tasks.workunit.client.0.smithi008.stdout:7/396: write d1/d15/d43/d54/d58/f67 [1008792,72543] 0 2024-04-12T05:55:29.069 INFO:tasks.workunit.client.0.smithi008.stdout:6/345: dwrite df/f1f [0,4194304] 0 2024-04-12T05:55:29.071 INFO:tasks.workunit.client.0.smithi008.stdout:5/498: sync 2024-04-12T05:55:29.071 INFO:tasks.workunit.client.0.smithi008.stdout:1/298: sync 2024-04-12T05:55:29.072 INFO:tasks.workunit.client.0.smithi008.stdout:4/192: sync 2024-04-12T05:55:29.072 INFO:tasks.workunit.client.0.smithi008.stdout:6/346: mknod df/d10/d15/d4f/d50/c69 0 2024-04-12T05:55:29.074 INFO:tasks.workunit.client.0.smithi008.stdout:7/397: dwrite d1/d5/da/d3c/d44/d8a/f3a [0,4194304] 0 2024-04-12T05:55:29.076 INFO:tasks.workunit.client.0.smithi008.stdout:7/398: write d1/d15/f66 [741746,124175] 0 2024-04-12T05:55:29.076 INFO:tasks.workunit.client.0.smithi008.stdout:6/347: creat df/d10/d20/f6a x:0 0 0 2024-04-12T05:55:29.078 INFO:tasks.workunit.client.0.smithi008.stdout:7/399: chown d1/d5/da/d11/f25 0 1 2024-04-12T05:55:29.082 INFO:tasks.workunit.client.0.smithi008.stdout:7/400: mknod d1/d5/da/d11/d56/c8c 0 2024-04-12T05:55:29.091 INFO:tasks.workunit.client.0.smithi008.stdout:6/348: dread df/d21/f43 [0,4194304] 0 2024-04-12T05:55:29.092 INFO:tasks.workunit.client.0.smithi008.stdout:7/401: rename d1/d5/da/d1d/l24 to d1/l8d 0 2024-04-12T05:55:29.094 INFO:tasks.workunit.client.0.smithi008.stdout:6/349: write df/d21/f57 [1190145,62979] 0 2024-04-12T05:55:29.096 INFO:tasks.workunit.client.0.smithi008.stdout:7/402: creat d1/d15/d43/d54/d58/f8e x:0 0 0 2024-04-12T05:55:29.098 INFO:tasks.workunit.client.0.smithi008.stdout:6/350: creat df/d21/d4c/f6b x:0 0 0 2024-04-12T05:55:29.103 INFO:tasks.workunit.client.0.smithi008.stdout:1/299: dwrite d4/d6/d11/f22 [0,4194304] 0 2024-04-12T05:55:29.103 INFO:tasks.workunit.client.0.smithi008.stdout:7/403: mkdir d1/d5/da/d11/d56/d8f 0 2024-04-12T05:55:29.108 INFO:tasks.workunit.client.0.smithi008.stdout:7/404: truncate d1/d5/da/d11/f69 5340081 0 2024-04-12T05:55:29.111 INFO:tasks.workunit.client.0.smithi008.stdout:7/405: mknod d1/d15/d43/d54/d58/d5e/c90 0 2024-04-12T05:55:29.112 INFO:tasks.workunit.client.0.smithi008.stdout:1/300: getdents d4/dd/d17/d28 0 2024-04-12T05:55:29.113 INFO:tasks.workunit.client.0.smithi008.stdout:1/301: dread - d4/dd/d17/f64 zero size 2024-04-12T05:55:29.118 INFO:tasks.workunit.client.0.smithi008.stdout:6/351: truncate df/d21/f57 298852 0 2024-04-12T05:55:29.119 INFO:tasks.workunit.client.0.smithi008.stdout:1/302: mknod d4/dd/d17/d26/c65 0 2024-04-12T05:55:29.120 INFO:tasks.workunit.client.0.smithi008.stdout:6/352: read - df/d10/f56 zero size 2024-04-12T05:55:29.124 INFO:tasks.workunit.client.0.smithi008.stdout:1/303: unlink d4/d6/f8 0 2024-04-12T05:55:29.125 INFO:tasks.workunit.client.0.smithi008.stdout:7/406: truncate d1/d5/da/d11/f69 3200988 0 2024-04-12T05:55:29.126 INFO:tasks.workunit.client.0.smithi008.stdout:1/304: write d4/d7/f56 [830587,75186] 0 2024-04-12T05:55:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:7/407: chown d1/d5/da/d1d/d68/l76 1573 1 2024-04-12T05:55:29.128 INFO:tasks.workunit.client.0.smithi008.stdout:4/193: dread d1/db/f18 [0,4194304] 0 2024-04-12T05:55:29.130 INFO:tasks.workunit.client.0.smithi008.stdout:1/305: readlink d4/dd/d17/l1e 0 2024-04-12T05:55:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:1/306: fsync d4/d7/ff 0 2024-04-12T05:55:29.131 INFO:tasks.workunit.client.0.smithi008.stdout:6/353: dread f4 [0,4194304] 0 2024-04-12T05:55:29.132 INFO:tasks.workunit.client.0.smithi008.stdout:7/408: mkdir d1/d15/d43/d5a/d91 0 2024-04-12T05:55:29.133 INFO:tasks.workunit.client.0.smithi008.stdout:4/194: symlink d1/db/d19/d28/l35 0 2024-04-12T05:55:29.134 INFO:tasks.workunit.client.0.smithi008.stdout:4/195: stat d1/db/d19/l2a 0 2024-04-12T05:55:29.135 INFO:tasks.workunit.client.0.smithi008.stdout:4/196: fdatasync d1/d8/f25 0 2024-04-12T05:55:29.136 INFO:tasks.workunit.client.0.smithi008.stdout:6/354: fsync df/f63 0 2024-04-12T05:55:29.142 INFO:tasks.workunit.client.0.smithi008.stdout:4/197: truncate d1/d8/f2d 1995237 0 2024-04-12T05:55:29.143 INFO:tasks.workunit.client.0.smithi008.stdout:4/198: fsync d1/db/d19/f24 0 2024-04-12T05:55:29.144 INFO:tasks.workunit.client.0.smithi008.stdout:6/355: creat df/d10/d15/d4f/d50/f6c x:0 0 0 2024-04-12T05:55:29.146 INFO:tasks.workunit.client.0.smithi008.stdout:1/307: mkdir d4/dd/d17/d66 0 2024-04-12T05:55:29.147 INFO:tasks.workunit.client.0.smithi008.stdout:7/409: rmdir d1/d15/d43/d5a/d91 0 2024-04-12T05:55:29.151 INFO:tasks.workunit.client.0.smithi008.stdout:4/199: write d1/db/f16 [672702,45440] 0 2024-04-12T05:55:29.152 INFO:tasks.workunit.client.0.smithi008.stdout:6/356: symlink df/l6d 0 2024-04-12T05:55:29.154 INFO:tasks.workunit.client.0.smithi008.stdout:7/410: write d1/f2d [7747768,73174] 0 2024-04-12T05:55:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:1/308: mknod d4/d33/d37/c67 0 2024-04-12T05:55:29.159 INFO:tasks.workunit.client.0.smithi008.stdout:4/200: dwrite d1/fe [0,4194304] 0 2024-04-12T05:55:29.165 INFO:tasks.workunit.client.0.smithi008.stdout:6/357: link df/d10/d15/d4f/d50/f6c df/d10/d15/d4f/d50/f6e 0 2024-04-12T05:55:29.166 INFO:tasks.workunit.client.0.smithi008.stdout:7/411: mknod d1/d15/c92 0 2024-04-12T05:55:29.169 INFO:tasks.workunit.client.0.smithi008.stdout:7/412: fsync d1/d5/f4e 0 2024-04-12T05:55:29.171 INFO:tasks.workunit.client.0.smithi008.stdout:4/201: creat d1/f36 x:0 0 0 2024-04-12T05:55:29.171 INFO:tasks.workunit.client.0.smithi008.stdout:6/358: mknod df/d10/c6f 0 2024-04-12T05:55:29.173 INFO:tasks.workunit.client.0.smithi008.stdout:6/359: chown df/d10/d20/f49 234828809 1 2024-04-12T05:55:29.174 INFO:tasks.workunit.client.0.smithi008.stdout:7/413: dread d1/f4f [0,4194304] 0 2024-04-12T05:55:29.174 INFO:tasks.workunit.client.0.smithi008.stdout:4/202: creat d1/d8/f37 x:0 0 0 2024-04-12T05:55:29.177 INFO:tasks.workunit.client.0.smithi008.stdout:6/360: fdatasync df/d21/d4c/f53 0 2024-04-12T05:55:29.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/414: truncate d1/d15/f66 1268824 0 2024-04-12T05:55:29.177 INFO:tasks.workunit.client.0.smithi008.stdout:1/309: truncate d4/d7/f56 736038 0 2024-04-12T05:55:29.178 INFO:tasks.workunit.client.0.smithi008.stdout:6/361: write df/d21/d4c/f58 [376946,20355] 0 2024-04-12T05:55:29.180 INFO:tasks.workunit.client.0.smithi008.stdout:4/203: rename d1/db/lf to d1/db/d19/l38 0 2024-04-12T05:55:29.180 INFO:tasks.workunit.client.0.smithi008.stdout:4/204: chown d1/d22 477949835 1 2024-04-12T05:55:29.180 INFO:tasks.workunit.client.0.smithi008.stdout:7/415: rmdir d1/d5/da/d1d/d68 39 2024-04-12T05:55:29.181 INFO:tasks.workunit.client.0.smithi008.stdout:1/310: rmdir d4/d1c/d4f 39 2024-04-12T05:55:29.183 INFO:tasks.workunit.client.0.smithi008.stdout:6/362: rename df/d10/d15/l1e to df/d10/d15/d4f/d50/l70 0 2024-04-12T05:55:29.184 INFO:tasks.workunit.client.0.smithi008.stdout:1/311: mknod d4/dd/d21/c68 0 2024-04-12T05:55:29.185 INFO:tasks.workunit.client.0.smithi008.stdout:0/329: sync 2024-04-12T05:55:29.185 INFO:tasks.workunit.client.0.smithi008.stdout:2/334: sync 2024-04-12T05:55:29.186 INFO:tasks.workunit.client.0.smithi008.stdout:1/312: fdatasync d4/d6/d11/f42 0 2024-04-12T05:55:29.186 INFO:tasks.workunit.client.0.smithi008.stdout:7/416: creat d1/d5/f93 x:0 0 0 2024-04-12T05:55:29.188 INFO:tasks.workunit.client.0.smithi008.stdout:4/205: dread d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:29.189 INFO:tasks.workunit.client.0.smithi008.stdout:7/417: chown d1/d5/da/d11/l36 49928 1 2024-04-12T05:55:29.192 INFO:tasks.workunit.client.0.smithi008.stdout:2/335: fsync d6/f21 0 2024-04-12T05:55:29.194 INFO:tasks.workunit.client.0.smithi008.stdout:6/363: rename df/d10/d20/d3e/c48 to df/d10/d15/d4f/c71 0 2024-04-12T05:55:29.199 INFO:tasks.workunit.client.0.smithi008.stdout:1/313: dread d4/d7/fe [0,4194304] 0 2024-04-12T05:55:29.200 INFO:tasks.workunit.client.0.smithi008.stdout:2/336: unlink d6/db/f14 0 2024-04-12T05:55:29.202 INFO:tasks.workunit.client.0.smithi008.stdout:7/418: mkdir d1/d5/da/d3c/d44/d8a/d94 0 2024-04-12T05:55:29.203 INFO:tasks.workunit.client.0.smithi008.stdout:6/364: creat df/d10/d20/d3e/f72 x:0 0 0 2024-04-12T05:55:29.204 INFO:tasks.workunit.client.0.smithi008.stdout:1/314: dread d4/d6/d11/f22 [0,4194304] 0 2024-04-12T05:55:29.209 INFO:tasks.workunit.client.0.smithi008.stdout:4/206: link d1/db/d19/d28/c2c d1/d8/c39 0 2024-04-12T05:55:29.210 INFO:tasks.workunit.client.0.smithi008.stdout:7/419: dwrite d1/d15/f66 [0,4194304] 0 2024-04-12T05:55:29.213 INFO:tasks.workunit.client.0.smithi008.stdout:2/337: mkdir d6/db/d6f 0 2024-04-12T05:55:29.217 INFO:tasks.workunit.client.0.smithi008.stdout:2/338: chown d6/db/d10/c20 17989 1 2024-04-12T05:55:29.218 INFO:tasks.workunit.client.0.smithi008.stdout:7/420: chown d1/l28 1347057913 1 2024-04-12T05:55:29.219 INFO:tasks.workunit.client.0.smithi008.stdout:7/421: read - d1/d5/da/f62 zero size 2024-04-12T05:55:29.225 INFO:tasks.workunit.client.0.smithi008.stdout:0/330: dwrite d2/d11/d36/d42/f62 [0,4194304] 0 2024-04-12T05:55:29.229 INFO:tasks.workunit.client.0.smithi008.stdout:6/365: dwrite fd [0,4194304] 0 2024-04-12T05:55:29.230 INFO:tasks.workunit.client.0.smithi008.stdout:7/422: mknod d1/d5/da/d11/d56/d8f/c95 0 2024-04-12T05:55:29.232 INFO:tasks.workunit.client.0.smithi008.stdout:7/423: stat d1/d15/f59 0 2024-04-12T05:55:29.233 INFO:tasks.workunit.client.0.smithi008.stdout:0/331: unlink d2/d11/l14 0 2024-04-12T05:55:29.234 INFO:tasks.workunit.client.0.smithi008.stdout:2/339: creat d6/db/d10/d23/d35/d3c/f70 x:0 0 0 2024-04-12T05:55:29.236 INFO:tasks.workunit.client.0.smithi008.stdout:6/366: creat df/d21/f73 x:0 0 0 2024-04-12T05:55:29.237 INFO:tasks.workunit.client.0.smithi008.stdout:4/207: write d1/d22/d29/d30/f31 [475888,57359] 0 2024-04-12T05:55:29.240 INFO:tasks.workunit.client.0.smithi008.stdout:0/332: write d2/d11/d36/d42/f57 [841717,41797] 0 2024-04-12T05:55:29.241 INFO:tasks.workunit.client.0.smithi008.stdout:1/315: dwrite d4/dd/d17/f63 [0,4194304] 0 2024-04-12T05:55:29.243 INFO:tasks.workunit.client.0.smithi008.stdout:4/208: truncate d1/d8/f23 68639 0 2024-04-12T05:55:29.248 INFO:tasks.workunit.client.0.smithi008.stdout:7/424: dwrite d1/d15/f59 [0,4194304] 0 2024-04-12T05:55:29.248 INFO:tasks.workunit.client.0.smithi008.stdout:6/367: write df/d10/d15/d4f/d50/f51 [384025,103223] 0 2024-04-12T05:55:29.250 INFO:tasks.workunit.client.0.smithi008.stdout:0/333: fdatasync d2/d11/d2a/f3e 0 2024-04-12T05:55:29.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/243: sync 2024-04-12T05:55:29.254 INFO:tasks.workunit.client.0.smithi008.stdout:9/385: sync 2024-04-12T05:55:29.256 INFO:tasks.workunit.client.0.smithi008.stdout:6/368: read - df/d10/d15/f24 zero size 2024-04-12T05:55:29.258 INFO:tasks.workunit.client.0.smithi008.stdout:9/386: fsync d3/df/f20 0 2024-04-12T05:55:29.259 INFO:tasks.workunit.client.0.smithi008.stdout:4/209: dwrite d1/d22/d29/d30/f31 [0,4194304] 0 2024-04-12T05:55:29.261 INFO:tasks.workunit.client.0.smithi008.stdout:0/334: creat d2/d11/d36/d42/f65 x:0 0 0 2024-04-12T05:55:29.264 INFO:tasks.workunit.client.0.smithi008.stdout:7/425: write d1/d5/da/d3c/d44/d8a/f86 [1042908,68279] 0 2024-04-12T05:55:29.266 INFO:tasks.workunit.client.0.smithi008.stdout:4/210: chown d1/db/l1b 1025792 1 2024-04-12T05:55:29.266 INFO:tasks.workunit.client.0.smithi008.stdout:1/316: dread d4/dd/f39 [0,4194304] 0 2024-04-12T05:55:29.269 INFO:tasks.workunit.client.0.smithi008.stdout:6/369: symlink df/d10/d15/d4f/d50/l74 0 2024-04-12T05:55:29.270 INFO:tasks.workunit.client.0.smithi008.stdout:1/317: read - d4/dd/d17/f64 zero size 2024-04-12T05:55:29.270 INFO:tasks.workunit.client.0.smithi008.stdout:7/426: creat d1/d15/d43/d54/d77/f96 x:0 0 0 2024-04-12T05:55:29.271 INFO:tasks.workunit.client.0.smithi008.stdout:4/211: creat d1/d22/d29/f3a x:0 0 0 2024-04-12T05:55:29.271 INFO:tasks.workunit.client.0.smithi008.stdout:9/387: mknod d3/d22/d2b/d58/d38/c85 0 2024-04-12T05:55:29.272 INFO:tasks.workunit.client.0.smithi008.stdout:6/370: fsync fd 0 2024-04-12T05:55:29.273 INFO:tasks.workunit.client.0.smithi008.stdout:0/335: fdatasync d2/fe 0 2024-04-12T05:55:29.276 INFO:tasks.workunit.client.0.smithi008.stdout:4/212: unlink d1/f13 0 2024-04-12T05:55:29.277 INFO:tasks.workunit.client.0.smithi008.stdout:8/244: dwrite dd/de/f47 [0,4194304] 0 2024-04-12T05:55:29.278 INFO:tasks.workunit.client.0.smithi008.stdout:0/336: chown d2/d11/f40 25202 1 2024-04-12T05:55:29.282 INFO:tasks.workunit.client.0.smithi008.stdout:9/388: chown d3/d22/d2b/d4a/d62/c7e 144 1 2024-04-12T05:55:29.285 INFO:tasks.workunit.client.0.smithi008.stdout:8/245: mkdir dd/d4f 0 2024-04-12T05:55:29.286 INFO:tasks.workunit.client.0.smithi008.stdout:9/389: chown d3/df/d13/d21/d2d/d32/c80 47269 1 2024-04-12T05:55:29.287 INFO:tasks.workunit.client.0.smithi008.stdout:1/318: link d4/d6/d11/f22 d4/d6/d11/f69 0 2024-04-12T05:55:29.288 INFO:tasks.workunit.client.0.smithi008.stdout:4/213: dwrite d1/db/f1a [0,4194304] 0 2024-04-12T05:55:29.291 INFO:tasks.workunit.client.0.smithi008.stdout:7/427: creat d1/d5/da/f97 x:0 0 0 2024-04-12T05:55:29.294 INFO:tasks.workunit.client.0.smithi008.stdout:4/214: chown d1/d8/f21 213 1 2024-04-12T05:55:29.294 INFO:tasks.workunit.client.0.smithi008.stdout:6/371: dwrite f5 [0,4194304] 0 2024-04-12T05:55:29.294 INFO:tasks.workunit.client.0.smithi008.stdout:8/246: mkdir dd/de/d16/d50 0 2024-04-12T05:55:29.295 INFO:tasks.workunit.client.0.smithi008.stdout:0/337: creat d2/f66 x:0 0 0 2024-04-12T05:55:29.298 INFO:tasks.workunit.client.0.smithi008.stdout:9/390: fdatasync d3/f25 0 2024-04-12T05:55:29.301 INFO:tasks.workunit.client.0.smithi008.stdout:0/338: rename d2/d11/d2a/f2d to d2/d11/d36/f67 0 2024-04-12T05:55:29.303 INFO:tasks.workunit.client.0.smithi008.stdout:3/339: sync 2024-04-12T05:55:29.304 INFO:tasks.workunit.client.0.smithi008.stdout:5/499: sync 2024-04-12T05:55:29.304 INFO:tasks.workunit.client.0.smithi008.stdout:7/428: dread d1/f3 [4194304,4194304] 0 2024-04-12T05:55:29.310 INFO:tasks.workunit.client.0.smithi008.stdout:1/319: creat d4/d1c/d4f/f6a x:0 0 0 2024-04-12T05:55:29.311 INFO:tasks.workunit.client.0.smithi008.stdout:8/247: dread dd/d30/f35 [0,4194304] 0 2024-04-12T05:55:29.314 INFO:tasks.workunit.client.0.smithi008.stdout:4/215: dread d1/db/f18 [0,4194304] 0 2024-04-12T05:55:29.314 INFO:tasks.workunit.client.0.smithi008.stdout:5/500: truncate d0/f97 658830 0 2024-04-12T05:55:29.322 INFO:tasks.workunit.client.0.smithi008.stdout:0/339: creat d2/d11/d36/d47/f68 x:0 0 0 2024-04-12T05:55:29.323 INFO:tasks.workunit.client.0.smithi008.stdout:8/248: dwrite fb [0,4194304] 0 2024-04-12T05:55:29.324 INFO:tasks.workunit.client.0.smithi008.stdout:3/340: creat d9/d44/f6e x:0 0 0 2024-04-12T05:55:29.326 INFO:tasks.workunit.client.0.smithi008.stdout:7/429: dread d1/d5/f2e [0,4194304] 0 2024-04-12T05:55:29.329 INFO:tasks.workunit.client.0.smithi008.stdout:5/501: rename d0/d2/f31 to d0/d29/f99 0 2024-04-12T05:55:29.330 INFO:tasks.workunit.client.0.smithi008.stdout:0/340: readlink d2/d24/l56 0 2024-04-12T05:55:29.331 INFO:tasks.workunit.client.0.smithi008.stdout:0/341: chown d2 76922013 1 2024-04-12T05:55:29.331 INFO:tasks.workunit.client.0.smithi008.stdout:5/502: dread - d0/dc/d13/d1a/f83 zero size 2024-04-12T05:55:29.334 INFO:tasks.workunit.client.0.smithi008.stdout:6/372: getdents df/d21 0 2024-04-12T05:55:29.343 INFO:tasks.workunit.client.0.smithi008.stdout:3/341: unlink d9/d10/d19/d3a/l5c 0 2024-04-12T05:55:29.343 INFO:tasks.workunit.client.0.smithi008.stdout:6/373: write df/f23 [1846263,104804] 0 2024-04-12T05:55:29.343 INFO:tasks.workunit.client.0.smithi008.stdout:9/391: link d3/d22/d2b/d58/f1d d3/df/f86 0 2024-04-12T05:55:29.347 INFO:tasks.workunit.client.0.smithi008.stdout:5/503: symlink d0/dc/d13/d1a/d3a/l9a 0 2024-04-12T05:55:29.349 INFO:tasks.workunit.client.0.smithi008.stdout:6/374: rmdir df 39 2024-04-12T05:55:29.352 INFO:tasks.workunit.client.0.smithi008.stdout:7/430: dread d1/d5/da/d3c/d44/d8a/f3a [0,4194304] 0 2024-04-12T05:55:29.354 INFO:tasks.workunit.client.0.smithi008.stdout:8/249: link dd/d30/d32/c46 dd/de/c51 0 2024-04-12T05:55:29.356 INFO:tasks.workunit.client.0.smithi008.stdout:5/504: write d0/d29/f99 [2313291,58826] 0 2024-04-12T05:55:29.357 INFO:tasks.workunit.client.0.smithi008.stdout:0/342: link d2/d11/d2a/f49 d2/d11/f69 0 2024-04-12T05:55:29.358 INFO:tasks.workunit.client.0.smithi008.stdout:9/392: creat d3/d22/d2b/d77/d82/f87 x:0 0 0 2024-04-12T05:55:29.360 INFO:tasks.workunit.client.0.smithi008.stdout:8/250: write dd/de/f2a [864151,27023] 0 2024-04-12T05:55:29.360 INFO:tasks.workunit.client.0.smithi008.stdout:6/375: mkdir df/d10/d15/d75 0 2024-04-12T05:55:29.360 INFO:tasks.workunit.client.0.smithi008.stdout:3/342: dwrite f6 [0,4194304] 0 2024-04-12T05:55:29.361 INFO:tasks.workunit.client.0.smithi008.stdout:1/320: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:29.363 INFO:tasks.workunit.client.0.smithi008.stdout:6/376: readlink df/d21/l45 0 2024-04-12T05:55:29.363 INFO:tasks.workunit.client.0.smithi008.stdout:7/431: truncate d1/d15/d43/d54/d77/f89 319790 0 2024-04-12T05:55:29.365 INFO:tasks.workunit.client.0.smithi008.stdout:7/432: readlink d1/d15/l6a 0 2024-04-12T05:55:29.366 INFO:tasks.workunit.client.0.smithi008.stdout:1/321: chown d4/dd/d17/d28/d46/d4c/c2a 3555387 1 2024-04-12T05:55:29.366 INFO:tasks.workunit.client.0.smithi008.stdout:9/393: symlink d3/d22/d2b/d4e/l88 0 2024-04-12T05:55:29.367 INFO:tasks.workunit.client.0.smithi008.stdout:3/343: symlink d9/d11/l6f 0 2024-04-12T05:55:29.368 INFO:tasks.workunit.client.0.smithi008.stdout:5/505: creat d0/d29/d96/f9b x:0 0 0 2024-04-12T05:55:29.368 INFO:tasks.workunit.client.0.smithi008.stdout:1/322: chown d4/dd/d17/d28/d30/f5d 2 1 2024-04-12T05:55:29.371 INFO:tasks.workunit.client.0.smithi008.stdout:0/343: creat d2/f6a x:0 0 0 2024-04-12T05:55:29.373 INFO:tasks.workunit.client.0.smithi008.stdout:7/433: chown d1/d5/da/d11/d56/l7c 21 1 2024-04-12T05:55:29.374 INFO:tasks.workunit.client.0.smithi008.stdout:9/394: unlink d3/df/d10/l64 0 2024-04-12T05:55:29.375 INFO:tasks.workunit.client.0.smithi008.stdout:3/344: creat d9/d2e/d3d/f70 x:0 0 0 2024-04-12T05:55:29.378 INFO:tasks.workunit.client.0.smithi008.stdout:5/506: write d0/dc/f2a [2925889,96407] 0 2024-04-12T05:55:29.380 INFO:tasks.workunit.client.0.smithi008.stdout:0/344: write d2/d24/f28 [3469375,18814] 0 2024-04-12T05:55:29.383 INFO:tasks.workunit.client.0.smithi008.stdout:7/434: creat d1/d15/d43/d5a/f98 x:0 0 0 2024-04-12T05:55:29.383 INFO:tasks.workunit.client.0.smithi008.stdout:5/507: chown d0/dd/c30 333 1 2024-04-12T05:55:29.386 INFO:tasks.workunit.client.0.smithi008.stdout:9/395: dread d3/d22/d2b/d58/d38/f54 [0,4194304] 0 2024-04-12T05:55:29.386 INFO:tasks.workunit.client.0.smithi008.stdout:5/508: rmdir d0/d29 39 2024-04-12T05:55:29.387 INFO:tasks.workunit.client.0.smithi008.stdout:3/345: dwrite d9/dd/f2f [0,4194304] 0 2024-04-12T05:55:29.388 INFO:tasks.workunit.client.0.smithi008.stdout:7/435: write d1/f55 [3127485,37608] 0 2024-04-12T05:55:29.391 INFO:tasks.workunit.client.0.smithi008.stdout:0/345: mknod d2/d24/c6b 0 2024-04-12T05:55:29.394 INFO:tasks.workunit.client.0.smithi008.stdout:0/346: write d2/d11/d36/d47/f21 [8090950,104487] 0 2024-04-12T05:55:29.397 INFO:tasks.workunit.client.0.smithi008.stdout:7/436: mknod d1/d5/da/d60/c99 0 2024-04-12T05:55:29.398 INFO:tasks.workunit.client.0.smithi008.stdout:0/347: chown d2/d11/d36/d47/f59 25095749 1 2024-04-12T05:55:29.399 INFO:tasks.workunit.client.0.smithi008.stdout:8/251: truncate dd/f39 223297 0 2024-04-12T05:55:29.399 INFO:tasks.workunit.client.0.smithi008.stdout:3/346: dread d9/dd/f2f [0,4194304] 0 2024-04-12T05:55:29.401 INFO:tasks.workunit.client.0.smithi008.stdout:6/377: truncate df/d21/f43 6780826 0 2024-04-12T05:55:29.401 INFO:tasks.workunit.client.0.smithi008.stdout:0/348: chown d2/f66 1172639 1 2024-04-12T05:55:29.401 INFO:tasks.workunit.client.0.smithi008.stdout:9/396: mkdir d3/d22/d2b/d58/d89 0 2024-04-12T05:55:29.406 INFO:tasks.workunit.client.0.smithi008.stdout:9/397: chown d3/d22/d2b/c59 25 1 2024-04-12T05:55:29.407 INFO:tasks.workunit.client.0.smithi008.stdout:3/347: symlink d9/d37/l71 0 2024-04-12T05:55:29.407 INFO:tasks.workunit.client.0.smithi008.stdout:7/437: symlink d1/d5/da/d1d/d68/l9a 0 2024-04-12T05:55:29.409 INFO:tasks.workunit.client.0.smithi008.stdout:5/509: dwrite d0/dc/d13/d1a/d35/f42 [4194304,4194304] 0 2024-04-12T05:55:29.410 INFO:tasks.workunit.client.0.smithi008.stdout:6/378: mknod df/d10/d20/c76 0 2024-04-12T05:55:29.415 INFO:tasks.workunit.client.0.smithi008.stdout:8/252: dread dd/de/f12 [0,4194304] 0 2024-04-12T05:55:29.418 INFO:tasks.workunit.client.0.smithi008.stdout:5/510: creat d0/d29/f9c x:0 0 0 2024-04-12T05:55:29.418 INFO:tasks.workunit.client.0.smithi008.stdout:9/398: chown d3/d22/d2b/d77/f84 7 1 2024-04-12T05:55:29.419 INFO:tasks.workunit.client.0.smithi008.stdout:7/438: chown d1/d5/da/d11/d56/l7c 806475 1 2024-04-12T05:55:29.419 INFO:tasks.workunit.client.0.smithi008.stdout:3/348: chown d9/d10/d19/d22/d2c/f5e 105828686 1 2024-04-12T05:55:29.420 INFO:tasks.workunit.client.0.smithi008.stdout:1/323: dwrite d4/d7/f56 [0,4194304] 0 2024-04-12T05:55:29.423 INFO:tasks.workunit.client.0.smithi008.stdout:9/399: dread - d3/d22/d2b/f72 zero size 2024-04-12T05:55:29.423 INFO:tasks.workunit.client.0.smithi008.stdout:1/324: truncate f3 5238964 0 2024-04-12T05:55:29.429 INFO:tasks.workunit.client.0.smithi008.stdout:1/325: unlink f3 0 2024-04-12T05:55:29.430 INFO:tasks.workunit.client.0.smithi008.stdout:7/439: truncate d1/d5/f2e 4813452 0 2024-04-12T05:55:29.430 INFO:tasks.workunit.client.0.smithi008.stdout:0/349: dread d2/ff [4194304,4194304] 0 2024-04-12T05:55:29.431 INFO:tasks.workunit.client.0.smithi008.stdout:5/511: fdatasync d0/d2/f4 0 2024-04-12T05:55:29.431 INFO:tasks.workunit.client.0.smithi008.stdout:1/326: write d4/f47 [63127,49010] 0 2024-04-12T05:55:29.433 INFO:tasks.workunit.client.0.smithi008.stdout:5/512: readlink d0/d2/l91 0 2024-04-12T05:55:29.433 INFO:tasks.workunit.client.0.smithi008.stdout:9/400: creat d3/d22/d2b/d77/d82/f8a x:0 0 0 2024-04-12T05:55:29.434 INFO:tasks.workunit.client.0.smithi008.stdout:8/253: mkdir dd/d4f/d52 0 2024-04-12T05:55:29.436 INFO:tasks.workunit.client.0.smithi008.stdout:3/349: symlink d9/d2e/l72 0 2024-04-12T05:55:29.436 INFO:tasks.workunit.client.0.smithi008.stdout:9/401: chown d3/d22/d2b/d77 1045218 1 2024-04-12T05:55:29.438 INFO:tasks.workunit.client.0.smithi008.stdout:5/513: write d0/dd/d11/d38/f43 [88049,53317] 0 2024-04-12T05:55:29.441 INFO:tasks.workunit.client.0.smithi008.stdout:8/254: symlink dd/d30/d32/l53 0 2024-04-12T05:55:29.443 INFO:tasks.workunit.client.0.smithi008.stdout:3/350: creat d9/d2e/d3d/f73 x:0 0 0 2024-04-12T05:55:29.444 INFO:tasks.workunit.client.0.smithi008.stdout:5/514: fsync d0/d2/f2c 0 2024-04-12T05:55:29.445 INFO:tasks.workunit.client.0.smithi008.stdout:7/440: dwrite d1/d15/f59 [0,4194304] 0 2024-04-12T05:55:29.448 INFO:tasks.workunit.client.0.smithi008.stdout:8/255: creat dd/f54 x:0 0 0 2024-04-12T05:55:29.452 INFO:tasks.workunit.client.0.smithi008.stdout:7/441: stat d1/d15/d43/d54/d58/d5e/c90 0 2024-04-12T05:55:29.452 INFO:tasks.workunit.client.0.smithi008.stdout:8/256: chown dd/de/d16/c3d 268444 1 2024-04-12T05:55:29.455 INFO:tasks.workunit.client.0.smithi008.stdout:3/351: readlink d9/d10/d19/l3b 0 2024-04-12T05:55:29.455 INFO:tasks.workunit.client.0.smithi008.stdout:6/379: dwrite df/d10/d15/d4f/d50/f51 [0,4194304] 0 2024-04-12T05:55:29.457 INFO:tasks.workunit.client.0.smithi008.stdout:5/515: rename d0/d29/d44 to d0/dc/d57/d9d 0 2024-04-12T05:55:29.459 INFO:tasks.workunit.client.0.smithi008.stdout:8/257: creat dd/d30/d3b/f55 x:0 0 0 2024-04-12T05:55:29.462 INFO:tasks.workunit.client.0.smithi008.stdout:7/442: creat d1/d5/da/d3c/d44/d8a/f9b x:0 0 0 2024-04-12T05:55:29.463 INFO:tasks.workunit.client.0.smithi008.stdout:6/380: stat fd 0 2024-04-12T05:55:29.468 INFO:tasks.workunit.client.0.smithi008.stdout:8/258: fdatasync dd/d30/d32/f3e 0 2024-04-12T05:55:29.470 INFO:tasks.workunit.client.0.smithi008.stdout:9/402: dread d3/d22/d2b/d58/d38/f4c [0,4194304] 0 2024-04-12T05:55:29.470 INFO:tasks.workunit.client.0.smithi008.stdout:5/516: dwrite d0/dc/d13/f72 [0,4194304] 0 2024-04-12T05:55:29.473 INFO:tasks.workunit.client.0.smithi008.stdout:6/381: creat df/d21/f77 x:0 0 0 2024-04-12T05:55:29.474 INFO:tasks.workunit.client.0.smithi008.stdout:3/352: link d9/d44/f63 d9/d2e/f74 0 2024-04-12T05:55:29.476 INFO:tasks.workunit.client.0.smithi008.stdout:8/259: mkdir dd/de/d16/d50/d56 0 2024-04-12T05:55:29.478 INFO:tasks.workunit.client.0.smithi008.stdout:7/443: dwrite d1/d5/da/d3c/d44/d8a/f3b [0,4194304] 0 2024-04-12T05:55:29.479 INFO:tasks.workunit.client.0.smithi008.stdout:5/517: creat d0/dc/d13/d1a/f9e x:0 0 0 2024-04-12T05:55:29.484 INFO:tasks.workunit.client.0.smithi008.stdout:5/518: mkdir d0/dc/d13/d9f 0 2024-04-12T05:55:29.488 INFO:tasks.workunit.client.0.smithi008.stdout:7/444: creat d1/f9c x:0 0 0 2024-04-12T05:55:29.495 INFO:tasks.workunit.client.0.smithi008.stdout:6/382: write f3 [9558209,43914] 0 2024-04-12T05:55:29.499 INFO:tasks.workunit.client.0.smithi008.stdout:8/260: dwrite dd/de/f2d [0,4194304] 0 2024-04-12T05:55:29.499 INFO:tasks.workunit.client.0.smithi008.stdout:9/403: dwrite d3/df/f20 [0,4194304] 0 2024-04-12T05:55:29.500 INFO:tasks.workunit.client.0.smithi008.stdout:3/353: dwrite d9/d2e/d3d/f73 [0,4194304] 0 2024-04-12T05:55:29.500 INFO:tasks.workunit.client.0.smithi008.stdout:7/445: write d1/d5/f32 [1227505,56115] 0 2024-04-12T05:55:29.508 INFO:tasks.workunit.client.0.smithi008.stdout:3/354: rename d9/d11/c14 to d9/d10/d19/d22/d2c/c75 0 2024-04-12T05:55:29.510 INFO:tasks.workunit.client.0.smithi008.stdout:8/261: mknod dd/de/c57 0 2024-04-12T05:55:29.512 INFO:tasks.workunit.client.0.smithi008.stdout:5/519: truncate d0/d2/f84 229951 0 2024-04-12T05:55:29.513 INFO:tasks.workunit.client.0.smithi008.stdout:9/404: rename d3/df/d13/d21/d2d/d32/l71 to d3/d22/d2b/d58/d38/l8b 0 2024-04-12T05:55:29.516 INFO:tasks.workunit.client.0.smithi008.stdout:3/355: creat d9/d11/d5b/f76 x:0 0 0 2024-04-12T05:55:29.516 INFO:tasks.workunit.client.0.smithi008.stdout:6/383: dwrite f9 [0,4194304] 0 2024-04-12T05:55:29.520 INFO:tasks.workunit.client.0.smithi008.stdout:8/262: symlink dd/de/d4c/l58 0 2024-04-12T05:55:29.523 INFO:tasks.workunit.client.0.smithi008.stdout:6/384: mkdir df/d10/d3a/d78 0 2024-04-12T05:55:29.526 INFO:tasks.workunit.client.0.smithi008.stdout:7/446: dwrite d1/d5/da/d11/f65 [0,4194304] 0 2024-04-12T05:55:29.530 INFO:tasks.workunit.client.0.smithi008.stdout:6/385: chown df/l2d 338908 1 2024-04-12T05:55:29.530 INFO:tasks.workunit.client.0.smithi008.stdout:9/405: creat d3/df/d13/d21/d2d/d75/d7c/f8c x:0 0 0 2024-04-12T05:55:29.536 INFO:tasks.workunit.client.0.smithi008.stdout:3/356: dwrite f7 [0,4194304] 0 2024-04-12T05:55:29.539 INFO:tasks.workunit.client.0.smithi008.stdout:5/520: rename d0/dc/d57/l36 to d0/la0 0 2024-04-12T05:55:29.541 INFO:tasks.workunit.client.0.smithi008.stdout:6/386: mkdir df/d10/d15/d79 0 2024-04-12T05:55:29.545 INFO:tasks.workunit.client.0.smithi008.stdout:8/263: dwrite dd/de/d16/f2e [0,4194304] 0 2024-04-12T05:55:29.549 INFO:tasks.workunit.client.0.smithi008.stdout:7/447: creat d1/d5/f9d x:0 0 0 2024-04-12T05:55:29.551 INFO:tasks.workunit.client.0.smithi008.stdout:8/264: symlink dd/d4f/l59 0 2024-04-12T05:55:29.556 INFO:tasks.workunit.client.0.smithi008.stdout:7/448: mknod d1/c9e 0 2024-04-12T05:55:29.557 INFO:tasks.workunit.client.0.smithi008.stdout:6/387: link df/d10/d15/d4f/d50/c69 df/d10/d15/d4f/c7a 0 2024-04-12T05:55:29.564 INFO:tasks.workunit.client.0.smithi008.stdout:6/388: chown df/d10/d15/l25 3 1 2024-04-12T05:55:29.568 INFO:tasks.workunit.client.0.smithi008.stdout:9/406: dwrite d3/f6 [4194304,4194304] 0 2024-04-12T05:55:29.570 INFO:tasks.workunit.client.0.smithi008.stdout:3/357: dwrite d9/d2e/f3f [0,4194304] 0 2024-04-12T05:55:29.570 INFO:tasks.workunit.client.0.smithi008.stdout:8/265: dwrite dd/f48 [0,4194304] 0 2024-04-12T05:55:29.574 INFO:tasks.workunit.client.0.smithi008.stdout:6/389: truncate df/d10/d15/f3f 762764 0 2024-04-12T05:55:29.581 INFO:tasks.workunit.client.0.smithi008.stdout:3/358: write d9/d10/d19/d22/f4d [1181094,65743] 0 2024-04-12T05:55:29.581 INFO:tasks.workunit.client.0.smithi008.stdout:8/266: rmdir dd/de/d16 39 2024-04-12T05:55:29.583 INFO:tasks.workunit.client.0.smithi008.stdout:9/407: stat d3/df/d10/c50 0 2024-04-12T05:55:29.586 INFO:tasks.workunit.client.0.smithi008.stdout:5/521: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:29.588 INFO:tasks.workunit.client.0.smithi008.stdout:9/408: rmdir d3/df 39 2024-04-12T05:55:29.589 INFO:tasks.workunit.client.0.smithi008.stdout:3/359: dwrite d9/dd/f36 [0,4194304] 0 2024-04-12T05:55:29.591 INFO:tasks.workunit.client.0.smithi008.stdout:3/360: fsync d9/d1e/f64 0 2024-04-12T05:55:29.593 INFO:tasks.workunit.client.0.smithi008.stdout:7/449: truncate d1/d5/da/d3c/d44/d8a/f3b 57314 0 2024-04-12T05:55:29.597 INFO:tasks.workunit.client.0.smithi008.stdout:5/522: creat d0/dd/fa1 x:0 0 0 2024-04-12T05:55:29.597 INFO:tasks.workunit.client.0.smithi008.stdout:3/361: write d9/d2e/d3d/f4c [4042450,115614] 0 2024-04-12T05:55:29.601 INFO:tasks.workunit.client.0.smithi008.stdout:6/390: link df/d10/l17 df/d10/d15/d4f/d50/l7b 0 2024-04-12T05:55:29.604 INFO:tasks.workunit.client.0.smithi008.stdout:9/409: dread d3/f1f [0,4194304] 0 2024-04-12T05:55:29.605 INFO:tasks.workunit.client.0.smithi008.stdout:8/267: dwrite dd/f39 [0,4194304] 0 2024-04-12T05:55:29.606 INFO:tasks.workunit.client.0.smithi008.stdout:7/450: mkdir d1/d5/da/d1d/d82/d9f 0 2024-04-12T05:55:29.609 INFO:tasks.workunit.client.0.smithi008.stdout:8/268: write fb [917586,35567] 0 2024-04-12T05:55:29.611 INFO:tasks.workunit.client.0.smithi008.stdout:7/451: fsync d1/fb 0 2024-04-12T05:55:29.613 INFO:tasks.workunit.client.0.smithi008.stdout:9/410: unlink d3/d22/d2b/c59 0 2024-04-12T05:55:29.614 INFO:tasks.workunit.client.0.smithi008.stdout:8/269: symlink dd/d4f/l5a 0 2024-04-12T05:55:29.615 INFO:tasks.workunit.client.0.smithi008.stdout:5/523: rmdir d0/dc/d13/d9f 0 2024-04-12T05:55:29.616 INFO:tasks.workunit.client.0.smithi008.stdout:6/391: dwrite df/d21/f43 [0,4194304] 0 2024-04-12T05:55:29.625 INFO:tasks.workunit.client.0.smithi008.stdout:6/392: rmdir df/d10/d20/d3e 39 2024-04-12T05:55:29.625 INFO:tasks.workunit.client.0.smithi008.stdout:3/362: dread d9/f25 [0,4194304] 0 2024-04-12T05:55:29.629 INFO:tasks.workunit.client.0.smithi008.stdout:3/363: chown d9/d10/d19/d3a/d67 0 1 2024-04-12T05:55:29.629 INFO:tasks.workunit.client.0.smithi008.stdout:9/411: dwrite d3/f35 [0,4194304] 0 2024-04-12T05:55:29.629 INFO:tasks.workunit.client.0.smithi008.stdout:7/452: rename d1/d5/da/d3c/d44/d8a/f86 to d1/d5/da/d3c/d44/fa0 0 2024-04-12T05:55:29.629 INFO:tasks.workunit.client.0.smithi008.stdout:3/364: chown d9 380 1 2024-04-12T05:55:29.631 INFO:tasks.workunit.client.0.smithi008.stdout:7/453: read d1/d15/f66 [783575,114441] 0 2024-04-12T05:55:29.633 INFO:tasks.workunit.client.0.smithi008.stdout:8/270: dread dd/f1e [0,4194304] 0 2024-04-12T05:55:29.637 INFO:tasks.workunit.client.0.smithi008.stdout:6/393: rename df/d21/f73 to df/d10/d15/f7c 0 2024-04-12T05:55:29.637 INFO:tasks.workunit.client.0.smithi008.stdout:5/524: unlink d0/dc/d13/d1a/d35/l86 0 2024-04-12T05:55:29.649 INFO:tasks.workunit.client.0.smithi008.stdout:8/271: chown dd/de/d16/l21 12316335 1 2024-04-12T05:55:29.650 INFO:tasks.workunit.client.0.smithi008.stdout:6/394: dwrite df/f31 [0,4194304] 0 2024-04-12T05:55:29.652 INFO:tasks.workunit.client.0.smithi008.stdout:6/395: write df/d10/d15/d4f/d50/f5b [575674,86640] 0 2024-04-12T05:55:29.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:29 smithi008 ceph-mon[39722]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:29.655 INFO:tasks.workunit.client.0.smithi008.stdout:8/272: symlink dd/d30/l5b 0 2024-04-12T05:55:29.657 INFO:tasks.workunit.client.0.smithi008.stdout:7/454: dread d1/d5/f4e [0,4194304] 0 2024-04-12T05:55:29.662 INFO:tasks.workunit.client.0.smithi008.stdout:9/412: getdents d3 0 2024-04-12T05:55:29.664 INFO:tasks.workunit.client.0.smithi008.stdout:6/396: stat fe 0 2024-04-12T05:55:29.665 INFO:tasks.workunit.client.0.smithi008.stdout:9/413: chown d3/d22/d2b/d58/d5d 13978 1 2024-04-12T05:55:29.666 INFO:tasks.workunit.client.0.smithi008.stdout:8/273: symlink dd/d4f/l5c 0 2024-04-12T05:55:29.667 INFO:tasks.workunit.client.0.smithi008.stdout:7/455: mkdir d1/d5/da/d60/da1 0 2024-04-12T05:55:29.667 INFO:tasks.workunit.client.0.smithi008.stdout:5/525: link d0/l6f d0/dd/la2 0 2024-04-12T05:55:29.669 INFO:tasks.workunit.client.0.smithi008.stdout:6/397: mknod df/d21/c7d 0 2024-04-12T05:55:29.671 INFO:tasks.workunit.client.0.smithi008.stdout:3/365: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:29.675 INFO:tasks.workunit.client.0.smithi008.stdout:8/274: write f7 [1465585,130414] 0 2024-04-12T05:55:29.676 INFO:tasks.workunit.client.0.smithi008.stdout:9/414: creat d3/d22/d2b/d58/d89/f8d x:0 0 0 2024-04-12T05:55:29.677 INFO:tasks.workunit.client.0.smithi008.stdout:6/398: rename df/d10/d15/d4f to df/d10/d15/d4f/d50/d7e 22 2024-04-12T05:55:29.678 INFO:tasks.workunit.client.0.smithi008.stdout:5/526: link d0/dd/d3d/d5d/l90 d0/d40/la3 0 2024-04-12T05:55:29.682 INFO:tasks.workunit.client.0.smithi008.stdout:8/275: fsync dd/de/f12 0 2024-04-12T05:55:29.684 INFO:tasks.workunit.client.0.smithi008.stdout:6/399: read f5 [1781766,120243] 0 2024-04-12T05:55:29.684 INFO:tasks.workunit.client.0.smithi008.stdout:7/456: dwrite d1/f12 [0,4194304] 0 2024-04-12T05:55:29.691 INFO:tasks.workunit.client.0.smithi008.stdout:6/400: dread - df/d21/f5a zero size 2024-04-12T05:55:29.693 INFO:tasks.workunit.client.0.smithi008.stdout:3/366: dwrite d9/d11/f48 [0,4194304] 0 2024-04-12T05:55:29.702 INFO:tasks.workunit.client.0.smithi008.stdout:9/415: dwrite d3/f4b [0,4194304] 0 2024-04-12T05:55:29.703 INFO:tasks.workunit.client.0.smithi008.stdout:5/527: dread d0/dc/d13/f54 [4194304,4194304] 0 2024-04-12T05:55:29.703 INFO:tasks.workunit.client.0.smithi008.stdout:3/367: getdents d9/d37/d69 0 2024-04-12T05:55:29.709 INFO:tasks.workunit.client.0.smithi008.stdout:5/528: creat d0/dc/d57/d9d/fa4 x:0 0 0 2024-04-12T05:55:29.710 INFO:tasks.workunit.client.0.smithi008.stdout:7/457: dwrite d1/d5/f29 [0,4194304] 0 2024-04-12T05:55:29.712 INFO:tasks.workunit.client.0.smithi008.stdout:3/368: symlink d9/d10/d19/d22/l77 0 2024-04-12T05:55:29.713 INFO:tasks.workunit.client.0.smithi008.stdout:3/369: chown d9/d11/d5b 22 1 2024-04-12T05:55:29.716 INFO:tasks.workunit.client.0.smithi008.stdout:3/370: write f1 [2917903,130148] 0 2024-04-12T05:55:29.717 INFO:tasks.workunit.client.0.smithi008.stdout:5/529: creat d0/dd/d11/d38/fa5 x:0 0 0 2024-04-12T05:55:29.717 INFO:tasks.workunit.client.0.smithi008.stdout:9/416: creat d3/df/d13/d21/d2d/f8e x:0 0 0 2024-04-12T05:55:29.717 INFO:tasks.workunit.client.0.smithi008.stdout:7/458: rename d1/d5/f93 to d1/d5/da/d3c/d44/fa2 0 2024-04-12T05:55:29.719 INFO:tasks.workunit.client.0.smithi008.stdout:3/371: chown d9/d10/d19/d22 914518979 1 2024-04-12T05:55:29.719 INFO:tasks.workunit.client.0.smithi008.stdout:8/276: dread dd/ff [0,4194304] 0 2024-04-12T05:55:29.720 INFO:tasks.workunit.client.0.smithi008.stdout:9/417: dread - d3/d22/d2b/d77/d82/f8a zero size 2024-04-12T05:55:29.723 INFO:tasks.workunit.client.0.smithi008.stdout:9/418: read d3/f35 [3600600,5788] 0 2024-04-12T05:55:29.725 INFO:tasks.workunit.client.0.smithi008.stdout:3/372: dread - d9/d10/d19/d22/f5f zero size 2024-04-12T05:55:29.727 INFO:tasks.workunit.client.0.smithi008.stdout:8/277: creat dd/de/d16/d50/d56/f5d x:0 0 0 2024-04-12T05:55:29.729 INFO:tasks.workunit.client.0.smithi008.stdout:7/459: getdents d1/d15/d43/d54/d58/d5e 0 2024-04-12T05:55:29.729 INFO:tasks.workunit.client.0.smithi008.stdout:9/419: dread - d3/df/d13/d21/d2d/f8e zero size 2024-04-12T05:55:29.732 INFO:tasks.workunit.client.0.smithi008.stdout:8/278: mkdir dd/de/d16/d50/d5e 0 2024-04-12T05:55:29.733 INFO:tasks.workunit.client.0.smithi008.stdout:5/530: dwrite d0/dd/d3d/f4b [0,4194304] 0 2024-04-12T05:55:29.736 INFO:tasks.workunit.client.0.smithi008.stdout:3/373: read d9/d10/d19/d22/f32 [3469111,73045] 0 2024-04-12T05:55:29.739 INFO:tasks.workunit.client.0.smithi008.stdout:5/531: dread d0/dd/d11/d38/f43 [0,4194304] 0 2024-04-12T05:55:29.741 INFO:tasks.workunit.client.0.smithi008.stdout:8/279: write dd/f1f [2633127,54613] 0 2024-04-12T05:55:29.743 INFO:tasks.workunit.client.0.smithi008.stdout:7/460: creat d1/d5/da/d3c/d44/d8a/d94/fa3 x:0 0 0 2024-04-12T05:55:29.745 INFO:tasks.workunit.client.0.smithi008.stdout:7/461: write d1/f12 [4770227,35656] 0 2024-04-12T05:55:29.747 INFO:tasks.workunit.client.0.smithi008.stdout:5/532: creat d0/dc/d13/d1a/d35/fa6 x:0 0 0 2024-04-12T05:55:29.747 INFO:tasks.workunit.client.0.smithi008.stdout:8/280: creat dd/d4f/f5f x:0 0 0 2024-04-12T05:55:29.749 INFO:tasks.workunit.client.0.smithi008.stdout:9/420: dread d3/f43 [0,4194304] 0 2024-04-12T05:55:29.750 INFO:tasks.workunit.client.0.smithi008.stdout:3/374: getdents d9/d10/d19/d22/d2c 0 2024-04-12T05:55:29.750 INFO:tasks.workunit.client.0.smithi008.stdout:5/533: write d0/d2/f2c [5021106,70113] 0 2024-04-12T05:55:29.753 INFO:tasks.workunit.client.0.smithi008.stdout:8/281: creat dd/d4f/d52/f60 x:0 0 0 2024-04-12T05:55:29.755 INFO:tasks.workunit.client.0.smithi008.stdout:8/282: write f7 [3123452,93230] 0 2024-04-12T05:55:29.756 INFO:tasks.workunit.client.0.smithi008.stdout:5/534: write d0/d40/d56/f67 [20754,118004] 0 2024-04-12T05:55:29.756 INFO:tasks.workunit.client.0.smithi008.stdout:3/375: symlink d9/d2e/l78 0 2024-04-12T05:55:29.758 INFO:tasks.workunit.client.0.smithi008.stdout:9/421: mkdir d3/df/d13/d21/d5c/d8f 0 2024-04-12T05:55:29.760 INFO:tasks.workunit.client.0.smithi008.stdout:7/462: dwrite d1/d5/da/d3c/d44/fa2 [0,4194304] 0 2024-04-12T05:55:29.767 INFO:tasks.workunit.client.0.smithi008.stdout:8/283: mkdir dd/de/d16/d50/d56/d61 0 2024-04-12T05:55:29.771 INFO:tasks.workunit.client.0.smithi008.stdout:3/376: getdents d9/d10/d19/d3a/d67 0 2024-04-12T05:55:29.772 INFO:tasks.workunit.client.0.smithi008.stdout:8/284: symlink dd/de/d16/d50/l62 0 2024-04-12T05:55:29.772 INFO:tasks.workunit.client.0.smithi008.stdout:7/463: dwrite d1/d49/f53 [0,4194304] 0 2024-04-12T05:55:29.776 INFO:tasks.workunit.client.0.smithi008.stdout:5/535: truncate d0/dd/f34 3778239 0 2024-04-12T05:55:29.777 INFO:tasks.workunit.client.0.smithi008.stdout:7/464: chown d1/d5/da/d11/d56/d8f/c95 439666970 1 2024-04-12T05:55:29.778 INFO:tasks.workunit.client.0.smithi008.stdout:5/536: dread - d0/d29/f9c zero size 2024-04-12T05:55:29.778 INFO:tasks.workunit.client.0.smithi008.stdout:9/422: getdents d3/d22/d2b/d4a 0 2024-04-12T05:55:29.780 INFO:tasks.workunit.client.0.smithi008.stdout:8/285: creat dd/de/d16/d50/d56/d61/f63 x:0 0 0 2024-04-12T05:55:29.781 INFO:tasks.workunit.client.0.smithi008.stdout:3/377: dread d9/dd/f2f [0,4194304] 0 2024-04-12T05:55:29.782 INFO:tasks.workunit.client.0.smithi008.stdout:5/537: creat d0/dd/d11/fa7 x:0 0 0 2024-04-12T05:55:29.783 INFO:tasks.workunit.client.0.smithi008.stdout:7/465: read d1/d5/da/d3c/d44/d8a/f3b [34030,31837] 0 2024-04-12T05:55:29.786 INFO:tasks.workunit.client.0.smithi008.stdout:5/538: read d0/dc/d57/f2f [2690599,59550] 0 2024-04-12T05:55:29.786 INFO:tasks.workunit.client.0.smithi008.stdout:9/423: mkdir d3/d22/d2b/d58/d90 0 2024-04-12T05:55:29.787 INFO:tasks.workunit.client.0.smithi008.stdout:7/466: rmdir d1/d15/d43/d54 39 2024-04-12T05:55:29.789 INFO:tasks.workunit.client.0.smithi008.stdout:8/286: creat dd/de/f64 x:0 0 0 2024-04-12T05:55:29.791 INFO:tasks.workunit.client.0.smithi008.stdout:5/539: write d0/dc/f68 [974509,33117] 0 2024-04-12T05:55:29.791 INFO:tasks.workunit.client.0.smithi008.stdout:7/467: creat d1/d5/da/d3c/d44/d8a/fa4 x:0 0 0 2024-04-12T05:55:29.793 INFO:tasks.workunit.client.0.smithi008.stdout:8/287: fdatasync dd/d30/f35 0 2024-04-12T05:55:29.793 INFO:tasks.workunit.client.0.smithi008.stdout:3/378: getdents d9/d10/d19 0 2024-04-12T05:55:29.794 INFO:tasks.workunit.client.0.smithi008.stdout:8/288: readlink dd/de/l20 0 2024-04-12T05:55:29.796 INFO:tasks.workunit.client.0.smithi008.stdout:3/379: write d9/d10/d19/d22/f1f [3626766,93131] 0 2024-04-12T05:55:29.796 INFO:tasks.workunit.client.0.smithi008.stdout:7/468: unlink d1/d5/da/d1d/c39 0 2024-04-12T05:55:29.800 INFO:tasks.workunit.client.0.smithi008.stdout:9/424: creat d3/df/d13/d21/f91 x:0 0 0 2024-04-12T05:55:29.802 INFO:tasks.workunit.client.0.smithi008.stdout:8/289: mkdir dd/d65 0 2024-04-12T05:55:29.804 INFO:tasks.workunit.client.0.smithi008.stdout:8/290: dread dd/f1e [0,4194304] 0 2024-04-12T05:55:29.806 INFO:tasks.workunit.client.0.smithi008.stdout:7/469: rename d1/f9c to d1/d5/da/d3c/d44/d8a/d94/fa5 0 2024-04-12T05:55:29.808 INFO:tasks.workunit.client.0.smithi008.stdout:3/380: dread d9/f1c [0,4194304] 0 2024-04-12T05:55:29.808 INFO:tasks.workunit.client.0.smithi008.stdout:5/540: creat d0/dc/fa8 x:0 0 0 2024-04-12T05:55:29.810 INFO:tasks.workunit.client.0.smithi008.stdout:3/381: fdatasync d9/d10/d19/f50 0 2024-04-12T05:55:29.813 INFO:tasks.workunit.client.0.smithi008.stdout:7/470: creat d1/d5/da/d1d/d68/fa6 x:0 0 0 2024-04-12T05:55:29.815 INFO:tasks.workunit.client.0.smithi008.stdout:8/291: dread dd/de/f27 [0,4194304] 0 2024-04-12T05:55:29.817 INFO:tasks.workunit.client.0.smithi008.stdout:5/541: creat d0/dc/fa9 x:0 0 0 2024-04-12T05:55:29.818 INFO:tasks.workunit.client.0.smithi008.stdout:3/382: dread d9/d2e/d3d/f56 [0,4194304] 0 2024-04-12T05:55:29.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:29 smithi194 ceph-mon[49771]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:29.820 INFO:tasks.workunit.client.0.smithi008.stdout:9/425: dwrite d3/f43 [4194304,4194304] 0 2024-04-12T05:55:29.820 INFO:tasks.workunit.client.0.smithi008.stdout:8/292: write dd/de/f2f [3934746,41097] 0 2024-04-12T05:55:29.821 INFO:tasks.workunit.client.0.smithi008.stdout:5/542: rmdir d0/d40 39 2024-04-12T05:55:29.824 INFO:tasks.workunit.client.0.smithi008.stdout:8/293: rmdir dd/de/d4c 39 2024-04-12T05:55:29.826 INFO:tasks.workunit.client.0.smithi008.stdout:7/471: rename d1/d5/da/d3c/d44/d8a/fa4 to d1/d5/da/d3c/d44/d8a/fa7 0 2024-04-12T05:55:29.826 INFO:tasks.workunit.client.0.smithi008.stdout:8/294: write dd/f1a [804229,22118] 0 2024-04-12T05:55:29.827 INFO:tasks.workunit.client.0.smithi008.stdout:9/426: truncate d3/d22/d2b/d77/d82/f8a 451695 0 2024-04-12T05:55:29.833 INFO:tasks.workunit.client.0.smithi008.stdout:3/383: dwrite d9/d11/f48 [0,4194304] 0 2024-04-12T05:55:29.833 INFO:tasks.workunit.client.0.smithi008.stdout:9/427: dread d3/d22/d2b/f41 [0,4194304] 0 2024-04-12T05:55:29.833 INFO:tasks.workunit.client.0.smithi008.stdout:8/295: write dd/de/f2f [1455493,45387] 0 2024-04-12T05:55:29.839 INFO:tasks.workunit.client.0.smithi008.stdout:7/472: unlink d1/d15/d43/d54/d58/f8e 0 2024-04-12T05:55:29.839 INFO:tasks.workunit.client.0.smithi008.stdout:3/384: mknod d9/d44/c79 0 2024-04-12T05:55:29.841 INFO:tasks.workunit.client.0.smithi008.stdout:3/385: read - d9/d11/d5b/f60 zero size 2024-04-12T05:55:29.842 INFO:tasks.workunit.client.0.smithi008.stdout:5/543: dread d0/dd/d11/d38/f2b [0,4194304] 0 2024-04-12T05:55:29.843 INFO:tasks.workunit.client.0.smithi008.stdout:8/296: write dd/d30/f41 [651262,70258] 0 2024-04-12T05:55:29.845 INFO:tasks.workunit.client.0.smithi008.stdout:7/473: symlink d1/d5/da/d3c/d44/la8 0 2024-04-12T05:55:29.845 INFO:tasks.workunit.client.0.smithi008.stdout:7/474: read d1/d15/f20 [2410402,91843] 0 2024-04-12T05:55:29.846 INFO:tasks.workunit.client.0.smithi008.stdout:7/475: chown d1/d5/f4e 0 1 2024-04-12T05:55:29.848 INFO:tasks.workunit.client.0.smithi008.stdout:3/386: creat d9/f7a x:0 0 0 2024-04-12T05:55:29.850 INFO:tasks.workunit.client.0.smithi008.stdout:9/428: write d3/d22/d2b/d58/d38/f54 [283965,53774] 0 2024-04-12T05:55:29.852 INFO:tasks.workunit.client.0.smithi008.stdout:7/476: rename d1/d5/da/d1d to d1/d5/da/d1d/da9 22 2024-04-12T05:55:29.855 INFO:tasks.workunit.client.0.smithi008.stdout:8/297: dwrite dd/de/d16/d50/d56/f5d [0,4194304] 0 2024-04-12T05:55:29.855 INFO:tasks.workunit.client.0.smithi008.stdout:9/429: mknod d3/d22/d2b/c92 0 2024-04-12T05:55:29.857 INFO:tasks.workunit.client.0.smithi008.stdout:9/430: chown d3/d6a 45258407 1 2024-04-12T05:55:29.857 INFO:tasks.workunit.client.0.smithi008.stdout:5/544: rename d0/dc/d13/d1a/f7d to d0/dd/d3d/d5d/faa 0 2024-04-12T05:55:29.860 INFO:tasks.workunit.client.0.smithi008.stdout:7/477: creat d1/d5/da/d78/faa x:0 0 0 2024-04-12T05:55:29.861 INFO:tasks.workunit.client.0.smithi008.stdout:9/431: stat d3/df/d13/d21/d2d/d32/d52 0 2024-04-12T05:55:29.863 INFO:tasks.workunit.client.0.smithi008.stdout:9/432: write d3/d22/d2b/f72 [206576,56085] 0 2024-04-12T05:55:29.867 INFO:tasks.workunit.client.0.smithi008.stdout:8/298: mkdir dd/de/d4c/d66 0 2024-04-12T05:55:29.867 INFO:tasks.workunit.client.0.smithi008.stdout:3/387: symlink d9/d11/d5b/l7b 0 2024-04-12T05:55:29.869 INFO:tasks.workunit.client.0.smithi008.stdout:9/433: symlink d3/d22/d2b/d4a/l93 0 2024-04-12T05:55:29.872 INFO:tasks.workunit.client.0.smithi008.stdout:3/388: fdatasync d9/d10/f33 0 2024-04-12T05:55:29.873 INFO:tasks.workunit.client.0.smithi008.stdout:5/545: readlink d0/dc/d13/d1a/d3a/l9a 0 2024-04-12T05:55:29.876 INFO:tasks.workunit.client.0.smithi008.stdout:8/299: symlink dd/de/d16/d50/d5e/l67 0 2024-04-12T05:55:29.876 INFO:tasks.workunit.client.0.smithi008.stdout:7/478: dwrite d1/d5/da/d78/f74 [4194304,4194304] 0 2024-04-12T05:55:29.878 INFO:tasks.workunit.client.0.smithi008.stdout:2/340: sync 2024-04-12T05:55:29.879 INFO:tasks.workunit.client.0.smithi008.stdout:7/479: chown d1/d5/da/c85 72968797 1 2024-04-12T05:55:29.879 INFO:tasks.workunit.client.0.smithi008.stdout:4/216: sync 2024-04-12T05:55:29.880 INFO:tasks.workunit.client.0.smithi008.stdout:9/434: link d3/df/d13/d21/d2d/f8e d3/df/d13/d21/d2d/d75/f94 0 2024-04-12T05:55:29.882 INFO:tasks.workunit.client.0.smithi008.stdout:5/546: mkdir d0/dab 0 2024-04-12T05:55:29.884 INFO:tasks.workunit.client.0.smithi008.stdout:5/547: read - d0/dc/fa8 zero size 2024-04-12T05:55:29.884 INFO:tasks.workunit.client.0.smithi008.stdout:8/300: unlink dd/de/f2d 0 2024-04-12T05:55:29.887 INFO:tasks.workunit.client.0.smithi008.stdout:8/301: chown dd/de/f2a 325483 1 2024-04-12T05:55:29.889 INFO:tasks.workunit.client.0.smithi008.stdout:7/480: fdatasync d1/d5/f4e 0 2024-04-12T05:55:29.894 INFO:tasks.workunit.client.0.smithi008.stdout:3/389: dwrite d9/f1c [4194304,4194304] 0 2024-04-12T05:55:29.896 INFO:tasks.workunit.client.0.smithi008.stdout:2/341: write d6/db/f56 [1565855,18772] 0 2024-04-12T05:55:29.898 INFO:tasks.workunit.client.0.smithi008.stdout:5/548: write d0/dc/d57/f2f [5579065,20837] 0 2024-04-12T05:55:29.902 INFO:tasks.workunit.client.0.smithi008.stdout:9/435: dwrite d3/df/d13/d21/d2d/d30/f4d [0,4194304] 0 2024-04-12T05:55:29.903 INFO:tasks.workunit.client.0.smithi008.stdout:3/390: write d9/d11/d5b/f68 [954794,506] 0 2024-04-12T05:55:29.903 INFO:tasks.workunit.client.0.smithi008.stdout:7/481: chown d1/d5/da/d3c/d44/d8a/fa7 10027891 1 2024-04-12T05:55:29.907 INFO:tasks.workunit.client.0.smithi008.stdout:2/342: symlink d6/db/d10/d23/d35/d3c/l71 0 2024-04-12T05:55:29.908 INFO:tasks.workunit.client.0.smithi008.stdout:3/391: fdatasync d9/d2e/f59 0 2024-04-12T05:55:29.909 INFO:tasks.workunit.client.0.smithi008.stdout:4/217: rename d1/db/d19/f24 to d1/f3b 0 2024-04-12T05:55:29.910 INFO:tasks.workunit.client.0.smithi008.stdout:8/302: dwrite dd/f54 [0,4194304] 0 2024-04-12T05:55:29.952 INFO:tasks.workunit.client.0.smithi008.stdout:7/482: unlink d1/l2 0 2024-04-12T05:55:29.955 INFO:tasks.workunit.client.0.smithi008.stdout:9/436: mkdir d3/df/d13/d14/d95 0 2024-04-12T05:55:29.959 INFO:tasks.workunit.client.0.smithi008.stdout:8/303: symlink dd/de/d4c/l68 0 2024-04-12T05:55:29.962 INFO:tasks.workunit.client.0.smithi008.stdout:2/343: symlink d6/db/d38/l72 0 2024-04-12T05:55:29.964 INFO:tasks.workunit.client.0.smithi008.stdout:8/304: creat dd/d4f/d52/f69 x:0 0 0 2024-04-12T05:55:29.966 INFO:tasks.workunit.client.0.smithi008.stdout:3/392: dwrite d9/d11/d5b/f68 [0,4194304] 0 2024-04-12T05:55:29.967 INFO:tasks.workunit.client.0.smithi008.stdout:2/344: creat d6/f73 x:0 0 0 2024-04-12T05:55:29.968 INFO:tasks.workunit.client.0.smithi008.stdout:4/218: dwrite d1/f36 [0,4194304] 0 2024-04-12T05:55:29.972 INFO:tasks.workunit.client.0.smithi008.stdout:9/437: link d3/df/d13/d21/d2d/d75/f94 d3/df/d10/f96 0 2024-04-12T05:55:29.975 INFO:tasks.workunit.client.0.smithi008.stdout:8/305: mkdir dd/de/d16/d50/d56/d61/d6a 0 2024-04-12T05:55:29.979 INFO:tasks.workunit.client.0.smithi008.stdout:0/350: sync 2024-04-12T05:55:29.979 INFO:tasks.workunit.client.0.smithi008.stdout:1/327: sync 2024-04-12T05:55:29.979 INFO:tasks.workunit.client.0.smithi008.stdout:6/401: sync 2024-04-12T05:55:29.980 INFO:tasks.workunit.client.0.smithi008.stdout:3/393: mkdir d9/d10/d19/d3a/d7c 0 2024-04-12T05:55:29.983 INFO:tasks.workunit.client.0.smithi008.stdout:4/219: mkdir d1/d3c 0 2024-04-12T05:55:29.984 INFO:tasks.workunit.client.0.smithi008.stdout:6/402: write df/f1f [3926037,126728] 0 2024-04-12T05:55:29.985 INFO:tasks.workunit.client.0.smithi008.stdout:8/306: mknod dd/d30/c6b 0 2024-04-12T05:55:29.986 INFO:tasks.workunit.client.0.smithi008.stdout:1/328: fdatasync d4/d7/fe 0 2024-04-12T05:55:30.108 INFO:tasks.workunit.client.0.smithi008.stdout:6/403: stat df/d21/l3b 0 2024-04-12T05:55:30.109 INFO:tasks.workunit.client.0.smithi008.stdout:7/483: dwrite d1/d15/f20 [0,4194304] 0 2024-04-12T05:55:30.109 INFO:tasks.workunit.client.0.smithi008.stdout:2/345: getdents d6 0 2024-04-12T05:55:30.112 INFO:tasks.workunit.client.0.smithi008.stdout:4/220: chown d1/db/fc 27 1 2024-04-12T05:55:30.112 INFO:tasks.workunit.client.0.smithi008.stdout:9/438: dwrite d3/d22/d2b/d58/f48 [0,4194304] 0 2024-04-12T05:55:30.121 INFO:tasks.workunit.client.0.smithi008.stdout:6/404: truncate f4 2019450 0 2024-04-12T05:55:30.129 INFO:tasks.workunit.client.0.smithi008.stdout:1/329: truncate d4/dd/d17/f64 726244 0 2024-04-12T05:55:30.130 INFO:tasks.workunit.client.0.smithi008.stdout:4/221: rename d1/d8/c1f to d1/db/c3d 0 2024-04-12T05:55:30.131 INFO:tasks.workunit.client.0.smithi008.stdout:8/307: dwrite dd/de/d16/f4a [0,4194304] 0 2024-04-12T05:55:30.133 INFO:tasks.workunit.client.0.smithi008.stdout:3/394: link d9/dd/l46 d9/d2e/l7d 0 2024-04-12T05:55:30.133 INFO:tasks.workunit.client.0.smithi008.stdout:6/405: write fe [753326,15735] 0 2024-04-12T05:55:30.135 INFO:tasks.workunit.client.0.smithi008.stdout:4/222: fdatasync d1/fe 0 2024-04-12T05:55:30.137 INFO:tasks.workunit.client.0.smithi008.stdout:2/346: rename d6/db/d60/d4f/f61 to d6/f74 0 2024-04-12T05:55:30.137 INFO:tasks.workunit.client.0.smithi008.stdout:9/439: truncate d3/f6 6606325 0 2024-04-12T05:55:30.140 INFO:tasks.workunit.client.0.smithi008.stdout:1/330: dread d4/dd/f39 [0,4194304] 0 2024-04-12T05:55:30.142 INFO:tasks.workunit.client.0.smithi008.stdout:8/308: unlink f7 0 2024-04-12T05:55:30.144 INFO:tasks.workunit.client.0.smithi008.stdout:7/484: dwrite d1/d5/da/d3c/d44/d8a/d94/fa3 [0,4194304] 0 2024-04-12T05:55:30.144 INFO:tasks.workunit.client.0.smithi008.stdout:3/395: chown d9/d10/d19/d22/f4d 32151 1 2024-04-12T05:55:30.145 INFO:tasks.workunit.client.0.smithi008.stdout:9/440: write d3/d22/d2b/d4e/f6b [404396,121567] 0 2024-04-12T05:55:30.146 INFO:tasks.workunit.client.0.smithi008.stdout:4/223: stat d1/d22/d29/c2e 0 2024-04-12T05:55:30.147 INFO:tasks.workunit.client.0.smithi008.stdout:9/441: fsync d3/f25 0 2024-04-12T05:55:30.147 INFO:tasks.workunit.client.0.smithi008.stdout:8/309: dread dd/de/f27 [0,4194304] 0 2024-04-12T05:55:30.150 INFO:tasks.workunit.client.0.smithi008.stdout:6/406: creat df/d10/d3a/f7f x:0 0 0 2024-04-12T05:55:30.151 INFO:tasks.workunit.client.0.smithi008.stdout:2/347: read - d6/db/d29/f6c zero size 2024-04-12T05:55:30.154 INFO:tasks.workunit.client.0.smithi008.stdout:7/485: mkdir d1/d5/da/d78/dab 0 2024-04-12T05:55:30.157 INFO:tasks.workunit.client.0.smithi008.stdout:9/442: creat d3/d22/d2b/d4a/f97 x:0 0 0 2024-04-12T05:55:30.160 INFO:tasks.workunit.client.0.smithi008.stdout:6/407: creat df/d21/d4c/f80 x:0 0 0 2024-04-12T05:55:30.161 INFO:tasks.workunit.client.0.smithi008.stdout:1/331: mkdir d4/dd/d17/d6b 0 2024-04-12T05:55:30.162 INFO:tasks.workunit.client.0.smithi008.stdout:6/408: chown df/d10/d3a/d78 413321 1 2024-04-12T05:55:30.165 INFO:tasks.workunit.client.0.smithi008.stdout:4/224: dwrite d1/db/f16 [0,4194304] 0 2024-04-12T05:55:30.167 INFO:tasks.workunit.client.0.smithi008.stdout:2/348: creat d6/db/d38/f75 x:0 0 0 2024-04-12T05:55:30.168 INFO:tasks.workunit.client.0.smithi008.stdout:8/310: getdents dd/d65 0 2024-04-12T05:55:30.168 INFO:tasks.workunit.client.0.smithi008.stdout:7/486: creat d1/d5/da/d3c/d44/d8a/d94/fac x:0 0 0 2024-04-12T05:55:30.169 INFO:tasks.workunit.client.0.smithi008.stdout:2/349: chown d6/db/d29 114 1 2024-04-12T05:55:30.170 INFO:tasks.workunit.client.0.smithi008.stdout:9/443: mknod d3/d22/d2b/d4e/d60/c98 0 2024-04-12T05:55:30.174 INFO:tasks.workunit.client.0.smithi008.stdout:6/409: creat df/d21/f81 x:0 0 0 2024-04-12T05:55:30.181 INFO:tasks.workunit.client.0.smithi008.stdout:4/225: rename d1/d8/d2f to d1/db/d19/d28/d3e 0 2024-04-12T05:55:30.184 INFO:tasks.workunit.client.0.smithi008.stdout:4/226: dread - d1/db/d19/f26 zero size 2024-04-12T05:55:30.184 INFO:tasks.workunit.client.0.smithi008.stdout:2/350: truncate d6/db/d38/f25 2786588 0 2024-04-12T05:55:30.188 INFO:tasks.workunit.client.0.smithi008.stdout:9/444: symlink d3/df/d10/l99 0 2024-04-12T05:55:30.196 INFO:tasks.workunit.client.0.smithi008.stdout:2/351: creat d6/db/d60/d57/f76 x:0 0 0 2024-04-12T05:55:30.197 INFO:tasks.workunit.client.0.smithi008.stdout:1/332: creat d4/d1a/f6c x:0 0 0 2024-04-12T05:55:30.198 INFO:tasks.workunit.client.0.smithi008.stdout:3/396: dwrite d9/d44/f63 [0,4194304] 0 2024-04-12T05:55:30.200 INFO:tasks.workunit.client.0.smithi008.stdout:3/397: readlink d9/d11/d5b/l7b 0 2024-04-12T05:55:30.205 INFO:tasks.workunit.client.0.smithi008.stdout:7/487: dwrite d1/d5/da/d3c/d44/d8a/d94/fa5 [0,4194304] 0 2024-04-12T05:55:30.209 INFO:tasks.workunit.client.0.smithi008.stdout:1/333: mkdir d4/d6/d11/d6d 0 2024-04-12T05:55:30.210 INFO:tasks.workunit.client.0.smithi008.stdout:2/352: creat d6/db/d38/d41/f77 x:0 0 0 2024-04-12T05:55:30.212 INFO:tasks.workunit.client.0.smithi008.stdout:9/445: link d3/df/d13/d21/f91 d3/d22/d2b/d4a/d62/f9a 0 2024-04-12T05:55:30.213 INFO:tasks.workunit.client.0.smithi008.stdout:8/311: fdatasync dd/ff 0 2024-04-12T05:55:30.223 INFO:tasks.workunit.client.0.smithi008.stdout:3/398: rename d9/dd/l15 to d9/d37/l7e 0 2024-04-12T05:55:30.223 INFO:tasks.workunit.client.0.smithi008.stdout:6/410: dwrite f4 [0,4194304] 0 2024-04-12T05:55:30.227 INFO:tasks.workunit.client.0.smithi008.stdout:4/227: dread d1/fe [0,4194304] 0 2024-04-12T05:55:30.228 INFO:tasks.workunit.client.0.smithi008.stdout:1/334: write d4/dd/d17/f19 [3075116,70162] 0 2024-04-12T05:55:30.230 INFO:tasks.workunit.client.0.smithi008.stdout:7/488: getdents d1/d49 0 2024-04-12T05:55:30.232 INFO:tasks.workunit.client.0.smithi008.stdout:9/446: rename d3/df/d13/d21/f44 to d3/d22/d2b/d58/d38/f9b 0 2024-04-12T05:55:30.233 INFO:tasks.workunit.client.0.smithi008.stdout:8/312: mkdir dd/de/d4c/d66/d6c 0 2024-04-12T05:55:30.234 INFO:tasks.workunit.client.0.smithi008.stdout:1/335: write d4/dd/d17/d28/d30/f5d [914068,43794] 0 2024-04-12T05:55:30.234 INFO:tasks.workunit.client.0.smithi008.stdout:4/228: write d1/db/d19/f26 [17133,109165] 0 2024-04-12T05:55:30.238 INFO:tasks.workunit.client.0.smithi008.stdout:2/353: dwrite d6/db/d10/f55 [0,4194304] 0 2024-04-12T05:55:30.238 INFO:tasks.workunit.client.0.smithi008.stdout:7/489: mknod d1/d5/da/d3c/d44/cad 0 2024-04-12T05:55:30.239 INFO:tasks.workunit.client.0.smithi008.stdout:4/229: chown d1/d8/f25 7 1 2024-04-12T05:55:30.241 INFO:tasks.workunit.client.0.smithi008.stdout:8/313: dread dd/f1e [0,4194304] 0 2024-04-12T05:55:30.242 INFO:tasks.workunit.client.0.smithi008.stdout:3/399: rename d9/d2e/d3d/f4c to d9/dd/f7f 0 2024-04-12T05:55:30.243 INFO:tasks.workunit.client.0.smithi008.stdout:2/354: write d6/db/d10/f55 [1566622,52439] 0 2024-04-12T05:55:30.250 INFO:tasks.workunit.client.0.smithi008.stdout:8/314: chown dd/d30/l5b 6626712 1 2024-04-12T05:55:30.251 INFO:tasks.workunit.client.0.smithi008.stdout:3/400: chown d9/d10/d19/d22/d2c 28146618 1 2024-04-12T05:55:30.252 INFO:tasks.workunit.client.0.smithi008.stdout:7/490: symlink d1/d5/da/d1d/lae 0 2024-04-12T05:55:30.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/315: fsync dd/de/f2a 0 2024-04-12T05:55:30.253 INFO:tasks.workunit.client.0.smithi008.stdout:6/411: rename df/f63 to df/d10/d15/d4f/d50/f82 0 2024-04-12T05:55:30.255 INFO:tasks.workunit.client.0.smithi008.stdout:2/355: creat d6/db/d60/f78 x:0 0 0 2024-04-12T05:55:30.256 INFO:tasks.workunit.client.0.smithi008.stdout:3/401: creat d9/d2e/d3d/f80 x:0 0 0 2024-04-12T05:55:30.258 INFO:tasks.workunit.client.0.smithi008.stdout:9/447: dwrite d3/d22/f3e [0,4194304] 0 2024-04-12T05:55:30.260 INFO:tasks.workunit.client.0.smithi008.stdout:9/448: readlink d3/df/d13/d14/l2c 0 2024-04-12T05:55:30.262 INFO:tasks.workunit.client.0.smithi008.stdout:8/316: mkdir dd/de/d4c/d66/d6d 0 2024-04-12T05:55:30.263 INFO:tasks.workunit.client.0.smithi008.stdout:4/230: dread d1/fe [0,4194304] 0 2024-04-12T05:55:30.271 INFO:tasks.workunit.client.0.smithi008.stdout:7/491: symlink d1/d5/da/d60/d7b/laf 0 2024-04-12T05:55:30.272 INFO:tasks.workunit.client.0.smithi008.stdout:9/449: creat d3/d22/d2b/d77/f9c x:0 0 0 2024-04-12T05:55:30.274 INFO:tasks.workunit.client.0.smithi008.stdout:2/356: dwrite d6/db/f69 [0,4194304] 0 2024-04-12T05:55:30.274 INFO:tasks.workunit.client.0.smithi008.stdout:6/412: mknod df/d10/d3a/d78/c83 0 2024-04-12T05:55:30.276 INFO:tasks.workunit.client.0.smithi008.stdout:3/402: creat d9/d10/d19/d65/f81 x:0 0 0 2024-04-12T05:55:30.279 INFO:tasks.workunit.client.0.smithi008.stdout:4/231: link d1/d22/d29/d30/f31 d1/db/d19/d28/f3f 0 2024-04-12T05:55:30.282 INFO:tasks.workunit.client.0.smithi008.stdout:8/317: truncate dd/f22 1908137 0 2024-04-12T05:55:30.288 INFO:tasks.workunit.client.0.smithi008.stdout:8/318: creat dd/de/d16/f6e x:0 0 0 2024-04-12T05:55:30.289 INFO:tasks.workunit.client.0.smithi008.stdout:7/492: dwrite d1/d5/da/d3c/d44/f51 [8388608,4194304] 0 2024-04-12T05:55:30.289 INFO:tasks.workunit.client.0.smithi008.stdout:4/232: rename d1/fe to d1/f40 0 2024-04-12T05:55:30.291 INFO:tasks.workunit.client.0.smithi008.stdout:4/233: stat d1/d8/l11 0 2024-04-12T05:55:30.291 INFO:tasks.workunit.client.0.smithi008.stdout:3/403: rename f6 to d9/f82 0 2024-04-12T05:55:30.292 INFO:tasks.workunit.client.0.smithi008.stdout:9/450: read d3/df/f86 [488849,87739] 0 2024-04-12T05:55:30.293 INFO:tasks.workunit.client.0.smithi008.stdout:8/319: readlink dd/l2b 0 2024-04-12T05:55:30.294 INFO:tasks.workunit.client.0.smithi008.stdout:8/320: dread - dd/de/d16/f45 zero size 2024-04-12T05:55:30.298 INFO:tasks.workunit.client.0.smithi008.stdout:2/357: dwrite d6/db/f2c [0,4194304] 0 2024-04-12T05:55:30.300 INFO:tasks.workunit.client.0.smithi008.stdout:6/413: rename df/d10/d15/d4f/c7a to df/d10/d20/d3e/c84 0 2024-04-12T05:55:30.308 INFO:tasks.workunit.client.0.smithi008.stdout:3/404: creat d9/d10/d19/d22/d2c/f83 x:0 0 0 2024-04-12T05:55:30.318 INFO:tasks.workunit.client.0.smithi008.stdout:7/493: rename d1/d5/c45 to d1/d5/da/d3c/d44/cb0 0 2024-04-12T05:55:30.319 INFO:tasks.workunit.client.0.smithi008.stdout:2/358: creat d6/db/d60/d57/f79 x:0 0 0 2024-04-12T05:55:30.322 INFO:tasks.workunit.client.0.smithi008.stdout:8/321: dwrite dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:30.324 INFO:tasks.workunit.client.0.smithi008.stdout:9/451: truncate d3/d22/d2b/d58/f27 1151534 0 2024-04-12T05:55:30.324 INFO:tasks.workunit.client.0.smithi008.stdout:4/234: rename d1/db/fc to d1/d22/d29/f41 0 2024-04-12T05:55:30.328 INFO:tasks.workunit.client.0.smithi008.stdout:9/452: fsync d3/f35 0 2024-04-12T05:55:30.328 INFO:tasks.workunit.client.0.smithi008.stdout:7/494: write d1/d5/da/d3c/d44/d8a/d94/fa5 [1367176,129080] 0 2024-04-12T05:55:30.328 INFO:tasks.workunit.client.0.smithi008.stdout:6/414: dwrite f3 [8388608,4194304] 0 2024-04-12T05:55:30.332 INFO:tasks.workunit.client.0.smithi008.stdout:4/235: creat d1/d8/f42 x:0 0 0 2024-04-12T05:55:30.334 INFO:tasks.workunit.client.0.smithi008.stdout:8/322: rename dd/f1e to dd/de/d16/d50/d56/d61/d6a/f6f 0 2024-04-12T05:55:30.338 INFO:tasks.workunit.client.0.smithi008.stdout:8/323: rename dd/l2b to dd/de/d16/d50/d56/d61/d6a/l70 0 2024-04-12T05:55:30.338 INFO:tasks.workunit.client.0.smithi008.stdout:6/415: symlink df/d10/d15/d79/l85 0 2024-04-12T05:55:30.339 INFO:tasks.workunit.client.0.smithi008.stdout:6/416: readlink df/d10/d15/l38 0 2024-04-12T05:55:30.340 INFO:tasks.workunit.client.0.smithi008.stdout:8/324: chown dd/de/d16/d50/d56/d61/f63 444543 1 2024-04-12T05:55:30.344 INFO:tasks.workunit.client.0.smithi008.stdout:2/359: write d6/ff [1717316,22381] 0 2024-04-12T05:55:30.345 INFO:tasks.workunit.client.0.smithi008.stdout:6/417: mknod df/d21/d4c/c86 0 2024-04-12T05:55:30.346 INFO:tasks.workunit.client.0.smithi008.stdout:7/495: link d1/d5/da/d60/c99 d1/d5/da/d1d/d82/cb1 0 2024-04-12T05:55:30.348 INFO:tasks.workunit.client.0.smithi008.stdout:8/325: dread dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:30.348 INFO:tasks.workunit.client.0.smithi008.stdout:4/236: getdents d1/db 0 2024-04-12T05:55:30.349 INFO:tasks.workunit.client.0.smithi008.stdout:2/360: rmdir d6/db/d38/d41 39 2024-04-12T05:55:30.349 INFO:tasks.workunit.client.0.smithi008.stdout:4/237: dread - d1/d22/d29/f3a zero size 2024-04-12T05:55:30.351 INFO:tasks.workunit.client.0.smithi008.stdout:4/238: chown d1/db/d19/f26 28 1 2024-04-12T05:55:30.352 INFO:tasks.workunit.client.0.smithi008.stdout:8/326: creat dd/d30/d32/f71 x:0 0 0 2024-04-12T05:55:30.353 INFO:tasks.workunit.client.0.smithi008.stdout:4/239: truncate d1/db/f1a 4789508 0 2024-04-12T05:55:30.353 INFO:tasks.workunit.client.0.smithi008.stdout:6/418: mknod df/d10/d15/d75/c87 0 2024-04-12T05:55:30.355 INFO:tasks.workunit.client.0.smithi008.stdout:2/361: symlink d6/db/d10/d23/l7a 0 2024-04-12T05:55:30.356 INFO:tasks.workunit.client.0.smithi008.stdout:6/419: fsync df/d10/d3a/f7f 0 2024-04-12T05:55:30.357 INFO:tasks.workunit.client.0.smithi008.stdout:7/496: symlink d1/d15/d43/d54/d58/lb2 0 2024-04-12T05:55:30.358 INFO:tasks.workunit.client.0.smithi008.stdout:6/420: chown df/d21 11437 1 2024-04-12T05:55:30.360 INFO:tasks.workunit.client.0.smithi008.stdout:4/240: mknod d1/d22/d29/d30/c43 0 2024-04-12T05:55:30.362 INFO:tasks.workunit.client.0.smithi008.stdout:2/362: mkdir d6/db/d10/d7b 0 2024-04-12T05:55:30.362 INFO:tasks.workunit.client.0.smithi008.stdout:6/421: write df/d21/d4c/f6b [477189,12473] 0 2024-04-12T05:55:30.365 INFO:tasks.workunit.client.0.smithi008.stdout:9/453: dread d3/d22/d2b/d58/f55 [0,4194304] 0 2024-04-12T05:55:30.365 INFO:tasks.workunit.client.0.smithi008.stdout:8/327: dwrite dd/de/f47 [4194304,4194304] 0 2024-04-12T05:55:30.367 INFO:tasks.workunit.client.0.smithi008.stdout:3/405: dread d9/d1e/f64 [0,4194304] 0 2024-04-12T05:55:30.370 INFO:tasks.workunit.client.0.smithi008.stdout:7/497: creat d1/d5/da/fb3 x:0 0 0 2024-04-12T05:55:30.370 INFO:tasks.workunit.client.0.smithi008.stdout:3/406: dread - d9/d10/d19/d65/f81 zero size 2024-04-12T05:55:30.371 INFO:tasks.workunit.client.0.smithi008.stdout:6/422: creat df/d10/d15/d4f/f88 x:0 0 0 2024-04-12T05:55:30.373 INFO:tasks.workunit.client.0.smithi008.stdout:7/498: write d1/d5/da/d1d/d68/fa6 [351511,47755] 0 2024-04-12T05:55:30.373 INFO:tasks.workunit.client.0.smithi008.stdout:9/454: symlink d3/d22/d2b/d77/l9d 0 2024-04-12T05:55:30.375 INFO:tasks.workunit.client.0.smithi008.stdout:4/241: write d1/d22/d29/f41 [4524129,52984] 0 2024-04-12T05:55:30.376 INFO:tasks.workunit.client.0.smithi008.stdout:3/407: fsync d9/d44/f6e 0 2024-04-12T05:55:30.376 INFO:tasks.workunit.client.0.smithi008.stdout:7/499: readlink d1/l8d 0 2024-04-12T05:55:30.377 INFO:tasks.workunit.client.0.smithi008.stdout:2/363: mknod d6/db/d10/d23/d35/c7c 0 2024-04-12T05:55:30.379 INFO:tasks.workunit.client.0.smithi008.stdout:6/423: link fd df/d10/d20/f89 0 2024-04-12T05:55:30.380 INFO:tasks.workunit.client.0.smithi008.stdout:3/408: write d9/d10/d19/d22/f5f [369084,115589] 0 2024-04-12T05:55:30.380 INFO:tasks.workunit.client.0.smithi008.stdout:4/242: rmdir d1/db/d19 39 2024-04-12T05:55:30.381 INFO:tasks.workunit.client.0.smithi008.stdout:9/455: dread d3/f4b [0,4194304] 0 2024-04-12T05:55:30.385 INFO:tasks.workunit.client.0.smithi008.stdout:4/243: unlink d1/d22/d29/d30/f31 0 2024-04-12T05:55:30.386 INFO:tasks.workunit.client.0.smithi008.stdout:3/409: symlink d9/d10/d19/d3a/l84 0 2024-04-12T05:55:30.386 INFO:tasks.workunit.client.0.smithi008.stdout:4/244: chown d1/d8/f42 49136 1 2024-04-12T05:55:30.388 INFO:tasks.workunit.client.0.smithi008.stdout:7/500: getdents d1/d5/da/d1d 0 2024-04-12T05:55:30.389 INFO:tasks.workunit.client.0.smithi008.stdout:3/410: stat d9/d2e/d3d/c58 0 2024-04-12T05:55:30.391 INFO:tasks.workunit.client.0.smithi008.stdout:3/411: truncate d9/d2e/d3d/f51 895678 0 2024-04-12T05:55:30.392 INFO:tasks.workunit.client.0.smithi008.stdout:7/501: write d1/f2d [3984093,124324] 0 2024-04-12T05:55:30.394 INFO:tasks.workunit.client.0.smithi008.stdout:3/412: readlink d9/d11/d5b/l7b 0 2024-04-12T05:55:30.396 INFO:tasks.workunit.client.0.smithi008.stdout:7/502: unlink d1/d5/da/d3c/d44/d8a/f3a 0 2024-04-12T05:55:30.397 INFO:tasks.workunit.client.0.smithi008.stdout:3/413: creat d9/d11/f85 x:0 0 0 2024-04-12T05:55:30.399 INFO:tasks.workunit.client.0.smithi008.stdout:4/245: rename d1/db/d19/l38 to d1/d3c/l44 0 2024-04-12T05:55:30.405 INFO:tasks.workunit.client.0.smithi008.stdout:9/456: truncate d3/df/d13/f5a 1651721 0 2024-04-12T05:55:30.406 INFO:tasks.workunit.client.0.smithi008.stdout:4/246: dread d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:30.410 INFO:tasks.workunit.client.0.smithi008.stdout:7/503: link d1/d5/da/d11/l36 d1/lb4 0 2024-04-12T05:55:30.412 INFO:tasks.workunit.client.0.smithi008.stdout:2/364: dwrite d6/db/f22 [0,4194304] 0 2024-04-12T05:55:30.414 INFO:tasks.workunit.client.0.smithi008.stdout:9/457: rename d3/d22/d2b/d58/d38/f67 to d3/d6a/f9e 0 2024-04-12T05:55:30.414 INFO:tasks.workunit.client.0.smithi008.stdout:4/247: symlink d1/db/l45 0 2024-04-12T05:55:30.418 INFO:tasks.workunit.client.0.smithi008.stdout:6/424: dwrite df/d10/d15/f2a [0,4194304] 0 2024-04-12T05:55:30.418 INFO:tasks.workunit.client.0.smithi008.stdout:3/414: dread f7 [0,4194304] 0 2024-04-12T05:55:30.419 INFO:tasks.workunit.client.0.smithi008.stdout:7/504: mknod d1/d5/da/d60/d7b/cb5 0 2024-04-12T05:55:30.422 INFO:tasks.workunit.client.0.smithi008.stdout:3/415: fsync d9/d44/f6e 0 2024-04-12T05:55:30.424 INFO:tasks.workunit.client.0.smithi008.stdout:4/248: rmdir d1/d22/d29/d30 39 2024-04-12T05:55:30.427 INFO:tasks.workunit.client.0.smithi008.stdout:9/458: dread d3/df/d13/d21/f76 [0,4194304] 0 2024-04-12T05:55:30.427 INFO:tasks.workunit.client.0.smithi008.stdout:4/249: dread d1/db/d19/f26 [0,4194304] 0 2024-04-12T05:55:30.427 INFO:tasks.workunit.client.0.smithi008.stdout:3/416: write d9/d10/f6a [163174,53040] 0 2024-04-12T05:55:30.429 INFO:tasks.workunit.client.0.smithi008.stdout:4/250: write d1/d22/d29/f41 [2855898,9163] 0 2024-04-12T05:55:30.430 INFO:tasks.workunit.client.0.smithi008.stdout:4/251: dread - d1/d8/f37 zero size 2024-04-12T05:55:30.433 INFO:tasks.workunit.client.0.smithi008.stdout:3/417: mknod d9/d37/c86 0 2024-04-12T05:55:30.434 INFO:tasks.workunit.client.0.smithi008.stdout:3/418: fdatasync d9/d11/f49 0 2024-04-12T05:55:30.436 INFO:tasks.workunit.client.0.smithi008.stdout:4/252: rename d1/d3c to d1/d22/d29/d46 0 2024-04-12T05:55:30.437 INFO:tasks.workunit.client.0.smithi008.stdout:7/505: dread d1/d5/da/d1d/d68/fa6 [0,4194304] 0 2024-04-12T05:55:30.438 INFO:tasks.workunit.client.0.smithi008.stdout:7/506: dread - d1/d5/f4d zero size 2024-04-12T05:55:30.443 INFO:tasks.workunit.client.0.smithi008.stdout:7/507: creat d1/d5/da/d3c/d44/d8a/d94/fb6 x:0 0 0 2024-04-12T05:55:30.445 INFO:tasks.workunit.client.0.smithi008.stdout:2/365: fsync d6/db/d10/d23/f4d 0 2024-04-12T05:55:30.445 INFO:tasks.workunit.client.0.smithi008.stdout:4/253: rename d1/db/d19 to d1/d22/d29/d30/d47 0 2024-04-12T05:55:30.451 INFO:tasks.workunit.client.0.smithi008.stdout:3/419: dwrite d9/dd/f2f [4194304,4194304] 0 2024-04-12T05:55:30.455 INFO:tasks.workunit.client.0.smithi008.stdout:7/508: rename d1/d5/da/d3c/d44/d8a/d94 to d1/d15/d43/d54/d58/db7 0 2024-04-12T05:55:30.460 INFO:tasks.workunit.client.0.smithi008.stdout:2/366: dwrite d6/f15 [0,4194304] 0 2024-04-12T05:55:30.460 INFO:tasks.workunit.client.0.smithi008.stdout:4/254: dwrite d1/f20 [0,4194304] 0 2024-04-12T05:55:30.466 INFO:tasks.workunit.client.0.smithi008.stdout:2/367: fdatasync d6/f43 0 2024-04-12T05:55:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:6/425: dread fe [0,4194304] 0 2024-04-12T05:55:30.515 INFO:tasks.workunit.client.0.smithi008.stdout:0/351: sync 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:5/549: sync 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:0/352: dread - d2/d11/f64 zero size 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/509: dwrite d1/d5/da/d11/f23 [0,4194304] 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:4/255: write d1/db/f1a [3147808,3483] 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/510: write d1/d5/f2e [3575587,37665] 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:5/550: creat d0/d2/fac x:0 0 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:0/353: truncate d2/d11/f23 1421920 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/511: dread - d1/d5/f4d zero size 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/512: stat d1/d5/da/c85 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:2/368: dwrite d6/f16 [4194304,4194304] 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/513: chown d1/d5/da/d11/d56/d8f 15655978 1 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:4/256: dread d1/f40 [0,4194304] 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:5/551: chown d0/dd/d3d/d5d/l90 263708457 1 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:0/354: symlink d2/d11/d36/d47/l6c 0 2024-04-12T05:55:30.516 INFO:tasks.workunit.client.0.smithi008.stdout:6/426: truncate df/d21/f43 4616462 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:4/257: creat d1/d22/d29/d30/d47/f48 x:0 0 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:2/369: mkdir d6/db/d10/d7b/d7d 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:0/355: fsync d2/d11/d36/d47/f68 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:0/356: stat d2/d11/d36/d47/f59 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:6/427: creat df/d10/d20/f8a x:0 0 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:2/370: write d6/db/d38/d41/f77 [644305,1580] 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:7/514: dwrite d1/d15/d43/d54/d58/db7/fac [0,4194304] 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:3/420: dread d9/f82 [0,4194304] 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:0/357: creat d2/d11/d36/d47/f6d x:0 0 0 2024-04-12T05:55:30.517 INFO:tasks.workunit.client.0.smithi008.stdout:5/552: dwrite d0/dc/d13/d1a/d35/f4c [0,4194304] 0 2024-04-12T05:55:30.519 INFO:tasks.workunit.client.0.smithi008.stdout:2/371: readlink d6/db/d60/l4e 0 2024-04-12T05:55:30.522 INFO:tasks.workunit.client.0.smithi008.stdout:4/258: truncate d1/f40 2245510 0 2024-04-12T05:55:30.525 INFO:tasks.workunit.client.0.smithi008.stdout:0/358: write d2/d11/d36/d47/f21 [2543470,29818] 0 2024-04-12T05:55:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:7/515: creat d1/fb8 x:0 0 0 2024-04-12T05:55:30.530 INFO:tasks.workunit.client.0.smithi008.stdout:4/259: chown d1/d22/d29/d30/f34 103817166 1 2024-04-12T05:55:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:6/428: dwrite df/d21/d4c/f80 [0,4194304] 0 2024-04-12T05:55:30.533 INFO:tasks.workunit.client.0.smithi008.stdout:5/553: chown d0/d40/l4d 20191 1 2024-04-12T05:55:30.535 INFO:tasks.workunit.client.0.smithi008.stdout:3/421: dwrite d9/d2e/d3d/f73 [0,4194304] 0 2024-04-12T05:55:30.536 INFO:tasks.workunit.client.0.smithi008.stdout:0/359: write d2/d11/d36/f46 [3426007,40903] 0 2024-04-12T05:55:30.538 INFO:tasks.workunit.client.0.smithi008.stdout:0/360: chown d2/f66 674645540 1 2024-04-12T05:55:30.541 INFO:tasks.workunit.client.0.smithi008.stdout:0/361: fsync d2/d11/f40 0 2024-04-12T05:55:30.542 INFO:tasks.workunit.client.0.smithi008.stdout:6/429: write df/d10/d3a/f7f [183124,115236] 0 2024-04-12T05:55:30.547 INFO:tasks.workunit.client.0.smithi008.stdout:7/516: rename d1/d5/da/d11/f23 to d1/d5/da/d1d/d82/fb9 0 2024-04-12T05:55:30.548 INFO:tasks.workunit.client.0.smithi008.stdout:3/422: chown d9/d10/l34 158 1 2024-04-12T05:55:30.554 INFO:tasks.workunit.client.0.smithi008.stdout:6/430: write df/d10/d15/d4f/d50/f5b [521349,34448] 0 2024-04-12T05:55:30.561 INFO:tasks.workunit.client.0.smithi008.stdout:5/554: dwrite d0/d40/d56/f67 [0,4194304] 0 2024-04-12T05:55:30.563 INFO:tasks.workunit.client.0.smithi008.stdout:4/260: dwrite d1/f3b [4194304,4194304] 0 2024-04-12T05:55:30.568 INFO:tasks.workunit.client.0.smithi008.stdout:2/372: dwrite d6/db/d10/d23/d35/f49 [0,4194304] 0 2024-04-12T05:55:30.568 INFO:tasks.workunit.client.0.smithi008.stdout:6/431: mkdir df/d10/d3a/d78/d8b 0 2024-04-12T05:55:30.573 INFO:tasks.workunit.client.0.smithi008.stdout:7/517: dwrite d1/d5/f32 [0,4194304] 0 2024-04-12T05:55:30.573 INFO:tasks.workunit.client.0.smithi008.stdout:5/555: write d0/d40/f77 [930593,82159] 0 2024-04-12T05:55:30.573 INFO:tasks.workunit.client.0.smithi008.stdout:6/432: fsync df/d21/d4c/f58 0 2024-04-12T05:55:30.580 INFO:tasks.workunit.client.0.smithi008.stdout:6/433: write df/f2e [118854,18883] 0 2024-04-12T05:55:30.580 INFO:tasks.workunit.client.0.smithi008.stdout:2/373: creat d6/db/d38/f7e x:0 0 0 2024-04-12T05:55:30.582 INFO:tasks.workunit.client.0.smithi008.stdout:3/423: truncate d9/d11/f48 222982 0 2024-04-12T05:55:30.585 INFO:tasks.workunit.client.0.smithi008.stdout:0/362: dwrite d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:30.585 INFO:tasks.workunit.client.0.smithi008.stdout:7/518: read d1/d15/d43/d54/d58/db7/fa5 [675476,86096] 0 2024-04-12T05:55:30.588 INFO:tasks.workunit.client.0.smithi008.stdout:6/434: dread - df/d10/d20/f6a zero size 2024-04-12T05:55:30.588 INFO:tasks.workunit.client.0.smithi008.stdout:3/424: chown d9/dd/f53 1132 1 2024-04-12T05:55:30.588 INFO:tasks.workunit.client.0.smithi008.stdout:5/556: getdents d0/dab 0 2024-04-12T05:55:30.591 INFO:tasks.workunit.client.0.smithi008.stdout:2/374: fdatasync d6/db/d38/f25 0 2024-04-12T05:55:30.593 INFO:tasks.workunit.client.0.smithi008.stdout:4/261: dwrite d1/f12 [4194304,4194304] 0 2024-04-12T05:55:30.593 INFO:tasks.workunit.client.0.smithi008.stdout:6/435: chown df/d10/d20/d3e/f72 63 1 2024-04-12T05:55:30.595 INFO:tasks.workunit.client.0.smithi008.stdout:2/375: fdatasync f0 0 2024-04-12T05:55:30.595 INFO:tasks.workunit.client.0.smithi008.stdout:7/519: chown d1/d15/d43/d54/d58/d5e/c90 880 1 2024-04-12T05:55:30.597 INFO:tasks.workunit.client.0.smithi008.stdout:0/363: mknod d2/d24/c6e 0 2024-04-12T05:55:30.598 INFO:tasks.workunit.client.0.smithi008.stdout:4/262: fsync d1/f20 0 2024-04-12T05:55:30.599 INFO:tasks.workunit.client.0.smithi008.stdout:5/557: mkdir d0/dd/dad 0 2024-04-12T05:55:30.600 INFO:tasks.workunit.client.0.smithi008.stdout:6/436: rmdir df/d10/d15/d75 39 2024-04-12T05:55:30.606 INFO:tasks.workunit.client.0.smithi008.stdout:4/263: stat d1/d8/f23 0 2024-04-12T05:55:30.606 INFO:tasks.workunit.client.0.smithi008.stdout:2/376: read d6/db/d38/d41/f77 [310147,129693] 0 2024-04-12T05:55:30.609 INFO:tasks.workunit.client.0.smithi008.stdout:3/425: dwrite d9/d10/f6a [0,4194304] 0 2024-04-12T05:55:30.611 INFO:tasks.workunit.client.0.smithi008.stdout:6/437: write df/d21/f5a [679407,99151] 0 2024-04-12T05:55:30.612 INFO:tasks.workunit.client.0.smithi008.stdout:7/520: creat d1/d15/d43/d54/d58/d5e/d7f/fba x:0 0 0 2024-04-12T05:55:30.612 INFO:tasks.workunit.client.0.smithi008.stdout:4/264: read - d1/d22/d29/d30/f34 zero size 2024-04-12T05:55:30.615 INFO:tasks.workunit.client.0.smithi008.stdout:5/558: mknod d0/dd/dad/cae 0 2024-04-12T05:55:30.623 INFO:tasks.workunit.client.0.smithi008.stdout:2/377: rename d6/db/d38/l1d to d6/db/d10/d4c/l7f 0 2024-04-12T05:55:30.627 INFO:tasks.workunit.client.0.smithi008.stdout:5/559: unlink d0/d40/l4d 0 2024-04-12T05:55:30.628 INFO:tasks.workunit.client.0.smithi008.stdout:4/265: rmdir d1/d22/d29/d30/d47 39 2024-04-12T05:55:30.629 INFO:tasks.workunit.client.0.smithi008.stdout:6/438: mkdir df/d8c 0 2024-04-12T05:55:30.631 INFO:tasks.workunit.client.0.smithi008.stdout:3/426: stat d9/d11/f48 0 2024-04-12T05:55:30.631 INFO:tasks.workunit.client.0.smithi008.stdout:6/439: readlink df/d10/d3a/l42 0 2024-04-12T05:55:30.633 INFO:tasks.workunit.client.0.smithi008.stdout:0/364: dread d2/d11/d2a/f5d [0,4194304] 0 2024-04-12T05:55:30.634 INFO:tasks.workunit.client.0.smithi008.stdout:6/440: chown df/c26 1 1 2024-04-12T05:55:30.637 INFO:tasks.workunit.client.0.smithi008.stdout:2/378: dwrite d6/f43 [0,4194304] 0 2024-04-12T05:55:30.638 INFO:tasks.workunit.client.0.smithi008.stdout:0/365: read d2/d11/d2a/f5c [61680,21230] 0 2024-04-12T05:55:30.639 INFO:tasks.workunit.client.0.smithi008.stdout:5/560: rename d0/d2 to d0/dc/d13/d1a/daf 0 2024-04-12T05:55:30.644 INFO:tasks.workunit.client.0.smithi008.stdout:6/441: creat df/d10/d15/d4f/d50/f8d x:0 0 0 2024-04-12T05:55:30.648 INFO:tasks.workunit.client.0.smithi008.stdout:0/366: mknod d2/d4b/c6f 0 2024-04-12T05:55:30.648 INFO:tasks.workunit.client.0.smithi008.stdout:3/427: dwrite d9/d11/f48 [0,4194304] 0 2024-04-12T05:55:30.659 INFO:tasks.workunit.client.0.smithi008.stdout:3/428: chown d9/d10/d19/d22/c3e 4361208 1 2024-04-12T05:55:30.660 INFO:tasks.workunit.client.0.smithi008.stdout:2/379: dread d6/f9 [4194304,4194304] 0 2024-04-12T05:55:30.661 INFO:tasks.workunit.client.0.smithi008.stdout:2/380: chown d6/db/d38/f75 258 1 2024-04-12T05:55:30.664 INFO:tasks.workunit.client.0.smithi008.stdout:6/442: getdents df/d10/d20 0 2024-04-12T05:55:30.664 INFO:tasks.workunit.client.0.smithi008.stdout:3/429: creat d9/d11/d5b/f87 x:0 0 0 2024-04-12T05:55:30.664 INFO:tasks.workunit.client.0.smithi008.stdout:4/266: truncate d1/d22/d29/f41 433797 0 2024-04-12T05:55:30.666 INFO:tasks.workunit.client.0.smithi008.stdout:2/381: rename d6/f73 to d6/db/d10/f80 0 2024-04-12T05:55:30.672 INFO:tasks.workunit.client.0.smithi008.stdout:5/561: dread d0/dc/d13/d1a/f22 [0,4194304] 0 2024-04-12T05:55:30.673 INFO:tasks.workunit.client.0.smithi008.stdout:2/382: symlink d6/db/d10/d23/l81 0 2024-04-12T05:55:30.673 INFO:tasks.workunit.client.0.smithi008.stdout:3/430: dread d9/d11/d5b/f68 [0,4194304] 0 2024-04-12T05:55:30.673 INFO:tasks.workunit.client.0.smithi008.stdout:6/443: symlink df/d10/d15/d75/l8e 0 2024-04-12T05:55:30.674 INFO:tasks.workunit.client.0.smithi008.stdout:4/267: mknod d1/d22/d29/d30/d47/d28/c49 0 2024-04-12T05:55:30.675 INFO:tasks.workunit.client.0.smithi008.stdout:2/383: chown d6/db/d10/d23/d35 507 1 2024-04-12T05:55:30.675 INFO:tasks.workunit.client.0.smithi008.stdout:3/431: readlink d9/d11/l29 0 2024-04-12T05:55:30.679 INFO:tasks.workunit.client.0.smithi008.stdout:4/268: mkdir d1/d22/d29/d30/d47/d28/d4a 0 2024-04-12T05:55:30.680 INFO:tasks.workunit.client.0.smithi008.stdout:6/444: truncate df/f23 402818 0 2024-04-12T05:55:30.681 INFO:tasks.workunit.client.0.smithi008.stdout:3/432: mknod d9/d10/c88 0 2024-04-12T05:55:30.682 INFO:tasks.workunit.client.0.smithi008.stdout:2/384: creat d6/db/d10/d4c/f82 x:0 0 0 2024-04-12T05:55:30.682 INFO:tasks.workunit.client.0.smithi008.stdout:0/367: dread d2/d24/d39/f53 [0,4194304] 0 2024-04-12T05:55:30.683 INFO:tasks.workunit.client.0.smithi008.stdout:4/269: read d1/db/f16 [895013,21979] 0 2024-04-12T05:55:30.687 INFO:tasks.workunit.client.0.smithi008.stdout:3/433: creat d9/d37/f89 x:0 0 0 2024-04-12T05:55:30.687 INFO:tasks.workunit.client.0.smithi008.stdout:5/562: dwrite d0/dc/d57/f66 [0,4194304] 0 2024-04-12T05:55:30.689 INFO:tasks.workunit.client.0.smithi008.stdout:2/385: fsync d6/db/d10/d23/f4d 0 2024-04-12T05:55:30.692 INFO:tasks.workunit.client.0.smithi008.stdout:2/386: dread - d6/db/d38/d41/f6b zero size 2024-04-12T05:55:30.692 INFO:tasks.workunit.client.0.smithi008.stdout:4/270: truncate d1/d8/f37 119200 0 2024-04-12T05:55:30.697 INFO:tasks.workunit.client.0.smithi008.stdout:4/271: chown d1/d22/d29/d30/l32 499 1 2024-04-12T05:55:30.697 INFO:tasks.workunit.client.0.smithi008.stdout:5/563: symlink d0/dc/lb0 0 2024-04-12T05:55:30.700 INFO:tasks.workunit.client.0.smithi008.stdout:6/445: dwrite df/d10/d15/f3f [0,4194304] 0 2024-04-12T05:55:30.701 INFO:tasks.workunit.client.0.smithi008.stdout:3/434: creat d9/d10/d19/d3a/d67/f8a x:0 0 0 2024-04-12T05:55:30.703 INFO:tasks.workunit.client.0.smithi008.stdout:3/435: stat d9/d11/f48 0 2024-04-12T05:55:30.705 INFO:tasks.workunit.client.0.smithi008.stdout:2/387: mknod d6/db/d10/d7b/d7d/c83 0 2024-04-12T05:55:30.707 INFO:tasks.workunit.client.0.smithi008.stdout:6/446: creat df/d10/d15/d79/f8f x:0 0 0 2024-04-12T05:55:30.709 INFO:tasks.workunit.client.0.smithi008.stdout:5/564: link d0/d29/f99 d0/dd/d11/d38/fb1 0 2024-04-12T05:55:30.710 INFO:tasks.workunit.client.0.smithi008.stdout:4/272: dwrite d1/d8/f23 [0,4194304] 0 2024-04-12T05:55:30.713 INFO:tasks.workunit.client.0.smithi008.stdout:6/447: truncate df/d10/d15/d79/f8f 29723 0 2024-04-12T05:55:30.715 INFO:tasks.workunit.client.0.smithi008.stdout:6/448: write df/d10/d15/d4f/f65 [1191543,55429] 0 2024-04-12T05:55:30.724 INFO:tasks.workunit.client.0.smithi008.stdout:3/436: readlink d9/d2e/l7d 0 2024-04-12T05:55:30.726 INFO:tasks.workunit.client.0.smithi008.stdout:2/388: mkdir d6/db/d60/d84 0 2024-04-12T05:55:30.727 INFO:tasks.workunit.client.0.smithi008.stdout:6/449: dwrite df/d10/d15/d79/f8f [0,4194304] 0 2024-04-12T05:55:30.728 INFO:tasks.workunit.client.0.smithi008.stdout:5/565: rmdir d0/dd/d3d 39 2024-04-12T05:55:30.733 INFO:tasks.workunit.client.0.smithi008.stdout:2/389: stat d6/db/fd 0 2024-04-12T05:55:30.734 INFO:tasks.workunit.client.0.smithi008.stdout:5/566: creat d0/d29/d96/fb2 x:0 0 0 2024-04-12T05:55:30.737 INFO:tasks.workunit.client.0.smithi008.stdout:6/450: rename df/d21/c7d to df/d21/c90 0 2024-04-12T05:55:30.740 INFO:tasks.workunit.client.0.smithi008.stdout:2/390: mknod d6/db/d10/d23/d35/d3c/c85 0 2024-04-12T05:55:30.741 INFO:tasks.workunit.client.0.smithi008.stdout:4/273: dread d1/d8/f2d [0,4194304] 0 2024-04-12T05:55:30.743 INFO:tasks.workunit.client.0.smithi008.stdout:5/567: unlink d0/dd/d3d/f53 0 2024-04-12T05:55:30.743 INFO:tasks.workunit.client.0.smithi008.stdout:0/368: dwrite d2/d11/d36/f67 [0,4194304] 0 2024-04-12T05:55:30.745 INFO:tasks.workunit.client.0.smithi008.stdout:6/451: creat df/d10/d3a/d78/f91 x:0 0 0 2024-04-12T05:55:30.746 INFO:tasks.workunit.client.0.smithi008.stdout:1/336: sync 2024-04-12T05:55:30.747 INFO:tasks.workunit.client.0.smithi008.stdout:0/369: fsync d2/d24/f32 0 2024-04-12T05:55:30.748 INFO:tasks.workunit.client.0.smithi008.stdout:3/437: rename d9/d2e/l7d to d9/d10/d19/d22/l8b 0 2024-04-12T05:55:30.754 INFO:tasks.workunit.client.0.smithi008.stdout:0/370: chown d2/d11/d2a/f5c 0 1 2024-04-12T05:55:30.754 INFO:tasks.workunit.client.0.smithi008.stdout:5/568: creat d0/dc/d57/d9d/fb3 x:0 0 0 2024-04-12T05:55:30.755 INFO:tasks.workunit.client.0.smithi008.stdout:6/452: creat df/f92 x:0 0 0 2024-04-12T05:55:30.756 INFO:tasks.workunit.client.0.smithi008.stdout:3/438: symlink d9/d10/d19/l8c 0 2024-04-12T05:55:30.759 INFO:tasks.workunit.client.0.smithi008.stdout:2/391: dwrite d6/f43 [0,4194304] 0 2024-04-12T05:55:30.760 INFO:tasks.workunit.client.0.smithi008.stdout:5/569: creat d0/d40/fb4 x:0 0 0 2024-04-12T05:55:30.761 INFO:tasks.workunit.client.0.smithi008.stdout:1/337: mknod d4/dd/d17/d66/c6e 0 2024-04-12T05:55:30.762 INFO:tasks.workunit.client.0.smithi008.stdout:0/371: truncate d2/d11/f2e 3551754 0 2024-04-12T05:55:30.762 INFO:tasks.workunit.client.0.smithi008.stdout:6/453: symlink df/d10/d3a/l93 0 2024-04-12T05:55:30.763 INFO:tasks.workunit.client.0.smithi008.stdout:3/439: dread - d9/d2e/f59 zero size 2024-04-12T05:55:30.766 INFO:tasks.workunit.client.0.smithi008.stdout:1/338: stat d4/dd/d17/d28/d30 0 2024-04-12T05:55:30.766 INFO:tasks.workunit.client.0.smithi008.stdout:2/392: symlink d6/db/d60/d57/l86 0 2024-04-12T05:55:30.766 INFO:tasks.workunit.client.0.smithi008.stdout:6/454: creat df/d21/f94 x:0 0 0 2024-04-12T05:55:30.768 INFO:tasks.workunit.client.0.smithi008.stdout:3/440: creat d9/d11/d5b/f8d x:0 0 0 2024-04-12T05:55:30.768 INFO:tasks.workunit.client.0.smithi008.stdout:5/570: write d0/dc/f65 [514698,85665] 0 2024-04-12T05:55:30.769 INFO:tasks.workunit.client.0.smithi008.stdout:0/372: symlink d2/d24/d61/l70 0 2024-04-12T05:55:30.772 INFO:tasks.workunit.client.0.smithi008.stdout:6/455: mkdir df/d10/d3a/d95 0 2024-04-12T05:55:30.772 INFO:tasks.workunit.client.0.smithi008.stdout:2/393: rmdir d6/db/d10/d23 39 2024-04-12T05:55:30.772 INFO:tasks.workunit.client.0.smithi008.stdout:0/373: dread - d2/d11/d36/d42/f65 zero size 2024-04-12T05:55:30.772 INFO:tasks.workunit.client.0.smithi008.stdout:4/274: dread d1/f12 [0,4194304] 0 2024-04-12T05:55:30.773 INFO:tasks.workunit.client.0.smithi008.stdout:1/339: symlink d4/dd/d17/d28/d46/d4c/l6f 0 2024-04-12T05:55:30.775 INFO:tasks.workunit.client.0.smithi008.stdout:5/571: write d0/dc/d57/d9d/fb3 [560405,70736] 0 2024-04-12T05:55:30.775 INFO:tasks.workunit.client.0.smithi008.stdout:0/374: symlink d2/d11/l71 0 2024-04-12T05:55:30.776 INFO:tasks.workunit.client.0.smithi008.stdout:6/456: creat df/d21/f96 x:0 0 0 2024-04-12T05:55:30.782 INFO:tasks.workunit.client.0.smithi008.stdout:5/572: write d0/dc/d13/d1a/d3a/f63 [51537,74052] 0 2024-04-12T05:55:30.785 INFO:tasks.workunit.client.0.smithi008.stdout:6/457: truncate df/d10/d15/d4f/d50/f6c 623154 0 2024-04-12T05:55:30.785 INFO:tasks.workunit.client.0.smithi008.stdout:2/394: dwrite d6/db/d38/f75 [0,4194304] 0 2024-04-12T05:55:30.790 INFO:tasks.workunit.client.0.smithi008.stdout:0/375: getdents d2/d24 0 2024-04-12T05:55:30.793 INFO:tasks.workunit.client.0.smithi008.stdout:2/395: symlink d6/db/d38/l87 0 2024-04-12T05:55:30.795 INFO:tasks.workunit.client.0.smithi008.stdout:3/441: dread - d9/d10/d19/d65/f81 zero size 2024-04-12T05:55:30.797 INFO:tasks.workunit.client.0.smithi008.stdout:0/376: unlink d2/d11/d36/d47/l6c 0 2024-04-12T05:55:30.800 INFO:tasks.workunit.client.0.smithi008.stdout:3/442: write f7 [540285,60345] 0 2024-04-12T05:55:30.804 INFO:tasks.workunit.client.0.smithi008.stdout:1/340: rename d4/dd/d17/d28 to d4/d24/d62/d70 0 2024-04-12T05:55:30.808 INFO:tasks.workunit.client.0.smithi008.stdout:0/377: link d2/d11/d36/f67 d2/f72 0 2024-04-12T05:55:30.809 INFO:tasks.workunit.client.0.smithi008.stdout:2/396: dwrite d6/db/d10/d23/f4d [0,4194304] 0 2024-04-12T05:55:30.813 INFO:tasks.workunit.client.0.smithi008.stdout:1/341: creat d4/d33/f71 x:0 0 0 2024-04-12T05:55:30.815 INFO:tasks.workunit.client.0.smithi008.stdout:3/443: dwrite d9/d2e/f3f [4194304,4194304] 0 2024-04-12T05:55:30.816 INFO:tasks.workunit.client.0.smithi008.stdout:1/342: dread - d4/d1c/d4f/f6a zero size 2024-04-12T05:55:30.818 INFO:tasks.workunit.client.0.smithi008.stdout:3/444: readlink d9/d11/l6f 0 2024-04-12T05:55:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:30 smithi194 ceph-mon[49771]: pgmap v11: 65 pgs: 65 active+clean; 624 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 85 MiB/s wr, 421 op/s 2024-04-12T05:55:30.819 INFO:tasks.workunit.client.0.smithi008.stdout:5/573: dwrite d0/d29/f99 [0,4194304] 0 2024-04-12T05:55:30.822 INFO:tasks.workunit.client.0.smithi008.stdout:3/445: write d9/d2e/d3d/f6b [794491,81112] 0 2024-04-12T05:55:30.824 INFO:tasks.workunit.client.0.smithi008.stdout:0/378: creat d2/d11/d2a/f73 x:0 0 0 2024-04-12T05:55:30.824 INFO:tasks.workunit.client.0.smithi008.stdout:5/574: mkdir d0/dc/d57/d9d/db5 0 2024-04-12T05:55:30.826 INFO:tasks.workunit.client.0.smithi008.stdout:5/575: stat d0/dd/d3d/d5d/l7f 0 2024-04-12T05:55:30.826 INFO:tasks.workunit.client.0.smithi008.stdout:2/397: creat d6/db/d6f/f88 x:0 0 0 2024-04-12T05:55:30.827 INFO:tasks.workunit.client.0.smithi008.stdout:5/576: stat d0/d40/c6e 0 2024-04-12T05:55:30.827 INFO:tasks.workunit.client.0.smithi008.stdout:0/379: creat d2/d11/d36/d47/f74 x:0 0 0 2024-04-12T05:55:30.829 INFO:tasks.workunit.client.0.smithi008.stdout:2/398: write d6/db/d10/d23/d35/f49 [2156345,98008] 0 2024-04-12T05:55:30.831 INFO:tasks.workunit.client.0.smithi008.stdout:0/380: fsync d2/d11/d36/d42/f62 0 2024-04-12T05:55:30.832 INFO:tasks.workunit.client.0.smithi008.stdout:5/577: write d0/dc/d13/d1a/d35/f42 [5374143,47514] 0 2024-04-12T05:55:30.835 INFO:tasks.workunit.client.0.smithi008.stdout:2/399: creat d6/db/f89 x:0 0 0 2024-04-12T05:55:30.836 INFO:tasks.workunit.client.0.smithi008.stdout:2/400: write f0 [4453301,90570] 0 2024-04-12T05:55:30.836 INFO:tasks.workunit.client.0.smithi008.stdout:0/381: unlink d2/d24/l56 0 2024-04-12T05:55:30.839 INFO:tasks.workunit.client.0.smithi008.stdout:5/578: mknod d0/dc/d13/d1a/daf/cb6 0 2024-04-12T05:55:30.841 INFO:tasks.workunit.client.0.smithi008.stdout:0/382: write d2/d11/d2a/f5d [624910,85450] 0 2024-04-12T05:55:30.845 INFO:tasks.workunit.client.0.smithi008.stdout:2/401: dwrite d6/db/f56 [0,4194304] 0 2024-04-12T05:55:30.855 INFO:tasks.workunit.client.0.smithi008.stdout:1/343: dwrite d4/d6/d11/f69 [0,4194304] 0 2024-04-12T05:55:30.857 INFO:tasks.workunit.client.0.smithi008.stdout:5/579: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:30.857 INFO:tasks.workunit.client.0.smithi008.stdout:1/344: read d4/d24/f32 [1440954,60336] 0 2024-04-12T05:55:30.861 INFO:tasks.workunit.client.0.smithi008.stdout:2/402: truncate d6/f74 237371 0 2024-04-12T05:55:30.862 INFO:tasks.workunit.client.0.smithi008.stdout:2/403: dread - d6/db/d60/d57/f76 zero size 2024-04-12T05:55:30.864 INFO:tasks.workunit.client.0.smithi008.stdout:5/580: symlink d0/dd/d11/d38/d55/d89/lb7 0 2024-04-12T05:55:30.864 INFO:tasks.workunit.client.0.smithi008.stdout:1/345: rename d4/d6/d11/d6d to d4/d24/d62/d70/d30/d72 0 2024-04-12T05:55:30.869 INFO:tasks.workunit.client.0.smithi008.stdout:1/346: dread - d4/d1a/f5f zero size 2024-04-12T05:55:30.871 INFO:tasks.workunit.client.0.smithi008.stdout:5/581: getdents d0/dc/d57 0 2024-04-12T05:55:30.871 INFO:tasks.workunit.client.0.smithi008.stdout:3/446: dread d9/dd/f7f [0,4194304] 0 2024-04-12T05:55:30.873 INFO:tasks.workunit.client.0.smithi008.stdout:5/582: unlink d0/dd/c82 0 2024-04-12T05:55:30.878 INFO:tasks.workunit.client.0.smithi008.stdout:1/347: dwrite d4/d1c/d4f/f6a [0,4194304] 0 2024-04-12T05:55:30.878 INFO:tasks.workunit.client.0.smithi008.stdout:5/583: truncate d0/dc/d13/d1a/daf/d37/f3e 6102976 0 2024-04-12T05:55:30.879 INFO:tasks.workunit.client.0.smithi008.stdout:1/348: rmdir d4/d33 39 2024-04-12T05:55:30.884 INFO:tasks.workunit.client.0.smithi008.stdout:1/349: unlink d4/dd/d17/c20 0 2024-04-12T05:55:30.887 INFO:tasks.workunit.client.0.smithi008.stdout:1/350: fsync d4/d7/f4e 0 2024-04-12T05:55:30.888 INFO:tasks.workunit.client.0.smithi008.stdout:3/447: truncate d9/d11/f48 3041691 0 2024-04-12T05:55:30.889 INFO:tasks.workunit.client.0.smithi008.stdout:5/584: dwrite d0/dd/d3d/d5d/f78 [0,4194304] 0 2024-04-12T05:55:30.890 INFO:tasks.workunit.client.0.smithi008.stdout:3/448: chown d9/d1e/f57 3 1 2024-04-12T05:55:30.893 INFO:tasks.workunit.client.0.smithi008.stdout:1/351: mknod d4/d1a/d57/c73 0 2024-04-12T05:55:30.894 INFO:tasks.workunit.client.0.smithi008.stdout:5/585: symlink d0/dc/d13/d1a/daf/d37/lb8 0 2024-04-12T05:55:30.896 INFO:tasks.workunit.client.0.smithi008.stdout:3/449: rename d9/d11/d5b/f76 to d9/d10/d19/d65/f8e 0 2024-04-12T05:55:30.899 INFO:tasks.workunit.client.0.smithi008.stdout:3/450: fdatasync d9/f25 0 2024-04-12T05:55:30.901 INFO:tasks.workunit.client.0.smithi008.stdout:3/451: mknod d9/d44/c8f 0 2024-04-12T05:55:30.902 INFO:tasks.workunit.client.0.smithi008.stdout:3/452: chown c5 445204 1 2024-04-12T05:55:30.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:30 smithi008 ceph-mon[39722]: pgmap v11: 65 pgs: 65 active+clean; 624 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 7.3 MiB/s rd, 85 MiB/s wr, 421 op/s 2024-04-12T05:55:30.903 INFO:tasks.workunit.client.0.smithi008.stdout:5/586: rmdir d0/dc/d13 39 2024-04-12T05:55:30.905 INFO:tasks.workunit.client.0.smithi008.stdout:5/587: write d0/dd/d3d/f4b [1509234,79930] 0 2024-04-12T05:55:30.906 INFO:tasks.workunit.client.0.smithi008.stdout:5/588: write d0/dd/d11/fa7 [639112,64511] 0 2024-04-12T05:55:30.907 INFO:tasks.workunit.client.0.smithi008.stdout:5/589: chown d0/dc/d57 3 1 2024-04-12T05:55:30.908 INFO:tasks.workunit.client.0.smithi008.stdout:3/453: link d9/d2e/f3f d9/d10/d19/d65/f90 0 2024-04-12T05:55:30.910 INFO:tasks.workunit.client.0.smithi008.stdout:3/454: write d9/d10/f6a [4637687,86894] 0 2024-04-12T05:55:30.915 INFO:tasks.workunit.client.0.smithi008.stdout:3/455: link d9/d10/d19/d22/f28 d9/d37/d69/f91 0 2024-04-12T05:55:30.919 INFO:tasks.workunit.client.0.smithi008.stdout:5/590: truncate d0/dc/d13/d1a/d35/f4c 2180702 0 2024-04-12T05:55:30.922 INFO:tasks.workunit.client.0.smithi008.stdout:3/456: dwrite d9/d2e/d3d/f70 [0,4194304] 0 2024-04-12T05:55:30.925 INFO:tasks.workunit.client.0.smithi008.stdout:3/457: readlink d9/dd/l46 0 2024-04-12T05:55:30.930 INFO:tasks.workunit.client.0.smithi008.stdout:5/591: truncate d0/dd/d3d/d5d/faa 376787 0 2024-04-12T05:55:30.930 INFO:tasks.workunit.client.0.smithi008.stdout:3/458: write d9/d2e/d3d/f56 [491447,65110] 0 2024-04-12T05:55:30.932 INFO:tasks.workunit.client.0.smithi008.stdout:5/592: mkdir d0/d29/d96/db9 0 2024-04-12T05:55:30.933 INFO:tasks.workunit.client.0.smithi008.stdout:3/459: creat d9/f92 x:0 0 0 2024-04-12T05:55:30.935 INFO:tasks.workunit.client.0.smithi008.stdout:3/460: creat d9/d10/d19/d65/f93 x:0 0 0 2024-04-12T05:55:30.943 INFO:tasks.workunit.client.0.smithi008.stdout:5/593: truncate d0/d29/f99 2982514 0 2024-04-12T05:55:30.944 INFO:tasks.workunit.client.0.smithi008.stdout:3/461: truncate d9/dd/f36 3532970 0 2024-04-12T05:55:30.945 INFO:tasks.workunit.client.0.smithi008.stdout:5/594: rmdir d0/dd/d11/d38 39 2024-04-12T05:55:30.951 INFO:tasks.workunit.client.0.smithi008.stdout:3/462: creat d9/d2e/f94 x:0 0 0 2024-04-12T05:55:30.953 INFO:tasks.workunit.client.0.smithi008.stdout:5/595: dwrite d0/dc/d13/d1a/f9e [0,4194304] 0 2024-04-12T05:55:30.955 INFO:tasks.workunit.client.0.smithi008.stdout:3/463: mknod d9/c95 0 2024-04-12T05:55:30.960 INFO:tasks.workunit.client.0.smithi008.stdout:3/464: truncate d9/d10/f55 5226749 0 2024-04-12T05:55:30.961 INFO:tasks.workunit.client.0.smithi008.stdout:3/465: stat d9/d1e/c21 0 2024-04-12T05:55:30.961 INFO:tasks.workunit.client.0.smithi008.stdout:5/596: unlink d0/dc/d13/d1a/f3f 0 2024-04-12T05:55:30.962 INFO:tasks.workunit.client.0.smithi008.stdout:3/466: read d9/d2e/d3d/f73 [140043,12718] 0 2024-04-12T05:55:30.965 INFO:tasks.workunit.client.0.smithi008.stdout:3/467: mknod d9/c96 0 2024-04-12T05:55:30.972 INFO:tasks.workunit.client.0.smithi008.stdout:3/468: creat d9/d10/d19/d3a/d67/f97 x:0 0 0 2024-04-12T05:55:30.973 INFO:tasks.workunit.client.0.smithi008.stdout:3/469: chown d9/d10/d19 1966 1 2024-04-12T05:55:30.976 INFO:tasks.workunit.client.0.smithi008.stdout:3/470: creat d9/d11/f98 x:0 0 0 2024-04-12T05:55:30.978 INFO:tasks.workunit.client.0.smithi008.stdout:2/404: dread d6/f33 [0,4194304] 0 2024-04-12T05:55:30.978 INFO:tasks.workunit.client.0.smithi008.stdout:3/471: symlink d9/d37/l99 0 2024-04-12T05:55:30.979 INFO:tasks.workunit.client.0.smithi008.stdout:5/597: dread d0/dc/d13/d1a/daf/f2c [4194304,4194304] 0 2024-04-12T05:55:30.979 INFO:tasks.workunit.client.0.smithi008.stdout:3/472: stat d9/d11/d5b/f60 0 2024-04-12T05:55:30.981 INFO:tasks.workunit.client.0.smithi008.stdout:2/405: rmdir d6/db/d60 39 2024-04-12T05:55:30.983 INFO:tasks.workunit.client.0.smithi008.stdout:5/598: creat d0/d29/fba x:0 0 0 2024-04-12T05:55:30.983 INFO:tasks.workunit.client.0.smithi008.stdout:3/473: creat d9/d10/d19/d22/d2c/f9a x:0 0 0 2024-04-12T05:55:30.984 INFO:tasks.workunit.client.0.smithi008.stdout:5/599: readlink d0/dc/d13/d1a/daf/d37/lb8 0 2024-04-12T05:55:30.987 INFO:tasks.workunit.client.0.smithi008.stdout:5/600: unlink d0/dd/d3d/f8e 0 2024-04-12T05:55:30.989 INFO:tasks.workunit.client.0.smithi008.stdout:3/474: unlink d9/d37/l7e 0 2024-04-12T05:55:30.989 INFO:tasks.workunit.client.0.smithi008.stdout:2/406: getdents d6/db/d60/d84 0 2024-04-12T05:55:30.992 INFO:tasks.workunit.client.0.smithi008.stdout:5/601: chown d0/dc/f68 23101465 1 2024-04-12T05:55:30.998 INFO:tasks.workunit.client.0.smithi008.stdout:2/407: creat d6/db/d60/d84/f8a x:0 0 0 2024-04-12T05:55:30.999 INFO:tasks.workunit.client.0.smithi008.stdout:2/408: fdatasync f0 0 2024-04-12T05:55:31.002 INFO:tasks.workunit.client.0.smithi008.stdout:3/475: getdents d9/d37 0 2024-04-12T05:55:31.004 INFO:tasks.workunit.client.0.smithi008.stdout:5/602: dwrite d0/dc/d13/d1a/daf/f2c [0,4194304] 0 2024-04-12T05:55:31.008 INFO:tasks.workunit.client.0.smithi008.stdout:2/409: dread d6/db/d10/d23/d35/f49 [0,4194304] 0 2024-04-12T05:55:31.009 INFO:tasks.workunit.client.0.smithi008.stdout:5/603: fsync d0/dc/d13/d1a/f24 0 2024-04-12T05:55:31.009 INFO:tasks.workunit.client.0.smithi008.stdout:3/476: rename d9/d2e/l78 to d9/d10/l9b 0 2024-04-12T05:55:31.010 INFO:tasks.workunit.client.0.smithi008.stdout:5/604: chown d0/dc/fa8 7 1 2024-04-12T05:55:31.058 INFO:tasks.workunit.client.0.smithi008.stdout:8/328: sync 2024-04-12T05:55:31.058 INFO:tasks.workunit.client.0.smithi008.stdout:3/477: creat d9/d10/f9c x:0 0 0 2024-04-12T05:55:31.061 INFO:tasks.workunit.client.0.smithi008.stdout:3/478: write d9/d10/d19/d22/d2c/f5e [274079,7658] 0 2024-04-12T05:55:31.061 INFO:tasks.workunit.client.0.smithi008.stdout:5/605: unlink d0/dc/d13/d1a/f9e 0 2024-04-12T05:55:31.062 INFO:tasks.workunit.client.0.smithi008.stdout:8/329: creat dd/de/d16/f72 x:0 0 0 2024-04-12T05:55:31.064 INFO:tasks.workunit.client.0.smithi008.stdout:3/479: mknod d9/d2e/d3d/c9d 0 2024-04-12T05:55:31.065 INFO:tasks.workunit.client.0.smithi008.stdout:8/330: mkdir dd/de/d16/d73 0 2024-04-12T05:55:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:8/331: creat dd/d30/d3b/f74 x:0 0 0 2024-04-12T05:55:31.068 INFO:tasks.workunit.client.0.smithi008.stdout:8/332: stat dd/de/c15 0 2024-04-12T05:55:31.069 INFO:tasks.workunit.client.0.smithi008.stdout:8/333: dread - dd/de/f64 zero size 2024-04-12T05:55:31.071 INFO:tasks.workunit.client.0.smithi008.stdout:3/480: link c5 d9/d10/c9e 0 2024-04-12T05:55:31.074 INFO:tasks.workunit.client.0.smithi008.stdout:3/481: creat d9/d44/f9f x:0 0 0 2024-04-12T05:55:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:8/334: link c8 dd/de/d4c/d66/d6d/c75 0 2024-04-12T05:55:31.076 INFO:tasks.workunit.client.0.smithi008.stdout:5/606: dwrite d0/dc/d13/f48 [0,4194304] 0 2024-04-12T05:55:31.077 INFO:tasks.workunit.client.0.smithi008.stdout:8/335: dread - dd/de/d16/f6e zero size 2024-04-12T05:55:31.084 INFO:tasks.workunit.client.0.smithi008.stdout:5/607: truncate d0/dd/d3d/d5d/faa 922901 0 2024-04-12T05:55:31.086 INFO:tasks.workunit.client.0.smithi008.stdout:8/336: dwrite dd/de/d16/f6e [0,4194304] 0 2024-04-12T05:55:31.092 INFO:tasks.workunit.client.0.smithi008.stdout:5/608: dwrite d0/d29/d96/fb2 [0,4194304] 0 2024-04-12T05:55:31.093 INFO:tasks.workunit.client.0.smithi008.stdout:5/609: chown d0/dc/d13/d1a/daf 10816 1 2024-04-12T05:55:31.094 INFO:tasks.workunit.client.0.smithi008.stdout:5/610: rmdir d0/d40 39 2024-04-12T05:55:31.102 INFO:tasks.workunit.client.0.smithi008.stdout:5/611: dwrite d0/dc/d13/d1a/daf/d37/f45 [4194304,4194304] 0 2024-04-12T05:55:31.104 INFO:tasks.workunit.client.0.smithi008.stdout:5/612: chown d0/dc/d13/d1a/daf/f5 1169162185 1 2024-04-12T05:55:31.108 INFO:tasks.workunit.client.0.smithi008.stdout:5/613: symlink d0/dd/d11/d38/d55/d89/lbb 0 2024-04-12T05:55:31.110 INFO:tasks.workunit.client.0.smithi008.stdout:5/614: dread - d0/d40/fb4 zero size 2024-04-12T05:55:31.111 INFO:tasks.workunit.client.0.smithi008.stdout:5/615: chown d0/dc/d13/d1a/daf/cb6 29759456 1 2024-04-12T05:55:31.124 INFO:tasks.workunit.client.0.smithi008.stdout:5/616: dread d0/dc/d13/f54 [0,4194304] 0 2024-04-12T05:55:31.126 INFO:tasks.workunit.client.0.smithi008.stdout:5/617: mkdir d0/dc/d57/d9d/db5/dbc 0 2024-04-12T05:55:31.132 INFO:tasks.workunit.client.0.smithi008.stdout:5/618: dread d0/dc/f4f [0,4194304] 0 2024-04-12T05:55:31.133 INFO:tasks.workunit.client.0.smithi008.stdout:8/337: dread dd/de/d16/d50/d56/f5d [0,4194304] 0 2024-04-12T05:55:31.135 INFO:tasks.workunit.client.0.smithi008.stdout:5/619: write d0/dc/d13/d1a/d35/f94 [967628,86170] 0 2024-04-12T05:55:31.135 INFO:tasks.workunit.client.0.smithi008.stdout:8/338: dread - dd/f49 zero size 2024-04-12T05:55:31.137 INFO:tasks.workunit.client.0.smithi008.stdout:8/339: chown dd/de/d4c/l68 6 1 2024-04-12T05:55:31.138 INFO:tasks.workunit.client.0.smithi008.stdout:5/620: mkdir d0/dd/d11/d38/dbd 0 2024-04-12T05:55:31.139 INFO:tasks.workunit.client.0.smithi008.stdout:5/621: chown d0/d40/f5e 138888808 1 2024-04-12T05:55:31.139 INFO:tasks.workunit.client.0.smithi008.stdout:8/340: dread - dd/d4f/f5f zero size 2024-04-12T05:55:31.139 INFO:tasks.workunit.client.0.smithi008.stdout:8/341: fsync dd/de/f64 0 2024-04-12T05:55:31.141 INFO:tasks.workunit.client.0.smithi008.stdout:5/622: rmdir d0/dc/d13 39 2024-04-12T05:55:31.141 INFO:tasks.workunit.client.0.smithi008.stdout:8/342: write dd/d4f/d52/f69 [391003,124007] 0 2024-04-12T05:55:31.144 INFO:tasks.workunit.client.0.smithi008.stdout:5/623: symlink d0/dd/d3d/lbe 0 2024-04-12T05:55:31.148 INFO:tasks.workunit.client.0.smithi008.stdout:5/624: write d0/dc/d13/d1a/d3a/f63 [129823,91199] 0 2024-04-12T05:55:31.148 INFO:tasks.workunit.client.0.smithi008.stdout:8/343: dwrite dd/d4f/d52/f69 [0,4194304] 0 2024-04-12T05:55:31.152 INFO:tasks.workunit.client.0.smithi008.stdout:8/344: symlink dd/de/d16/d50/d56/d61/d6a/l76 0 2024-04-12T05:55:31.175 INFO:tasks.workunit.client.0.smithi008.stdout:5/625: dwrite d0/dc/d13/d1a/daf/f1b [0,4194304] 0 2024-04-12T05:55:31.182 INFO:tasks.workunit.client.0.smithi008.stdout:8/345: dread dd/f36 [0,4194304] 0 2024-04-12T05:55:31.185 INFO:tasks.workunit.client.0.smithi008.stdout:8/346: mkdir dd/de/d16/d50/d56/d61/d77 0 2024-04-12T05:55:31.187 INFO:tasks.workunit.client.0.smithi008.stdout:5/626: dread d0/dd/d3d/d5d/faa [0,4194304] 0 2024-04-12T05:55:31.188 INFO:tasks.workunit.client.0.smithi008.stdout:8/347: creat dd/d30/f78 x:0 0 0 2024-04-12T05:55:31.189 INFO:tasks.workunit.client.0.smithi008.stdout:5/627: creat d0/d29/fbf x:0 0 0 2024-04-12T05:55:31.190 INFO:tasks.workunit.client.0.smithi008.stdout:8/348: creat dd/d4f/d52/f79 x:0 0 0 2024-04-12T05:55:31.191 INFO:tasks.workunit.client.0.smithi008.stdout:8/349: read - dd/de/f64 zero size 2024-04-12T05:55:31.192 INFO:tasks.workunit.client.0.smithi008.stdout:5/628: rmdir d0/dc/d13/d1a 39 2024-04-12T05:55:31.193 INFO:tasks.workunit.client.0.smithi008.stdout:8/350: truncate dd/d30/f35 796176 0 2024-04-12T05:55:31.194 INFO:tasks.workunit.client.0.smithi008.stdout:8/351: chown dd/d4f/d52/f60 1034228271 1 2024-04-12T05:55:31.196 INFO:tasks.workunit.client.0.smithi008.stdout:5/629: unlink d0/dc/d13/d1a/daf/d37/c61 0 2024-04-12T05:55:31.198 INFO:tasks.workunit.client.0.smithi008.stdout:5/630: mknod d0/dd/d5c/cc0 0 2024-04-12T05:55:31.201 INFO:tasks.workunit.client.0.smithi008.stdout:8/352: rename dd/c1b to dd/de/d16/d50/d5e/c7a 0 2024-04-12T05:55:31.202 INFO:tasks.workunit.client.0.smithi008.stdout:9/459: sync 2024-04-12T05:55:31.202 INFO:tasks.workunit.client.0.smithi008.stdout:7/521: sync 2024-04-12T05:55:31.202 INFO:tasks.workunit.client.0.smithi008.stdout:6/458: sync 2024-04-12T05:55:31.208 INFO:tasks.workunit.client.0.smithi008.stdout:6/459: unlink df/d10/d20/f6a 0 2024-04-12T05:55:31.209 INFO:tasks.workunit.client.0.smithi008.stdout:6/460: chown df/d10/d3a 1908890 1 2024-04-12T05:55:31.209 INFO:tasks.workunit.client.0.smithi008.stdout:8/353: dwrite dd/de/f47 [0,4194304] 0 2024-04-12T05:55:31.211 INFO:tasks.workunit.client.0.smithi008.stdout:5/631: rename d0/dd/d3d/d5d/f78 to d0/dc/fc1 0 2024-04-12T05:55:31.215 INFO:tasks.workunit.client.0.smithi008.stdout:9/460: creat d3/df/d13/d21/d2d/d32/d52/f9f x:0 0 0 2024-04-12T05:55:31.218 INFO:tasks.workunit.client.0.smithi008.stdout:6/461: mknod df/d21/d4c/c97 0 2024-04-12T05:55:31.219 INFO:tasks.workunit.client.0.smithi008.stdout:8/354: creat dd/d4f/f7b x:0 0 0 2024-04-12T05:55:31.221 INFO:tasks.workunit.client.0.smithi008.stdout:9/461: unlink d3/df/d13/c42 0 2024-04-12T05:55:31.223 INFO:tasks.workunit.client.0.smithi008.stdout:9/462: readlink d3/d22/d2b/d58/l6f 0 2024-04-12T05:55:31.224 INFO:tasks.workunit.client.0.smithi008.stdout:8/355: link dd/d30/l37 dd/de/d4c/d66/d6c/l7c 0 2024-04-12T05:55:31.225 INFO:tasks.workunit.client.0.smithi008.stdout:9/463: chown d3/d22/d2b/d4e/l88 785571098 1 2024-04-12T05:55:31.227 INFO:tasks.workunit.client.0.smithi008.stdout:5/632: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:31.230 INFO:tasks.workunit.client.0.smithi008.stdout:9/464: symlink d3/la0 0 2024-04-12T05:55:31.232 INFO:tasks.workunit.client.0.smithi008.stdout:5/633: write d0/d29/fbf [715658,62089] 0 2024-04-12T05:55:31.232 INFO:tasks.workunit.client.0.smithi008.stdout:7/522: write d1/d5/da/d3c/d44/d8a/f3b [732058,57107] 0 2024-04-12T05:55:31.235 INFO:tasks.workunit.client.0.smithi008.stdout:5/634: fdatasync d0/d40/fb4 0 2024-04-12T05:55:31.239 INFO:tasks.workunit.client.0.smithi008.stdout:8/356: dwrite dd/de/d16/f72 [0,4194304] 0 2024-04-12T05:55:31.248 INFO:tasks.workunit.client.0.smithi008.stdout:7/523: symlink d1/d5/da/d60/da1/lbb 0 2024-04-12T05:55:31.249 INFO:tasks.workunit.client.0.smithi008.stdout:5/635: write d0/dc/d13/d1a/daf/f21 [3055191,86822] 0 2024-04-12T05:55:31.250 INFO:tasks.workunit.client.0.smithi008.stdout:5/636: dread - d0/dc/fa9 zero size 2024-04-12T05:55:31.254 INFO:tasks.workunit.client.0.smithi008.stdout:8/357: creat dd/de/d16/d50/d56/d61/d6a/f7d x:0 0 0 2024-04-12T05:55:31.259 INFO:tasks.workunit.client.0.smithi008.stdout:5/637: creat d0/d29/d96/fc2 x:0 0 0 2024-04-12T05:55:31.260 INFO:tasks.workunit.client.0.smithi008.stdout:7/524: rename d1/d15/d43/d54/d58/db7/fa3 to d1/d5/da/d11/fbc 0 2024-04-12T05:55:31.261 INFO:tasks.workunit.client.0.smithi008.stdout:9/465: mknod d3/df/d13/d21/d2d/ca1 0 2024-04-12T05:55:31.264 INFO:tasks.workunit.client.0.smithi008.stdout:5/638: dread - d0/dc/d13/d1a/d35/fa6 zero size 2024-04-12T05:55:31.268 INFO:tasks.workunit.client.0.smithi008.stdout:8/358: getdents dd/de/d16/d50/d56/d61/d6a 0 2024-04-12T05:55:31.269 INFO:tasks.workunit.client.0.smithi008.stdout:6/462: dwrite df/d10/d20/f89 [4194304,4194304] 0 2024-04-12T05:55:31.273 INFO:tasks.workunit.client.0.smithi008.stdout:9/466: unlink d3/f5 0 2024-04-12T05:55:31.275 INFO:tasks.workunit.client.0.smithi008.stdout:8/359: creat dd/d30/d32/f7e x:0 0 0 2024-04-12T05:55:31.277 INFO:tasks.workunit.client.0.smithi008.stdout:7/525: dwrite d1/d5/da/f62 [0,4194304] 0 2024-04-12T05:55:31.283 INFO:tasks.workunit.client.0.smithi008.stdout:5/639: creat d0/dc/d13/d1a/daf/fc3 x:0 0 0 2024-04-12T05:55:31.286 INFO:tasks.workunit.client.0.smithi008.stdout:9/467: read d3/df/d13/d21/d2d/d32/f34 [2810548,31886] 0 2024-04-12T05:55:31.289 INFO:tasks.workunit.client.0.smithi008.stdout:8/360: dread dd/d30/d32/f3e [0,4194304] 0 2024-04-12T05:55:31.289 INFO:tasks.workunit.client.0.smithi008.stdout:5/640: getdents d0/d29/d96/db9 0 2024-04-12T05:55:31.290 INFO:tasks.workunit.client.0.smithi008.stdout:7/526: rename d1/d15/f66 to d1/d5/da/d78/fbd 0 2024-04-12T05:55:31.291 INFO:tasks.workunit.client.0.smithi008.stdout:6/463: dwrite df/d10/d15/d79/f8f [0,4194304] 0 2024-04-12T05:55:31.295 INFO:tasks.workunit.client.0.smithi008.stdout:8/361: dread - dd/d30/d32/f7e zero size 2024-04-12T05:55:31.296 INFO:tasks.workunit.client.0.smithi008.stdout:9/468: rename d3/d22/f3e to d3/df/d13/d21/d2d/d32/d52/fa2 0 2024-04-12T05:55:31.299 INFO:tasks.workunit.client.0.smithi008.stdout:9/469: dread - d3/df/d13/d21/f91 zero size 2024-04-12T05:55:31.300 INFO:tasks.workunit.client.0.smithi008.stdout:5/641: symlink d0/dd/lc4 0 2024-04-12T05:55:31.304 INFO:tasks.workunit.client.0.smithi008.stdout:6/464: dwrite df/d21/f81 [0,4194304] 0 2024-04-12T05:55:31.305 INFO:tasks.workunit.client.0.smithi008.stdout:5/642: write d0/dc/d13/d1a/daf/d37/f45 [7849614,2690] 0 2024-04-12T05:55:31.307 INFO:tasks.workunit.client.0.smithi008.stdout:9/470: creat d3/df/d13/d14/fa3 x:0 0 0 2024-04-12T05:55:31.308 INFO:tasks.workunit.client.0.smithi008.stdout:9/471: chown d3/df/d13/d14/l2c 13622 1 2024-04-12T05:55:31.309 INFO:tasks.workunit.client.0.smithi008.stdout:6/465: rmdir df/d21/d4c 39 2024-04-12T05:55:31.310 INFO:tasks.workunit.client.0.smithi008.stdout:6/466: stat cc 0 2024-04-12T05:55:31.311 INFO:tasks.workunit.client.0.smithi008.stdout:9/472: mkdir d3/df/d13/d21/d5c/da4 0 2024-04-12T05:55:31.312 INFO:tasks.workunit.client.0.smithi008.stdout:5/643: getdents d0/dd/d11/d38/dbd 0 2024-04-12T05:55:31.312 INFO:tasks.workunit.client.0.smithi008.stdout:5/644: fsync d0/f97 0 2024-04-12T05:55:31.313 INFO:tasks.workunit.client.0.smithi008.stdout:6/467: rmdir df/d10/d3a/d78 39 2024-04-12T05:55:31.322 INFO:tasks.workunit.client.0.smithi008.stdout:9/473: symlink d3/df/d13/d21/d5c/la5 0 2024-04-12T05:55:31.323 INFO:tasks.workunit.client.0.smithi008.stdout:5/645: creat d0/dd/d11/d38/d55/d89/fc5 x:0 0 0 2024-04-12T05:55:31.323 INFO:tasks.workunit.client.0.smithi008.stdout:7/527: write d1/d5/da/d78/fbd [225811,9791] 0 2024-04-12T05:55:31.324 INFO:tasks.workunit.client.0.smithi008.stdout:9/474: read d3/d22/d2b/f41 [71179,129363] 0 2024-04-12T05:55:31.327 INFO:tasks.workunit.client.0.smithi008.stdout:6/468: creat df/d10/d3a/f98 x:0 0 0 2024-04-12T05:55:31.332 INFO:tasks.workunit.client.0.smithi008.stdout:5/646: rename d0/dd/d11/f19 to d0/dc/d13/fc6 0 2024-04-12T05:55:31.334 INFO:tasks.workunit.client.0.smithi008.stdout:9/475: link d3/df/d13/d21/d2d/d32/f7d d3/df/d13/d21/d2d/fa6 0 2024-04-12T05:55:31.334 INFO:tasks.workunit.client.0.smithi008.stdout:7/528: dwrite d1/d5/da/f2a [0,4194304] 0 2024-04-12T05:55:31.336 INFO:tasks.workunit.client.0.smithi008.stdout:6/469: mknod df/d10/d3a/c99 0 2024-04-12T05:55:31.336 INFO:tasks.workunit.client.0.smithi008.stdout:5/647: dread - d0/dd/d11/d38/fa5 zero size 2024-04-12T05:55:31.340 INFO:tasks.workunit.client.0.smithi008.stdout:6/470: creat df/d8c/f9a x:0 0 0 2024-04-12T05:55:31.343 INFO:tasks.workunit.client.0.smithi008.stdout:7/529: creat d1/d5/da/d3c/fbe x:0 0 0 2024-04-12T05:55:31.345 INFO:tasks.workunit.client.0.smithi008.stdout:9/476: dwrite d3/df/d13/f51 [0,4194304] 0 2024-04-12T05:55:31.348 INFO:tasks.workunit.client.0.smithi008.stdout:6/471: mkdir df/d10/d3a/d9b 0 2024-04-12T05:55:31.349 INFO:tasks.workunit.client.0.smithi008.stdout:5/648: dread d0/dc/d13/f54 [0,4194304] 0 2024-04-12T05:55:31.355 INFO:tasks.workunit.client.0.smithi008.stdout:9/477: symlink d3/df/d13/d21/d5c/d8f/la7 0 2024-04-12T05:55:31.358 INFO:tasks.workunit.client.0.smithi008.stdout:6/472: dread df/d21/f43 [0,4194304] 0 2024-04-12T05:55:31.360 INFO:tasks.workunit.client.0.smithi008.stdout:9/478: chown d3/f25 1550536 1 2024-04-12T05:55:31.360 INFO:tasks.workunit.client.0.smithi008.stdout:7/530: dwrite d1/d5/da/d3c/d44/d8a/f9b [0,4194304] 0 2024-04-12T05:55:31.362 INFO:tasks.workunit.client.0.smithi008.stdout:6/473: fsync df/f67 0 2024-04-12T05:55:31.367 INFO:tasks.workunit.client.0.smithi008.stdout:7/531: rmdir d1/d5/da/d60/da1 39 2024-04-12T05:55:31.367 INFO:tasks.workunit.client.0.smithi008.stdout:9/479: mkdir d3/d22/d2b/d58/da8 0 2024-04-12T05:55:31.368 INFO:tasks.workunit.client.0.smithi008.stdout:6/474: write df/f92 [590418,51314] 0 2024-04-12T05:55:31.369 INFO:tasks.workunit.client.0.smithi008.stdout:7/532: chown d1/f55 2788040 1 2024-04-12T05:55:31.373 INFO:tasks.workunit.client.0.smithi008.stdout:7/533: fdatasync d1/d5/da/d3c/d44/f51 0 2024-04-12T05:55:31.374 INFO:tasks.workunit.client.0.smithi008.stdout:9/480: dread - d3/df/d10/f96 zero size 2024-04-12T05:55:31.375 INFO:tasks.workunit.client.0.smithi008.stdout:6/475: write df/d21/d4c/f52 [1988332,31289] 0 2024-04-12T05:55:31.376 INFO:tasks.workunit.client.0.smithi008.stdout:5/649: dwrite d0/dd/d11/d38/f43 [0,4194304] 0 2024-04-12T05:55:31.380 INFO:tasks.workunit.client.0.smithi008.stdout:9/481: rmdir d3/df/d13/d21/d5c/d8f 39 2024-04-12T05:55:31.383 INFO:tasks.workunit.client.0.smithi008.stdout:5/650: truncate d0/d29/d96/fc2 739469 0 2024-04-12T05:55:31.386 INFO:tasks.workunit.client.0.smithi008.stdout:5/651: chown d0/dd/d11/d38/c76 985094132 1 2024-04-12T05:55:31.387 INFO:tasks.workunit.client.0.smithi008.stdout:7/534: dwrite d1/d5/da/f97 [0,4194304] 0 2024-04-12T05:55:31.388 INFO:tasks.workunit.client.0.smithi008.stdout:5/652: chown d0/dd/c49 6 1 2024-04-12T05:55:31.389 INFO:tasks.workunit.client.0.smithi008.stdout:6/476: truncate df/d10/d15/f13 3744853 0 2024-04-12T05:55:31.391 INFO:tasks.workunit.client.0.smithi008.stdout:9/482: rename d3/df/d13/f7a to d3/d22/d2b/d58/da8/fa9 0 2024-04-12T05:55:31.393 INFO:tasks.workunit.client.0.smithi008.stdout:9/483: chown d3/d22/c69 103924800 1 2024-04-12T05:55:31.400 INFO:tasks.workunit.client.0.smithi008.stdout:6/477: fdatasync df/d10/d15/d4f/d50/f51 0 2024-04-12T05:55:31.401 INFO:tasks.workunit.client.0.smithi008.stdout:7/535: dwrite d1/d15/d43/d54/d58/db7/fac [4194304,4194304] 0 2024-04-12T05:55:31.402 INFO:tasks.workunit.client.0.smithi008.stdout:9/484: chown d3/f35 1729357 1 2024-04-12T05:55:31.405 INFO:tasks.workunit.client.0.smithi008.stdout:5/653: getdents d0/dd 0 2024-04-12T05:55:31.406 INFO:tasks.workunit.client.0.smithi008.stdout:6/478: chown df/d10/d3a/c5f 1 1 2024-04-12T05:55:31.408 INFO:tasks.workunit.client.0.smithi008.stdout:7/536: symlink d1/d15/d43/d54/d58/d5e/d7f/lbf 0 2024-04-12T05:55:31.409 INFO:tasks.workunit.client.0.smithi008.stdout:6/479: getdents df/d10/d3a/d9b 0 2024-04-12T05:55:31.410 INFO:tasks.workunit.client.0.smithi008.stdout:7/537: fdatasync d1/d5/da/d1d/f5c 0 2024-04-12T05:55:31.413 INFO:tasks.workunit.client.0.smithi008.stdout:6/480: symlink df/d10/d3a/d78/l9c 0 2024-04-12T05:55:31.416 INFO:tasks.workunit.client.0.smithi008.stdout:6/481: creat df/d10/d15/f9d x:0 0 0 2024-04-12T05:55:31.417 INFO:tasks.workunit.client.0.smithi008.stdout:9/485: write d3/df/d13/d21/d2d/d32/f7d [790282,51595] 0 2024-04-12T05:55:31.419 INFO:tasks.workunit.client.0.smithi008.stdout:9/486: stat d3/df/d13/d14/l5f 0 2024-04-12T05:55:31.419 INFO:tasks.workunit.client.0.smithi008.stdout:6/482: symlink df/d10/d15/d4f/d50/l9e 0 2024-04-12T05:55:31.420 INFO:tasks.workunit.client.0.smithi008.stdout:9/487: readlink d3/df/d13/d21/d5c/l5e 0 2024-04-12T05:55:31.421 INFO:tasks.workunit.client.0.smithi008.stdout:6/483: creat df/d10/d15/d75/f9f x:0 0 0 2024-04-12T05:55:31.422 INFO:tasks.workunit.client.0.smithi008.stdout:9/488: truncate d3/d22/d2b/d77/f9c 1040395 0 2024-04-12T05:55:31.425 INFO:tasks.workunit.client.0.smithi008.stdout:6/484: creat df/d10/d3a/fa0 x:0 0 0 2024-04-12T05:55:31.427 INFO:tasks.workunit.client.0.smithi008.stdout:9/489: truncate d3/df/d13/f7f 612564 0 2024-04-12T05:55:31.428 INFO:tasks.workunit.client.0.smithi008.stdout:6/485: dread df/d10/d15/d4f/d50/f6e [0,4194304] 0 2024-04-12T05:55:31.430 INFO:tasks.workunit.client.0.smithi008.stdout:9/490: creat d3/df/d13/d21/d2d/d30/faa x:0 0 0 2024-04-12T05:55:31.434 INFO:tasks.workunit.client.0.smithi008.stdout:9/491: mkdir d3/df/d13/d21/d5c/d8f/dab 0 2024-04-12T05:55:31.437 INFO:tasks.workunit.client.0.smithi008.stdout:9/492: symlink d3/df/d13/d21/d5c/d8f/dab/lac 0 2024-04-12T05:55:31.438 INFO:tasks.workunit.client.0.smithi008.stdout:9/493: symlink d3/d22/d2b/d58/d36/lad 0 2024-04-12T05:55:31.440 INFO:tasks.workunit.client.0.smithi008.stdout:9/494: mkdir d3/dae 0 2024-04-12T05:55:31.447 INFO:tasks.workunit.client.0.smithi008.stdout:9/495: write d3/d22/d2b/d4a/d62/f9a [55112,14529] 0 2024-04-12T05:55:31.449 INFO:tasks.workunit.client.0.smithi008.stdout:9/496: write d3/d22/d2b/d4a/d62/f9a [997162,87941] 0 2024-04-12T05:55:31.451 INFO:tasks.workunit.client.0.smithi008.stdout:9/497: creat d3/d22/d2b/d4e/faf x:0 0 0 2024-04-12T05:55:31.453 INFO:tasks.workunit.client.0.smithi008.stdout:9/498: mkdir d3/df/d13/d21/d2d/d32/d52/db0 0 2024-04-12T05:55:31.455 INFO:tasks.workunit.client.0.smithi008.stdout:9/499: write d3/d6a/f9e [691572,128923] 0 2024-04-12T05:55:31.457 INFO:tasks.workunit.client.0.smithi008.stdout:9/500: fdatasync d3/d22/d2b/d4a/d62/f9a 0 2024-04-12T05:55:31.459 INFO:tasks.workunit.client.0.smithi008.stdout:9/501: rmdir d3/df/d13/d21/d2d/d30 39 2024-04-12T05:55:31.466 INFO:tasks.workunit.client.0.smithi008.stdout:9/502: dwrite d3/fd [0,4194304] 0 2024-04-12T05:55:31.473 INFO:tasks.workunit.client.0.smithi008.stdout:4/275: sync 2024-04-12T05:55:31.476 INFO:tasks.workunit.client.0.smithi008.stdout:9/503: dwrite d3/df/f28 [0,4194304] 0 2024-04-12T05:55:31.478 INFO:tasks.workunit.client.0.smithi008.stdout:9/504: readlink d3/df/d13/d21/d5c/d8f/la7 0 2024-04-12T05:55:31.486 INFO:tasks.workunit.client.0.smithi008.stdout:9/505: write d3/d22/d2b/d58/d36/f3c [1041234,23395] 0 2024-04-12T05:55:31.487 INFO:tasks.workunit.client.0.smithi008.stdout:4/276: dread d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:31.490 INFO:tasks.workunit.client.0.smithi008.stdout:9/506: mkdir d3/d22/d2b/d58/d90/db1 0 2024-04-12T05:55:31.492 INFO:tasks.workunit.client.0.smithi008.stdout:4/277: rmdir d1/d22/d29/d30/d47/d28/d3e 0 2024-04-12T05:55:31.494 INFO:tasks.workunit.client.0.smithi008.stdout:9/507: getdents d3/d22/d2b/d4a/d62 0 2024-04-12T05:55:31.496 INFO:tasks.workunit.client.0.smithi008.stdout:4/278: dread d1/f12 [4194304,4194304] 0 2024-04-12T05:55:31.498 INFO:tasks.workunit.client.0.smithi008.stdout:4/279: dread d1/d8/f37 [0,4194304] 0 2024-04-12T05:55:31.500 INFO:tasks.workunit.client.0.smithi008.stdout:4/280: mknod d1/d22/d29/c4b 0 2024-04-12T05:55:31.510 INFO:tasks.workunit.client.0.smithi008.stdout:4/281: dwrite d1/f40 [0,4194304] 0 2024-04-12T05:55:31.512 INFO:tasks.workunit.client.0.smithi008.stdout:4/282: creat d1/d22/f4c x:0 0 0 2024-04-12T05:55:31.521 INFO:tasks.workunit.client.0.smithi008.stdout:4/283: write d1/d22/d29/f41 [986331,65853] 0 2024-04-12T05:55:31.522 INFO:tasks.workunit.client.0.smithi008.stdout:4/284: write d1/f20 [1821250,100249] 0 2024-04-12T05:55:31.531 INFO:tasks.workunit.client.0.smithi008.stdout:4/285: dwrite d1/db/f18 [0,4194304] 0 2024-04-12T05:55:31.539 INFO:tasks.workunit.client.0.smithi008.stdout:4/286: dwrite d1/d22/d29/d30/d47/f48 [0,4194304] 0 2024-04-12T05:55:31.541 INFO:tasks.workunit.client.0.smithi008.stdout:4/287: write d1/d22/f4c [285465,104578] 0 2024-04-12T05:55:31.551 INFO:tasks.workunit.client.0.smithi008.stdout:4/288: truncate d1/d22/d29/d30/d47/f48 3834347 0 2024-04-12T05:55:31.553 INFO:tasks.workunit.client.0.smithi008.stdout:4/289: mknod d1/d8/c4d 0 2024-04-12T05:55:31.555 INFO:tasks.workunit.client.0.smithi008.stdout:4/290: symlink d1/l4e 0 2024-04-12T05:55:31.558 INFO:tasks.workunit.client.0.smithi008.stdout:4/291: write d1/d22/d29/d30/d47/f48 [3332946,36602] 0 2024-04-12T05:55:31.561 INFO:tasks.workunit.client.0.smithi008.stdout:4/292: mkdir d1/d4f 0 2024-04-12T05:55:31.562 INFO:tasks.workunit.client.0.smithi008.stdout:4/293: chown d1/d8/f23 2770 1 2024-04-12T05:55:31.564 INFO:tasks.workunit.client.0.smithi008.stdout:4/294: symlink d1/d8/l50 0 2024-04-12T05:55:31.566 INFO:tasks.workunit.client.0.smithi008.stdout:4/295: mknod d1/d22/d29/d30/c51 0 2024-04-12T05:55:31.567 INFO:tasks.workunit.client.0.smithi008.stdout:4/296: truncate d1/d8/f25 238686 0 2024-04-12T05:55:31.568 INFO:tasks.workunit.client.0.smithi008.stdout:4/297: fsync d1/d8/f42 0 2024-04-12T05:55:31.569 INFO:tasks.workunit.client.0.smithi008.stdout:4/298: write d1/d22/d29/f41 [162695,12055] 0 2024-04-12T05:55:31.573 INFO:tasks.workunit.client.0.smithi008.stdout:4/299: creat d1/d22/d29/d30/f52 x:0 0 0 2024-04-12T05:55:31.574 INFO:tasks.workunit.client.0.smithi008.stdout:4/300: dread - d1/d22/d29/d30/f34 zero size 2024-04-12T05:55:31.577 INFO:tasks.workunit.client.0.smithi008.stdout:4/301: link d1/d22/d29/d30/d47/f48 d1/d4f/f53 0 2024-04-12T05:55:31.578 INFO:tasks.workunit.client.0.smithi008.stdout:4/302: readlink d1/l7 0 2024-04-12T05:55:31.578 INFO:tasks.workunit.client.0.smithi008.stdout:4/303: chown d1/d22/d29/d30/d47/d28/l35 441948 1 2024-04-12T05:55:31.588 INFO:tasks.workunit.client.0.smithi008.stdout:4/304: dread d1/f20 [0,4194304] 0 2024-04-12T05:55:31.591 INFO:tasks.workunit.client.0.smithi008.stdout:4/305: rename d1/d22/d29/d30/d47/d28/l35 to d1/d22/d29/d46/l54 0 2024-04-12T05:55:31.592 INFO:tasks.workunit.client.0.smithi008.stdout:4/306: read d1/f40 [2519617,119143] 0 2024-04-12T05:55:31.600 INFO:tasks.workunit.client.0.smithi008.stdout:4/307: dwrite d1/f20 [0,4194304] 0 2024-04-12T05:55:31.602 INFO:tasks.workunit.client.0.smithi008.stdout:4/308: read d1/d8/f25 [207383,71578] 0 2024-04-12T05:55:31.604 INFO:tasks.workunit.client.0.smithi008.stdout:4/309: creat d1/d8/f55 x:0 0 0 2024-04-12T05:55:31.609 INFO:tasks.workunit.client.0.smithi008.stdout:4/310: dread d1/f20 [0,4194304] 0 2024-04-12T05:55:31.616 INFO:tasks.workunit.client.0.smithi008.stdout:4/311: write d1/d4f/f53 [1130340,51469] 0 2024-04-12T05:55:31.617 INFO:tasks.workunit.client.0.smithi008.stdout:4/312: write d1/f20 [2515057,97453] 0 2024-04-12T05:55:31.620 INFO:tasks.workunit.client.0.smithi008.stdout:4/313: truncate d1/d22/d29/d30/f52 420349 0 2024-04-12T05:55:31.622 INFO:tasks.workunit.client.0.smithi008.stdout:4/314: mkdir d1/d22/d29/d46/d56 0 2024-04-12T05:55:31.625 INFO:tasks.workunit.client.0.smithi008.stdout:4/315: getdents d1/d22/d29 0 2024-04-12T05:55:31.628 INFO:tasks.workunit.client.0.smithi008.stdout:4/316: dread d1/d8/f2d [0,4194304] 0 2024-04-12T05:55:31.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/317: dwrite d1/f20 [0,4194304] 0 2024-04-12T05:55:31.635 INFO:tasks.workunit.client.0.smithi008.stdout:4/318: chown d1/l7 671 1 2024-04-12T05:55:31.644 INFO:tasks.workunit.client.0.smithi008.stdout:4/319: rename d1/d4f/f53 to d1/d22/d29/d30/f57 0 2024-04-12T05:55:31.644 INFO:tasks.workunit.client.0.smithi008.stdout:4/320: readlink d1/d22/d29/l33 0 2024-04-12T05:55:31.646 INFO:tasks.workunit.client.0.smithi008.stdout:4/321: write d1/d8/f15 [1093447,20328] 0 2024-04-12T05:55:31.651 INFO:tasks.workunit.client.0.smithi008.stdout:4/322: write d1/db/f16 [3623994,92937] 0 2024-04-12T05:55:31.652 INFO:tasks.workunit.client.0.smithi008.stdout:4/323: write d1/d22/f4c [199863,126835] 0 2024-04-12T05:55:31.653 INFO:tasks.workunit.client.0.smithi008.stdout:4/324: truncate d1/d22/d29/f3a 398605 0 2024-04-12T05:55:31.654 INFO:tasks.workunit.client.0.smithi008.stdout:3/482: sync 2024-04-12T05:55:31.655 INFO:tasks.workunit.client.0.smithi008.stdout:8/362: sync 2024-04-12T05:55:31.655 INFO:tasks.workunit.client.0.smithi008.stdout:1/352: sync 2024-04-12T05:55:31.655 INFO:tasks.workunit.client.0.smithi008.stdout:0/383: sync 2024-04-12T05:55:31.655 INFO:tasks.workunit.client.0.smithi008.stdout:2/410: sync 2024-04-12T05:55:31.655 INFO:tasks.workunit.client.0.smithi008.stdout:1/353: readlink d4/d24/l3a 0 2024-04-12T05:55:31.656 INFO:tasks.workunit.client.0.smithi008.stdout:4/325: unlink d1/d22/l2b 0 2024-04-12T05:55:31.657 INFO:tasks.workunit.client.0.smithi008.stdout:2/411: write f0 [2957891,6225] 0 2024-04-12T05:55:31.660 INFO:tasks.workunit.client.0.smithi008.stdout:3/483: symlink d9/d10/d19/d3a/d67/la0 0 2024-04-12T05:55:31.660 INFO:tasks.workunit.client.0.smithi008.stdout:1/354: read - d4/d33/d37/f55 zero size 2024-04-12T05:55:31.661 INFO:tasks.workunit.client.0.smithi008.stdout:3/484: fsync d9/f25 0 2024-04-12T05:55:31.663 INFO:tasks.workunit.client.0.smithi008.stdout:4/326: mknod d1/d22/d29/d30/c58 0 2024-04-12T05:55:31.664 INFO:tasks.workunit.client.0.smithi008.stdout:2/412: symlink d6/db/d10/l8b 0 2024-04-12T05:55:31.665 INFO:tasks.workunit.client.0.smithi008.stdout:8/363: dread dd/f36 [0,4194304] 0 2024-04-12T05:55:31.666 INFO:tasks.workunit.client.0.smithi008.stdout:2/413: read d6/db/f56 [657864,94964] 0 2024-04-12T05:55:31.668 INFO:tasks.workunit.client.0.smithi008.stdout:1/355: rename d4/d24/d62/d70 to d4/d33/d74 0 2024-04-12T05:55:31.668 INFO:tasks.workunit.client.0.smithi008.stdout:2/414: stat d6/db/d38/f63 0 2024-04-12T05:55:31.668 INFO:tasks.workunit.client.0.smithi008.stdout:8/364: truncate dd/d30/d32/f3e 1284072 0 2024-04-12T05:55:31.671 INFO:tasks.workunit.client.0.smithi008.stdout:4/327: symlink d1/d22/d29/d30/l59 0 2024-04-12T05:55:31.672 INFO:tasks.workunit.client.0.smithi008.stdout:0/384: dread d2/d11/d2a/f5d [0,4194304] 0 2024-04-12T05:55:31.673 INFO:tasks.workunit.client.0.smithi008.stdout:3/485: link d9/d44/c79 d9/d11/ca1 0 2024-04-12T05:55:31.674 INFO:tasks.workunit.client.0.smithi008.stdout:0/385: readlink d2/d11/l71 0 2024-04-12T05:55:31.678 INFO:tasks.workunit.client.0.smithi008.stdout:8/365: rename dd/de/f64 to dd/de/d16/d50/f7f 0 2024-04-12T05:55:31.681 INFO:tasks.workunit.client.0.smithi008.stdout:4/328: creat d1/d4f/f5a x:0 0 0 2024-04-12T05:55:31.685 INFO:tasks.workunit.client.0.smithi008.stdout:3/486: dwrite d9/d10/d19/d3a/d67/f8a [0,4194304] 0 2024-04-12T05:55:31.690 INFO:tasks.workunit.client.0.smithi008.stdout:0/386: mknod d2/c75 0 2024-04-12T05:55:31.693 INFO:tasks.workunit.client.0.smithi008.stdout:2/415: dread d6/db/d60/f42 [0,4194304] 0 2024-04-12T05:55:31.693 INFO:tasks.workunit.client.0.smithi008.stdout:3/487: creat d9/d11/fa2 x:0 0 0 2024-04-12T05:55:31.695 INFO:tasks.workunit.client.0.smithi008.stdout:1/356: dread d4/d6/f59 [0,4194304] 0 2024-04-12T05:55:31.698 INFO:tasks.workunit.client.0.smithi008.stdout:4/329: truncate d1/f3b 7079844 0 2024-04-12T05:55:31.700 INFO:tasks.workunit.client.0.smithi008.stdout:2/416: mknod d6/db/d10/d23/d35/d3c/c8c 0 2024-04-12T05:55:31.700 INFO:tasks.workunit.client.0.smithi008.stdout:8/366: dwrite dd/f1f [0,4194304] 0 2024-04-12T05:55:31.702 INFO:tasks.workunit.client.0.smithi008.stdout:3/488: link d9/d11/f85 d9/d44/fa3 0 2024-04-12T05:55:31.703 INFO:tasks.workunit.client.0.smithi008.stdout:3/489: chown d9/dd/l16 5093 1 2024-04-12T05:55:31.703 INFO:tasks.workunit.client.0.smithi008.stdout:2/417: truncate d6/db/d38/f7e 951625 0 2024-04-12T05:55:31.705 INFO:tasks.workunit.client.0.smithi008.stdout:3/490: write d9/d10/d19/d22/f1f [1321114,24372] 0 2024-04-12T05:55:31.705 INFO:tasks.workunit.client.0.smithi008.stdout:2/418: write d6/db/d60/d57/f76 [460079,52227] 0 2024-04-12T05:55:31.707 INFO:tasks.workunit.client.0.smithi008.stdout:8/367: chown c8 24 1 2024-04-12T05:55:31.708 INFO:tasks.workunit.client.0.smithi008.stdout:4/330: rename d1/d8/f21 to d1/d4f/f5b 0 2024-04-12T05:55:31.709 INFO:tasks.workunit.client.0.smithi008.stdout:2/419: read - d6/db/d38/d41/f6b zero size 2024-04-12T05:55:31.710 INFO:tasks.workunit.client.0.smithi008.stdout:8/368: write dd/de/d16/f6e [4338877,54660] 0 2024-04-12T05:55:31.710 INFO:tasks.workunit.client.0.smithi008.stdout:1/357: creat d4/dd/d17/f75 x:0 0 0 2024-04-12T05:55:31.712 INFO:tasks.workunit.client.0.smithi008.stdout:3/491: rmdir d9/dd 39 2024-04-12T05:55:31.713 INFO:tasks.workunit.client.0.smithi008.stdout:2/420: readlink d6/db/d38/l2f 0 2024-04-12T05:55:31.713 INFO:tasks.workunit.client.0.smithi008.stdout:1/358: write d4/d6/f10 [889694,55498] 0 2024-04-12T05:55:31.714 INFO:tasks.workunit.client.0.smithi008.stdout:3/492: readlink d9/d11/l29 0 2024-04-12T05:55:31.714 INFO:tasks.workunit.client.0.smithi008.stdout:4/331: fsync d1/d22/d29/d30/d47/f48 0 2024-04-12T05:55:31.718 INFO:tasks.workunit.client.0.smithi008.stdout:1/359: stat d4/dd 0 2024-04-12T05:55:31.719 INFO:tasks.workunit.client.0.smithi008.stdout:0/387: link d2/d24/f28 d2/d24/f76 0 2024-04-12T05:55:31.719 INFO:tasks.workunit.client.0.smithi008.stdout:3/493: symlink d9/d10/d19/d22/d2c/la4 0 2024-04-12T05:55:31.722 INFO:tasks.workunit.client.0.smithi008.stdout:0/388: write d2/d11/d36/d42/f57 [1827840,124078] 0 2024-04-12T05:55:31.722 INFO:tasks.workunit.client.0.smithi008.stdout:4/332: creat d1/d22/d29/d30/d47/d28/f5c x:0 0 0 2024-04-12T05:55:31.726 INFO:tasks.workunit.client.0.smithi008.stdout:3/494: unlink d9/d11/fa2 0 2024-04-12T05:55:31.727 INFO:tasks.workunit.client.0.smithi008.stdout:4/333: fdatasync d1/d22/d29/d30/f52 0 2024-04-12T05:55:31.730 INFO:tasks.workunit.client.0.smithi008.stdout:8/369: dwrite dd/de/d16/d50/d56/d61/d6a/f6f [0,4194304] 0 2024-04-12T05:55:31.734 INFO:tasks.workunit.client.0.smithi008.stdout:8/370: chown dd/f1a 7 1 2024-04-12T05:55:31.734 INFO:tasks.workunit.client.0.smithi008.stdout:1/360: dread d4/d6/d11/f42 [0,4194304] 0 2024-04-12T05:55:31.735 INFO:tasks.workunit.client.0.smithi008.stdout:4/334: mknod d1/c5d 0 2024-04-12T05:55:31.736 INFO:tasks.workunit.client.0.smithi008.stdout:8/371: write dd/de/d16/f6e [416426,50395] 0 2024-04-12T05:55:31.740 INFO:tasks.workunit.client.0.smithi008.stdout:4/335: dread d1/d8/f25 [0,4194304] 0 2024-04-12T05:55:31.742 INFO:tasks.workunit.client.0.smithi008.stdout:8/372: unlink dd/d30/f41 0 2024-04-12T05:55:31.744 INFO:tasks.workunit.client.0.smithi008.stdout:0/389: write d2/d11/d36/d47/f4f [756221,41282] 0 2024-04-12T05:55:31.744 INFO:tasks.workunit.client.0.smithi008.stdout:1/361: dwrite d4/d33/d37/f55 [0,4194304] 0 2024-04-12T05:55:31.745 INFO:tasks.workunit.client.0.smithi008.stdout:4/336: symlink d1/d8/l5e 0 2024-04-12T05:55:31.751 INFO:tasks.workunit.client.0.smithi008.stdout:0/390: mknod d2/d11/d36/c77 0 2024-04-12T05:55:31.752 INFO:tasks.workunit.client.0.smithi008.stdout:8/373: dread dd/de/d16/f6e [0,4194304] 0 2024-04-12T05:55:31.755 INFO:tasks.workunit.client.0.smithi008.stdout:8/374: creat dd/de/d16/f80 x:0 0 0 2024-04-12T05:55:31.756 INFO:tasks.workunit.client.0.smithi008.stdout:4/337: dread d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:31.757 INFO:tasks.workunit.client.0.smithi008.stdout:1/362: link d4/d6/f53 d4/dd/d17/d26/f76 0 2024-04-12T05:55:31.758 INFO:tasks.workunit.client.0.smithi008.stdout:1/363: stat d4/d6/d11 0 2024-04-12T05:55:31.758 INFO:tasks.workunit.client.0.smithi008.stdout:0/391: symlink d2/l78 0 2024-04-12T05:55:31.760 INFO:tasks.workunit.client.0.smithi008.stdout:8/375: symlink dd/de/d16/d50/d5e/l81 0 2024-04-12T05:55:31.761 INFO:tasks.workunit.client.0.smithi008.stdout:1/364: dread d4/dd/d17/d26/f35 [0,4194304] 0 2024-04-12T05:55:31.762 INFO:tasks.workunit.client.0.smithi008.stdout:1/365: chown d4/d24/f45 14840017 1 2024-04-12T05:55:31.764 INFO:tasks.workunit.client.0.smithi008.stdout:8/376: symlink dd/de/d16/d50/d56/d61/l82 0 2024-04-12T05:55:31.765 INFO:tasks.workunit.client.0.smithi008.stdout:1/366: symlink d4/dd/d17/d66/l77 0 2024-04-12T05:55:31.766 INFO:tasks.workunit.client.0.smithi008.stdout:1/367: chown d4/d33/d74/d30/c4d 245242575 1 2024-04-12T05:55:31.767 INFO:tasks.workunit.client.0.smithi008.stdout:0/392: rmdir d2/d11 39 2024-04-12T05:55:31.768 INFO:tasks.workunit.client.0.smithi008.stdout:1/368: dread - d4/d6/f53 zero size 2024-04-12T05:55:31.769 INFO:tasks.workunit.client.0.smithi008.stdout:8/377: rename dd/d30/d32/f3e to dd/d30/d32/f83 0 2024-04-12T05:55:31.771 INFO:tasks.workunit.client.0.smithi008.stdout:4/338: write d1/d8/f37 [323877,71679] 0 2024-04-12T05:55:31.772 INFO:tasks.workunit.client.0.smithi008.stdout:4/339: chown d1/d8 28431627 1 2024-04-12T05:55:31.774 INFO:tasks.workunit.client.0.smithi008.stdout:0/393: dread - d2/d11/d36/d42/f65 zero size 2024-04-12T05:55:31.776 INFO:tasks.workunit.client.0.smithi008.stdout:8/378: creat dd/de/d4c/d66/f84 x:0 0 0 2024-04-12T05:55:31.776 INFO:tasks.workunit.client.0.smithi008.stdout:8/379: chown dd 331113631 1 2024-04-12T05:55:31.777 INFO:tasks.workunit.client.0.smithi008.stdout:4/340: write d1/d22/d29/d30/d47/d28/f3f [5424149,42841] 0 2024-04-12T05:55:31.778 INFO:tasks.workunit.client.0.smithi008.stdout:8/380: chown dd/de/d16/d50/d56/d61 12 1 2024-04-12T05:55:31.780 INFO:tasks.workunit.client.0.smithi008.stdout:4/341: dread - d1/d22/d29/d30/f34 zero size 2024-04-12T05:55:31.780 INFO:tasks.workunit.client.0.smithi008.stdout:1/369: write d4/d33/d74/f43 [757422,117864] 0 2024-04-12T05:55:31.781 INFO:tasks.workunit.client.0.smithi008.stdout:4/342: dread - d1/d8/f55 zero size 2024-04-12T05:55:31.783 INFO:tasks.workunit.client.0.smithi008.stdout:0/394: truncate d2/d11/f23 1936588 0 2024-04-12T05:55:31.784 INFO:tasks.workunit.client.0.smithi008.stdout:1/370: truncate d4/d24/f32 412980 0 2024-04-12T05:55:31.785 INFO:tasks.workunit.client.0.smithi008.stdout:5/654: sync 2024-04-12T05:55:31.788 INFO:tasks.workunit.client.0.smithi008.stdout:0/395: chown d2/c15 0 1 2024-04-12T05:55:31.788 INFO:tasks.workunit.client.0.smithi008.stdout:4/343: dread d1/d8/f2d [0,4194304] 0 2024-04-12T05:55:31.789 INFO:tasks.workunit.client.0.smithi008.stdout:1/371: write d4/dd/d17/f31 [1624304,40460] 0 2024-04-12T05:55:31.791 INFO:tasks.workunit.client.0.smithi008.stdout:5/655: dread d0/f97 [0,4194304] 0 2024-04-12T05:55:31.792 INFO:tasks.workunit.client.0.smithi008.stdout:8/381: dwrite dd/d30/d32/f83 [0,4194304] 0 2024-04-12T05:55:31.795 INFO:tasks.workunit.client.0.smithi008.stdout:5/656: dread - d0/d29/f9c zero size 2024-04-12T05:55:31.799 INFO:tasks.workunit.client.0.smithi008.stdout:4/344: creat d1/d8/f5f x:0 0 0 2024-04-12T05:55:31.802 INFO:tasks.workunit.client.0.smithi008.stdout:1/372: symlink d4/d1c/l78 0 2024-04-12T05:55:31.805 INFO:tasks.workunit.client.0.smithi008.stdout:0/396: dwrite d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:31.807 INFO:tasks.workunit.client.0.smithi008.stdout:8/382: dread dd/f1f [0,4194304] 0 2024-04-12T05:55:31.814 INFO:tasks.workunit.client.0.smithi008.stdout:0/397: creat d2/d24/f79 x:0 0 0 2024-04-12T05:55:31.815 INFO:tasks.workunit.client.0.smithi008.stdout:5/657: dwrite d0/dc/d13/f48 [0,4194304] 0 2024-04-12T05:55:31.822 INFO:tasks.workunit.client.0.smithi008.stdout:8/383: creat dd/de/d16/d50/d5e/f85 x:0 0 0 2024-04-12T05:55:31.826 INFO:tasks.workunit.client.0.smithi008.stdout:5/658: unlink d0/dc/d13/d1a/daf/c7 0 2024-04-12T05:55:31.828 INFO:tasks.workunit.client.0.smithi008.stdout:4/345: dwrite d1/d8/f37 [0,4194304] 0 2024-04-12T05:55:31.828 INFO:tasks.workunit.client.0.smithi008.stdout:1/373: dwrite d4/d7/f56 [0,4194304] 0 2024-04-12T05:55:31.829 INFO:tasks.workunit.client.0.smithi008.stdout:8/384: readlink dd/de/d16/d50/d5e/l67 0 2024-04-12T05:55:31.832 INFO:tasks.workunit.client.0.smithi008.stdout:5/659: mkdir d0/dd/dc7 0 2024-04-12T05:55:31.833 INFO:tasks.workunit.client.0.smithi008.stdout:4/346: stat d1/d22/d29/d30/d47/d28 0 2024-04-12T05:55:31.839 INFO:tasks.workunit.client.0.smithi008.stdout:0/398: dwrite d2/f66 [0,4194304] 0 2024-04-12T05:55:31.839 INFO:tasks.workunit.client.0.smithi008.stdout:4/347: truncate d1/d8/f42 557584 0 2024-04-12T05:55:31.846 INFO:tasks.workunit.client.0.smithi008.stdout:0/399: write d2/d24/d39/f3a [3399715,19444] 0 2024-04-12T05:55:31.857 INFO:tasks.workunit.client.0.smithi008.stdout:4/348: mkdir d1/d22/d29/d30/d60 0 2024-04-12T05:55:31.858 INFO:tasks.workunit.client.0.smithi008.stdout:1/374: link d4/l14 d4/dd/d17/d26/l79 0 2024-04-12T05:55:31.858 INFO:tasks.workunit.client.0.smithi008.stdout:0/400: unlink d2/d11/d2a/f3e 0 2024-04-12T05:55:31.859 INFO:tasks.workunit.client.0.smithi008.stdout:0/401: fdatasync d2/d11/f64 0 2024-04-12T05:55:31.860 INFO:tasks.workunit.client.0.smithi008.stdout:8/385: getdents dd/de/d4c/d66/d6c 0 2024-04-12T05:55:31.866 INFO:tasks.workunit.client.0.smithi008.stdout:5/660: link d0/dc/d13/d1a/d35/f85 d0/dd/dad/fc8 0 2024-04-12T05:55:31.867 INFO:tasks.workunit.client.0.smithi008.stdout:4/349: mknod d1/d22/d29/d30/d47/d28/c61 0 2024-04-12T05:55:31.873 INFO:tasks.workunit.client.0.smithi008.stdout:1/375: dread d4/dd/d17/f31 [0,4194304] 0 2024-04-12T05:55:31.874 INFO:tasks.workunit.client.0.smithi008.stdout:8/386: rename dd/d30/c6b to dd/c86 0 2024-04-12T05:55:31.876 INFO:tasks.workunit.client.0.smithi008.stdout:5/661: mkdir d0/dd/d3d/dc9 0 2024-04-12T05:55:31.877 INFO:tasks.workunit.client.0.smithi008.stdout:4/350: dread d1/f40 [0,4194304] 0 2024-04-12T05:55:31.877 INFO:tasks.workunit.client.0.smithi008.stdout:1/376: rmdir d4/d33/d37 39 2024-04-12T05:55:31.878 INFO:tasks.workunit.client.0.smithi008.stdout:8/387: rmdir dd/de/d16/d50 39 2024-04-12T05:55:31.879 INFO:tasks.workunit.client.0.smithi008.stdout:5/662: symlink d0/d29/d96/lca 0 2024-04-12T05:55:31.879 INFO:tasks.workunit.client.0.smithi008.stdout:0/402: dread d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:31.880 INFO:tasks.workunit.client.0.smithi008.stdout:4/351: unlink d1/d8/f2d 0 2024-04-12T05:55:31.881 INFO:tasks.workunit.client.0.smithi008.stdout:0/403: truncate d2/d4b/f5f 1318809 0 2024-04-12T05:55:31.882 INFO:tasks.workunit.client.0.smithi008.stdout:8/388: unlink dd/d30/d32/c42 0 2024-04-12T05:55:31.885 INFO:tasks.workunit.client.0.smithi008.stdout:5/663: symlink d0/dd/dc7/lcb 0 2024-04-12T05:55:31.886 INFO:tasks.workunit.client.0.smithi008.stdout:5/664: write d0/f97 [1612988,24070] 0 2024-04-12T05:55:31.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/404: symlink d2/d11/d36/d42/l7a 0 2024-04-12T05:55:31.891 INFO:tasks.workunit.client.0.smithi008.stdout:0/405: read d2/d11/f69 [2404910,33022] 0 2024-04-12T05:55:31.894 INFO:tasks.workunit.client.0.smithi008.stdout:5/665: unlink d0/la0 0 2024-04-12T05:55:31.896 INFO:tasks.workunit.client.0.smithi008.stdout:4/352: dread d1/d8/f23 [0,4194304] 0 2024-04-12T05:55:31.896 INFO:tasks.workunit.client.0.smithi008.stdout:8/389: dwrite dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:31.898 INFO:tasks.workunit.client.0.smithi008.stdout:0/406: readlink d2/d11/d2a/l43 0 2024-04-12T05:55:31.899 INFO:tasks.workunit.client.0.smithi008.stdout:5/666: fdatasync d0/dd/d11/d38/f43 0 2024-04-12T05:55:31.899 INFO:tasks.workunit.client.0.smithi008.stdout:1/377: truncate d4/dd/d17/f19 997423 0 2024-04-12T05:55:31.904 INFO:tasks.workunit.client.0.smithi008.stdout:1/378: mkdir d4/d33/d74/d7a 0 2024-04-12T05:55:31.906 INFO:tasks.workunit.client.0.smithi008.stdout:8/390: dwrite dd/de/f29 [0,4194304] 0 2024-04-12T05:55:31.907 INFO:tasks.workunit.client.0.smithi008.stdout:5/667: creat d0/dd/fcc x:0 0 0 2024-04-12T05:55:31.911 INFO:tasks.workunit.client.0.smithi008.stdout:5/668: write d0/dc/d13/d1a/d3a/f63 [1375653,66809] 0 2024-04-12T05:55:31.916 INFO:tasks.workunit.client.0.smithi008.stdout:0/407: rename d2/d11/d36/d47/f59 to d2/d11/d2a/f7b 0 2024-04-12T05:55:31.917 INFO:tasks.workunit.client.0.smithi008.stdout:1/379: unlink d4/dd/d17/c60 0 2024-04-12T05:55:31.919 INFO:tasks.workunit.client.0.smithi008.stdout:1/380: write d4/d7/fc [2364569,114888] 0 2024-04-12T05:55:31.921 INFO:tasks.workunit.client.0.smithi008.stdout:8/391: creat dd/de/d4c/d66/d6c/f87 x:0 0 0 2024-04-12T05:55:31.923 INFO:tasks.workunit.client.0.smithi008.stdout:5/669: fsync d0/dc/d13/d1a/d35/f4c 0 2024-04-12T05:55:31.926 INFO:tasks.workunit.client.0.smithi008.stdout:0/408: dwrite d2/d24/d39/f3a [0,4194304] 0 2024-04-12T05:55:31.928 INFO:tasks.workunit.client.0.smithi008.stdout:0/409: read d2/d24/d39/f3a [823979,98375] 0 2024-04-12T05:55:31.935 INFO:tasks.workunit.client.0.smithi008.stdout:1/381: getdents d4/dd/d17/d61 0 2024-04-12T05:55:31.937 INFO:tasks.workunit.client.0.smithi008.stdout:8/392: getdents dd/d30/d3b 0 2024-04-12T05:55:31.938 INFO:tasks.workunit.client.0.smithi008.stdout:0/410: dwrite d2/d24/f79 [0,4194304] 0 2024-04-12T05:55:31.939 INFO:tasks.workunit.client.0.smithi008.stdout:1/382: write d4/d1a/f6c [563296,67070] 0 2024-04-12T05:55:31.941 INFO:tasks.workunit.client.0.smithi008.stdout:1/383: fsync d4/d33/d74/d30/f5d 0 2024-04-12T05:55:31.941 INFO:tasks.workunit.client.0.smithi008.stdout:0/411: write d2/f6a [1047490,36744] 0 2024-04-12T05:55:31.943 INFO:tasks.workunit.client.0.smithi008.stdout:5/670: link d0/d29/l46 d0/d29/d96/db9/lcd 0 2024-04-12T05:55:31.948 INFO:tasks.workunit.client.0.smithi008.stdout:1/384: rename d4/dd/d17/d66/c6e to d4/d6/d11/c7b 0 2024-04-12T05:55:31.950 INFO:tasks.workunit.client.0.smithi008.stdout:5/671: creat d0/dd/dc7/fce x:0 0 0 2024-04-12T05:55:31.952 INFO:tasks.workunit.client.0.smithi008.stdout:0/412: link d2/f6a d2/d11/d36/f7c 0 2024-04-12T05:55:31.953 INFO:tasks.workunit.client.0.smithi008.stdout:8/393: getdents dd 0 2024-04-12T05:55:31.953 INFO:tasks.workunit.client.0.smithi008.stdout:0/413: chown d2/f72 11128 1 2024-04-12T05:55:31.955 INFO:tasks.workunit.client.0.smithi008.stdout:8/394: fsync dd/d30/d3b/f74 0 2024-04-12T05:55:31.957 INFO:tasks.workunit.client.0.smithi008.stdout:8/395: write dd/f36 [3078890,59854] 0 2024-04-12T05:55:31.958 INFO:tasks.workunit.client.0.smithi008.stdout:0/414: fdatasync d2/d11/f23 0 2024-04-12T05:55:31.959 INFO:tasks.workunit.client.0.smithi008.stdout:0/415: chown d2/d11/c20 254006083 1 2024-04-12T05:55:31.960 INFO:tasks.workunit.client.0.smithi008.stdout:5/672: rename d0/dc/d13/d1a/d35/f42 to d0/dd/d3d/d5d/fcf 0 2024-04-12T05:55:31.961 INFO:tasks.workunit.client.0.smithi008.stdout:8/396: symlink dd/d30/d3b/l88 0 2024-04-12T05:55:31.969 INFO:tasks.workunit.client.0.smithi008.stdout:5/673: dwrite d0/dd/dc7/fce [0,4194304] 0 2024-04-12T05:55:31.971 INFO:tasks.workunit.client.0.smithi008.stdout:7/538: sync 2024-04-12T05:55:31.971 INFO:tasks.workunit.client.0.smithi008.stdout:5/674: write d0/dc/f68 [2027321,94810] 0 2024-04-12T05:55:31.972 INFO:tasks.workunit.client.0.smithi008.stdout:5/675: readlink d0/d40/la3 0 2024-04-12T05:55:31.973 INFO:tasks.workunit.client.0.smithi008.stdout:8/397: mkdir dd/de/d4c/d66/d6c/d89 0 2024-04-12T05:55:31.974 INFO:tasks.workunit.client.0.smithi008.stdout:1/385: truncate d4/dd/d17/f31 3264274 0 2024-04-12T05:55:31.976 INFO:tasks.workunit.client.0.smithi008.stdout:0/416: symlink d2/d24/d61/l7d 0 2024-04-12T05:55:31.982 INFO:tasks.workunit.client.0.smithi008.stdout:8/398: readlink dd/de/d16/d50/d56/d61/d6a/l76 0 2024-04-12T05:55:31.983 INFO:tasks.workunit.client.0.smithi008.stdout:1/386: mkdir d4/d33/d74/d30/d72/d7c 0 2024-04-12T05:55:31.987 INFO:tasks.workunit.client.0.smithi008.stdout:8/399: chown dd/d30/d32/f3a 8724787 1 2024-04-12T05:55:31.990 INFO:tasks.workunit.client.0.smithi008.stdout:7/539: dwrite d1/d5/da/d3c/d44/f6f [0,4194304] 0 2024-04-12T05:55:31.991 INFO:tasks.workunit.client.0.smithi008.stdout:0/417: creat d2/d11/d2a/f7e x:0 0 0 2024-04-12T05:55:31.992 INFO:tasks.workunit.client.0.smithi008.stdout:7/540: fdatasync d1/d5/da/d11/f57 0 2024-04-12T05:55:31.993 INFO:tasks.workunit.client.0.smithi008.stdout:0/418: stat d2/d11/d36/d42 0 2024-04-12T05:55:31.996 INFO:tasks.workunit.client.0.smithi008.stdout:5/676: write d0/dc/d13/d1a/daf/f84 [18052,109737] 0 2024-04-12T05:55:31.998 INFO:tasks.workunit.client.0.smithi008.stdout:0/419: rmdir d2/d4b 39 2024-04-12T05:55:32.001 INFO:tasks.workunit.client.0.smithi008.stdout:0/420: mkdir d2/d11/d36/d7f 0 2024-04-12T05:55:32.002 INFO:tasks.workunit.client.0.smithi008.stdout:8/400: link dd/d4f/l5c dd/de/d16/d50/d56/d61/l8a 0 2024-04-12T05:55:32.003 INFO:tasks.workunit.client.0.smithi008.stdout:8/401: fsync dd/de/d16/f45 0 2024-04-12T05:55:32.004 INFO:tasks.workunit.client.0.smithi008.stdout:1/387: truncate d4/dd/d17/d26/f36 960398 0 2024-04-12T05:55:32.008 INFO:tasks.workunit.client.0.smithi008.stdout:7/541: rename d1/f12 to d1/fc0 0 2024-04-12T05:55:32.009 INFO:tasks.workunit.client.0.smithi008.stdout:8/402: chown dd/d30/l37 103820375 1 2024-04-12T05:55:32.010 INFO:tasks.workunit.client.0.smithi008.stdout:8/403: stat dd/de/d4c 0 2024-04-12T05:55:32.015 INFO:tasks.workunit.client.0.smithi008.stdout:5/677: write d0/dd/f34 [1675053,31131] 0 2024-04-12T05:55:32.018 INFO:tasks.workunit.client.0.smithi008.stdout:8/404: mknod dd/d65/c8b 0 2024-04-12T05:55:32.018 INFO:tasks.workunit.client.0.smithi008.stdout:7/542: dwrite d1/d5/da/f62 [4194304,4194304] 0 2024-04-12T05:55:32.022 INFO:tasks.workunit.client.0.smithi008.stdout:1/388: getdents d4/d6 0 2024-04-12T05:55:32.022 INFO:tasks.workunit.client.0.smithi008.stdout:0/421: dread d2/d24/f28 [0,4194304] 0 2024-04-12T05:55:32.026 INFO:tasks.workunit.client.0.smithi008.stdout:5/678: creat d0/dd/d3d/d5d/fd0 x:0 0 0 2024-04-12T05:55:32.030 INFO:tasks.workunit.client.0.smithi008.stdout:1/389: creat d4/d24/d62/f7d x:0 0 0 2024-04-12T05:55:32.030 INFO:tasks.workunit.client.0.smithi008.stdout:7/543: creat d1/d5/da/d3c/fc1 x:0 0 0 2024-04-12T05:55:32.031 INFO:tasks.workunit.client.0.smithi008.stdout:5/679: symlink d0/d40/d56/ld1 0 2024-04-12T05:55:32.038 INFO:tasks.workunit.client.0.smithi008.stdout:1/390: creat d4/d33/d74/d46/f7e x:0 0 0 2024-04-12T05:55:32.039 INFO:tasks.workunit.client.0.smithi008.stdout:1/391: fsync d4/dd/d17/f64 0 2024-04-12T05:55:32.042 INFO:tasks.workunit.client.0.smithi008.stdout:0/422: dread d2/d11/f69 [4194304,4194304] 0 2024-04-12T05:55:32.042 INFO:tasks.workunit.client.0.smithi008.stdout:7/544: dwrite d1/d49/f53 [0,4194304] 0 2024-04-12T05:55:32.042 INFO:tasks.workunit.client.0.smithi008.stdout:1/392: write d4/d24/d62/f7d [576566,1671] 0 2024-04-12T05:55:32.052 INFO:tasks.workunit.client.0.smithi008.stdout:1/393: rename d4/d6/f10 to d4/f7f 0 2024-04-12T05:55:32.056 INFO:tasks.workunit.client.0.smithi008.stdout:5/680: dread d0/dc/d57/f2f [0,4194304] 0 2024-04-12T05:55:32.056 INFO:tasks.workunit.client.0.smithi008.stdout:0/423: dwrite d2/d24/d39/f3a [0,4194304] 0 2024-04-12T05:55:32.057 INFO:tasks.workunit.client.0.smithi008.stdout:0/424: stat d2/d11/d36/c77 0 2024-04-12T05:55:32.059 INFO:tasks.workunit.client.0.smithi008.stdout:0/425: dread - d2/d11/d36/d42/f65 zero size 2024-04-12T05:55:32.063 INFO:tasks.workunit.client.0.smithi008.stdout:5/681: rename d0/dd/d11/d38/c6b to d0/dc/d13/d1a/cd2 0 2024-04-12T05:55:32.064 INFO:tasks.workunit.client.0.smithi008.stdout:0/426: creat d2/d11/d36/f80 x:0 0 0 2024-04-12T05:55:32.068 INFO:tasks.workunit.client.0.smithi008.stdout:5/682: rename d0/dd/d3d/d5d/fcf to d0/dd/d11/d38/d55/fd3 0 2024-04-12T05:55:32.071 INFO:tasks.workunit.client.0.smithi008.stdout:5/683: fsync d0/dc/d13/d1a/daf/f84 0 2024-04-12T05:55:32.072 INFO:tasks.workunit.client.0.smithi008.stdout:5/684: chown d0/dd/d11 6671 1 2024-04-12T05:55:32.073 INFO:tasks.workunit.client.0.smithi008.stdout:0/427: dwrite d2/d11/d36/d42/f65 [0,4194304] 0 2024-04-12T05:55:32.075 INFO:tasks.workunit.client.0.smithi008.stdout:7/545: dread d1/d5/f32 [0,4194304] 0 2024-04-12T05:55:32.077 INFO:tasks.workunit.client.0.smithi008.stdout:5/685: write d0/dc/d13/d1a/daf/f95 [233581,116998] 0 2024-04-12T05:55:32.077 INFO:tasks.workunit.client.0.smithi008.stdout:7/546: fdatasync d1/fb8 0 2024-04-12T05:55:32.082 INFO:tasks.workunit.client.0.smithi008.stdout:5/686: creat d0/dc/d57/d9d/db5/fd4 x:0 0 0 2024-04-12T05:55:32.083 INFO:tasks.workunit.client.0.smithi008.stdout:7/547: mkdir d1/d15/d43/d54/d58/dc2 0 2024-04-12T05:55:32.083 INFO:tasks.workunit.client.0.smithi008.stdout:0/428: link d2/d11/f69 d2/d11/d36/f81 0 2024-04-12T05:55:32.084 INFO:tasks.workunit.client.0.smithi008.stdout:0/429: stat d2/d11/l1e 0 2024-04-12T05:55:32.085 INFO:tasks.workunit.client.0.smithi008.stdout:7/548: mkdir d1/d5/da/d1d/dc3 0 2024-04-12T05:55:32.086 INFO:tasks.workunit.client.0.smithi008.stdout:0/430: mkdir d2/d11/d36/d82 0 2024-04-12T05:55:32.087 INFO:tasks.workunit.client.0.smithi008.stdout:5/687: rmdir d0/d29 39 2024-04-12T05:55:32.088 INFO:tasks.workunit.client.0.smithi008.stdout:5/688: write d0/d40/f77 [106458,30378] 0 2024-04-12T05:55:32.090 INFO:tasks.workunit.client.0.smithi008.stdout:7/549: creat d1/d15/d43/d54/fc4 x:0 0 0 2024-04-12T05:55:32.090 INFO:tasks.workunit.client.0.smithi008.stdout:5/689: fdatasync d0/dc/d13/d1a/f22 0 2024-04-12T05:55:32.090 INFO:tasks.workunit.client.0.smithi008.stdout:0/431: rename d2/d11/d2a to d2/d11/d2a/d83 22 2024-04-12T05:55:32.091 INFO:tasks.workunit.client.0.smithi008.stdout:7/550: stat d1/d5/da/d3c/d44/d8a/fa7 0 2024-04-12T05:55:32.092 INFO:tasks.workunit.client.0.smithi008.stdout:0/432: write d2/d11/d36/f46 [2283068,112095] 0 2024-04-12T05:55:32.094 INFO:tasks.workunit.client.0.smithi008.stdout:0/433: write d2/d11/f18 [312866,5371] 0 2024-04-12T05:55:32.101 INFO:tasks.workunit.client.0.smithi008.stdout:7/551: creat d1/d5/da/d1d/fc5 x:0 0 0 2024-04-12T05:55:32.102 INFO:tasks.workunit.client.0.smithi008.stdout:5/690: truncate d0/dc/d57/f2f 5044063 0 2024-04-12T05:55:32.102 INFO:tasks.workunit.client.0.smithi008.stdout:0/434: dwrite d2/d11/d36/d47/f21 [0,4194304] 0 2024-04-12T05:55:32.105 INFO:tasks.workunit.client.0.smithi008.stdout:0/435: write d2/d24/f79 [1730983,11756] 0 2024-04-12T05:55:32.111 INFO:tasks.workunit.client.0.smithi008.stdout:7/552: dwrite d1/d49/f53 [0,4194304] 0 2024-04-12T05:55:32.113 INFO:tasks.workunit.client.0.smithi008.stdout:5/691: link d0/dd/d11/d38/f2b d0/dd/fd5 0 2024-04-12T05:55:32.121 INFO:tasks.workunit.client.0.smithi008.stdout:5/692: mkdir d0/d29/dd6 0 2024-04-12T05:55:32.125 INFO:tasks.workunit.client.0.smithi008.stdout:7/553: dwrite d1/d15/d43/d54/d58/db7/fa5 [0,4194304] 0 2024-04-12T05:55:32.126 INFO:tasks.workunit.client.0.smithi008.stdout:0/436: dwrite d2/fe [0,4194304] 0 2024-04-12T05:55:32.133 INFO:tasks.workunit.client.0.smithi008.stdout:0/437: mknod d2/d11/d36/d47/c84 0 2024-04-12T05:55:32.134 INFO:tasks.workunit.client.0.smithi008.stdout:7/554: write d1/d5/da/d3c/d44/fa0 [613851,92163] 0 2024-04-12T05:55:32.138 INFO:tasks.workunit.client.0.smithi008.stdout:0/438: mkdir d2/d24/d85 0 2024-04-12T05:55:32.139 INFO:tasks.workunit.client.0.smithi008.stdout:5/693: dwrite d0/d40/f5e [4194304,4194304] 0 2024-04-12T05:55:32.140 INFO:tasks.workunit.client.0.smithi008.stdout:5/694: fsync d0/dd/d11/d38/d55/f5b 0 2024-04-12T05:55:32.141 INFO:tasks.workunit.client.0.smithi008.stdout:7/555: mknod d1/d5/da/d60/da1/cc6 0 2024-04-12T05:55:32.144 INFO:tasks.workunit.client.0.smithi008.stdout:5/695: unlink d0/dc/d13/d1a/daf/d37/f3e 0 2024-04-12T05:55:32.151 INFO:tasks.workunit.client.0.smithi008.stdout:5/696: dwrite d0/dc/d13/f54 [4194304,4194304] 0 2024-04-12T05:55:32.153 INFO:tasks.workunit.client.0.smithi008.stdout:5/697: readlink d0/dd/lc4 0 2024-04-12T05:55:32.155 INFO:tasks.workunit.client.0.smithi008.stdout:5/698: mknod d0/d29/d96/cd7 0 2024-04-12T05:55:32.157 INFO:tasks.workunit.client.0.smithi008.stdout:5/699: rmdir d0/dd/d11/d38/d55 39 2024-04-12T05:55:32.159 INFO:tasks.workunit.client.0.smithi008.stdout:5/700: dread d0/d29/d96/fc2 [0,4194304] 0 2024-04-12T05:55:32.161 INFO:tasks.workunit.client.0.smithi008.stdout:5/701: creat d0/dab/fd8 x:0 0 0 2024-04-12T05:55:32.186 INFO:tasks.workunit.client.0.smithi008.stdout:5/702: dread d0/dc/d13/d1a/f24 [0,4194304] 0 2024-04-12T05:55:32.193 INFO:tasks.workunit.client.0.smithi008.stdout:5/703: dwrite d0/dd/d11/d38/fa5 [0,4194304] 0 2024-04-12T05:55:32.198 INFO:tasks.workunit.client.0.smithi008.stdout:5/704: write d0/dc/f2a [6473503,29625] 0 2024-04-12T05:55:32.200 INFO:tasks.workunit.client.0.smithi008.stdout:5/705: chown d0/c1 55 1 2024-04-12T05:55:32.229 INFO:tasks.workunit.client.0.smithi008.stdout:3/495: sync 2024-04-12T05:55:32.229 INFO:tasks.workunit.client.0.smithi008.stdout:2/421: sync 2024-04-12T05:55:32.229 INFO:tasks.workunit.client.0.smithi008.stdout:6/486: sync 2024-04-12T05:55:32.230 INFO:tasks.workunit.client.0.smithi008.stdout:9/508: sync 2024-04-12T05:55:32.231 INFO:tasks.workunit.client.0.smithi008.stdout:6/487: fsync df/d10/d3a/f7f 0 2024-04-12T05:55:32.232 INFO:tasks.workunit.client.0.smithi008.stdout:9/509: chown d3/df/d13/d14/l5f 256034170 1 2024-04-12T05:55:32.233 INFO:tasks.workunit.client.0.smithi008.stdout:2/422: mknod d6/db/d60/d57/c8d 0 2024-04-12T05:55:32.234 INFO:tasks.workunit.client.0.smithi008.stdout:2/423: fdatasync d6/db/f22 0 2024-04-12T05:55:32.242 INFO:tasks.workunit.client.0.smithi008.stdout:5/706: truncate d0/dc/d13/d1a/daf/f1b 3424905 0 2024-04-12T05:55:32.243 INFO:tasks.workunit.client.0.smithi008.stdout:6/488: rename f3 to df/d10/d20/d3e/fa1 0 2024-04-12T05:55:32.243 INFO:tasks.workunit.client.0.smithi008.stdout:9/510: symlink d3/df/d13/d14/d95/lb2 0 2024-04-12T05:55:32.244 INFO:tasks.workunit.client.0.smithi008.stdout:2/424: symlink d6/db/d10/d23/l8e 0 2024-04-12T05:55:32.249 INFO:tasks.workunit.client.0.smithi008.stdout:9/511: rename d3/d22/d2b/f33 to d3/d22/d2b/d58/d90/db1/fb3 0 2024-04-12T05:55:32.255 INFO:tasks.workunit.client.0.smithi008.stdout:6/489: creat df/d10/d15/d75/fa2 x:0 0 0 2024-04-12T05:55:32.255 INFO:tasks.workunit.client.0.smithi008.stdout:5/707: creat d0/dd/fd9 x:0 0 0 2024-04-12T05:55:32.257 INFO:tasks.workunit.client.0.smithi008.stdout:3/496: dread d9/dd/f2f [4194304,4194304] 0 2024-04-12T05:55:32.258 INFO:tasks.workunit.client.0.smithi008.stdout:2/425: link d6/ff d6/db/d10/d23/d35/d3c/f8f 0 2024-04-12T05:55:32.260 INFO:tasks.workunit.client.0.smithi008.stdout:2/426: read d6/ff [476395,16858] 0 2024-04-12T05:55:32.261 INFO:tasks.workunit.client.0.smithi008.stdout:6/490: unlink df/l6d 0 2024-04-12T05:55:32.262 INFO:tasks.workunit.client.0.smithi008.stdout:2/427: read d6/db/d60/f42 [3280100,93533] 0 2024-04-12T05:55:32.263 INFO:tasks.workunit.client.0.smithi008.stdout:5/708: mknod d0/dc/cda 0 2024-04-12T05:55:32.264 INFO:tasks.workunit.client.0.smithi008.stdout:2/428: write d6/db/d29/f48 [4625029,89627] 0 2024-04-12T05:55:32.267 INFO:tasks.workunit.client.0.smithi008.stdout:5/709: fsync d0/dd/d3d/d5d/faa 0 2024-04-12T05:55:32.271 INFO:tasks.workunit.client.0.smithi008.stdout:3/497: dread d9/dd/f7f [0,4194304] 0 2024-04-12T05:55:32.272 INFO:tasks.workunit.client.0.smithi008.stdout:3/498: readlink d9/d10/d19/d22/d2c/la4 0 2024-04-12T05:55:32.273 INFO:tasks.workunit.client.0.smithi008.stdout:5/710: creat d0/dd/d11/d38/fdb x:0 0 0 2024-04-12T05:55:32.274 INFO:tasks.workunit.client.0.smithi008.stdout:3/499: write d9/d44/f63 [166583,53938] 0 2024-04-12T05:55:32.277 INFO:tasks.workunit.client.0.smithi008.stdout:5/711: creat d0/dd/dad/fdc x:0 0 0 2024-04-12T05:55:32.278 INFO:tasks.workunit.client.0.smithi008.stdout:2/429: dread d6/db/d29/f48 [0,4194304] 0 2024-04-12T05:55:32.279 INFO:tasks.workunit.client.0.smithi008.stdout:6/491: write df/d10/d15/d4f/d50/f6c [938115,17672] 0 2024-04-12T05:55:32.282 INFO:tasks.workunit.client.0.smithi008.stdout:2/430: chown d6/db/d10/d23/l7a 192517238 1 2024-04-12T05:55:32.282 INFO:tasks.workunit.client.0.smithi008.stdout:5/712: mknod d0/dc/cdd 0 2024-04-12T05:55:32.283 INFO:tasks.workunit.client.0.smithi008.stdout:2/431: readlink d6/db/d38/l72 0 2024-04-12T05:55:32.284 INFO:tasks.workunit.client.0.smithi008.stdout:6/492: creat df/d8c/fa3 x:0 0 0 2024-04-12T05:55:32.284 INFO:tasks.workunit.client.0.smithi008.stdout:3/500: link d9/d11/f85 d9/d10/d19/d22/fa5 0 2024-04-12T05:55:32.284 INFO:tasks.workunit.client.0.smithi008.stdout:5/713: write d0/dd/d3d/f4b [844272,104884] 0 2024-04-12T05:55:32.287 INFO:tasks.workunit.client.0.smithi008.stdout:2/432: creat d6/db/d38/d41/f90 x:0 0 0 2024-04-12T05:55:32.288 INFO:tasks.workunit.client.0.smithi008.stdout:2/433: chown d6/db/d60/d57/f76 849546 1 2024-04-12T05:55:32.289 INFO:tasks.workunit.client.0.smithi008.stdout:5/714: symlink d0/dab/lde 0 2024-04-12T05:55:32.290 INFO:tasks.workunit.client.0.smithi008.stdout:3/501: unlink d9/d10/d19/d22/l8b 0 2024-04-12T05:55:32.291 INFO:tasks.workunit.client.0.smithi008.stdout:5/715: write d0/dc/d13/f48 [2705521,49424] 0 2024-04-12T05:55:32.294 INFO:tasks.workunit.client.0.smithi008.stdout:2/434: creat d6/db/d29/f91 x:0 0 0 2024-04-12T05:55:32.297 INFO:tasks.workunit.client.0.smithi008.stdout:2/435: rename d6/db/d38/l2f to d6/l92 0 2024-04-12T05:55:32.300 INFO:tasks.workunit.client.0.smithi008.stdout:3/502: creat d9/dd/fa6 x:0 0 0 2024-04-12T05:55:32.300 INFO:tasks.workunit.client.0.smithi008.stdout:2/436: read d6/f15 [3731131,34873] 0 2024-04-12T05:55:32.303 INFO:tasks.workunit.client.0.smithi008.stdout:3/503: read - d9/f7a zero size 2024-04-12T05:55:32.304 INFO:tasks.workunit.client.0.smithi008.stdout:5/716: dwrite d0/d29/fba [0,4194304] 0 2024-04-12T05:55:32.308 INFO:tasks.workunit.client.0.smithi008.stdout:2/437: creat d6/db/d38/d41/f93 x:0 0 0 2024-04-12T05:55:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:2/438: stat d6/db/d60/d57/f76 0 2024-04-12T05:55:32.310 INFO:tasks.workunit.client.0.smithi008.stdout:3/504: dread - d9/d10/d19/d65/f8e zero size 2024-04-12T05:55:32.311 INFO:tasks.workunit.client.0.smithi008.stdout:6/493: dwrite df/d21/f22 [0,4194304] 0 2024-04-12T05:55:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:3/505: readlink d9/d1e/l30 0 2024-04-12T05:55:32.312 INFO:tasks.workunit.client.0.smithi008.stdout:5/717: rmdir d0/dc/d13/d1a 39 2024-04-12T05:55:32.314 INFO:tasks.workunit.client.0.smithi008.stdout:2/439: symlink d6/db/d60/d84/l94 0 2024-04-12T05:55:32.315 INFO:tasks.workunit.client.0.smithi008.stdout:6/494: write df/d10/d15/d75/fa2 [554552,55049] 0 2024-04-12T05:55:32.320 INFO:tasks.workunit.client.0.smithi008.stdout:3/506: rename d9/dd/f36 to d9/d10/d4b/fa7 0 2024-04-12T05:55:32.331 INFO:tasks.workunit.client.0.smithi008.stdout:2/440: symlink d6/db/d6f/l95 0 2024-04-12T05:55:32.333 INFO:tasks.workunit.client.0.smithi008.stdout:6/495: dread df/d21/f5a [0,4194304] 0 2024-04-12T05:55:32.336 INFO:tasks.workunit.client.0.smithi008.stdout:2/441: chown d6/db/d29/f48 330294 1 2024-04-12T05:55:32.336 INFO:tasks.workunit.client.0.smithi008.stdout:6/496: mkdir df/d8c/da4 0 2024-04-12T05:55:32.338 INFO:tasks.workunit.client.0.smithi008.stdout:3/507: mkdir d9/d10/d19/d22/da8 0 2024-04-12T05:55:32.338 INFO:tasks.workunit.client.0.smithi008.stdout:5/718: dwrite d0/dc/d13/d1a/d3a/f93 [0,4194304] 0 2024-04-12T05:55:32.340 INFO:tasks.workunit.client.0.smithi008.stdout:2/442: chown d6/db/d60/l4e 307309 1 2024-04-12T05:55:32.344 INFO:tasks.workunit.client.0.smithi008.stdout:2/443: rmdir d6/db/d60/d4f 39 2024-04-12T05:55:32.352 INFO:tasks.workunit.client.0.smithi008.stdout:5/719: unlink d0/d29/l8d 0 2024-04-12T05:55:32.352 INFO:tasks.workunit.client.0.smithi008.stdout:6/497: dwrite df/d10/f62 [0,4194304] 0 2024-04-12T05:55:32.356 INFO:tasks.workunit.client.0.smithi008.stdout:5/720: chown d0/dc/d13/d1a/daf/d37/lb8 110526 1 2024-04-12T05:55:32.357 INFO:tasks.workunit.client.0.smithi008.stdout:4/353: sync 2024-04-12T05:55:32.357 INFO:tasks.workunit.client.0.smithi008.stdout:8/405: sync 2024-04-12T05:55:32.359 INFO:tasks.workunit.client.0.smithi008.stdout:2/444: dwrite d6/db/f2c [0,4194304] 0 2024-04-12T05:55:32.359 INFO:tasks.workunit.client.0.smithi008.stdout:5/721: fdatasync d0/dc/d13/f54 0 2024-04-12T05:55:32.361 INFO:tasks.workunit.client.0.smithi008.stdout:4/354: chown d1/f20 1938858 1 2024-04-12T05:55:32.361 INFO:tasks.workunit.client.0.smithi008.stdout:6/498: stat df/d10/c59 0 2024-04-12T05:55:32.364 INFO:tasks.workunit.client.0.smithi008.stdout:8/406: creat dd/de/d16/d50/d5e/f8c x:0 0 0 2024-04-12T05:55:32.365 INFO:tasks.workunit.client.0.smithi008.stdout:2/445: fdatasync d6/db/d38/f7e 0 2024-04-12T05:55:32.368 INFO:tasks.workunit.client.0.smithi008.stdout:8/407: dread - dd/de/d16/d50/f7f zero size 2024-04-12T05:55:32.369 INFO:tasks.workunit.client.0.smithi008.stdout:6/499: dread - df/d10/d3a/d78/f91 zero size 2024-04-12T05:55:32.369 INFO:tasks.workunit.client.0.smithi008.stdout:4/355: mkdir d1/d22/d29/d30/d47/d28/d4a/d62 0 2024-04-12T05:55:32.371 INFO:tasks.workunit.client.0.smithi008.stdout:2/446: chown d6/db/d60/d57/c8d 4082351 1 2024-04-12T05:55:32.372 INFO:tasks.workunit.client.0.smithi008.stdout:4/356: write d1/db/f18 [197878,69876] 0 2024-04-12T05:55:32.373 INFO:tasks.workunit.client.0.smithi008.stdout:8/408: creat dd/d4f/d52/f8d x:0 0 0 2024-04-12T05:55:32.373 INFO:tasks.workunit.client.0.smithi008.stdout:8/409: dread - dd/f49 zero size 2024-04-12T05:55:32.375 INFO:tasks.workunit.client.0.smithi008.stdout:8/410: dread - dd/de/d16/f80 zero size 2024-04-12T05:55:32.377 INFO:tasks.workunit.client.0.smithi008.stdout:5/722: link d0/dd/d3d/d5d/f7e d0/dd/d11/d38/fdf 0 2024-04-12T05:55:32.378 INFO:tasks.workunit.client.0.smithi008.stdout:6/500: rename df/d10/d20/d3e/c47 to df/d10/d3a/d78/d8b/ca5 0 2024-04-12T05:55:32.380 INFO:tasks.workunit.client.0.smithi008.stdout:5/723: write d0/dc/d57/d9d/db5/fd4 [255916,11203] 0 2024-04-12T05:55:32.380 INFO:tasks.workunit.client.0.smithi008.stdout:8/411: mkdir dd/de/d4c/d66/d6d/d8e 0 2024-04-12T05:55:32.387 INFO:tasks.workunit.client.0.smithi008.stdout:8/412: symlink dd/de/d16/d50/d5e/l8f 0 2024-04-12T05:55:32.388 INFO:tasks.workunit.client.0.smithi008.stdout:4/357: write d1/d22/d29/d30/d47/f26 [286007,122402] 0 2024-04-12T05:55:32.390 INFO:tasks.workunit.client.0.smithi008.stdout:3/508: dread d9/d37/d69/f91 [0,4194304] 0 2024-04-12T05:55:32.391 INFO:tasks.workunit.client.0.smithi008.stdout:8/413: chown dd/de/d16/d50/d56/d61/d6a/l70 20692890 1 2024-04-12T05:55:32.392 INFO:tasks.workunit.client.0.smithi008.stdout:4/358: write d1/d22/d29/d30/f57 [2034286,60563] 0 2024-04-12T05:55:32.395 INFO:tasks.workunit.client.0.smithi008.stdout:8/414: chown dd/de/d16/d50/d56/d61/d6a 238 1 2024-04-12T05:55:32.395 INFO:tasks.workunit.client.0.smithi008.stdout:5/724: dwrite d0/dd/d11/d38/d55/f5b [0,4194304] 0 2024-04-12T05:55:32.396 INFO:tasks.workunit.client.0.smithi008.stdout:3/509: symlink d9/d10/d19/d22/d2c/la9 0 2024-04-12T05:55:32.397 INFO:tasks.workunit.client.0.smithi008.stdout:8/415: chown dd/de/d16/d50/d56/d61 2292237 1 2024-04-12T05:55:32.398 INFO:tasks.workunit.client.0.smithi008.stdout:8/416: chown dd/de/d16/f80 125518 1 2024-04-12T05:55:32.399 INFO:tasks.workunit.client.0.smithi008.stdout:5/725: rmdir d0/dc/d57 39 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:32 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.406 INFO:tasks.workunit.client.0.smithi008.stdout:8/417: creat dd/de/d16/d50/d56/f90 x:0 0 0 2024-04-12T05:55:32.409 INFO:tasks.workunit.client.0.smithi008.stdout:3/510: rename d9/d1e/c2d to d9/d10/caa 0 2024-04-12T05:55:32.410 INFO:tasks.workunit.client.0.smithi008.stdout:5/726: dread d0/dd/d11/d38/d55/f5b [0,4194304] 0 2024-04-12T05:55:32.412 INFO:tasks.workunit.client.0.smithi008.stdout:5/727: chown d0/dd/d3d/lbe 249069 1 2024-04-12T05:55:32.414 INFO:tasks.workunit.client.0.smithi008.stdout:4/359: dread d1/db/f1a [0,4194304] 0 2024-04-12T05:55:32.417 INFO:tasks.workunit.client.0.smithi008.stdout:3/511: link d9/d10/d19/d22/d2c/la9 d9/d44/lab 0 2024-04-12T05:55:32.418 INFO:tasks.workunit.client.0.smithi008.stdout:8/418: truncate dd/de/d16/f2e 3704399 0 2024-04-12T05:55:32.421 INFO:tasks.workunit.client.0.smithi008.stdout:4/360: rename d1/d22/d29/f3a to d1/d22/d29/d46/d56/f63 0 2024-04-12T05:55:32.421 INFO:tasks.workunit.client.0.smithi008.stdout:3/512: mknod d9/d10/d19/d3a/cac 0 2024-04-12T05:55:32.422 INFO:tasks.workunit.client.0.smithi008.stdout:4/361: dread d1/d8/f25 [0,4194304] 0 2024-04-12T05:55:32.423 INFO:tasks.workunit.client.0.smithi008.stdout:3/513: dread - d9/d11/d5b/f60 zero size 2024-04-12T05:55:32.426 INFO:tasks.workunit.client.0.smithi008.stdout:8/419: mknod dd/de/d16/d73/c91 0 2024-04-12T05:55:32.429 INFO:tasks.workunit.client.0.smithi008.stdout:3/514: mkdir d9/d10/d19/d3a/dad 0 2024-04-12T05:55:32.430 INFO:tasks.workunit.client.0.smithi008.stdout:4/362: mknod d1/db/c64 0 2024-04-12T05:55:32.431 INFO:tasks.workunit.client.0.smithi008.stdout:5/728: dwrite d0/dd/d3d/d5d/f7e [4194304,4194304] 0 2024-04-12T05:55:32.432 INFO:tasks.workunit.client.0.smithi008.stdout:8/420: creat dd/de/d16/d50/d5e/f92 x:0 0 0 2024-04-12T05:55:32.434 INFO:tasks.workunit.client.0.smithi008.stdout:3/515: chown d9/d10/c23 989597520 1 2024-04-12T05:55:32.435 INFO:tasks.workunit.client.0.smithi008.stdout:8/421: symlink dd/d65/l93 0 2024-04-12T05:55:32.437 INFO:tasks.workunit.client.0.smithi008.stdout:3/516: creat d9/d10/d19/d3a/d7c/fae x:0 0 0 2024-04-12T05:55:32.441 INFO:tasks.workunit.client.0.smithi008.stdout:3/517: creat d9/d10/d19/d22/d2c/faf x:0 0 0 2024-04-12T05:55:32.445 INFO:tasks.workunit.client.0.smithi008.stdout:3/518: symlink d9/d10/d19/d22/lb0 0 2024-04-12T05:55:32.446 INFO:tasks.workunit.client.0.smithi008.stdout:3/519: chown d9/d11/d5b/f8d 89657353 1 2024-04-12T05:55:32.449 INFO:tasks.workunit.client.0.smithi008.stdout:4/363: dwrite d1/d8/f25 [0,4194304] 0 2024-04-12T05:55:32.450 INFO:tasks.workunit.client.0.smithi008.stdout:8/422: write dd/de/f2f [3605430,67450] 0 2024-04-12T05:55:32.454 INFO:tasks.workunit.client.0.smithi008.stdout:8/423: mknod dd/d4f/d52/c94 0 2024-04-12T05:55:32.454 INFO:tasks.workunit.client.0.smithi008.stdout:4/364: fsync d1/d22/d29/d30/f52 0 2024-04-12T05:55:32.456 INFO:tasks.workunit.client.0.smithi008.stdout:4/365: write d1/d4f/f5a [604210,63782] 0 2024-04-12T05:55:32.460 INFO:tasks.workunit.client.0.smithi008.stdout:8/424: rename dd/de/c51 to dd/de/d16/d50/d56/d61/c95 0 2024-04-12T05:55:32.461 INFO:tasks.workunit.client.0.smithi008.stdout:4/366: mknod d1/db/c65 0 2024-04-12T05:55:32.461 INFO:tasks.workunit.client.0.smithi008.stdout:3/520: dread d9/d10/d19/d65/f90 [0,4194304] 0 2024-04-12T05:55:32.466 INFO:tasks.workunit.client.0.smithi008.stdout:4/367: mknod d1/d22/d29/d30/c66 0 2024-04-12T05:55:32.468 INFO:tasks.workunit.client.0.smithi008.stdout:4/368: symlink d1/l67 0 2024-04-12T05:55:32.468 INFO:tasks.workunit.client.0.smithi008.stdout:3/521: dwrite d9/d44/f63 [0,4194304] 0 2024-04-12T05:55:32.470 INFO:tasks.workunit.client.0.smithi008.stdout:4/369: write d1/d8/f55 [157605,115224] 0 2024-04-12T05:55:32.476 INFO:tasks.workunit.client.0.smithi008.stdout:8/425: rmdir dd/d4f 39 2024-04-12T05:55:32.478 INFO:tasks.workunit.client.0.smithi008.stdout:3/522: dread d9/d2e/f3f [0,4194304] 0 2024-04-12T05:55:32.480 INFO:tasks.workunit.client.0.smithi008.stdout:4/370: write d1/d22/d29/d46/d56/f63 [732734,116235] 0 2024-04-12T05:55:32.483 INFO:tasks.workunit.client.0.smithi008.stdout:8/426: creat dd/d4f/f96 x:0 0 0 2024-04-12T05:55:32.483 INFO:tasks.workunit.client.0.smithi008.stdout:3/523: unlink d9/d11/ca1 0 2024-04-12T05:55:32.485 INFO:tasks.workunit.client.0.smithi008.stdout:3/524: symlink d9/d11/d5b/lb1 0 2024-04-12T05:55:32.486 INFO:tasks.workunit.client.0.smithi008.stdout:3/525: read - d9/d10/d19/d3a/d67/f97 zero size 2024-04-12T05:55:32.505 INFO:tasks.workunit.client.0.smithi008.stdout:3/526: dread d9/d1e/f64 [0,4194304] 0 2024-04-12T05:55:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:4/371: truncate d1/d8/f25 963571 0 2024-04-12T05:55:32.513 INFO:tasks.workunit.client.0.smithi008.stdout:8/427: write dd/d30/f35 [581248,24820] 0 2024-04-12T05:55:32.514 INFO:tasks.workunit.client.0.smithi008.stdout:3/527: write d9/d11/f85 [487813,80988] 0 2024-04-12T05:55:32.516 INFO:tasks.workunit.client.0.smithi008.stdout:3/528: write d9/d10/f33 [1698696,23115] 0 2024-04-12T05:55:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:3/529: fsync d9/d44/f6e 0 2024-04-12T05:55:32.518 INFO:tasks.workunit.client.0.smithi008.stdout:8/428: read - dd/de/d16/d50/d5e/f85 zero size 2024-04-12T05:55:32.519 INFO:tasks.workunit.client.0.smithi008.stdout:4/372: dread d1/d22/d29/d46/d56/f63 [0,4194304] 0 2024-04-12T05:55:32.520 INFO:tasks.workunit.client.0.smithi008.stdout:8/429: write dd/d4f/d52/f69 [71494,62455] 0 2024-04-12T05:55:32.528 INFO:tasks.workunit.client.0.smithi008.stdout:4/373: dwrite d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:32.529 INFO:tasks.workunit.client.0.smithi008.stdout:4/374: dread - d1/d8/f5f zero size 2024-04-12T05:55:32.533 INFO:tasks.workunit.client.0.smithi008.stdout:4/375: mknod d1/d22/d29/d30/d47/c68 0 2024-04-12T05:55:32.534 INFO:tasks.workunit.client.0.smithi008.stdout:8/430: truncate dd/f1a 1379729 0 2024-04-12T05:55:32.536 INFO:tasks.workunit.client.0.smithi008.stdout:8/431: readlink dd/l14 0 2024-04-12T05:55:32.536 INFO:tasks.workunit.client.0.smithi008.stdout:4/376: write d1/f12 [5473897,90009] 0 2024-04-12T05:55:32.537 INFO:tasks.workunit.client.0.smithi008.stdout:3/530: dwrite d9/d11/f48 [0,4194304] 0 2024-04-12T05:55:32.539 INFO:tasks.workunit.client.0.smithi008.stdout:3/531: write f7 [1894619,12821] 0 2024-04-12T05:55:32.541 INFO:tasks.workunit.client.0.smithi008.stdout:8/432: mkdir dd/de/d4c/d66/d6d/d8e/d97 0 2024-04-12T05:55:32.545 INFO:tasks.workunit.client.0.smithi008.stdout:4/377: symlink d1/d22/d29/d30/d60/l69 0 2024-04-12T05:55:32.546 INFO:tasks.workunit.client.0.smithi008.stdout:3/532: dread - d9/d2e/d3d/f80 zero size 2024-04-12T05:55:32.546 INFO:tasks.workunit.client.0.smithi008.stdout:8/433: fsync dd/f48 0 2024-04-12T05:55:32.548 INFO:tasks.workunit.client.0.smithi008.stdout:4/378: stat d1/d22/d29/d30/c58 0 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:32.550 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:32 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:32.551 INFO:tasks.workunit.client.0.smithi008.stdout:3/533: creat d9/d10/d19/d22/da8/fb2 x:0 0 0 2024-04-12T05:55:32.551 INFO:tasks.workunit.client.0.smithi008.stdout:4/379: mknod d1/d22/c6a 0 2024-04-12T05:55:32.553 INFO:tasks.workunit.client.0.smithi008.stdout:8/434: creat dd/de/f98 x:0 0 0 2024-04-12T05:55:32.561 INFO:tasks.workunit.client.0.smithi008.stdout:4/380: dwrite d1/d22/d29/d30/f34 [0,4194304] 0 2024-04-12T05:55:32.562 INFO:tasks.workunit.client.0.smithi008.stdout:3/534: dwrite d9/d11/d5b/f68 [0,4194304] 0 2024-04-12T05:55:32.569 INFO:tasks.workunit.client.0.smithi008.stdout:4/381: symlink d1/d22/d29/d30/d47/d28/l6b 0 2024-04-12T05:55:32.572 INFO:tasks.workunit.client.0.smithi008.stdout:8/435: dwrite dd/de/f27 [0,4194304] 0 2024-04-12T05:55:32.574 INFO:tasks.workunit.client.0.smithi008.stdout:8/436: readlink dd/d30/d3b/l88 0 2024-04-12T05:55:32.574 INFO:tasks.workunit.client.0.smithi008.stdout:3/535: rename d9/d11/f48 to d9/d10/fb3 0 2024-04-12T05:55:32.575 INFO:tasks.workunit.client.0.smithi008.stdout:4/382: creat d1/d4f/f6c x:0 0 0 2024-04-12T05:55:32.576 INFO:tasks.workunit.client.0.smithi008.stdout:8/437: fsync dd/de/f29 0 2024-04-12T05:55:32.582 INFO:tasks.workunit.client.0.smithi008.stdout:4/383: mknod d1/d22/d29/d46/c6d 0 2024-04-12T05:55:32.585 INFO:tasks.workunit.client.0.smithi008.stdout:8/438: read dd/de/f2a [3789084,80455] 0 2024-04-12T05:55:32.585 INFO:tasks.workunit.client.0.smithi008.stdout:3/536: dread d9/d10/d19/d22/f4d [0,4194304] 0 2024-04-12T05:55:32.587 INFO:tasks.workunit.client.0.smithi008.stdout:8/439: write dd/d4f/f96 [104382,116163] 0 2024-04-12T05:55:32.590 INFO:tasks.workunit.client.0.smithi008.stdout:4/384: dwrite d1/f36 [0,4194304] 0 2024-04-12T05:55:32.591 INFO:tasks.workunit.client.0.smithi008.stdout:8/440: creat dd/f99 x:0 0 0 2024-04-12T05:55:32.594 INFO:tasks.workunit.client.0.smithi008.stdout:8/441: creat dd/d4f/f9a x:0 0 0 2024-04-12T05:55:32.595 INFO:tasks.workunit.client.0.smithi008.stdout:4/385: mknod d1/db/c6e 0 2024-04-12T05:55:32.596 INFO:tasks.workunit.client.0.smithi008.stdout:4/386: write d1/db/f16 [2853861,124750] 0 2024-04-12T05:55:32.603 INFO:tasks.workunit.client.0.smithi008.stdout:8/442: dwrite dd/d30/d3b/f55 [0,4194304] 0 2024-04-12T05:55:32.603 INFO:tasks.workunit.client.0.smithi008.stdout:4/387: link d1/d8/c1d d1/db/c6f 0 2024-04-12T05:55:32.604 INFO:tasks.workunit.client.0.smithi008.stdout:4/388: write d1/d22/d29/d30/d47/f48 [3412165,111648] 0 2024-04-12T05:55:32.614 INFO:tasks.workunit.client.0.smithi008.stdout:4/389: dread d1/d4f/f5b [0,4194304] 0 2024-04-12T05:55:32.616 INFO:tasks.workunit.client.0.smithi008.stdout:4/390: mknod d1/d4f/c70 0 2024-04-12T05:55:32.616 INFO:tasks.workunit.client.0.smithi008.stdout:4/391: truncate d1/f20 5094615 0 2024-04-12T05:55:32.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/392: dread d1/d8/f23 [0,4194304] 0 2024-04-12T05:55:32.622 INFO:tasks.workunit.client.0.smithi008.stdout:4/393: getdents d1/d22/d29/d30/d47/d28/d4a 0 2024-04-12T05:55:32.623 INFO:tasks.workunit.client.0.smithi008.stdout:4/394: write d1/d4f/f5a [1243711,6416] 0 2024-04-12T05:55:32.629 INFO:tasks.workunit.client.0.smithi008.stdout:4/395: truncate d1/d8/f42 265561 0 2024-04-12T05:55:32.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/396: truncate d1/d8/f37 2329575 0 2024-04-12T05:55:32.635 INFO:tasks.workunit.client.0.smithi008.stdout:4/397: write d1/d22/f4c [884911,18260] 0 2024-04-12T05:55:32.638 INFO:tasks.workunit.client.0.smithi008.stdout:4/398: mknod d1/d22/c71 0 2024-04-12T05:55:32.638 INFO:tasks.workunit.client.0.smithi008.stdout:4/399: stat d1/d8/l10 0 2024-04-12T05:55:32.643 INFO:tasks.workunit.client.0.smithi008.stdout:4/400: rmdir d1/d22 39 2024-04-12T05:55:32.643 INFO:tasks.workunit.client.0.smithi008.stdout:4/401: readlink d1/d8/l5e 0 2024-04-12T05:55:32.647 INFO:tasks.workunit.client.0.smithi008.stdout:4/402: write d1/d22/d29/f41 [1083827,108799] 0 2024-04-12T05:55:32.650 INFO:tasks.workunit.client.0.smithi008.stdout:4/403: link d1/d22/d29/c2e d1/d22/d29/d30/d47/c72 0 2024-04-12T05:55:32.651 INFO:tasks.workunit.client.0.smithi008.stdout:4/404: chown d1/d22/d29/d30/d47/d28/l6b 3959279 1 2024-04-12T05:55:32.652 INFO:tasks.workunit.client.0.smithi008.stdout:4/405: write d1/d22/d29/d30/d47/f48 [3953723,117569] 0 2024-04-12T05:55:32.656 INFO:tasks.workunit.client.0.smithi008.stdout:4/406: dread d1/d4f/f5b [0,4194304] 0 2024-04-12T05:55:32.657 INFO:tasks.workunit.client.0.smithi008.stdout:4/407: fsync d1/d22/d29/d30/f34 0 2024-04-12T05:55:32.658 INFO:tasks.workunit.client.0.smithi008.stdout:4/408: dread - d1/d22/d29/d30/d47/d28/f5c zero size 2024-04-12T05:55:32.667 INFO:tasks.workunit.client.0.smithi008.stdout:4/409: rename d1/f3b to d1/d22/d29/d30/d47/d28/f73 0 2024-04-12T05:55:32.675 INFO:tasks.workunit.client.0.smithi008.stdout:3/537: dread f7 [0,4194304] 0 2024-04-12T05:55:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:4/410: dwrite d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:32.727 INFO:tasks.workunit.client.0.smithi008.stdout:3/538: write d9/d10/f6a [4773794,47612] 0 2024-04-12T05:55:32.729 INFO:tasks.workunit.client.0.smithi008.stdout:1/394: sync 2024-04-12T05:55:32.736 INFO:tasks.workunit.client.0.smithi008.stdout:3/539: dread d9/d1e/f64 [0,4194304] 0 2024-04-12T05:55:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:3/540: symlink d9/d10/d19/d22/d2c/lb4 0 2024-04-12T05:55:32.739 INFO:tasks.workunit.client.0.smithi008.stdout:1/395: dread d4/d1a/f51 [0,4194304] 0 2024-04-12T05:55:32.741 INFO:tasks.workunit.client.0.smithi008.stdout:3/541: symlink d9/d2e/lb5 0 2024-04-12T05:55:32.742 INFO:tasks.workunit.client.0.smithi008.stdout:1/396: creat d4/d33/d74/d30/d72/d7c/f80 x:0 0 0 2024-04-12T05:55:32.743 INFO:tasks.workunit.client.0.smithi008.stdout:3/542: mknod d9/d37/d69/cb6 0 2024-04-12T05:55:32.746 INFO:tasks.workunit.client.0.smithi008.stdout:3/543: mkdir d9/db7 0 2024-04-12T05:55:32.748 INFO:tasks.workunit.client.0.smithi008.stdout:4/411: dread d1/f40 [0,4194304] 0 2024-04-12T05:55:32.750 INFO:tasks.workunit.client.0.smithi008.stdout:3/544: dread d9/d2e/f3f [0,4194304] 0 2024-04-12T05:55:32.751 INFO:tasks.workunit.client.0.smithi008.stdout:4/412: mkdir d1/d22/d29/d30/d47/d28/d74 0 2024-04-12T05:55:32.751 INFO:tasks.workunit.client.0.smithi008.stdout:3/545: read d9/d44/f63 [3312607,129991] 0 2024-04-12T05:55:32.756 INFO:tasks.workunit.client.0.smithi008.stdout:1/397: dwrite d4/d33/d74/d30/f49 [0,4194304] 0 2024-04-12T05:55:32.757 INFO:tasks.workunit.client.0.smithi008.stdout:3/546: unlink d9/d10/d19/d22/c24 0 2024-04-12T05:55:32.759 INFO:tasks.workunit.client.0.smithi008.stdout:1/398: read d4/dd/d17/f64 [82724,50786] 0 2024-04-12T05:55:32.761 INFO:tasks.workunit.client.0.smithi008.stdout:3/547: symlink d9/lb8 0 2024-04-12T05:55:32.762 INFO:tasks.workunit.client.0.smithi008.stdout:3/548: write d9/d44/f9f [145064,3013] 0 2024-04-12T05:55:32.899 INFO:tasks.workunit.client.0.smithi008.stdout:4/413: getdents d1/d22/d29/d30/d47/d28 0 2024-04-12T05:55:32.900 INFO:tasks.workunit.client.0.smithi008.stdout:3/549: symlink d9/d37/lb9 0 2024-04-12T05:55:32.902 INFO:tasks.workunit.client.0.smithi008.stdout:1/399: link d4/d1a/f51 d4/d7/d18/f81 0 2024-04-12T05:55:32.902 INFO:tasks.workunit.client.0.smithi008.stdout:3/550: truncate d9/d10/d19/d65/f8e 247160 0 2024-04-12T05:55:32.904 INFO:tasks.workunit.client.0.smithi008.stdout:1/400: symlink d4/d6/d11/l82 0 2024-04-12T05:55:32.905 INFO:tasks.workunit.client.0.smithi008.stdout:3/551: truncate d9/dd/fa6 844040 0 2024-04-12T05:55:32.905 INFO:tasks.workunit.client.0.smithi008.stdout:4/414: dread d1/d22/f4c [0,4194304] 0 2024-04-12T05:55:32.906 INFO:tasks.workunit.client.0.smithi008.stdout:3/552: write d9/d10/d19/d22/fa5 [980331,72961] 0 2024-04-12T05:55:32.912 INFO:tasks.workunit.client.0.smithi008.stdout:4/415: dwrite d1/d22/d29/d30/f34 [0,4194304] 0 2024-04-12T05:55:32.913 INFO:tasks.workunit.client.0.smithi008.stdout:4/416: chown d1/d22/d29/d30/d60 14702916 1 2024-04-12T05:55:32.914 INFO:tasks.workunit.client.0.smithi008.stdout:4/417: write d1/db/f18 [3748595,71860] 0 2024-04-12T05:55:32.921 INFO:tasks.workunit.client.0.smithi008.stdout:0/439: sync 2024-04-12T05:55:32.921 INFO:tasks.workunit.client.0.smithi008.stdout:7/556: sync 2024-04-12T05:55:32.921 INFO:tasks.workunit.client.0.smithi008.stdout:2/447: sync 2024-04-12T05:55:32.921 INFO:tasks.workunit.client.0.smithi008.stdout:9/512: sync 2024-04-12T05:55:32.921 INFO:tasks.workunit.client.0.smithi008.stdout:9/513: chown d3/df 29075 1 2024-04-12T05:55:32.927 INFO:tasks.workunit.client.0.smithi008.stdout:1/401: creat d4/d6/d11/f83 x:0 0 0 2024-04-12T05:55:32.936 INFO:tasks.workunit.client.0.smithi008.stdout:1/402: write d4/d6/f59 [809941,123701] 0 2024-04-12T05:55:32.936 INFO:tasks.workunit.client.0.smithi008.stdout:9/514: chown d3/df/d13/f7f 355511 1 2024-04-12T05:55:32.937 INFO:tasks.workunit.client.0.smithi008.stdout:7/557: dread d1/d5/f42 [0,4194304] 0 2024-04-12T05:55:32.940 INFO:tasks.workunit.client.0.smithi008.stdout:1/403: write d4/d6/d11/f69 [1547987,35647] 0 2024-04-12T05:55:32.941 INFO:tasks.workunit.client.0.smithi008.stdout:7/558: mknod d1/d5/da/d60/da1/cc7 0 2024-04-12T05:55:32.942 INFO:tasks.workunit.client.0.smithi008.stdout:4/418: dread d1/d8/f37 [0,4194304] 0 2024-04-12T05:55:32.943 INFO:tasks.workunit.client.0.smithi008.stdout:4/419: chown d1/d8/l10 4963 1 2024-04-12T05:55:32.943 INFO:tasks.workunit.client.0.smithi008.stdout:9/515: truncate d3/d22/d2b/f41 331352 0 2024-04-12T05:55:32.946 INFO:tasks.workunit.client.0.smithi008.stdout:1/404: unlink d4/dd/d21/f5c 0 2024-04-12T05:55:32.948 INFO:tasks.workunit.client.0.smithi008.stdout:7/559: readlink d1/d5/da/d1d/l2b 0 2024-04-12T05:55:32.952 INFO:tasks.workunit.client.0.smithi008.stdout:3/553: truncate d9/d11/d5b/f68 774962 0 2024-04-12T05:55:32.955 INFO:tasks.workunit.client.0.smithi008.stdout:1/405: creat d4/dd/d21/f84 x:0 0 0 2024-04-12T05:55:32.955 INFO:tasks.workunit.client.0.smithi008.stdout:0/440: truncate d2/d11/d2a/f7b 473089 0 2024-04-12T05:55:32.958 INFO:tasks.workunit.client.0.smithi008.stdout:9/516: creat d3/df/d13/d21/d5c/da4/fb4 x:0 0 0 2024-04-12T05:55:32.959 INFO:tasks.workunit.client.0.smithi008.stdout:2/448: dwrite d6/db/d10/d23/d35/d3c/f8f [0,4194304] 0 2024-04-12T05:55:32.960 INFO:tasks.workunit.client.0.smithi008.stdout:3/554: write d9/d10/d19/d22/f32 [2108926,59917] 0 2024-04-12T05:55:32.960 INFO:tasks.workunit.client.0.smithi008.stdout:4/420: dread d1/d22/d29/d30/f57 [0,4194304] 0 2024-04-12T05:55:32.963 INFO:tasks.workunit.client.0.smithi008.stdout:3/555: chown d9/dd/l16 132175301 1 2024-04-12T05:55:32.964 INFO:tasks.workunit.client.0.smithi008.stdout:6/501: sync 2024-04-12T05:55:32.965 INFO:tasks.workunit.client.0.smithi008.stdout:1/406: chown d4/d33/d37/c5b 608966 1 2024-04-12T05:55:32.966 INFO:tasks.workunit.client.0.smithi008.stdout:3/556: stat d9/d2e/f94 0 2024-04-12T05:55:32.970 INFO:tasks.workunit.client.0.smithi008.stdout:0/441: dwrite d2/ff [4194304,4194304] 0 2024-04-12T05:55:32.971 INFO:tasks.workunit.client.0.smithi008.stdout:6/502: creat df/d10/d3a/d78/fa6 x:0 0 0 2024-04-12T05:55:32.971 INFO:tasks.workunit.client.0.smithi008.stdout:0/442: readlink d2/d11/l1e 0 2024-04-12T05:55:32.973 INFO:tasks.workunit.client.0.smithi008.stdout:2/449: rename d6/db/d29/c5d to d6/db/d6f/c96 0 2024-04-12T05:55:32.977 INFO:tasks.workunit.client.0.smithi008.stdout:0/443: mkdir d2/d11/d36/d42/d86 0 2024-04-12T05:55:32.978 INFO:tasks.workunit.client.0.smithi008.stdout:7/560: truncate d1/d5/da/d1d/d82/fb9 7785931 0 2024-04-12T05:55:32.979 INFO:tasks.workunit.client.0.smithi008.stdout:4/421: getdents d1/d22/d29/d30/d47/d28/d4a/d62 0 2024-04-12T05:55:32.980 INFO:tasks.workunit.client.0.smithi008.stdout:6/503: creat df/d10/d3a/d95/fa7 x:0 0 0 2024-04-12T05:55:32.981 INFO:tasks.workunit.client.0.smithi008.stdout:3/557: dread d9/d1e/f64 [0,4194304] 0 2024-04-12T05:55:32.982 INFO:tasks.workunit.client.0.smithi008.stdout:6/504: write df/d21/d4c/f6b [311280,48530] 0 2024-04-12T05:55:32.984 INFO:tasks.workunit.client.0.smithi008.stdout:6/505: write df/d10/d20/d3e/f72 [179254,50466] 0 2024-04-12T05:55:32.984 INFO:tasks.workunit.client.0.smithi008.stdout:2/450: rename d6/ff to d6/db/d10/d7b/d7d/f97 0 2024-04-12T05:55:32.986 INFO:tasks.workunit.client.0.smithi008.stdout:2/451: chown d6/db/d10/d23/c26 32223 1 2024-04-12T05:55:32.986 INFO:tasks.workunit.client.0.smithi008.stdout:0/444: creat d2/d11/d36/d47/f87 x:0 0 0 2024-04-12T05:55:32.988 INFO:tasks.workunit.client.0.smithi008.stdout:2/452: chown d6/db/d38/f63 102829479 1 2024-04-12T05:55:32.989 INFO:tasks.workunit.client.0.smithi008.stdout:3/558: write f7 [2121151,33136] 0 2024-04-12T05:55:32.990 INFO:tasks.workunit.client.0.smithi008.stdout:2/453: readlink d6/db/d10/d23/l7a 0 2024-04-12T05:55:32.991 INFO:tasks.workunit.client.0.smithi008.stdout:3/559: write d9/d2e/d3d/f80 [216507,1643] 0 2024-04-12T05:55:32.993 INFO:tasks.workunit.client.0.smithi008.stdout:7/561: creat d1/d15/fc8 x:0 0 0 2024-04-12T05:55:32.998 INFO:tasks.workunit.client.0.smithi008.stdout:7/562: readlink d1/d5/da/d1d/l6b 0 2024-04-12T05:55:32.999 INFO:tasks.workunit.client.0.smithi008.stdout:4/422: link d1/d22/d29/c2e d1/d22/d29/d30/d47/c75 0 2024-04-12T05:55:33.004 INFO:tasks.workunit.client.0.smithi008.stdout:4/423: creat d1/d22/d29/d46/f76 x:0 0 0 2024-04-12T05:55:33.006 INFO:tasks.workunit.client.0.smithi008.stdout:4/424: chown d1/d22/d29/d46/d56 0 1 2024-04-12T05:55:33.006 INFO:tasks.workunit.client.0.smithi008.stdout:7/563: creat d1/d5/fc9 x:0 0 0 2024-04-12T05:55:33.007 INFO:tasks.workunit.client.0.smithi008.stdout:0/445: fsync d2/d4b/f5f 0 2024-04-12T05:55:33.009 INFO:tasks.workunit.client.0.smithi008.stdout:7/564: truncate d1/d5/fc9 213000 0 2024-04-12T05:55:33.012 INFO:tasks.workunit.client.0.smithi008.stdout:3/560: dread d9/d1e/f42 [0,4194304] 0 2024-04-12T05:55:33.016 INFO:tasks.workunit.client.0.smithi008.stdout:4/425: truncate d1/d8/f25 1332132 0 2024-04-12T05:55:33.016 INFO:tasks.workunit.client.0.smithi008.stdout:2/454: dread d6/f9 [0,4194304] 0 2024-04-12T05:55:33.018 INFO:tasks.workunit.client.0.smithi008.stdout:3/561: creat d9/d37/fba x:0 0 0 2024-04-12T05:55:33.020 INFO:tasks.workunit.client.0.smithi008.stdout:0/446: rename d2/d24/f28 to d2/d11/d2a/f88 0 2024-04-12T05:55:33.022 INFO:tasks.workunit.client.0.smithi008.stdout:4/426: truncate d1/d22/d29/d30/d47/d28/f3f 6453574 0 2024-04-12T05:55:33.022 INFO:tasks.workunit.client.0.smithi008.stdout:2/455: truncate d6/db/fd 659888 0 2024-04-12T05:55:33.023 INFO:tasks.workunit.client.0.smithi008.stdout:0/447: chown d2/d11/c2b 4 1 2024-04-12T05:55:33.025 INFO:tasks.workunit.client.0.smithi008.stdout:3/562: link d9/d10/d19/d22/d2c/la4 d9/d1e/lbb 0 2024-04-12T05:55:33.025 INFO:tasks.workunit.client.0.smithi008.stdout:0/448: readlink d2/l78 0 2024-04-12T05:55:33.025 INFO:tasks.workunit.client.0.smithi008.stdout:2/456: readlink d6/l92 0 2024-04-12T05:55:33.029 INFO:tasks.workunit.client.0.smithi008.stdout:7/565: dwrite d1/d49/f40 [0,4194304] 0 2024-04-12T05:55:33.029 INFO:tasks.workunit.client.0.smithi008.stdout:0/449: dread - d2/d11/d2a/f7e zero size 2024-04-12T05:55:33.029 INFO:tasks.workunit.client.0.smithi008.stdout:4/427: getdents d1/d22/d29/d46/d56 0 2024-04-12T05:55:33.030 INFO:tasks.workunit.client.0.smithi008.stdout:2/457: readlink d6/db/d10/d23/d35/d3c/l62 0 2024-04-12T05:55:33.040 INFO:tasks.workunit.client.0.smithi008.stdout:2/458: dwrite f0 [0,4194304] 0 2024-04-12T05:55:33.040 INFO:tasks.workunit.client.0.smithi008.stdout:7/566: symlink d1/d15/d43/d54/d58/dc2/lca 0 2024-04-12T05:55:33.042 INFO:tasks.workunit.client.0.smithi008.stdout:2/459: read d6/db/d10/d23/f4d [4246516,42253] 0 2024-04-12T05:55:33.042 INFO:tasks.workunit.client.0.smithi008.stdout:2/460: chown d6/db/d6f/f88 448 1 2024-04-12T05:55:33.044 INFO:tasks.workunit.client.0.smithi008.stdout:0/450: rename d2/d11/d2a/f49 to d2/d24/d61/f89 0 2024-04-12T05:55:33.044 INFO:tasks.workunit.client.0.smithi008.stdout:4/428: truncate d1/d8/f25 2216981 0 2024-04-12T05:55:33.045 INFO:tasks.workunit.client.0.smithi008.stdout:3/563: link d9/d37/l54 d9/d2e/lbc 0 2024-04-12T05:55:33.045 INFO:tasks.workunit.client.0.smithi008.stdout:4/429: chown d1/d22/d29/d46 30117 1 2024-04-12T05:55:33.046 INFO:tasks.workunit.client.0.smithi008.stdout:3/564: write f7 [3546558,129561] 0 2024-04-12T05:55:33.049 INFO:tasks.workunit.client.0.smithi008.stdout:2/461: creat d6/db/d10/d7b/d7d/f98 x:0 0 0 2024-04-12T05:55:33.049 INFO:tasks.workunit.client.0.smithi008.stdout:3/565: read d9/d1e/f35 [193013,76235] 0 2024-04-12T05:55:33.051 INFO:tasks.workunit.client.0.smithi008.stdout:7/567: write d1/f4f [362654,104148] 0 2024-04-12T05:55:33.052 INFO:tasks.workunit.client.0.smithi008.stdout:4/430: mknod d1/d22/d29/d30/d47/d28/d4a/d62/c77 0 2024-04-12T05:55:33.054 INFO:tasks.workunit.client.0.smithi008.stdout:4/431: read d1/d22/d29/d30/d47/f48 [1060682,95796] 0 2024-04-12T05:55:33.055 INFO:tasks.workunit.client.0.smithi008.stdout:2/462: rename d6/db/d60/f42 to d6/db/d38/d41/f99 0 2024-04-12T05:55:33.055 INFO:tasks.workunit.client.0.smithi008.stdout:7/568: fdatasync d1/d15/d43/d54/d77/f96 0 2024-04-12T05:55:33.056 INFO:tasks.workunit.client.0.smithi008.stdout:4/432: chown d1/db/c65 3 1 2024-04-12T05:55:33.057 INFO:tasks.workunit.client.0.smithi008.stdout:7/569: write d1/d5/da/f97 [649329,46635] 0 2024-04-12T05:55:33.058 INFO:tasks.workunit.client.0.smithi008.stdout:2/463: chown d6/db/d38/d41/l34 11476 1 2024-04-12T05:55:33.059 INFO:tasks.workunit.client.0.smithi008.stdout:0/451: link d2/d24/d39/c5b d2/d24/d61/c8a 0 2024-04-12T05:55:33.063 INFO:tasks.workunit.client.0.smithi008.stdout:4/433: dread d1/d8/f42 [0,4194304] 0 2024-04-12T05:55:33.063 INFO:tasks.workunit.client.0.smithi008.stdout:7/570: chown d1/d5/da/d60/c99 0 1 2024-04-12T05:55:33.067 INFO:tasks.workunit.client.0.smithi008.stdout:3/566: dread d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:33.068 INFO:tasks.workunit.client.0.smithi008.stdout:3/567: chown d9/c96 2283 1 2024-04-12T05:55:33.069 INFO:tasks.workunit.client.0.smithi008.stdout:3/568: chown d9/d44/c8f 9515907 1 2024-04-12T05:55:33.071 INFO:tasks.workunit.client.0.smithi008.stdout:4/434: getdents d1/d22/d29/d30/d47/d28/d4a 0 2024-04-12T05:55:33.074 INFO:tasks.workunit.client.0.smithi008.stdout:3/569: rename c4 to d9/d10/d19/d3a/d67/cbd 0 2024-04-12T05:55:33.076 INFO:tasks.workunit.client.0.smithi008.stdout:2/464: write d6/f6d [1504389,19910] 0 2024-04-12T05:55:33.078 INFO:tasks.workunit.client.0.smithi008.stdout:7/571: truncate d1/d5/da/d11/fbc 1129151 0 2024-04-12T05:55:33.079 INFO:tasks.workunit.client.0.smithi008.stdout:7/572: fsync d1/d5/da/d78/faa 0 2024-04-12T05:55:33.080 INFO:tasks.workunit.client.0.smithi008.stdout:4/435: creat d1/d22/d29/d30/f78 x:0 0 0 2024-04-12T05:55:33.080 INFO:tasks.workunit.client.0.smithi008.stdout:0/452: dread d2/d11/d36/d42/f62 [0,4194304] 0 2024-04-12T05:55:33.081 INFO:tasks.workunit.client.0.smithi008.stdout:2/465: symlink d6/db/d38/d41/l9a 0 2024-04-12T05:55:33.082 INFO:tasks.workunit.client.0.smithi008.stdout:0/453: fdatasync d2/d11/d36/d47/f87 0 2024-04-12T05:55:33.083 INFO:tasks.workunit.client.0.smithi008.stdout:2/466: creat d6/db/d6f/f9b x:0 0 0 2024-04-12T05:55:33.084 INFO:tasks.workunit.client.0.smithi008.stdout:7/573: truncate d1/fb 1773466 0 2024-04-12T05:55:33.086 INFO:tasks.workunit.client.0.smithi008.stdout:2/467: rmdir d6/db/d10 39 2024-04-12T05:55:33.089 INFO:tasks.workunit.client.0.smithi008.stdout:0/454: link d2/f66 d2/d11/d36/d47/f8b 0 2024-04-12T05:55:33.090 INFO:tasks.workunit.client.0.smithi008.stdout:7/574: fsync d1/d5/da/d3c/d44/d8a/fa7 0 2024-04-12T05:55:33.090 INFO:tasks.workunit.client.0.smithi008.stdout:0/455: read - d2/d11/d2a/f73 zero size 2024-04-12T05:55:33.092 INFO:tasks.workunit.client.0.smithi008.stdout:3/570: dread d9/d2e/d3d/f73 [0,4194304] 0 2024-04-12T05:55:33.093 INFO:tasks.workunit.client.0.smithi008.stdout:0/456: truncate d2/d11/d36/d47/f2c 4666802 0 2024-04-12T05:55:33.094 INFO:tasks.workunit.client.0.smithi008.stdout:2/468: getdents d6/db/d60/d4f 0 2024-04-12T05:55:33.096 INFO:tasks.workunit.client.0.smithi008.stdout:3/571: mknod d9/d2e/cbe 0 2024-04-12T05:55:33.098 INFO:tasks.workunit.client.0.smithi008.stdout:2/469: fdatasync d6/db/d38/f63 0 2024-04-12T05:55:33.100 INFO:tasks.workunit.client.0.smithi008.stdout:3/572: creat d9/d10/d19/d22/d2c/fbf x:0 0 0 2024-04-12T05:55:33.101 INFO:tasks.workunit.client.0.smithi008.stdout:3/573: readlink d9/d37/l99 0 2024-04-12T05:55:33.105 INFO:tasks.workunit.client.0.smithi008.stdout:2/470: dwrite d6/f43 [0,4194304] 0 2024-04-12T05:55:33.106 INFO:tasks.workunit.client.0.smithi008.stdout:2/471: chown d6/db/d38/d41/l34 32 1 2024-04-12T05:55:33.108 INFO:tasks.workunit.client.0.smithi008.stdout:3/574: link d9/d10/d19/d22/c3e d9/d44/cc0 0 2024-04-12T05:55:33.109 INFO:tasks.workunit.client.0.smithi008.stdout:3/575: chown d9/dd/f7f 80777 1 2024-04-12T05:55:33.112 INFO:tasks.workunit.client.0.smithi008.stdout:3/576: dread d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:33.119 INFO:tasks.workunit.client.0.smithi008.stdout:3/577: dwrite d9/d10/d19/d22/fa5 [0,4194304] 0 2024-04-12T05:55:33.122 INFO:tasks.workunit.client.0.smithi008.stdout:7/575: dread d1/fb [0,4194304] 0 2024-04-12T05:55:33.129 INFO:tasks.workunit.client.0.smithi008.stdout:7/576: dwrite d1/d5/da/f2a [0,4194304] 0 2024-04-12T05:55:33.133 INFO:tasks.workunit.client.0.smithi008.stdout:7/577: fdatasync d1/d5/da/d78/f88 0 2024-04-12T05:55:33.134 INFO:tasks.workunit.client.0.smithi008.stdout:7/578: chown d1/d15/d43/d54/d58/db7/fb6 10938487 1 2024-04-12T05:55:33.137 INFO:tasks.workunit.client.0.smithi008.stdout:7/579: mknod d1/d15/d43/d54/d58/db7/ccb 0 2024-04-12T05:55:33.140 INFO:tasks.workunit.client.0.smithi008.stdout:7/580: symlink d1/d5/da/d60/lcc 0 2024-04-12T05:55:33.147 INFO:tasks.workunit.client.0.smithi008.stdout:7/581: dwrite d1/d5/da/d3c/d44/f51 [12582912,4194304] 0 2024-04-12T05:55:33.149 INFO:tasks.workunit.client.0.smithi008.stdout:7/582: write d1/d5/da/d78/f74 [3731377,69699] 0 2024-04-12T05:55:33.153 INFO:tasks.workunit.client.0.smithi008.stdout:7/583: symlink d1/d5/da/d1d/d82/d9f/lcd 0 2024-04-12T05:55:33.154 INFO:tasks.workunit.client.0.smithi008.stdout:7/584: fsync d1/d5/da/d78/faa 0 2024-04-12T05:55:33.157 INFO:tasks.workunit.client.0.smithi008.stdout:7/585: truncate d1/d5/da/d1d/d68/fa6 1023355 0 2024-04-12T05:55:33.176 INFO:tasks.workunit.client.0.smithi008.stdout:8/443: sync 2024-04-12T05:55:33.176 INFO:tasks.workunit.client.0.smithi008.stdout:5/729: sync 2024-04-12T05:55:33.177 INFO:tasks.workunit.client.0.smithi008.stdout:5/730: write d0/dd/fcc [317963,37806] 0 2024-04-12T05:55:33.178 INFO:tasks.workunit.client.0.smithi008.stdout:8/444: creat dd/d30/d3b/f9b x:0 0 0 2024-04-12T05:55:33.178 INFO:tasks.workunit.client.0.smithi008.stdout:5/731: chown d0/dc/d13/d1a/f22 10796911 1 2024-04-12T05:55:33.178 INFO:tasks.workunit.client.0.smithi008.stdout:8/445: dread - dd/d4f/f7b zero size 2024-04-12T05:55:33.180 INFO:tasks.workunit.client.0.smithi008.stdout:5/732: truncate d0/dd/d11/d38/d55/d89/fc5 343249 0 2024-04-12T05:55:33.181 INFO:tasks.workunit.client.0.smithi008.stdout:8/446: read dd/de/d16/d50/d56/d61/d6a/f6f [1203525,66261] 0 2024-04-12T05:55:33.181 INFO:tasks.workunit.client.0.smithi008.stdout:5/733: truncate d0/dd/fd9 603046 0 2024-04-12T05:55:33.182 INFO:tasks.workunit.client.0.smithi008.stdout:8/447: chown dd/de/d16/f2e 416328013 1 2024-04-12T05:55:33.189 INFO:tasks.workunit.client.0.smithi008.stdout:5/734: dwrite d0/d29/fbf [0,4194304] 0 2024-04-12T05:55:33.190 INFO:tasks.workunit.client.0.smithi008.stdout:8/448: dwrite dd/d4f/f96 [0,4194304] 0 2024-04-12T05:55:33.195 INFO:tasks.workunit.client.0.smithi008.stdout:8/449: mkdir dd/de/d16/d50/d56/d61/d9c 0 2024-04-12T05:55:33.197 INFO:tasks.workunit.client.0.smithi008.stdout:5/735: rename d0/dc/d13/d1a/daf/f21 to d0/d29/d96/db9/fe0 0 2024-04-12T05:55:33.199 INFO:tasks.workunit.client.0.smithi008.stdout:5/736: rmdir d0/dc/d13 39 2024-04-12T05:55:33.202 INFO:tasks.workunit.client.0.smithi008.stdout:5/737: mknod d0/d29/ce1 0 2024-04-12T05:55:33.202 INFO:tasks.workunit.client.0.smithi008.stdout:8/450: dwrite dd/de/d4c/d66/d6c/f87 [0,4194304] 0 2024-04-12T05:55:33.206 INFO:tasks.workunit.client.0.smithi008.stdout:8/451: symlink dd/de/d16/d50/d56/d61/l9d 0 2024-04-12T05:55:33.212 INFO:tasks.workunit.client.0.smithi008.stdout:8/452: link dd/de/d4c/l68 dd/de/d4c/l9e 0 2024-04-12T05:55:33.217 INFO:tasks.workunit.client.0.smithi008.stdout:5/738: dwrite d0/dc/d13/d1a/d3a/f63 [0,4194304] 0 2024-04-12T05:55:33.221 INFO:tasks.workunit.client.0.smithi008.stdout:5/739: read d0/dd/f34 [1827076,61135] 0 2024-04-12T05:55:33.223 INFO:tasks.workunit.client.0.smithi008.stdout:5/740: chown d0/dd/d11/d38 432910 1 2024-04-12T05:55:33.223 INFO:tasks.workunit.client.0.smithi008.stdout:8/453: dwrite dd/f4e [0,4194304] 0 2024-04-12T05:55:33.231 INFO:tasks.workunit.client.0.smithi008.stdout:5/741: dread d0/dd/fcc [0,4194304] 0 2024-04-12T05:55:33.232 INFO:tasks.workunit.client.0.smithi008.stdout:8/454: creat dd/de/d4c/d66/d6d/d8e/f9f x:0 0 0 2024-04-12T05:55:33.236 INFO:tasks.workunit.client.0.smithi008.stdout:8/455: creat dd/de/d4c/d66/d6c/fa0 x:0 0 0 2024-04-12T05:55:33.236 INFO:tasks.workunit.client.0.smithi008.stdout:8/456: read - dd/de/f98 zero size 2024-04-12T05:55:33.238 INFO:tasks.workunit.client.0.smithi008.stdout:8/457: write dd/f36 [4094720,5568] 0 2024-04-12T05:55:33.242 INFO:tasks.workunit.client.0.smithi008.stdout:8/458: fsync dd/f22 0 2024-04-12T05:55:33.246 INFO:tasks.workunit.client.0.smithi008.stdout:5/742: dread d0/dd/d3d/f4b [0,4194304] 0 2024-04-12T05:55:33.253 INFO:tasks.workunit.client.0.smithi008.stdout:8/459: dwrite dd/f22 [0,4194304] 0 2024-04-12T05:55:33.257 INFO:tasks.workunit.client.0.smithi008.stdout:5/743: dwrite d0/dc/d13/f92 [4194304,4194304] 0 2024-04-12T05:55:33.259 INFO:tasks.workunit.client.0.smithi008.stdout:8/460: getdents dd/de/d4c/d66/d6d/d8e 0 2024-04-12T05:55:33.261 INFO:tasks.workunit.client.0.smithi008.stdout:8/461: write dd/f36 [3267771,92301] 0 2024-04-12T05:55:33.267 INFO:tasks.workunit.client.0.smithi008.stdout:5/744: dwrite d0/f97 [0,4194304] 0 2024-04-12T05:55:33.276 INFO:tasks.workunit.client.0.smithi008.stdout:8/462: dwrite dd/de/d16/d50/d56/d61/d6a/f7d [0,4194304] 0 2024-04-12T05:55:33.278 INFO:tasks.workunit.client.0.smithi008.stdout:8/463: rmdir dd/d65 39 2024-04-12T05:55:33.286 INFO:tasks.workunit.client.0.smithi008.stdout:5/745: dread d0/dc/d13/d1a/daf/f2c [0,4194304] 0 2024-04-12T05:55:33.289 INFO:tasks.workunit.client.0.smithi008.stdout:8/464: dwrite dd/de/d16/d50/d56/d61/d6a/f7d [0,4194304] 0 2024-04-12T05:55:33.289 INFO:tasks.workunit.client.0.smithi008.stdout:5/746: chown d0/dc/d13/d1a/daf/d37/c73 17912 1 2024-04-12T05:55:33.290 INFO:tasks.workunit.client.0.smithi008.stdout:8/465: fsync dd/f36 0 2024-04-12T05:55:33.290 INFO:tasks.workunit.client.0.smithi008.stdout:5/747: stat d0/dc/d13/l3b 0 2024-04-12T05:55:33.292 INFO:tasks.workunit.client.0.smithi008.stdout:5/748: write d0/dd/fd9 [229699,65670] 0 2024-04-12T05:55:33.294 INFO:tasks.workunit.client.0.smithi008.stdout:8/466: mknod dd/de/d16/d50/d56/ca1 0 2024-04-12T05:55:33.297 INFO:tasks.workunit.client.0.smithi008.stdout:5/749: rename d0/dc/d13/d1a/daf/d37/l69 to d0/dd/d5c/le2 0 2024-04-12T05:55:33.297 INFO:tasks.workunit.client.0.smithi008.stdout:5/750: chown d0/dc/d13 0 1 2024-04-12T05:55:33.299 INFO:tasks.workunit.client.0.smithi008.stdout:5/751: write d0/dd/fcc [1344563,93554] 0 2024-04-12T05:55:33.300 INFO:tasks.workunit.client.0.smithi008.stdout:5/752: chown d0/dc/d13/d1a/d35/f64 162184 1 2024-04-12T05:55:33.302 INFO:tasks.workunit.client.0.smithi008.stdout:5/753: mkdir d0/dc/d13/d1a/daf/de3 0 2024-04-12T05:55:33.305 INFO:tasks.workunit.client.0.smithi008.stdout:5/754: creat d0/dd/d11/fe4 x:0 0 0 2024-04-12T05:55:33.308 INFO:tasks.workunit.client.0.smithi008.stdout:5/755: mknod d0/dc/d57/d9d/db5/dbc/ce5 0 2024-04-12T05:55:33.313 INFO:tasks.workunit.client.0.smithi008.stdout:5/756: dwrite d0/d40/f77 [0,4194304] 0 2024-04-12T05:55:33.320 INFO:tasks.workunit.client.0.smithi008.stdout:5/757: dwrite d0/dc/d13/d1a/d35/f64 [0,4194304] 0 2024-04-12T05:55:33.321 INFO:tasks.workunit.client.0.smithi008.stdout:5/758: chown d0/dd/d11/d38 231276 1 2024-04-12T05:55:33.325 INFO:tasks.workunit.client.0.smithi008.stdout:5/759: mkdir d0/dc/d13/d1a/d3a/de6 0 2024-04-12T05:55:33.326 INFO:tasks.workunit.client.0.smithi008.stdout:5/760: write d0/dc/fa8 [585226,82839] 0 2024-04-12T05:55:33.342 INFO:tasks.workunit.client.0.smithi008.stdout:5/761: truncate d0/dc/d57/f66 1824229 0 2024-04-12T05:55:33.351 INFO:tasks.workunit.client.0.smithi008.stdout:5/762: dread d0/dd/d11/d38/f2b [0,4194304] 0 2024-04-12T05:55:33.353 INFO:tasks.workunit.client.0.smithi008.stdout:5/763: symlink d0/dd/d11/d38/d55/d89/le7 0 2024-04-12T05:55:33.355 INFO:tasks.workunit.client.0.smithi008.stdout:5/764: read d0/dd/d11/d38/f43 [3924866,31294] 0 2024-04-12T05:55:33.359 INFO:tasks.workunit.client.0.smithi008.stdout:5/765: rename d0/d40/d56/f67 to d0/d29/fe8 0 2024-04-12T05:55:33.364 INFO:tasks.workunit.client.0.smithi008.stdout:5/766: getdents d0/d29/d96 0 2024-04-12T05:55:33.373 INFO:tasks.workunit.client.0.smithi008.stdout:5/767: truncate d0/f12 492016 0 2024-04-12T05:55:33.374 INFO:tasks.workunit.client.0.smithi008.stdout:5/768: chown d0/dc/d13/d1a/d35/f85 233 1 2024-04-12T05:55:33.376 INFO:tasks.workunit.client.0.smithi008.stdout:5/769: fdatasync d0/dd/d11/d38/d55/d89/fc5 0 2024-04-12T05:55:33.384 INFO:tasks.workunit.client.0.smithi008.stdout:5/770: truncate d0/dd/d11/d38/fa5 3393010 0 2024-04-12T05:55:33.386 INFO:tasks.workunit.client.0.smithi008.stdout:9/517: sync 2024-04-12T05:55:33.387 INFO:tasks.workunit.client.0.smithi008.stdout:5/771: chown d0/dc/d13/d1a/daf/l91 174 1 2024-04-12T05:55:33.391 INFO:tasks.workunit.client.0.smithi008.stdout:9/518: mkdir d3/df/d13/d21/d5c/da4/db5 0 2024-04-12T05:55:33.392 INFO:tasks.workunit.client.0.smithi008.stdout:9/519: readlink d3/df/d13/d21/d5c/l5e 0 2024-04-12T05:55:33.398 INFO:tasks.workunit.client.0.smithi008.stdout:9/520: rmdir d3/df/d13 39 2024-04-12T05:55:33.401 INFO:tasks.workunit.client.0.smithi008.stdout:9/521: mkdir d3/d22/d2b/d58/da8/db6 0 2024-04-12T05:55:33.404 INFO:tasks.workunit.client.0.smithi008.stdout:5/772: dread d0/dd/d11/d38/fa5 [0,4194304] 0 2024-04-12T05:55:33.407 INFO:tasks.workunit.client.0.smithi008.stdout:5/773: write d0/dd/d11/d38/f2b [558975,42224] 0 2024-04-12T05:55:33.411 INFO:tasks.workunit.client.0.smithi008.stdout:5/774: rename d0/d29/d96/db9 to d0/d29/de9 0 2024-04-12T05:55:33.413 INFO:tasks.workunit.client.0.smithi008.stdout:9/522: truncate d3/df/d13/d21/d2d/d30/f4d 2118811 0 2024-04-12T05:55:33.417 INFO:tasks.workunit.client.0.smithi008.stdout:5/775: dread d0/dc/d13/d1a/d35/f64 [0,4194304] 0 2024-04-12T05:55:33.419 INFO:tasks.workunit.client.0.smithi008.stdout:5/776: unlink d0/l5f 0 2024-04-12T05:55:33.420 INFO:tasks.workunit.client.0.smithi008.stdout:9/523: dwrite d3/df/d13/d21/d2d/d30/faa [0,4194304] 0 2024-04-12T05:55:33.425 INFO:tasks.workunit.client.0.smithi008.stdout:9/524: unlink d3/fd 0 2024-04-12T05:55:33.426 INFO:tasks.workunit.client.0.smithi008.stdout:5/777: symlink d0/dd/d11/d38/lea 0 2024-04-12T05:55:33.429 INFO:tasks.workunit.client.0.smithi008.stdout:9/525: unlink d3/d22/d2b/d77/l9d 0 2024-04-12T05:55:33.431 INFO:tasks.workunit.client.0.smithi008.stdout:5/778: mknod d0/dc/d13/d1a/daf/de3/ceb 0 2024-04-12T05:55:33.433 INFO:tasks.workunit.client.0.smithi008.stdout:5/779: creat d0/d29/de9/fec x:0 0 0 2024-04-12T05:55:33.435 INFO:tasks.workunit.client.0.smithi008.stdout:5/780: rename d0/dc/d13/d1a/daf/l91 to d0/dd/dad/led 0 2024-04-12T05:55:33.438 INFO:tasks.workunit.client.0.smithi008.stdout:9/526: dread d3/df/d13/f7f [0,4194304] 0 2024-04-12T05:55:33.440 INFO:tasks.workunit.client.0.smithi008.stdout:9/527: write d3/d22/d2b/d58/d89/f8d [1008124,56706] 0 2024-04-12T05:55:33.442 INFO:tasks.workunit.client.0.smithi008.stdout:9/528: rmdir d3/df/d13/d14/d95 39 2024-04-12T05:55:33.444 INFO:tasks.workunit.client.0.smithi008.stdout:9/529: write d3/d22/d2b/d58/f55 [3637063,47252] 0 2024-04-12T05:55:33.454 INFO:tasks.workunit.client.0.smithi008.stdout:9/530: dwrite d3/df/d13/d21/d5c/da4/fb4 [0,4194304] 0 2024-04-12T05:55:33.454 INFO:tasks.workunit.client.0.smithi008.stdout:5/781: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:33.458 INFO:tasks.workunit.client.0.smithi008.stdout:5/782: rename d0/d29/d96/lca to d0/dd/d5c/lee 0 2024-04-12T05:55:33.462 INFO:tasks.workunit.client.0.smithi008.stdout:9/531: symlink d3/df/d13/d21/d2d/d75/lb7 0 2024-04-12T05:55:33.462 INFO:tasks.workunit.client.0.smithi008.stdout:5/783: read d0/dd/d11/d38/d55/d89/fc5 [304683,37493] 0 2024-04-12T05:55:33.462 INFO:tasks.workunit.client.0.smithi008.stdout:9/532: read d3/d22/d2b/d58/d89/f8d [515907,53651] 0 2024-04-12T05:55:33.479 INFO:tasks.workunit.client.0.smithi008.stdout:5/784: truncate d0/dd/d11/d38/fa5 1561790 0 2024-04-12T05:55:33.482 INFO:tasks.workunit.client.0.smithi008.stdout:9/533: dwrite d3/f6 [0,4194304] 0 2024-04-12T05:55:33.486 INFO:tasks.workunit.client.0.smithi008.stdout:9/534: symlink d3/d22/d2b/d77/lb8 0 2024-04-12T05:55:33.488 INFO:tasks.workunit.client.0.smithi008.stdout:9/535: write d3/d22/d2b/d77/d82/f87 [573316,17494] 0 2024-04-12T05:55:33.492 INFO:tasks.workunit.client.0.smithi008.stdout:9/536: symlink d3/df/d13/lb9 0 2024-04-12T05:55:33.493 INFO:tasks.workunit.client.0.smithi008.stdout:9/537: fdatasync d3/df/d13/d21/d2d/d30/d74/f83 0 2024-04-12T05:55:33.494 INFO:tasks.workunit.client.0.smithi008.stdout:9/538: stat d3/d22/d2b/d77/f84 0 2024-04-12T05:55:33.497 INFO:tasks.workunit.client.0.smithi008.stdout:9/539: fsync d3/d22/d2b/f41 0 2024-04-12T05:55:33.499 INFO:tasks.workunit.client.0.smithi008.stdout:5/785: dread d0/dc/d13/d1a/d3a/f93 [0,4194304] 0 2024-04-12T05:55:33.500 INFO:tasks.workunit.client.0.smithi008.stdout:9/540: mkdir d3/dae/dba 0 2024-04-12T05:55:33.502 INFO:tasks.workunit.client.0.smithi008.stdout:5/786: creat d0/dab/fef x:0 0 0 2024-04-12T05:55:33.503 INFO:tasks.workunit.client.0.smithi008.stdout:5/787: truncate d0/d29/f9c 182987 0 2024-04-12T05:55:33.507 INFO:tasks.workunit.client.0.smithi008.stdout:9/541: rename d3/d22/d2b/d4a/d62 to d3/df/d13/d21/d2d/dbb 0 2024-04-12T05:55:33.509 INFO:tasks.workunit.client.0.smithi008.stdout:5/788: getdents d0/d40 0 2024-04-12T05:55:33.512 INFO:tasks.workunit.client.0.smithi008.stdout:5/789: rename d0/dc/cdd to d0/dc/d13/d1a/daf/d37/cf0 0 2024-04-12T05:55:33.513 INFO:tasks.workunit.client.0.smithi008.stdout:5/790: fdatasync d0/dc/d57/d9d/fa4 0 2024-04-12T05:55:33.515 INFO:tasks.workunit.client.0.smithi008.stdout:9/542: truncate d3/d22/d2b/d58/d36/f3c 205615 0 2024-04-12T05:55:33.519 INFO:tasks.workunit.client.0.smithi008.stdout:5/791: rename d0/d29/fe8 to d0/dc/ff1 0 2024-04-12T05:55:33.520 INFO:tasks.workunit.client.0.smithi008.stdout:9/543: write d3/fb [1250697,63674] 0 2024-04-12T05:55:33.521 INFO:tasks.workunit.client.0.smithi008.stdout:5/792: dread - d0/dd/d11/fe4 zero size 2024-04-12T05:55:33.523 INFO:tasks.workunit.client.0.smithi008.stdout:5/793: symlink d0/dd/d11/d38/dbd/lf2 0 2024-04-12T05:55:33.524 INFO:tasks.workunit.client.0.smithi008.stdout:5/794: dread - d0/d40/fb4 zero size 2024-04-12T05:55:33.526 INFO:tasks.workunit.client.0.smithi008.stdout:9/544: link d3/f43 d3/d22/d2b/d77/d82/fbc 0 2024-04-12T05:55:33.527 INFO:tasks.workunit.client.0.smithi008.stdout:5/795: stat d0/dd/c30 0 2024-04-12T05:55:33.529 INFO:tasks.workunit.client.0.smithi008.stdout:5/796: mkdir d0/dd/d11/d38/d55/d89/df3 0 2024-04-12T05:55:33.529 INFO:tasks.workunit.client.0.smithi008.stdout:9/545: getdents d3/df/d13/d21/d2d/dbb 0 2024-04-12T05:55:33.530 INFO:tasks.workunit.client.0.smithi008.stdout:5/797: write d0/dd/d11/fe4 [345796,34099] 0 2024-04-12T05:55:33.542 INFO:tasks.workunit.client.0.smithi008.stdout:9/546: write d3/df/d13/f5a [2673748,112656] 0 2024-04-12T05:55:33.542 INFO:tasks.workunit.client.0.smithi008.stdout:5/798: write d0/dc/f4f [1495090,33249] 0 2024-04-12T05:55:33.544 INFO:tasks.workunit.client.0.smithi008.stdout:9/547: creat d3/d22/d2b/d77/fbd x:0 0 0 2024-04-12T05:55:33.546 INFO:tasks.workunit.client.0.smithi008.stdout:5/799: getdents d0/d29/d96 0 2024-04-12T05:55:33.547 INFO:tasks.workunit.client.0.smithi008.stdout:5/800: read d0/dc/d13/d1a/daf/f2c [2449044,63536] 0 2024-04-12T05:55:33.550 INFO:tasks.workunit.client.0.smithi008.stdout:5/801: creat d0/dd/d11/d38/d55/ff4 x:0 0 0 2024-04-12T05:55:33.553 INFO:tasks.workunit.client.0.smithi008.stdout:5/802: symlink d0/dd/d11/d38/lf5 0 2024-04-12T05:55:33.555 INFO:tasks.workunit.client.0.smithi008.stdout:5/803: mknod d0/dc/d13/d1a/daf/d37/cf6 0 2024-04-12T05:55:33.558 INFO:tasks.workunit.client.0.smithi008.stdout:5/804: rename d0/dd/dad/led to d0/d40/d56/lf7 0 2024-04-12T05:55:33.559 INFO:tasks.workunit.client.0.smithi008.stdout:5/805: fdatasync d0/dc/d13/d1a/f22 0 2024-04-12T05:55:33.562 INFO:tasks.workunit.client.0.smithi008.stdout:1/407: sync 2024-04-12T05:55:33.562 INFO:tasks.workunit.client.0.smithi008.stdout:6/506: sync 2024-04-12T05:55:33.567 INFO:tasks.workunit.client.0.smithi008.stdout:9/548: dread d3/df/f11 [0,4194304] 0 2024-04-12T05:55:33.567 INFO:tasks.workunit.client.0.smithi008.stdout:5/806: dwrite d0/d29/d96/fb2 [0,4194304] 0 2024-04-12T05:55:33.568 INFO:tasks.workunit.client.0.smithi008.stdout:1/408: truncate d4/f9 1827142 0 2024-04-12T05:55:33.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:33 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi194 to 2251M 2024-04-12T05:55:33.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:33 smithi194 ceph-mon[49771]: pgmap v12: 65 pgs: 65 active+clean; 840 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 124 MiB/s wr, 547 op/s 2024-04-12T05:55:33.572 INFO:tasks.workunit.client.0.smithi008.stdout:5/807: creat d0/d29/ff8 x:0 0 0 2024-04-12T05:55:33.575 INFO:tasks.workunit.client.0.smithi008.stdout:6/507: getdents df/d10/d3a/d95 0 2024-04-12T05:55:33.577 INFO:tasks.workunit.client.0.smithi008.stdout:5/808: creat d0/dc/d57/d9d/db5/ff9 x:0 0 0 2024-04-12T05:55:33.578 INFO:tasks.workunit.client.0.smithi008.stdout:6/508: dread - df/d21/f55 zero size 2024-04-12T05:55:33.583 INFO:tasks.workunit.client.0.smithi008.stdout:1/409: write d4/dd/d17/f1f [584230,104667] 0 2024-04-12T05:55:33.591 INFO:tasks.workunit.client.0.smithi008.stdout:9/549: dwrite d3/f25 [0,4194304] 0 2024-04-12T05:55:33.593 INFO:tasks.workunit.client.0.smithi008.stdout:1/410: rename d4/d6/d11/f83 to d4/dd/f85 0 2024-04-12T05:55:33.594 INFO:tasks.workunit.client.0.smithi008.stdout:6/509: dwrite df/d10/d15/f9d [0,4194304] 0 2024-04-12T05:55:33.595 INFO:tasks.workunit.client.0.smithi008.stdout:1/411: write d4/dd/d21/f84 [72677,102137] 0 2024-04-12T05:55:33.600 INFO:tasks.workunit.client.0.smithi008.stdout:5/809: dread d0/dc/d13/f72 [0,4194304] 0 2024-04-12T05:55:33.604 INFO:tasks.workunit.client.0.smithi008.stdout:1/412: creat d4/d33/d74/d46/f86 x:0 0 0 2024-04-12T05:55:33.606 INFO:tasks.workunit.client.0.smithi008.stdout:6/510: dread df/f92 [0,4194304] 0 2024-04-12T05:55:33.608 INFO:tasks.workunit.client.0.smithi008.stdout:5/810: fsync d0/dd/d3d/d5d/faa 0 2024-04-12T05:55:33.608 INFO:tasks.workunit.client.0.smithi008.stdout:6/511: fsync df/d21/f81 0 2024-04-12T05:55:33.611 INFO:tasks.workunit.client.0.smithi008.stdout:1/413: mknod d4/dd/d17/c87 0 2024-04-12T05:55:33.612 INFO:tasks.workunit.client.0.smithi008.stdout:1/414: readlink d4/l16 0 2024-04-12T05:55:33.616 INFO:tasks.workunit.client.0.smithi008.stdout:1/415: creat d4/d6/f88 x:0 0 0 2024-04-12T05:55:33.617 INFO:tasks.workunit.client.0.smithi008.stdout:9/550: truncate d3/df/d13/d21/d2d/d32/d52/fa2 2333091 0 2024-04-12T05:55:33.617 INFO:tasks.workunit.client.0.smithi008.stdout:6/512: link df/d10/c6f df/d10/d15/d4f/ca8 0 2024-04-12T05:55:33.618 INFO:tasks.workunit.client.0.smithi008.stdout:5/811: getdents d0/dd/d3d/d5d 0 2024-04-12T05:55:33.621 INFO:tasks.workunit.client.0.smithi008.stdout:5/812: write d0/f12 [1091583,27373] 0 2024-04-12T05:55:33.621 INFO:tasks.workunit.client.0.smithi008.stdout:9/551: unlink d3/df/d13/c29 0 2024-04-12T05:55:33.624 INFO:tasks.workunit.client.0.smithi008.stdout:5/813: write d0/dd/d11/d38/f43 [3557244,96523] 0 2024-04-12T05:55:33.630 INFO:tasks.workunit.client.0.smithi008.stdout:6/513: creat df/d10/d3a/fa9 x:0 0 0 2024-04-12T05:55:33.631 INFO:tasks.workunit.client.0.smithi008.stdout:1/416: dread d4/d1c/d4f/f6a [0,4194304] 0 2024-04-12T05:55:33.632 INFO:tasks.workunit.client.0.smithi008.stdout:5/814: dread d0/dc/f4f [0,4194304] 0 2024-04-12T05:55:33.633 INFO:tasks.workunit.client.0.smithi008.stdout:9/552: link d3/d22/d2b/d58/f3f d3/df/d13/d21/d2d/d75/d7c/fbe 0 2024-04-12T05:55:33.634 INFO:tasks.workunit.client.0.smithi008.stdout:6/514: symlink df/d10/d3a/d78/laa 0 2024-04-12T05:55:33.638 INFO:tasks.workunit.client.0.smithi008.stdout:5/815: dread d0/dd/d3d/d5d/faa [0,4194304] 0 2024-04-12T05:55:33.640 INFO:tasks.workunit.client.0.smithi008.stdout:1/417: rename d4/d1c/f2b to d4/dd/f89 0 2024-04-12T05:55:33.646 INFO:tasks.workunit.client.0.smithi008.stdout:5/816: symlink d0/dd/d3d/dc9/lfa 0 2024-04-12T05:55:33.647 INFO:tasks.workunit.client.0.smithi008.stdout:6/515: link df/d10/d20/d3e/f5c df/d10/d20/d3e/fab 0 2024-04-12T05:55:33.651 INFO:tasks.workunit.client.0.smithi008.stdout:9/553: rename d3/d22/d2b/d58/d36/lad to d3/df/d13/d21/d2d/d32/lbf 0 2024-04-12T05:55:33.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:33 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi194 to 2251M 2024-04-12T05:55:33.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:33 smithi008 ceph-mon[39722]: pgmap v12: 65 pgs: 65 active+clean; 840 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 124 MiB/s wr, 547 op/s 2024-04-12T05:55:33.654 INFO:tasks.workunit.client.0.smithi008.stdout:6/516: dread df/f23 [0,4194304] 0 2024-04-12T05:55:33.656 INFO:tasks.workunit.client.0.smithi008.stdout:1/418: rename d4/d33/d74/d30/c3d to d4/d33/d74/c8a 0 2024-04-12T05:55:33.657 INFO:tasks.workunit.client.0.smithi008.stdout:5/817: dwrite d0/dc/f65 [0,4194304] 0 2024-04-12T05:55:33.658 INFO:tasks.workunit.client.0.smithi008.stdout:9/554: creat d3/d22/d2b/d58/da8/db6/fc0 x:0 0 0 2024-04-12T05:55:33.661 INFO:tasks.workunit.client.0.smithi008.stdout:1/419: write d4/d24/d62/f7d [755294,11734] 0 2024-04-12T05:55:33.667 INFO:tasks.workunit.client.0.smithi008.stdout:5/818: creat d0/dc/d13/ffb x:0 0 0 2024-04-12T05:55:33.667 INFO:tasks.workunit.client.0.smithi008.stdout:6/517: dread df/d10/d20/d3e/f72 [0,4194304] 0 2024-04-12T05:55:33.667 INFO:tasks.workunit.client.0.smithi008.stdout:1/420: truncate d4/d7/d18/f81 2625020 0 2024-04-12T05:55:33.668 INFO:tasks.workunit.client.0.smithi008.stdout:1/421: chown d4/dd/d17/d26 29357 1 2024-04-12T05:55:33.668 INFO:tasks.workunit.client.0.smithi008.stdout:5/819: chown d0/d40/la3 161 1 2024-04-12T05:55:33.669 INFO:tasks.workunit.client.0.smithi008.stdout:6/518: read - df/d10/d15/d4f/d50/f8d zero size 2024-04-12T05:55:33.671 INFO:tasks.workunit.client.0.smithi008.stdout:1/422: mkdir d4/dd/d8b 0 2024-04-12T05:55:33.672 INFO:tasks.workunit.client.0.smithi008.stdout:5/820: chown d0/dc/d13/d1a/daf/f1b 4973 1 2024-04-12T05:55:33.672 INFO:tasks.workunit.client.0.smithi008.stdout:1/423: truncate d4/d6/d11/f42 4636880 0 2024-04-12T05:55:33.673 INFO:tasks.workunit.client.0.smithi008.stdout:6/519: unlink df/d10/d15/c66 0 2024-04-12T05:55:33.674 INFO:tasks.workunit.client.0.smithi008.stdout:6/520: readlink df/d10/d3a/l93 0 2024-04-12T05:55:33.675 INFO:tasks.workunit.client.0.smithi008.stdout:5/821: unlink d0/dc/d57/d9d/fb3 0 2024-04-12T05:55:33.677 INFO:tasks.workunit.client.0.smithi008.stdout:6/521: symlink df/d10/d15/d79/lac 0 2024-04-12T05:55:33.679 INFO:tasks.workunit.client.0.smithi008.stdout:5/822: fdatasync d0/dc/d13/d1a/f83 0 2024-04-12T05:55:33.680 INFO:tasks.workunit.client.0.smithi008.stdout:1/424: fsync d4/dd/f89 0 2024-04-12T05:55:33.684 INFO:tasks.workunit.client.0.smithi008.stdout:1/425: creat d4/d33/d74/d30/d72/d7c/f8c x:0 0 0 2024-04-12T05:55:33.687 INFO:tasks.workunit.client.0.smithi008.stdout:5/823: rename d0/dc/d13/d1a/d35/l7c to d0/dd/d3d/lfc 0 2024-04-12T05:55:33.688 INFO:tasks.workunit.client.0.smithi008.stdout:5/824: fsync d0/dd/d11/d38/d55/d89/fc5 0 2024-04-12T05:55:33.690 INFO:tasks.workunit.client.0.smithi008.stdout:1/426: creat d4/dd/d17/d61/f8d x:0 0 0 2024-04-12T05:55:33.690 INFO:tasks.workunit.client.0.smithi008.stdout:5/825: truncate d0/d29/de9/fec 146765 0 2024-04-12T05:55:33.691 INFO:tasks.workunit.client.0.smithi008.stdout:9/555: truncate d3/df/d13/f5a 1071565 0 2024-04-12T05:55:33.692 INFO:tasks.workunit.client.0.smithi008.stdout:9/556: readlink d3/d22/l81 0 2024-04-12T05:55:33.693 INFO:tasks.workunit.client.0.smithi008.stdout:1/427: mknod d4/d33/c8e 0 2024-04-12T05:55:33.694 INFO:tasks.workunit.client.0.smithi008.stdout:5/826: mknod d0/dc/d13/d1a/daf/d37/cfd 0 2024-04-12T05:55:33.696 INFO:tasks.workunit.client.0.smithi008.stdout:9/557: symlink d3/d22/d2b/d58/d89/lc1 0 2024-04-12T05:55:33.699 INFO:tasks.workunit.client.0.smithi008.stdout:5/827: symlink d0/d29/lfe 0 2024-04-12T05:55:33.701 INFO:tasks.workunit.client.0.smithi008.stdout:6/522: write df/d10/f2c [230207,109940] 0 2024-04-12T05:55:33.703 INFO:tasks.workunit.client.0.smithi008.stdout:9/558: mknod d3/df/d13/d14/d95/cc2 0 2024-04-12T05:55:33.703 INFO:tasks.workunit.client.0.smithi008.stdout:1/428: creat d4/d1c/f8f x:0 0 0 2024-04-12T05:55:33.704 INFO:tasks.workunit.client.0.smithi008.stdout:1/429: write d4/d7/ff [2215163,116157] 0 2024-04-12T05:55:33.705 INFO:tasks.workunit.client.0.smithi008.stdout:5/828: rename d0/dc/d13/d1a/d3a/f63 to d0/d29/dd6/fff 0 2024-04-12T05:55:33.706 INFO:tasks.workunit.client.0.smithi008.stdout:1/430: fsync d4/d33/d74/d30/d72/d7c/f80 0 2024-04-12T05:55:33.707 INFO:tasks.workunit.client.0.smithi008.stdout:6/523: mkdir df/d8c/da4/dad 0 2024-04-12T05:55:33.709 INFO:tasks.workunit.client.0.smithi008.stdout:1/431: fsync d4/d6/f40 0 2024-04-12T05:55:33.713 INFO:tasks.workunit.client.0.smithi008.stdout:5/829: mknod d0/dd/d11/d38/d55/d89/df3/c100 0 2024-04-12T05:55:33.715 INFO:tasks.workunit.client.0.smithi008.stdout:1/432: rmdir d4/d33/d74 39 2024-04-12T05:55:33.721 INFO:tasks.workunit.client.0.smithi008.stdout:1/433: rename d4/dd/d17/d61 to d4/d6/d90 0 2024-04-12T05:55:33.727 INFO:tasks.workunit.client.0.smithi008.stdout:1/434: mkdir d4/dd/d17/d91 0 2024-04-12T05:55:33.728 INFO:tasks.workunit.client.0.smithi008.stdout:5/830: dread d0/dd/d11/d38/d55/f5b [0,4194304] 0 2024-04-12T05:55:33.729 INFO:tasks.workunit.client.0.smithi008.stdout:1/435: chown d4/d7/fc 52116 1 2024-04-12T05:55:33.734 INFO:tasks.workunit.client.0.smithi008.stdout:1/436: creat d4/d33/d74/d7a/f92 x:0 0 0 2024-04-12T05:55:33.734 INFO:tasks.workunit.client.0.smithi008.stdout:6/524: dwrite df/d10/d15/d4f/f65 [0,4194304] 0 2024-04-12T05:55:33.737 INFO:tasks.workunit.client.0.smithi008.stdout:6/525: write df/d21/d4c/f52 [4969773,43461] 0 2024-04-12T05:55:33.745 INFO:tasks.workunit.client.0.smithi008.stdout:5/831: dwrite d0/d29/de9/fec [0,4194304] 0 2024-04-12T05:55:33.750 INFO:tasks.workunit.client.0.smithi008.stdout:6/526: creat df/d10/d20/fae x:0 0 0 2024-04-12T05:55:33.750 INFO:tasks.workunit.client.0.smithi008.stdout:6/527: write df/d21/d4c/f58 [671213,111182] 0 2024-04-12T05:55:33.755 INFO:tasks.workunit.client.0.smithi008.stdout:5/832: dread d0/dc/d13/d1a/f22 [4194304,4194304] 0 2024-04-12T05:55:33.756 INFO:tasks.workunit.client.0.smithi008.stdout:1/437: dwrite d4/d7/d18/f81 [0,4194304] 0 2024-04-12T05:55:33.757 INFO:tasks.workunit.client.0.smithi008.stdout:6/528: unlink df/d10/c2b 0 2024-04-12T05:55:33.760 INFO:tasks.workunit.client.0.smithi008.stdout:1/438: truncate d4/d7/f4e 612154 0 2024-04-12T05:55:33.760 INFO:tasks.workunit.client.0.smithi008.stdout:6/529: unlink df/d21/f5a 0 2024-04-12T05:55:33.764 INFO:tasks.workunit.client.0.smithi008.stdout:5/833: link d0/d29/ff8 d0/d29/de9/f101 0 2024-04-12T05:55:33.766 INFO:tasks.workunit.client.0.smithi008.stdout:6/530: creat df/d10/d20/d3e/faf x:0 0 0 2024-04-12T05:55:33.769 INFO:tasks.workunit.client.0.smithi008.stdout:6/531: chown f5 574 1 2024-04-12T05:55:33.770 INFO:tasks.workunit.client.0.smithi008.stdout:1/439: dwrite d4/d1a/f6c [0,4194304] 0 2024-04-12T05:55:33.774 INFO:tasks.workunit.client.0.smithi008.stdout:5/834: dread d0/dc/f2a [4194304,4194304] 0 2024-04-12T05:55:33.774 INFO:tasks.workunit.client.0.smithi008.stdout:6/532: write df/f92 [876764,93831] 0 2024-04-12T05:55:33.775 INFO:tasks.workunit.client.0.smithi008.stdout:6/533: chown df/d10/d15/f4b 3872903 1 2024-04-12T05:55:33.777 INFO:tasks.workunit.client.0.smithi008.stdout:1/440: mknod d4/d24/d62/c93 0 2024-04-12T05:55:33.778 INFO:tasks.workunit.client.0.smithi008.stdout:1/441: stat d4/d6/f88 0 2024-04-12T05:55:33.778 INFO:tasks.workunit.client.0.smithi008.stdout:6/534: read df/d21/d4c/f52 [167049,59549] 0 2024-04-12T05:55:33.779 INFO:tasks.workunit.client.0.smithi008.stdout:5/835: mknod d0/dd/d11/d38/c102 0 2024-04-12T05:55:33.781 INFO:tasks.workunit.client.0.smithi008.stdout:1/442: symlink d4/d33/d37/l94 0 2024-04-12T05:55:33.782 INFO:tasks.workunit.client.0.smithi008.stdout:1/443: write d4/dd/d17/f64 [331474,11252] 0 2024-04-12T05:55:33.782 INFO:tasks.workunit.client.0.smithi008.stdout:6/535: unlink df/d10/d15/f7c 0 2024-04-12T05:55:33.783 INFO:tasks.workunit.client.0.smithi008.stdout:5/836: rmdir d0/dd/dad 39 2024-04-12T05:55:33.785 INFO:tasks.workunit.client.0.smithi008.stdout:1/444: mknod d4/dd/c95 0 2024-04-12T05:55:33.787 INFO:tasks.workunit.client.0.smithi008.stdout:5/837: mkdir d0/dc/d13/d1a/daf/d103 0 2024-04-12T05:55:33.818 INFO:tasks.workunit.client.0.smithi008.stdout:1/445: creat d4/d33/d74/d30/d72/f96 x:0 0 0 2024-04-12T05:55:33.819 INFO:tasks.workunit.client.0.smithi008.stdout:1/446: stat d4/d6/d11 0 2024-04-12T05:55:33.819 INFO:tasks.workunit.client.0.smithi008.stdout:1/447: chown d4/dd/d17 125 1 2024-04-12T05:55:33.820 INFO:tasks.workunit.client.0.smithi008.stdout:1/448: readlink d4/dd/l52 0 2024-04-12T05:55:33.821 INFO:tasks.workunit.client.0.smithi008.stdout:5/838: mkdir d0/dc/d57/d9d/d104 0 2024-04-12T05:55:33.830 INFO:tasks.workunit.client.0.smithi008.stdout:5/839: symlink d0/dd/d11/l105 0 2024-04-12T05:55:33.831 INFO:tasks.workunit.client.0.smithi008.stdout:5/840: read d0/d29/d96/fc2 [711771,28193] 0 2024-04-12T05:55:33.832 INFO:tasks.workunit.client.0.smithi008.stdout:5/841: read d0/dc/d13/f92 [7354277,92007] 0 2024-04-12T05:55:33.833 INFO:tasks.workunit.client.0.smithi008.stdout:1/449: link d4/dd/l52 d4/d33/d74/d30/d72/l97 0 2024-04-12T05:55:33.834 INFO:tasks.workunit.client.0.smithi008.stdout:5/842: write d0/dd/d11/d38/f2b [4762856,65847] 0 2024-04-12T05:55:33.847 INFO:tasks.workunit.client.0.smithi008.stdout:4/436: sync 2024-04-12T05:55:33.847 INFO:tasks.workunit.client.0.smithi008.stdout:0/457: sync 2024-04-12T05:55:33.850 INFO:tasks.workunit.client.0.smithi008.stdout:0/458: write d2/d11/d2a/f5d [4544010,48260] 0 2024-04-12T05:55:33.854 INFO:tasks.workunit.client.0.smithi008.stdout:4/437: getdents d1/d22/d29/d30/d47/d28/d74 0 2024-04-12T05:55:33.858 INFO:tasks.workunit.client.0.smithi008.stdout:4/438: creat d1/d22/d29/d30/d47/d28/f79 x:0 0 0 2024-04-12T05:55:33.859 INFO:tasks.workunit.client.0.smithi008.stdout:0/459: rmdir d2/d11/d36/d7f 0 2024-04-12T05:55:33.863 INFO:tasks.workunit.client.0.smithi008.stdout:0/460: rename d2/d24/d61/l7d to d2/d11/d36/l8c 0 2024-04-12T05:55:33.865 INFO:tasks.workunit.client.0.smithi008.stdout:0/461: truncate d2/d11/d36/d42/f62 4922414 0 2024-04-12T05:55:33.867 INFO:tasks.workunit.client.0.smithi008.stdout:0/462: truncate d2/d24/f63 587274 0 2024-04-12T05:55:33.869 INFO:tasks.workunit.client.0.smithi008.stdout:0/463: creat d2/d24/d85/f8d x:0 0 0 2024-04-12T05:55:33.872 INFO:tasks.workunit.client.0.smithi008.stdout:0/464: mknod d2/d24/c8e 0 2024-04-12T05:55:33.874 INFO:tasks.workunit.client.0.smithi008.stdout:6/536: creat df/d21/fb0 x:0 0 0 2024-04-12T05:55:33.875 INFO:tasks.workunit.client.0.smithi008.stdout:0/465: symlink d2/d4b/l8f 0 2024-04-12T05:55:33.875 INFO:tasks.workunit.client.0.smithi008.stdout:0/466: readlink d2/d11/d36/l58 0 2024-04-12T05:55:33.879 INFO:tasks.workunit.client.0.smithi008.stdout:0/467: dread - d2/d24/d39/f60 zero size 2024-04-12T05:55:33.880 INFO:tasks.workunit.client.0.smithi008.stdout:1/450: write d4/f27 [4778840,103458] 0 2024-04-12T05:55:33.882 INFO:tasks.workunit.client.0.smithi008.stdout:0/468: dread - d2/d24/d39/f60 zero size 2024-04-12T05:55:33.884 INFO:tasks.workunit.client.0.smithi008.stdout:0/469: chown d2/d11/d36/f7c 89179939 1 2024-04-12T05:55:33.885 INFO:tasks.workunit.client.0.smithi008.stdout:6/537: dwrite df/d10/d20/d3e/f72 [0,4194304] 0 2024-04-12T05:55:33.887 INFO:tasks.workunit.client.0.smithi008.stdout:5/843: mkdir d0/dc/d106 0 2024-04-12T05:55:33.887 INFO:tasks.workunit.client.0.smithi008.stdout:6/538: write df/d10/d3a/fa0 [758206,41277] 0 2024-04-12T05:55:33.890 INFO:tasks.workunit.client.0.smithi008.stdout:1/451: rename d4/dd/d17/d6b to d4/dd/d21/d98 0 2024-04-12T05:55:33.891 INFO:tasks.workunit.client.0.smithi008.stdout:1/452: fsync d4/d33/d74/d30/d72/d7c/f80 0 2024-04-12T05:55:33.894 INFO:tasks.workunit.client.0.smithi008.stdout:6/539: mkdir df/d8c/db1 0 2024-04-12T05:55:33.897 INFO:tasks.workunit.client.0.smithi008.stdout:1/453: chown d4/d33/d74/c8a 3 1 2024-04-12T05:55:33.900 INFO:tasks.workunit.client.0.smithi008.stdout:4/439: write d1/d4f/f5b [4039003,112775] 0 2024-04-12T05:55:33.903 INFO:tasks.workunit.client.0.smithi008.stdout:0/470: getdents d2 0 2024-04-12T05:55:33.904 INFO:tasks.workunit.client.0.smithi008.stdout:5/844: rename d0/dc/d13/d1a/cd2 to d0/dd/d3d/c107 0 2024-04-12T05:55:33.906 INFO:tasks.workunit.client.0.smithi008.stdout:5/845: write d0/dc/d13/f54 [7480231,38698] 0 2024-04-12T05:55:33.906 INFO:tasks.workunit.client.0.smithi008.stdout:4/440: symlink d1/d22/d29/d46/l7a 0 2024-04-12T05:55:33.910 INFO:tasks.workunit.client.0.smithi008.stdout:6/540: rmdir df/d10/d15/d4f 39 2024-04-12T05:55:33.913 INFO:tasks.workunit.client.0.smithi008.stdout:1/454: creat d4/dd/f99 x:0 0 0 2024-04-12T05:55:33.914 INFO:tasks.workunit.client.0.smithi008.stdout:5/846: dread d0/dc/d13/d1a/daf/f95 [0,4194304] 0 2024-04-12T05:55:33.916 INFO:tasks.workunit.client.0.smithi008.stdout:6/541: creat df/d10/d20/d3e/fb2 x:0 0 0 2024-04-12T05:55:33.916 INFO:tasks.workunit.client.0.smithi008.stdout:0/471: dread d2/d11/f23 [0,4194304] 0 2024-04-12T05:55:33.917 INFO:tasks.workunit.client.0.smithi008.stdout:1/455: stat d4/d1a/d57 0 2024-04-12T05:55:33.918 INFO:tasks.workunit.client.0.smithi008.stdout:4/441: dwrite d1/d22/f4c [0,4194304] 0 2024-04-12T05:55:33.919 INFO:tasks.workunit.client.0.smithi008.stdout:6/542: write df/f1f [36476,90812] 0 2024-04-12T05:55:33.920 INFO:tasks.workunit.client.0.smithi008.stdout:1/456: chown d4/d7/d18/c29 511 1 2024-04-12T05:55:33.920 INFO:tasks.workunit.client.0.smithi008.stdout:5/847: unlink d0/dc/d13/fc6 0 2024-04-12T05:55:33.922 INFO:tasks.workunit.client.0.smithi008.stdout:0/472: readlink d2/d24/l2f 0 2024-04-12T05:55:33.923 INFO:tasks.workunit.client.0.smithi008.stdout:1/457: creat d4/dd/f9a x:0 0 0 2024-04-12T05:55:33.924 INFO:tasks.workunit.client.0.smithi008.stdout:0/473: fsync d2/d11/d2a/f4a 0 2024-04-12T05:55:33.925 INFO:tasks.workunit.client.0.smithi008.stdout:6/543: creat df/d10/d3a/d95/fb3 x:0 0 0 2024-04-12T05:55:33.928 INFO:tasks.workunit.client.0.smithi008.stdout:5/848: unlink d0/dc/d13/d1a/d35/f85 0 2024-04-12T05:55:33.928 INFO:tasks.workunit.client.0.smithi008.stdout:0/474: truncate d2/d11/d36/d47/f68 473385 0 2024-04-12T05:55:33.929 INFO:tasks.workunit.client.0.smithi008.stdout:5/849: chown d0/dd/f34 14958 1 2024-04-12T05:55:33.930 INFO:tasks.workunit.client.0.smithi008.stdout:1/458: write d4/d1c/d4f/f6a [1711685,54053] 0 2024-04-12T05:55:33.932 INFO:tasks.workunit.client.0.smithi008.stdout:5/850: read d0/dd/d3d/f4b [2102240,71489] 0 2024-04-12T05:55:33.932 INFO:tasks.workunit.client.0.smithi008.stdout:4/442: rename d1/d22/d29/d30/d47/c72 to d1/d22/d29/d46/d56/c7b 0 2024-04-12T05:55:33.934 INFO:tasks.workunit.client.0.smithi008.stdout:5/851: stat d0/d29/de9/fec 0 2024-04-12T05:55:33.936 INFO:tasks.workunit.client.0.smithi008.stdout:5/852: stat d0/dd/d11/d38/d55/d89/df3 0 2024-04-12T05:55:33.937 INFO:tasks.workunit.client.0.smithi008.stdout:5/853: chown d0/dd/d11/d38/c102 12741 1 2024-04-12T05:55:33.938 INFO:tasks.workunit.client.0.smithi008.stdout:1/459: creat d4/dd/d17/d26/f9b x:0 0 0 2024-04-12T05:55:33.939 INFO:tasks.workunit.client.0.smithi008.stdout:5/854: chown d0/dd/d11/d38/lea 661953460 1 2024-04-12T05:55:33.941 INFO:tasks.workunit.client.0.smithi008.stdout:6/544: truncate df/d10/d15/f3f 3050031 0 2024-04-12T05:55:33.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/475: truncate d2/d11/d2a/f5d 3809921 0 2024-04-12T05:55:33.945 INFO:tasks.workunit.client.0.smithi008.stdout:6/545: mkdir df/d10/d20/d3e/db4 0 2024-04-12T05:55:33.946 INFO:tasks.workunit.client.0.smithi008.stdout:6/546: write df/d10/d20/fae [63986,24738] 0 2024-04-12T05:55:33.947 INFO:tasks.workunit.client.0.smithi008.stdout:4/443: rename d1/d22/d29/d30/f78 to d1/d22/d29/f7c 0 2024-04-12T05:55:33.950 INFO:tasks.workunit.client.0.smithi008.stdout:6/547: dread - df/d10/d3a/d95/fb3 zero size 2024-04-12T05:55:33.957 INFO:tasks.workunit.client.0.smithi008.stdout:4/444: dwrite d1/d22/f4c [0,4194304] 0 2024-04-12T05:55:33.957 INFO:tasks.workunit.client.0.smithi008.stdout:5/855: creat d0/dd/d3d/f108 x:0 0 0 2024-04-12T05:55:33.959 INFO:tasks.workunit.client.0.smithi008.stdout:4/445: write d1/d22/d29/f41 [1252888,95661] 0 2024-04-12T05:55:33.965 INFO:tasks.workunit.client.0.smithi008.stdout:1/460: rename d4/dd/d17/f75 to d4/d1a/f9c 0 2024-04-12T05:55:33.965 INFO:tasks.workunit.client.0.smithi008.stdout:5/856: fdatasync d0/d29/de9/fe0 0 2024-04-12T05:55:33.966 INFO:tasks.workunit.client.0.smithi008.stdout:1/461: readlink d4/d7/l13 0 2024-04-12T05:55:33.966 INFO:tasks.workunit.client.0.smithi008.stdout:4/446: symlink d1/d22/d29/d30/d47/l7d 0 2024-04-12T05:55:33.967 INFO:tasks.workunit.client.0.smithi008.stdout:1/462: stat d4/d7/d18/l34 0 2024-04-12T05:55:33.969 INFO:tasks.workunit.client.0.smithi008.stdout:0/476: getdents d2 0 2024-04-12T05:55:33.969 INFO:tasks.workunit.client.0.smithi008.stdout:6/548: creat df/d10/fb5 x:0 0 0 2024-04-12T05:55:33.969 INFO:tasks.workunit.client.0.smithi008.stdout:1/463: truncate d4/dd/f39 1600753 0 2024-04-12T05:55:33.973 INFO:tasks.workunit.client.0.smithi008.stdout:1/464: write d4/d7/fe [1124239,55113] 0 2024-04-12T05:55:33.974 INFO:tasks.workunit.client.0.smithi008.stdout:6/549: mknod df/d8c/da4/cb6 0 2024-04-12T05:55:33.978 INFO:tasks.workunit.client.0.smithi008.stdout:4/447: dwrite d1/d4f/f6c [0,4194304] 0 2024-04-12T05:55:33.978 INFO:tasks.workunit.client.0.smithi008.stdout:1/465: symlink d4/d33/l9d 0 2024-04-12T05:55:33.983 INFO:tasks.workunit.client.0.smithi008.stdout:6/550: creat df/d10/d15/d4f/d50/fb7 x:0 0 0 2024-04-12T05:55:33.985 INFO:tasks.workunit.client.0.smithi008.stdout:1/466: mknod d4/d24/d62/c9e 0 2024-04-12T05:55:33.986 INFO:tasks.workunit.client.0.smithi008.stdout:4/448: mknod d1/d22/d29/d30/c7e 0 2024-04-12T05:55:33.987 INFO:tasks.workunit.client.0.smithi008.stdout:1/467: dread - d4/d6/f53 zero size 2024-04-12T05:55:33.991 INFO:tasks.workunit.client.0.smithi008.stdout:6/551: link df/d10/d15/c5d df/d8c/da4/cb8 0 2024-04-12T05:55:33.991 INFO:tasks.workunit.client.0.smithi008.stdout:4/449: dread d1/d22/d29/d30/f52 [0,4194304] 0 2024-04-12T05:55:33.993 INFO:tasks.workunit.client.0.smithi008.stdout:1/468: write d4/d6/d90/f8d [399207,36904] 0 2024-04-12T05:55:33.996 INFO:tasks.workunit.client.0.smithi008.stdout:1/469: chown d4/dd/f85 786 1 2024-04-12T05:55:33.997 INFO:tasks.workunit.client.0.smithi008.stdout:0/477: dread d2/d11/d2a/f5d [0,4194304] 0 2024-04-12T05:55:33.998 INFO:tasks.workunit.client.0.smithi008.stdout:5/857: dwrite d0/dc/d13/d1a/daf/f4 [0,4194304] 0 2024-04-12T05:55:34.002 INFO:tasks.workunit.client.0.smithi008.stdout:5/858: mkdir d0/dab/d109 0 2024-04-12T05:55:34.003 INFO:tasks.workunit.client.0.smithi008.stdout:2/472: sync 2024-04-12T05:55:34.003 INFO:tasks.workunit.client.0.smithi008.stdout:7/586: sync 2024-04-12T05:55:34.004 INFO:tasks.workunit.client.0.smithi008.stdout:8/467: sync 2024-04-12T05:55:34.004 INFO:tasks.workunit.client.0.smithi008.stdout:3/578: sync 2024-04-12T05:55:34.006 INFO:tasks.workunit.client.0.smithi008.stdout:8/468: chown dd/f48 36527613 1 2024-04-12T05:55:34.006 INFO:tasks.workunit.client.0.smithi008.stdout:4/450: getdents d1/d22/d29/d46/d56 0 2024-04-12T05:55:34.006 INFO:tasks.workunit.client.0.smithi008.stdout:0/478: symlink d2/d24/l90 0 2024-04-12T05:55:34.007 INFO:tasks.workunit.client.0.smithi008.stdout:5/859: chown d0/d29/lfe 615771329 1 2024-04-12T05:55:34.009 INFO:tasks.workunit.client.0.smithi008.stdout:3/579: stat d9/d10/d19/d3a/d67/la0 0 2024-04-12T05:55:34.011 INFO:tasks.workunit.client.0.smithi008.stdout:2/473: fsync d6/db/d38/f63 0 2024-04-12T05:55:34.011 INFO:tasks.workunit.client.0.smithi008.stdout:0/479: fdatasync d2/d24/d85/f8d 0 2024-04-12T05:55:34.012 INFO:tasks.workunit.client.0.smithi008.stdout:8/469: creat dd/de/d4c/d66/fa2 x:0 0 0 2024-04-12T05:55:34.013 INFO:tasks.workunit.client.0.smithi008.stdout:5/860: truncate d0/dc/fc1 4784967 0 2024-04-12T05:55:34.013 INFO:tasks.workunit.client.0.smithi008.stdout:4/451: creat d1/d4f/f7f x:0 0 0 2024-04-12T05:55:34.014 INFO:tasks.workunit.client.0.smithi008.stdout:6/552: dwrite df/d10/d15/f9d [0,4194304] 0 2024-04-12T05:55:34.020 INFO:tasks.workunit.client.0.smithi008.stdout:7/587: write d1/d5/da/d78/fbd [234387,68559] 0 2024-04-12T05:55:34.021 INFO:tasks.workunit.client.0.smithi008.stdout:1/470: truncate d4/dd/d21/f84 34699 0 2024-04-12T05:55:34.022 INFO:tasks.workunit.client.0.smithi008.stdout:5/861: stat d0/dc/d57/l25 0 2024-04-12T05:55:34.024 INFO:tasks.workunit.client.0.smithi008.stdout:4/452: creat d1/d22/d29/d46/d56/f80 x:0 0 0 2024-04-12T05:55:34.026 INFO:tasks.workunit.client.0.smithi008.stdout:6/553: chown df/d21/d4c/f58 1394 1 2024-04-12T05:55:34.027 INFO:tasks.workunit.client.0.smithi008.stdout:3/580: link d9/d2e/d3d/f80 d9/db7/fc1 0 2024-04-12T05:55:34.029 INFO:tasks.workunit.client.0.smithi008.stdout:6/554: fdatasync df/d21/fb0 0 2024-04-12T05:55:34.030 INFO:tasks.workunit.client.0.smithi008.stdout:7/588: symlink d1/d5/da/d60/da1/lce 0 2024-04-12T05:55:34.030 INFO:tasks.workunit.client.0.smithi008.stdout:2/474: write d6/db/d38/f5b [3068171,87016] 0 2024-04-12T05:55:34.032 INFO:tasks.workunit.client.0.smithi008.stdout:5/862: fdatasync d0/dc/d13/d1a/f83 0 2024-04-12T05:55:34.033 INFO:tasks.workunit.client.0.smithi008.stdout:8/470: getdents dd/d30/d32 0 2024-04-12T05:55:34.034 INFO:tasks.workunit.client.0.smithi008.stdout:0/480: dwrite d2/d11/d36/d47/f21 [0,4194304] 0 2024-04-12T05:55:34.040 INFO:tasks.workunit.client.0.smithi008.stdout:2/475: mknod d6/db/d29/c9c 0 2024-04-12T05:55:34.042 INFO:tasks.workunit.client.0.smithi008.stdout:0/481: creat d2/d11/d36/d47/f91 x:0 0 0 2024-04-12T05:55:34.044 INFO:tasks.workunit.client.0.smithi008.stdout:7/589: write d1/d49/f40 [3527469,112835] 0 2024-04-12T05:55:34.044 INFO:tasks.workunit.client.0.smithi008.stdout:2/476: chown d6/db/d29/f48 1 1 2024-04-12T05:55:34.045 INFO:tasks.workunit.client.0.smithi008.stdout:6/555: rename df/d10/c68 to df/cb9 0 2024-04-12T05:55:34.048 INFO:tasks.workunit.client.0.smithi008.stdout:5/863: mkdir d0/d29/d96/d10a 0 2024-04-12T05:55:34.050 INFO:tasks.workunit.client.0.smithi008.stdout:0/482: creat d2/d24/d61/f92 x:0 0 0 2024-04-12T05:55:34.051 INFO:tasks.workunit.client.0.smithi008.stdout:6/556: write df/d21/f81 [2886616,66938] 0 2024-04-12T05:55:34.052 INFO:tasks.workunit.client.0.smithi008.stdout:8/471: rename dd/de/d4c/d66/f84 to dd/de/d4c/d66/d6d/d8e/d97/fa3 0 2024-04-12T05:55:34.054 INFO:tasks.workunit.client.0.smithi008.stdout:5/864: fdatasync d0/dc/d57/d9d/db5/fd4 0 2024-04-12T05:55:34.056 INFO:tasks.workunit.client.0.smithi008.stdout:1/471: dwrite d4/d24/f45 [0,4194304] 0 2024-04-12T05:55:34.057 INFO:tasks.workunit.client.0.smithi008.stdout:0/483: write d2/d11/d36/d47/f87 [549088,121185] 0 2024-04-12T05:55:34.058 INFO:tasks.workunit.client.0.smithi008.stdout:7/590: creat d1/d15/d43/d54/d58/db7/fcf x:0 0 0 2024-04-12T05:55:34.062 INFO:tasks.workunit.client.0.smithi008.stdout:3/581: dread d9/d10/fb3 [0,4194304] 0 2024-04-12T05:55:34.063 INFO:tasks.workunit.client.0.smithi008.stdout:8/472: mknod dd/de/d4c/d66/ca4 0 2024-04-12T05:55:34.063 INFO:tasks.workunit.client.0.smithi008.stdout:6/557: creat df/d10/d15/d75/fba x:0 0 0 2024-04-12T05:55:34.064 INFO:tasks.workunit.client.0.smithi008.stdout:5/865: mknod d0/dc/c10b 0 2024-04-12T05:55:34.066 INFO:tasks.workunit.client.0.smithi008.stdout:7/591: chown d1/d5/da/d1d/f5c 70983 1 2024-04-12T05:55:34.067 INFO:tasks.workunit.client.0.smithi008.stdout:8/473: mkdir dd/d4f/d52/da5 0 2024-04-12T05:55:34.068 INFO:tasks.workunit.client.0.smithi008.stdout:2/477: symlink d6/db/d10/d23/d35/l9d 0 2024-04-12T05:55:34.068 INFO:tasks.workunit.client.0.smithi008.stdout:0/484: mknod d2/d11/d36/d82/c93 0 2024-04-12T05:55:34.071 INFO:tasks.workunit.client.0.smithi008.stdout:5/866: creat d0/d29/f10c x:0 0 0 2024-04-12T05:55:34.075 INFO:tasks.workunit.client.0.smithi008.stdout:7/592: rename d1/d5/da/d11/l5b to d1/d5/da/d1d/d68/ld0 0 2024-04-12T05:55:34.077 INFO:tasks.workunit.client.0.smithi008.stdout:2/478: write d6/db/d10/d23/d35/d3c/f8f [1404385,82378] 0 2024-04-12T05:55:34.078 INFO:tasks.workunit.client.0.smithi008.stdout:8/474: dwrite dd/de/f27 [0,4194304] 0 2024-04-12T05:55:34.082 INFO:tasks.workunit.client.0.smithi008.stdout:5/867: write d0/dd/fa1 [645270,95291] 0 2024-04-12T05:55:34.082 INFO:tasks.workunit.client.0.smithi008.stdout:0/485: truncate d2/d24/d39/f60 439041 0 2024-04-12T05:55:34.084 INFO:tasks.workunit.client.0.smithi008.stdout:1/472: write d4/dd/d17/f19 [1271153,35662] 0 2024-04-12T05:55:34.088 INFO:tasks.workunit.client.0.smithi008.stdout:6/558: getdents df/d21/d4c 0 2024-04-12T05:55:34.089 INFO:tasks.workunit.client.0.smithi008.stdout:5/868: mknod d0/d40/d56/c10d 0 2024-04-12T05:55:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:0/486: chown d2/d11/d36/d47/f6d 871412 1 2024-04-12T05:55:34.092 INFO:tasks.workunit.client.0.smithi008.stdout:3/582: dread d9/f82 [0,4194304] 0 2024-04-12T05:55:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:5/869: fdatasync d0/dc/d57/d9d/fa4 0 2024-04-12T05:55:34.095 INFO:tasks.workunit.client.0.smithi008.stdout:8/475: dwrite dd/f4e [4194304,4194304] 0 2024-04-12T05:55:34.096 INFO:tasks.workunit.client.0.smithi008.stdout:1/473: mkdir d4/dd/d21/d9f 0 2024-04-12T05:55:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:1/474: write d4/f9 [3374362,30166] 0 2024-04-12T05:55:34.098 INFO:tasks.workunit.client.0.smithi008.stdout:5/870: fsync d0/dc/d13/ffb 0 2024-04-12T05:55:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:7/593: dread d1/d15/f20 [0,4194304] 0 2024-04-12T05:55:34.101 INFO:tasks.workunit.client.0.smithi008.stdout:5/871: fsync d0/dc/d13/ffb 0 2024-04-12T05:55:34.102 INFO:tasks.workunit.client.0.smithi008.stdout:0/487: creat d2/d11/d36/d47/f94 x:0 0 0 2024-04-12T05:55:34.103 INFO:tasks.workunit.client.0.smithi008.stdout:1/475: fsync d4/d33/d74/d46/f86 0 2024-04-12T05:55:34.105 INFO:tasks.workunit.client.0.smithi008.stdout:8/476: symlink dd/d30/d32/la6 0 2024-04-12T05:55:34.105 INFO:tasks.workunit.client.0.smithi008.stdout:7/594: readlink d1/d5/da/d1d/l6b 0 2024-04-12T05:55:34.106 INFO:tasks.workunit.client.0.smithi008.stdout:6/559: creat df/d10/d15/fbb x:0 0 0 2024-04-12T05:55:34.110 INFO:tasks.workunit.client.0.smithi008.stdout:5/872: mknod d0/dc/d13/d1a/daf/de3/c10e 0 2024-04-12T05:55:34.110 INFO:tasks.workunit.client.0.smithi008.stdout:3/583: write d9/d11/d5b/f68 [977700,121130] 0 2024-04-12T05:55:34.110 INFO:tasks.workunit.client.0.smithi008.stdout:0/488: mkdir d2/d11/d36/d82/d95 0 2024-04-12T05:55:34.112 INFO:tasks.workunit.client.0.smithi008.stdout:6/560: write df/d10/d20/f89 [7421415,64202] 0 2024-04-12T05:55:34.113 INFO:tasks.workunit.client.0.smithi008.stdout:0/489: fdatasync d2/d24/f76 0 2024-04-12T05:55:34.115 INFO:tasks.workunit.client.0.smithi008.stdout:3/584: write d9/dd/fa6 [245384,110455] 0 2024-04-12T05:55:34.118 INFO:tasks.workunit.client.0.smithi008.stdout:1/476: rename d4/d1c/l78 to d4/d1a/la0 0 2024-04-12T05:55:34.119 INFO:tasks.workunit.client.0.smithi008.stdout:7/595: rename d1/d5/f4e to d1/d5/fd1 0 2024-04-12T05:55:34.119 INFO:tasks.workunit.client.0.smithi008.stdout:0/490: truncate d2/d11/d36/d47/f94 133813 0 2024-04-12T05:55:34.122 INFO:tasks.workunit.client.0.smithi008.stdout:6/561: getdents df/d8c/db1 0 2024-04-12T05:55:34.123 INFO:tasks.workunit.client.0.smithi008.stdout:7/596: write d1/d5/da/d78/f88 [104920,88492] 0 2024-04-12T05:55:34.125 INFO:tasks.workunit.client.0.smithi008.stdout:2/479: dread d6/db/d10/d23/d35/f49 [0,4194304] 0 2024-04-12T05:55:34.126 INFO:tasks.workunit.client.0.smithi008.stdout:8/477: rename dd/de/d4c/d66/d6c/f87 to dd/de/d16/fa7 0 2024-04-12T05:55:34.129 INFO:tasks.workunit.client.0.smithi008.stdout:5/873: dwrite d0/dc/d13/d1a/daf/f5 [8388608,4194304] 0 2024-04-12T05:55:34.132 INFO:tasks.workunit.client.0.smithi008.stdout:6/562: creat df/d10/d3a/d9b/fbc x:0 0 0 2024-04-12T05:55:34.134 INFO:tasks.workunit.client.0.smithi008.stdout:0/491: dwrite d2/fe [0,4194304] 0 2024-04-12T05:55:34.135 INFO:tasks.workunit.client.0.smithi008.stdout:6/563: chown df/d21/d4c/c86 34646181 1 2024-04-12T05:55:34.137 INFO:tasks.workunit.client.0.smithi008.stdout:0/492: chown d2/d11/c38 885 1 2024-04-12T05:55:34.140 INFO:tasks.workunit.client.0.smithi008.stdout:0/493: write d2/d11/d36/f80 [503766,44287] 0 2024-04-12T05:55:34.144 INFO:tasks.workunit.client.0.smithi008.stdout:8/478: fsync dd/de/d4c/d66/d6d/d8e/d97/fa3 0 2024-04-12T05:55:34.145 INFO:tasks.workunit.client.0.smithi008.stdout:1/477: getdents d4/d33/d74/d46/d4c 0 2024-04-12T05:55:34.147 INFO:tasks.workunit.client.0.smithi008.stdout:7/597: creat d1/d5/da/fd2 x:0 0 0 2024-04-12T05:55:34.147 INFO:tasks.workunit.client.0.smithi008.stdout:5/874: dread d0/dc/d13/d1a/daf/f84 [0,4194304] 0 2024-04-12T05:55:34.147 INFO:tasks.workunit.client.0.smithi008.stdout:2/480: creat d6/db/f9e x:0 0 0 2024-04-12T05:55:34.151 INFO:tasks.workunit.client.0.smithi008.stdout:6/564: dwrite df/d10/d20/d3e/fb2 [0,4194304] 0 2024-04-12T05:55:34.153 INFO:tasks.workunit.client.0.smithi008.stdout:0/494: creat d2/d11/d36/d42/d86/f96 x:0 0 0 2024-04-12T05:55:34.156 INFO:tasks.workunit.client.0.smithi008.stdout:5/875: chown d0/dc/lb0 247196 1 2024-04-12T05:55:34.157 INFO:tasks.workunit.client.0.smithi008.stdout:1/478: rmdir d4/d1a/d57 39 2024-04-12T05:55:34.158 INFO:tasks.workunit.client.0.smithi008.stdout:6/565: mknod df/d8c/da4/cbd 0 2024-04-12T05:55:34.161 INFO:tasks.workunit.client.0.smithi008.stdout:5/876: creat d0/d29/de9/f10f x:0 0 0 2024-04-12T05:55:34.163 INFO:tasks.workunit.client.0.smithi008.stdout:7/598: mkdir d1/d5/da/d3c/dd3 0 2024-04-12T05:55:34.163 INFO:tasks.workunit.client.0.smithi008.stdout:1/479: creat d4/d33/d74/d30/d72/d7c/fa1 x:0 0 0 2024-04-12T05:55:34.164 INFO:tasks.workunit.client.0.smithi008.stdout:6/566: readlink df/d10/l41 0 2024-04-12T05:55:34.165 INFO:tasks.workunit.client.0.smithi008.stdout:2/481: write d6/db/f56 [4575519,78659] 0 2024-04-12T05:55:34.166 INFO:tasks.workunit.client.0.smithi008.stdout:5/877: stat d0/l6f 0 2024-04-12T05:55:34.166 INFO:tasks.workunit.client.0.smithi008.stdout:1/480: mkdir d4/d33/d74/da2 0 2024-04-12T05:55:34.170 INFO:tasks.workunit.client.0.smithi008.stdout:6/567: creat df/d21/d4c/fbe x:0 0 0 2024-04-12T05:55:34.172 INFO:tasks.workunit.client.0.smithi008.stdout:1/481: symlink d4/d33/d74/d30/d72/la3 0 2024-04-12T05:55:34.174 INFO:tasks.workunit.client.0.smithi008.stdout:2/482: truncate d6/db/fd 555715 0 2024-04-12T05:55:34.176 INFO:tasks.workunit.client.0.smithi008.stdout:5/878: dread d0/dc/f2a [4194304,4194304] 0 2024-04-12T05:55:34.176 INFO:tasks.workunit.client.0.smithi008.stdout:0/495: dread d2/d11/f69 [4194304,4194304] 0 2024-04-12T05:55:34.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/599: getdents d1/d5/da/d3c/d44/d8a 0 2024-04-12T05:55:34.177 INFO:tasks.workunit.client.0.smithi008.stdout:9/559: sync 2024-04-12T05:55:34.179 INFO:tasks.workunit.client.0.smithi008.stdout:2/483: write d6/db/d38/d41/f99 [4093885,28039] 0 2024-04-12T05:55:34.181 INFO:tasks.workunit.client.0.smithi008.stdout:8/479: dread dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:34.182 INFO:tasks.workunit.client.0.smithi008.stdout:0/496: mknod d2/d24/d61/c97 0 2024-04-12T05:55:34.184 INFO:tasks.workunit.client.0.smithi008.stdout:5/879: symlink d0/dc/d13/d1a/daf/de3/l110 0 2024-04-12T05:55:34.185 INFO:tasks.workunit.client.0.smithi008.stdout:2/484: symlink d6/db/d60/d84/l9f 0 2024-04-12T05:55:34.190 INFO:tasks.workunit.client.0.smithi008.stdout:8/480: symlink dd/la8 0 2024-04-12T05:55:34.191 INFO:tasks.workunit.client.0.smithi008.stdout:1/482: truncate d4/d7/fe 1110738 0 2024-04-12T05:55:34.191 INFO:tasks.workunit.client.0.smithi008.stdout:5/880: dread - d0/dc/d13/d1a/daf/fc3 zero size 2024-04-12T05:55:34.193 INFO:tasks.workunit.client.0.smithi008.stdout:8/481: write dd/d30/d32/f71 [653014,27221] 0 2024-04-12T05:55:34.193 INFO:tasks.workunit.client.0.smithi008.stdout:9/560: dwrite d3/d22/d2b/d4e/faf [0,4194304] 0 2024-04-12T05:55:34.194 INFO:tasks.workunit.client.0.smithi008.stdout:0/497: creat d2/d11/d36/d82/d95/f98 x:0 0 0 2024-04-12T05:55:34.196 INFO:tasks.workunit.client.0.smithi008.stdout:2/485: rename d6/db/d10/l6e to d6/db/d10/d7b/d7d/la0 0 2024-04-12T05:55:34.198 INFO:tasks.workunit.client.0.smithi008.stdout:9/561: dread - d3/df/d13/d21/d2d/d75/f94 zero size 2024-04-12T05:55:34.198 INFO:tasks.workunit.client.0.smithi008.stdout:5/881: mknod d0/dd/d5c/c111 0 2024-04-12T05:55:34.202 INFO:tasks.workunit.client.0.smithi008.stdout:1/483: mkdir d4/dd/d17/d91/da4 0 2024-04-12T05:55:34.202 INFO:tasks.workunit.client.0.smithi008.stdout:0/498: creat d2/d4b/f99 x:0 0 0 2024-04-12T05:55:34.206 INFO:tasks.workunit.client.0.smithi008.stdout:9/562: creat d3/df/d13/d21/d5c/d8f/dab/fc3 x:0 0 0 2024-04-12T05:55:34.208 INFO:tasks.workunit.client.0.smithi008.stdout:0/499: chown d2/d11/d36/d82/d95 221350 1 2024-04-12T05:55:34.211 INFO:tasks.workunit.client.0.smithi008.stdout:5/882: symlink d0/dc/d13/d1a/daf/l112 0 2024-04-12T05:55:34.213 INFO:tasks.workunit.client.0.smithi008.stdout:9/563: fsync d3/df/d13/d21/d2d/dbb/f9a 0 2024-04-12T05:55:34.213 INFO:tasks.workunit.client.0.smithi008.stdout:1/484: unlink d4/d6/d11/c7b 0 2024-04-12T05:55:34.214 INFO:tasks.workunit.client.0.smithi008.stdout:2/486: dwrite d6/db/d38/d41/f6b [0,4194304] 0 2024-04-12T05:55:34.216 INFO:tasks.workunit.client.0.smithi008.stdout:8/482: dwrite dd/d4f/f5f [0,4194304] 0 2024-04-12T05:55:34.217 INFO:tasks.workunit.client.0.smithi008.stdout:0/500: unlink d2/d24/l2f 0 2024-04-12T05:55:34.218 INFO:tasks.workunit.client.0.smithi008.stdout:1/485: fdatasync d4/d7/d18/f81 0 2024-04-12T05:55:34.218 INFO:tasks.workunit.client.0.smithi008.stdout:8/483: write dd/f99 [105973,64552] 0 2024-04-12T05:55:34.219 INFO:tasks.workunit.client.0.smithi008.stdout:1/486: stat d4/f9 0 2024-04-12T05:55:34.220 INFO:tasks.workunit.client.0.smithi008.stdout:8/484: readlink dd/la8 0 2024-04-12T05:55:34.220 INFO:tasks.workunit.client.0.smithi008.stdout:5/883: symlink d0/dc/d13/d1a/d3a/l113 0 2024-04-12T05:55:34.221 INFO:tasks.workunit.client.0.smithi008.stdout:2/487: write d6/db/d10/f80 [932232,87005] 0 2024-04-12T05:55:34.223 INFO:tasks.workunit.client.0.smithi008.stdout:9/564: creat d3/df/d13/d21/d5c/d8f/fc4 x:0 0 0 2024-04-12T05:55:34.224 INFO:tasks.workunit.client.0.smithi008.stdout:0/501: symlink d2/d24/d85/l9a 0 2024-04-12T05:55:34.227 INFO:tasks.workunit.client.0.smithi008.stdout:5/884: rmdir d0/dc/d13 39 2024-04-12T05:55:34.228 INFO:tasks.workunit.client.0.smithi008.stdout:1/487: write d4/f7f [155021,96358] 0 2024-04-12T05:55:34.229 INFO:tasks.workunit.client.0.smithi008.stdout:5/885: chown d0/dd/d3d/dc9 11079 1 2024-04-12T05:55:34.233 INFO:tasks.workunit.client.0.smithi008.stdout:8/485: rename dd/de/d16/d50/d56/d61/d6a to dd/de/d4c/d66/da9 0 2024-04-12T05:55:34.234 INFO:tasks.workunit.client.0.smithi008.stdout:8/486: fdatasync dd/de/d4c/d66/d6c/fa0 0 2024-04-12T05:55:34.235 INFO:tasks.workunit.client.0.smithi008.stdout:2/488: unlink d6/db/d10/d23/d35/d3c/l66 0 2024-04-12T05:55:34.236 INFO:tasks.workunit.client.0.smithi008.stdout:2/489: dread - d6/db/d60/d57/f79 zero size 2024-04-12T05:55:34.237 INFO:tasks.workunit.client.0.smithi008.stdout:5/886: creat d0/dc/d13/d1a/daf/de3/f114 x:0 0 0 2024-04-12T05:55:34.237 INFO:tasks.workunit.client.0.smithi008.stdout:1/488: creat d4/d33/fa5 x:0 0 0 2024-04-12T05:55:34.238 INFO:tasks.workunit.client.0.smithi008.stdout:5/887: readlink d0/l6f 0 2024-04-12T05:55:34.240 INFO:tasks.workunit.client.0.smithi008.stdout:1/489: chown d4/dd/f89 7 1 2024-04-12T05:55:34.242 INFO:tasks.workunit.client.0.smithi008.stdout:5/888: readlink d0/dc/d13/d1a/d3a/l9a 0 2024-04-12T05:55:34.242 INFO:tasks.workunit.client.0.smithi008.stdout:2/490: truncate d6/db/d10/f55 4241971 0 2024-04-12T05:55:34.244 INFO:tasks.workunit.client.0.smithi008.stdout:2/491: chown d6/l1a 1533 1 2024-04-12T05:55:34.247 INFO:tasks.workunit.client.0.smithi008.stdout:8/487: dwrite dd/de/f47 [8388608,4194304] 0 2024-04-12T05:55:34.247 INFO:tasks.workunit.client.0.smithi008.stdout:9/565: write d3/d22/d2b/d58/f3f [1578401,97872] 0 2024-04-12T05:55:34.250 INFO:tasks.workunit.client.0.smithi008.stdout:2/492: write d6/db/d38/d41/f90 [408779,64377] 0 2024-04-12T05:55:34.255 INFO:tasks.workunit.client.0.smithi008.stdout:2/493: symlink d6/la1 0 2024-04-12T05:55:34.258 INFO:tasks.workunit.client.0.smithi008.stdout:1/490: dread d4/d33/d37/f55 [0,4194304] 0 2024-04-12T05:55:34.259 INFO:tasks.workunit.client.0.smithi008.stdout:8/488: getdents dd/de/d4c/d66 0 2024-04-12T05:55:34.262 INFO:tasks.workunit.client.0.smithi008.stdout:5/889: dread d0/dc/d13/d1a/d35/f4c [0,4194304] 0 2024-04-12T05:55:34.266 INFO:tasks.workunit.client.0.smithi008.stdout:8/489: stat dd/de/d16/d50/d5e/c7a 0 2024-04-12T05:55:34.271 INFO:tasks.workunit.client.0.smithi008.stdout:5/890: rmdir d0/dd/dad 39 2024-04-12T05:55:34.271 INFO:tasks.workunit.client.0.smithi008.stdout:1/491: link d4/d1a/f6c d4/dd/d21/d9f/fa6 0 2024-04-12T05:55:34.272 INFO:tasks.workunit.client.0.smithi008.stdout:9/566: dwrite d3/df/d13/d21/d2d/d75/f94 [0,4194304] 0 2024-04-12T05:55:34.273 INFO:tasks.workunit.client.0.smithi008.stdout:8/490: write dd/de/d4c/d66/d6d/d8e/d97/fa3 [931750,17473] 0 2024-04-12T05:55:34.273 INFO:tasks.workunit.client.0.smithi008.stdout:2/494: dwrite d6/db/d10/d7b/d7d/f98 [0,4194304] 0 2024-04-12T05:55:34.275 INFO:tasks.workunit.client.0.smithi008.stdout:1/492: chown d4/dd/f85 15400 1 2024-04-12T05:55:34.276 INFO:tasks.workunit.client.0.smithi008.stdout:5/891: symlink d0/dd/d5c/l115 0 2024-04-12T05:55:34.277 INFO:tasks.workunit.client.0.smithi008.stdout:5/892: stat d0/d40/f5e 0 2024-04-12T05:55:34.278 INFO:tasks.workunit.client.0.smithi008.stdout:2/495: write d6/db/d10/d4c/f82 [884023,53806] 0 2024-04-12T05:55:34.279 INFO:tasks.workunit.client.0.smithi008.stdout:8/491: unlink dd/de/f98 0 2024-04-12T05:55:34.281 INFO:tasks.workunit.client.0.smithi008.stdout:5/893: rmdir d0/dd/d5c 39 2024-04-12T05:55:34.283 INFO:tasks.workunit.client.0.smithi008.stdout:8/492: truncate dd/d30/d3b/f74 617187 0 2024-04-12T05:55:34.284 INFO:tasks.workunit.client.0.smithi008.stdout:1/493: dread d4/d1a/f51 [0,4194304] 0 2024-04-12T05:55:34.285 INFO:tasks.workunit.client.0.smithi008.stdout:8/493: chown dd/de/d4c/d66/da9/f6f 2264 1 2024-04-12T05:55:34.286 INFO:tasks.workunit.client.0.smithi008.stdout:2/496: creat d6/db/d10/d4c/fa2 x:0 0 0 2024-04-12T05:55:34.287 INFO:tasks.workunit.client.0.smithi008.stdout:9/567: link d3/d22/d2b/d58/d89/f8d d3/fc5 0 2024-04-12T05:55:34.289 INFO:tasks.workunit.client.0.smithi008.stdout:1/494: creat d4/d6/d90/fa7 x:0 0 0 2024-04-12T05:55:34.291 INFO:tasks.workunit.client.0.smithi008.stdout:2/497: symlink d6/db/d6f/la3 0 2024-04-12T05:55:34.291 INFO:tasks.workunit.client.0.smithi008.stdout:9/568: readlink d3/df/d13/d21/d2d/d32/d52/l57 0 2024-04-12T05:55:34.293 INFO:tasks.workunit.client.0.smithi008.stdout:8/494: write dd/f39 [280948,70967] 0 2024-04-12T05:55:34.295 INFO:tasks.workunit.client.0.smithi008.stdout:1/495: creat d4/dd/d17/d66/fa8 x:0 0 0 2024-04-12T05:55:34.296 INFO:tasks.workunit.client.0.smithi008.stdout:5/894: dwrite d0/dd/d3d/d5d/fd0 [0,4194304] 0 2024-04-12T05:55:34.298 INFO:tasks.workunit.client.0.smithi008.stdout:8/495: write fb [341363,3911] 0 2024-04-12T05:55:34.298 INFO:tasks.workunit.client.0.smithi008.stdout:9/569: symlink d3/df/d13/d21/d5c/lc6 0 2024-04-12T05:55:34.311 INFO:tasks.workunit.client.0.smithi008.stdout:9/570: symlink d3/d22/d2b/d58/d38/lc7 0 2024-04-12T05:55:34.312 INFO:tasks.workunit.client.0.smithi008.stdout:2/498: dwrite d6/db/f89 [0,4194304] 0 2024-04-12T05:55:34.317 INFO:tasks.workunit.client.0.smithi008.stdout:5/895: dwrite d0/dc/d13/f72 [12582912,4194304] 0 2024-04-12T05:55:34.318 INFO:tasks.workunit.client.0.smithi008.stdout:9/571: mkdir d3/d22/d2b/d4a/dc8 0 2024-04-12T05:55:34.319 INFO:tasks.workunit.client.0.smithi008.stdout:8/496: dwrite dd/d4f/d52/f8d [0,4194304] 0 2024-04-12T05:55:34.321 INFO:tasks.workunit.client.0.smithi008.stdout:1/496: truncate d4/d7/f4e 209924 0 2024-04-12T05:55:34.322 INFO:tasks.workunit.client.0.smithi008.stdout:5/896: read d0/dc/d13/d1a/daf/d37/f45 [3720619,79968] 0 2024-04-12T05:55:34.322 INFO:tasks.workunit.client.0.smithi008.stdout:9/572: dread - d3/df/d13/d21/d2d/d30/d74/f83 zero size 2024-04-12T05:55:34.325 INFO:tasks.workunit.client.0.smithi008.stdout:1/497: write d4/d7/ff [3514119,123653] 0 2024-04-12T05:55:34.326 INFO:tasks.workunit.client.0.smithi008.stdout:8/497: rename dd/de/d16/d73/c91 to dd/d30/caa 0 2024-04-12T05:55:34.326 INFO:tasks.workunit.client.0.smithi008.stdout:9/573: fsync d3/d6a/f9e 0 2024-04-12T05:55:34.331 INFO:tasks.workunit.client.0.smithi008.stdout:1/498: creat d4/dd/d17/d26/fa9 x:0 0 0 2024-04-12T05:55:34.332 INFO:tasks.workunit.client.0.smithi008.stdout:9/574: fsync d3/df/f11 0 2024-04-12T05:55:34.333 INFO:tasks.workunit.client.0.smithi008.stdout:8/498: dread dd/d30/d32/f71 [0,4194304] 0 2024-04-12T05:55:34.334 INFO:tasks.workunit.client.0.smithi008.stdout:5/897: truncate d0/dc/d13/d1a/daf/f1b 228197 0 2024-04-12T05:55:34.345 INFO:tasks.workunit.client.0.smithi008.stdout:1/499: chown d4/l14 1663437123 1 2024-04-12T05:55:34.350 INFO:tasks.workunit.client.0.smithi008.stdout:2/499: dwrite d6/f16 [4194304,4194304] 0 2024-04-12T05:55:34.350 INFO:tasks.workunit.client.0.smithi008.stdout:9/575: creat d3/df/d13/d21/d5c/da4/db5/fc9 x:0 0 0 2024-04-12T05:55:34.352 INFO:tasks.workunit.client.0.smithi008.stdout:8/499: mkdir dd/dab 0 2024-04-12T05:55:34.354 INFO:tasks.workunit.client.0.smithi008.stdout:9/576: stat d3/d22/d2b/d77 0 2024-04-12T05:55:34.355 INFO:tasks.workunit.client.0.smithi008.stdout:2/500: stat d6/db/d10/d4c/f5a 0 2024-04-12T05:55:34.356 INFO:tasks.workunit.client.0.smithi008.stdout:8/500: unlink c8 0 2024-04-12T05:55:34.356 INFO:tasks.workunit.client.0.smithi008.stdout:1/500: dwrite d4/d7/f56 [0,4194304] 0 2024-04-12T05:55:34.360 INFO:tasks.workunit.client.0.smithi008.stdout:8/501: mknod dd/de/d4c/d66/d6d/d8e/cac 0 2024-04-12T05:55:34.365 INFO:tasks.workunit.client.0.smithi008.stdout:5/898: dwrite d0/d40/f5e [0,4194304] 0 2024-04-12T05:55:34.366 INFO:tasks.workunit.client.0.smithi008.stdout:8/502: chown dd/de/d16/d50/d56/d61/d9c 135812238 1 2024-04-12T05:55:34.369 INFO:tasks.workunit.client.0.smithi008.stdout:5/899: mknod d0/d29/c116 0 2024-04-12T05:55:34.371 INFO:tasks.workunit.client.0.smithi008.stdout:5/900: write d0/dab/fd8 [720452,130509] 0 2024-04-12T05:55:34.374 INFO:tasks.workunit.client.0.smithi008.stdout:8/503: symlink dd/de/d4c/d66/lad 0 2024-04-12T05:55:34.383 INFO:tasks.workunit.client.0.smithi008.stdout:9/577: dread d3/df/d13/d21/d2d/d32/d52/fa2 [0,4194304] 0 2024-04-12T05:55:34.455 INFO:tasks.workunit.client.0.smithi008.stdout:9/578: symlink d3/d22/d2b/d58/d38/lca 0 2024-04-12T05:55:34.457 INFO:tasks.workunit.client.0.smithi008.stdout:9/579: write d3/d22/d2b/d58/f3f [1893536,14659] 0 2024-04-12T05:55:34.457 INFO:tasks.workunit.client.0.smithi008.stdout:8/504: symlink dd/de/d16/lae 0 2024-04-12T05:55:34.461 INFO:tasks.workunit.client.0.smithi008.stdout:9/580: creat d3/d22/d2b/d58/da8/db6/fcb x:0 0 0 2024-04-12T05:55:34.462 INFO:tasks.workunit.client.0.smithi008.stdout:8/505: symlink dd/de/d16/d50/d56/laf 0 2024-04-12T05:55:34.464 INFO:tasks.workunit.client.0.smithi008.stdout:9/581: dread d3/d22/d2b/d77/f9c [0,4194304] 0 2024-04-12T05:55:34.472 INFO:tasks.workunit.client.0.smithi008.stdout:9/582: rename d3/df/d13/d21/d2d/d32/c80 to d3/df/d13/d21/d2d/d75/d7c/ccc 0 2024-04-12T05:55:34.474 INFO:tasks.workunit.client.0.smithi008.stdout:9/583: chown d3/d22/d2b/d77/lb8 1657665 1 2024-04-12T05:55:34.476 INFO:tasks.workunit.client.0.smithi008.stdout:1/501: write d4/d6/f40 [39780,111545] 0 2024-04-12T05:55:34.477 INFO:tasks.workunit.client.0.smithi008.stdout:8/506: dread dd/d30/d32/f83 [0,4194304] 0 2024-04-12T05:55:34.483 INFO:tasks.workunit.client.0.smithi008.stdout:9/584: write d3/f4b [1952366,36424] 0 2024-04-12T05:55:34.487 INFO:tasks.workunit.client.0.smithi008.stdout:1/502: rename d4/d6/d11/f69 to d4/d24/faa 0 2024-04-12T05:55:34.489 INFO:tasks.workunit.client.0.smithi008.stdout:2/501: dwrite d6/db/d38/f63 [0,4194304] 0 2024-04-12T05:55:34.490 INFO:tasks.workunit.client.0.smithi008.stdout:8/507: mknod dd/de/d16/d50/d56/d61/d9c/cb0 0 2024-04-12T05:55:34.495 INFO:tasks.workunit.client.0.smithi008.stdout:5/901: dwrite d0/dc/d57/f2f [4194304,4194304] 0 2024-04-12T05:55:34.496 INFO:tasks.workunit.client.0.smithi008.stdout:1/503: creat d4/d33/d74/fab x:0 0 0 2024-04-12T05:55:34.499 INFO:tasks.workunit.client.0.smithi008.stdout:8/508: dread dd/f4e [4194304,4194304] 0 2024-04-12T05:55:34.504 INFO:tasks.workunit.client.0.smithi008.stdout:1/504: mkdir d4/dd/d21/d9f/dac 0 2024-04-12T05:55:34.504 INFO:tasks.workunit.client.0.smithi008.stdout:5/902: dread - d0/dc/d13/d1a/daf/fac zero size 2024-04-12T05:55:34.510 INFO:tasks.workunit.client.0.smithi008.stdout:1/505: rename d4/dd/d17/d26/c65 to d4/dd/d8b/cad 0 2024-04-12T05:55:34.510 INFO:tasks.workunit.client.0.smithi008.stdout:5/903: getdents d0/dd/d11/d38/dbd 0 2024-04-12T05:55:34.513 INFO:tasks.workunit.client.0.smithi008.stdout:1/506: rename d4/d33/d74/d46/d4c/c2a to d4/d33/d74/da2/cae 0 2024-04-12T05:55:34.515 INFO:tasks.workunit.client.0.smithi008.stdout:1/507: write d4/d33/d74/d30/f49 [2227967,106820] 0 2024-04-12T05:55:34.516 INFO:tasks.workunit.client.0.smithi008.stdout:2/502: write d6/db/fd [79739,65316] 0 2024-04-12T05:55:34.521 INFO:tasks.workunit.client.0.smithi008.stdout:2/503: dread d6/f74 [0,4194304] 0 2024-04-12T05:55:34.524 INFO:tasks.workunit.client.0.smithi008.stdout:1/508: dwrite d4/d33/fa5 [0,4194304] 0 2024-04-12T05:55:34.525 INFO:tasks.workunit.client.0.smithi008.stdout:2/504: creat d6/db/d10/d23/fa4 x:0 0 0 2024-04-12T05:55:34.531 INFO:tasks.workunit.client.0.smithi008.stdout:2/505: rename d6/db/d60/d57/f79 to d6/db/d38/d41/fa5 0 2024-04-12T05:55:34.535 INFO:tasks.workunit.client.0.smithi008.stdout:2/506: mkdir d6/db/d6f/da6 0 2024-04-12T05:55:34.535 INFO:tasks.workunit.client.0.smithi008.stdout:1/509: link d4/d33/d37/l41 d4/d1a/d57/laf 0 2024-04-12T05:55:34.537 INFO:tasks.workunit.client.0.smithi008.stdout:2/507: mknod d6/db/d10/d23/d35/d3c/ca7 0 2024-04-12T05:55:34.539 INFO:tasks.workunit.client.0.smithi008.stdout:1/510: creat d4/dd/d21/d9f/dac/fb0 x:0 0 0 2024-04-12T05:55:34.540 INFO:tasks.workunit.client.0.smithi008.stdout:2/508: symlink d6/db/d10/d7b/d7d/la8 0 2024-04-12T05:55:34.543 INFO:tasks.workunit.client.0.smithi008.stdout:2/509: symlink d6/db/d60/d4f/la9 0 2024-04-12T05:55:34.548 INFO:tasks.workunit.client.0.smithi008.stdout:1/511: link d4/dd/d21/f84 d4/d7/d18/fb1 0 2024-04-12T05:55:34.549 INFO:tasks.workunit.client.0.smithi008.stdout:1/512: truncate d4/d33/d74/d46/f86 439856 0 2024-04-12T05:55:34.550 INFO:tasks.workunit.client.0.smithi008.stdout:2/510: link d6/db/d10/d23/d35/d3c/c3e d6/db/d38/d41/caa 0 2024-04-12T05:55:34.551 INFO:tasks.workunit.client.0.smithi008.stdout:2/511: chown d6/db/d10/d4c 457688 1 2024-04-12T05:55:34.551 INFO:tasks.workunit.client.0.smithi008.stdout:1/513: chown d4/d33/d74/d30/d72/l97 307487501 1 2024-04-12T05:55:34.553 INFO:tasks.workunit.client.0.smithi008.stdout:1/514: write d4/d33/d74/fab [1023257,117600] 0 2024-04-12T05:55:34.553 INFO:tasks.workunit.client.0.smithi008.stdout:2/512: truncate d6/db/d29/f91 764145 0 2024-04-12T05:55:34.556 INFO:tasks.workunit.client.0.smithi008.stdout:1/515: rmdir d4/d33/d74 39 2024-04-12T05:55:34.558 INFO:tasks.workunit.client.0.smithi008.stdout:2/513: mknod d6/db/d10/d7b/cab 0 2024-04-12T05:55:34.562 INFO:tasks.workunit.client.0.smithi008.stdout:1/516: creat d4/dd/d17/fb2 x:0 0 0 2024-04-12T05:55:34.562 INFO:tasks.workunit.client.0.smithi008.stdout:2/514: dread d6/db/d29/f48 [4194304,4194304] 0 2024-04-12T05:55:34.563 INFO:tasks.workunit.client.0.smithi008.stdout:1/517: chown d4/d24/f45 164737 1 2024-04-12T05:55:34.565 INFO:tasks.workunit.client.0.smithi008.stdout:2/515: truncate d6/db/d60/f59 806395 0 2024-04-12T05:55:34.568 INFO:tasks.workunit.client.0.smithi008.stdout:2/516: rename d6/db/d10/d7b/cab to d6/cac 0 2024-04-12T05:55:34.570 INFO:tasks.workunit.client.0.smithi008.stdout:2/517: creat d6/db/d38/d41/fad x:0 0 0 2024-04-12T05:55:34.570 INFO:tasks.workunit.client.0.smithi008.stdout:2/518: dread - d6/db/d6f/f88 zero size 2024-04-12T05:55:34.574 INFO:tasks.workunit.client.0.smithi008.stdout:2/519: mkdir d6/db/dae 0 2024-04-12T05:55:34.577 INFO:tasks.workunit.client.0.smithi008.stdout:2/520: creat d6/db/d60/faf x:0 0 0 2024-04-12T05:55:34.577 INFO:tasks.workunit.client.0.smithi008.stdout:2/521: readlink d6/db/d38/d41/l9a 0 2024-04-12T05:55:34.577 INFO:tasks.workunit.client.0.smithi008.stdout:2/522: fsync d6/f15 0 2024-04-12T05:55:34.579 INFO:tasks.workunit.client.0.smithi008.stdout:2/523: mknod d6/cb0 0 2024-04-12T05:55:34.580 INFO:tasks.workunit.client.0.smithi008.stdout:2/524: creat d6/fb1 x:0 0 0 2024-04-12T05:55:34.587 INFO:tasks.workunit.client.0.smithi008.stdout:2/525: dwrite d6/f43 [4194304,4194304] 0 2024-04-12T05:55:34.590 INFO:tasks.workunit.client.0.smithi008.stdout:2/526: fdatasync d6/db/d10/d23/f4d 0 2024-04-12T05:55:34.591 INFO:tasks.workunit.client.0.smithi008.stdout:2/527: chown d6/db/d38/d41/f99 29 1 2024-04-12T05:55:34.598 INFO:tasks.workunit.client.0.smithi008.stdout:2/528: dwrite d6/db/d6f/f88 [0,4194304] 0 2024-04-12T05:55:34.601 INFO:tasks.workunit.client.0.smithi008.stdout:2/529: write d6/db/d10/d7b/d7d/f97 [4935098,44366] 0 2024-04-12T05:55:34.609 INFO:tasks.workunit.client.0.smithi008.stdout:4/453: sync 2024-04-12T05:55:34.609 INFO:tasks.workunit.client.0.smithi008.stdout:3/585: sync 2024-04-12T05:55:34.611 INFO:tasks.workunit.client.0.smithi008.stdout:2/530: dwrite d6/db/d60/faf [0,4194304] 0 2024-04-12T05:55:34.617 INFO:tasks.workunit.client.0.smithi008.stdout:3/586: mkdir d9/d10/d19/d3a/d67/dc2 0 2024-04-12T05:55:34.617 INFO:tasks.workunit.client.0.smithi008.stdout:4/454: symlink d1/d22/d29/d46/d56/l81 0 2024-04-12T05:55:34.618 INFO:tasks.workunit.client.0.smithi008.stdout:2/531: mkdir d6/db/d10/db2 0 2024-04-12T05:55:34.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/455: stat d1/d22/d29/d30/d60 0 2024-04-12T05:55:34.620 INFO:tasks.workunit.client.0.smithi008.stdout:3/587: creat d9/d44/fc3 x:0 0 0 2024-04-12T05:55:34.620 INFO:tasks.workunit.client.0.smithi008.stdout:2/532: chown l4 113013 1 2024-04-12T05:55:34.622 INFO:tasks.workunit.client.0.smithi008.stdout:2/533: chown d6/db/d6f/c96 14252420 1 2024-04-12T05:55:34.624 INFO:tasks.workunit.client.0.smithi008.stdout:2/534: unlink d6/f33 0 2024-04-12T05:55:34.624 INFO:tasks.workunit.client.0.smithi008.stdout:4/456: read d1/db/f16 [3291060,113338] 0 2024-04-12T05:55:34.626 INFO:tasks.workunit.client.0.smithi008.stdout:2/535: write d6/db/f56 [3878095,5457] 0 2024-04-12T05:55:34.630 INFO:tasks.workunit.client.0.smithi008.stdout:4/457: rename d1/f12 to d1/d22/d29/d30/d47/f82 0 2024-04-12T05:55:34.631 INFO:tasks.workunit.client.0.smithi008.stdout:4/458: dread - d1/d8/f5f zero size 2024-04-12T05:55:34.631 INFO:tasks.workunit.client.0.smithi008.stdout:2/536: rmdir d6/db/d10/d4c 39 2024-04-12T05:55:34.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/459: mknod d1/d22/d29/d30/d60/c83 0 2024-04-12T05:55:34.637 INFO:tasks.workunit.client.0.smithi008.stdout:4/460: creat d1/d8/f84 x:0 0 0 2024-04-12T05:55:34.639 INFO:tasks.workunit.client.0.smithi008.stdout:4/461: chown d1/db/c17 4592147 1 2024-04-12T05:55:34.641 INFO:tasks.workunit.client.0.smithi008.stdout:4/462: chown d1/d22/d29/d30/l59 2110 1 2024-04-12T05:55:34.644 INFO:tasks.workunit.client.0.smithi008.stdout:4/463: truncate d1/d8/f37 1141821 0 2024-04-12T05:55:34.647 INFO:tasks.workunit.client.0.smithi008.stdout:4/464: stat d1/d22/d29/l33 0 2024-04-12T05:55:34.650 INFO:tasks.workunit.client.0.smithi008.stdout:2/537: dwrite d6/db/d60/f59 [0,4194304] 0 2024-04-12T05:55:34.656 INFO:tasks.workunit.client.0.smithi008.stdout:2/538: rmdir d6/db/d10/d7b/d7d 39 2024-04-12T05:55:34.658 INFO:tasks.workunit.client.0.smithi008.stdout:2/539: chown d6/db/d10/d4c/fa2 17 1 2024-04-12T05:55:34.659 INFO:tasks.workunit.client.0.smithi008.stdout:2/540: fdatasync f0 0 2024-04-12T05:55:34.659 INFO:tasks.workunit.client.0.smithi008.stdout:4/465: truncate d1/d4f/f6c 724195 0 2024-04-12T05:55:34.661 INFO:tasks.workunit.client.0.smithi008.stdout:4/466: write d1/d22/d29/d30/d47/d28/f3f [2237222,123676] 0 2024-04-12T05:55:34.662 INFO:tasks.workunit.client.0.smithi008.stdout:4/467: read d1/d22/d29/d30/f52 [142105,78382] 0 2024-04-12T05:55:34.663 INFO:tasks.workunit.client.0.smithi008.stdout:2/541: mknod d6/db/cb3 0 2024-04-12T05:55:34.668 INFO:tasks.workunit.client.0.smithi008.stdout:4/468: getdents d1/d22/d29/d30/d60 0 2024-04-12T05:55:34.671 INFO:tasks.workunit.client.0.smithi008.stdout:4/469: symlink d1/d22/d29/d30/d47/d28/l85 0 2024-04-12T05:55:34.673 INFO:tasks.workunit.client.0.smithi008.stdout:2/542: dwrite d6/f15 [4194304,4194304] 0 2024-04-12T05:55:34.677 INFO:tasks.workunit.client.0.smithi008.stdout:2/543: unlink d6/db/d29/f91 0 2024-04-12T05:55:34.680 INFO:tasks.workunit.client.0.smithi008.stdout:4/470: dwrite d1/d8/f55 [0,4194304] 0 2024-04-12T05:55:34.681 INFO:tasks.workunit.client.0.smithi008.stdout:4/471: chown d1/d8/l14 10 1 2024-04-12T05:55:34.684 INFO:tasks.workunit.client.0.smithi008.stdout:2/544: creat d6/db/d10/d7b/fb4 x:0 0 0 2024-04-12T05:55:34.686 INFO:tasks.workunit.client.0.smithi008.stdout:4/472: mknod d1/d22/d29/d46/d56/c86 0 2024-04-12T05:55:34.687 INFO:tasks.workunit.client.0.smithi008.stdout:2/545: mkdir d6/db/d10/d23/d35/d3c/db5 0 2024-04-12T05:55:34.688 INFO:tasks.workunit.client.0.smithi008.stdout:4/473: symlink d1/l87 0 2024-04-12T05:55:34.691 INFO:tasks.workunit.client.0.smithi008.stdout:4/474: fsync d1/d22/d29/d30/d47/f48 0 2024-04-12T05:55:34.695 INFO:tasks.workunit.client.0.smithi008.stdout:4/475: dread d1/d22/f4c [0,4194304] 0 2024-04-12T05:55:34.699 INFO:tasks.workunit.client.0.smithi008.stdout:4/476: creat d1/d22/d29/d30/f88 x:0 0 0 2024-04-12T05:55:34.701 INFO:tasks.workunit.client.0.smithi008.stdout:4/477: creat d1/d4f/f89 x:0 0 0 2024-04-12T05:55:34.703 INFO:tasks.workunit.client.0.smithi008.stdout:4/478: unlink d1/d22/d29/d30/c66 0 2024-04-12T05:55:34.705 INFO:tasks.workunit.client.0.smithi008.stdout:4/479: write d1/db/f16 [1323154,61985] 0 2024-04-12T05:55:34.705 INFO:tasks.workunit.client.0.smithi008.stdout:2/546: truncate d6/f16 1821122 0 2024-04-12T05:55:34.706 INFO:tasks.workunit.client.0.smithi008.stdout:4/480: truncate d1/d22/f4c 4290186 0 2024-04-12T05:55:34.708 INFO:tasks.workunit.client.0.smithi008.stdout:4/481: rmdir d1/d22/d29/d30/d47 39 2024-04-12T05:55:34.715 INFO:tasks.workunit.client.0.smithi008.stdout:2/547: dwrite d6/db/d38/d41/f77 [0,4194304] 0 2024-04-12T05:55:34.716 INFO:tasks.workunit.client.0.smithi008.stdout:4/482: dread d1/db/f1a [4194304,4194304] 0 2024-04-12T05:55:34.718 INFO:tasks.workunit.client.0.smithi008.stdout:2/548: mknod d6/db/d6f/cb6 0 2024-04-12T05:55:34.720 INFO:tasks.workunit.client.0.smithi008.stdout:4/483: write d1/d22/f4c [2037467,65060] 0 2024-04-12T05:55:34.721 INFO:tasks.workunit.client.0.smithi008.stdout:2/549: symlink d6/db/d10/d23/lb7 0 2024-04-12T05:55:34.725 INFO:tasks.workunit.client.0.smithi008.stdout:4/484: symlink d1/d22/d29/d30/d47/d28/l8a 0 2024-04-12T05:55:34.725 INFO:tasks.workunit.client.0.smithi008.stdout:2/550: dread d6/db/d38/d41/f90 [0,4194304] 0 2024-04-12T05:55:34.727 INFO:tasks.workunit.client.0.smithi008.stdout:2/551: truncate d6/db/f69 4269060 0 2024-04-12T05:55:34.731 INFO:tasks.workunit.client.0.smithi008.stdout:2/552: getdents d6/db/d29 0 2024-04-12T05:55:34.734 INFO:tasks.workunit.client.0.smithi008.stdout:2/553: creat d6/db/fb8 x:0 0 0 2024-04-12T05:55:34.736 INFO:tasks.workunit.client.0.smithi008.stdout:4/485: write d1/f40 [4010564,75711] 0 2024-04-12T05:55:34.739 INFO:tasks.workunit.client.0.smithi008.stdout:4/486: creat d1/d22/f8b x:0 0 0 2024-04-12T05:55:34.742 INFO:tasks.workunit.client.0.smithi008.stdout:2/554: dwrite d6/db/d38/d41/f6b [0,4194304] 0 2024-04-12T05:55:34.743 INFO:tasks.workunit.client.0.smithi008.stdout:2/555: fsync d6/db/d38/f5b 0 2024-04-12T05:55:34.744 INFO:tasks.workunit.client.0.smithi008.stdout:2/556: stat d6/db/d10/d23/d35/d3c/c8c 0 2024-04-12T05:55:34.746 INFO:tasks.workunit.client.0.smithi008.stdout:2/557: symlink d6/lb9 0 2024-04-12T05:55:34.747 INFO:tasks.workunit.client.0.smithi008.stdout:2/558: fdatasync d6/db/d10/d4c/fa2 0 2024-04-12T05:55:34.747 INFO:tasks.workunit.client.0.smithi008.stdout:4/487: truncate d1/db/f18 1357340 0 2024-04-12T05:55:34.748 INFO:tasks.workunit.client.0.smithi008.stdout:2/559: chown d6/db/d38/f25 212770 1 2024-04-12T05:55:34.750 INFO:tasks.workunit.client.0.smithi008.stdout:4/488: write d1/d22/d29/d30/d47/d28/f3f [6000335,7151] 0 2024-04-12T05:55:34.751 INFO:tasks.workunit.client.0.smithi008.stdout:2/560: creat d6/db/d10/db2/fba x:0 0 0 2024-04-12T05:55:34.754 INFO:tasks.workunit.client.0.smithi008.stdout:4/489: rename d1/c5d to d1/d22/d29/d30/d47/d28/c8c 0 2024-04-12T05:55:34.755 INFO:tasks.workunit.client.0.smithi008.stdout:2/561: symlink d6/db/d10/d23/d35/d3c/db5/lbb 0 2024-04-12T05:55:34.757 INFO:tasks.workunit.client.0.smithi008.stdout:2/562: read - d6/db/d10/d23/d35/d3c/f70 zero size 2024-04-12T05:55:34.757 INFO:tasks.workunit.client.0.smithi008.stdout:2/563: chown d6/fb1 3397 1 2024-04-12T05:55:34.761 INFO:tasks.workunit.client.0.smithi008.stdout:4/490: link d1/d8/f37 d1/d22/d29/d30/d47/d28/d4a/f8d 0 2024-04-12T05:55:34.803 INFO:tasks.workunit.client.0.smithi008.stdout:0/502: sync 2024-04-12T05:55:34.803 INFO:tasks.workunit.client.0.smithi008.stdout:6/568: sync 2024-04-12T05:55:34.803 INFO:tasks.workunit.client.0.smithi008.stdout:7/600: sync 2024-04-12T05:55:34.831 INFO:tasks.workunit.client.0.smithi008.stdout:0/503: stat d2/d11/d36/f46 0 2024-04-12T05:55:34.831 INFO:tasks.workunit.client.0.smithi008.stdout:4/491: chown d1/d22/d29/d30/d47/f82 7859 1 2024-04-12T05:55:34.832 INFO:tasks.workunit.client.0.smithi008.stdout:7/601: stat d1/l16 0 2024-04-12T05:55:34.833 INFO:tasks.workunit.client.0.smithi008.stdout:6/569: fdatasync df/d10/d20/d3e/f61 0 2024-04-12T05:55:34.837 INFO:tasks.workunit.client.0.smithi008.stdout:2/564: dwrite d6/db/f69 [0,4194304] 0 2024-04-12T05:55:34.839 INFO:tasks.workunit.client.0.smithi008.stdout:4/492: truncate d1/d22/d29/d30/f34 5121597 0 2024-04-12T05:55:34.841 INFO:tasks.workunit.client.0.smithi008.stdout:6/570: unlink df/d10/d20/d3e/fab 0 2024-04-12T05:55:34.845 INFO:tasks.workunit.client.0.smithi008.stdout:7/602: dwrite d1/d49/f40 [4194304,4194304] 0 2024-04-12T05:55:34.845 INFO:tasks.workunit.client.0.smithi008.stdout:6/571: write df/f31 [2053344,307] 0 2024-04-12T05:55:34.849 INFO:tasks.workunit.client.0.smithi008.stdout:4/493: symlink d1/d8/l8e 0 2024-04-12T05:55:34.850 INFO:tasks.workunit.client.0.smithi008.stdout:7/603: stat d1/d5/da/d11/d56/d8f 0 2024-04-12T05:55:34.854 INFO:tasks.workunit.client.0.smithi008.stdout:6/572: rename df/d10/d15/l25 to df/d10/d20/d3e/lbf 0 2024-04-12T05:55:34.855 INFO:tasks.workunit.client.0.smithi008.stdout:0/504: truncate d2/d11/d36/d47/f4f 425698 0 2024-04-12T05:55:34.856 INFO:tasks.workunit.client.0.smithi008.stdout:2/565: mkdir d6/db/d10/dbc 0 2024-04-12T05:55:34.857 INFO:tasks.workunit.client.0.smithi008.stdout:0/505: chown d2/d11/d36/d47/f21 935366315 1 2024-04-12T05:55:34.859 INFO:tasks.workunit.client.0.smithi008.stdout:4/494: chown d1/d22/d29/c2e 138 1 2024-04-12T05:55:34.862 INFO:tasks.workunit.client.0.smithi008.stdout:7/604: getdents d1/d5/da 0 2024-04-12T05:55:34.862 INFO:tasks.workunit.client.0.smithi008.stdout:6/573: mknod df/d10/d20/d3e/db4/cc0 0 2024-04-12T05:55:34.863 INFO:tasks.workunit.client.0.smithi008.stdout:2/566: symlink d6/db/lbd 0 2024-04-12T05:55:34.865 INFO:tasks.workunit.client.0.smithi008.stdout:4/495: unlink d1/l67 0 2024-04-12T05:55:34.867 INFO:tasks.workunit.client.0.smithi008.stdout:7/605: rename d1/d5/da/d11 to d1/d5/da/d11/d56/d8f/dd4 22 2024-04-12T05:55:34.868 INFO:tasks.workunit.client.0.smithi008.stdout:6/574: write f4 [2817632,33073] 0 2024-04-12T05:55:34.869 INFO:tasks.workunit.client.0.smithi008.stdout:0/506: getdents d2/d24/d85 0 2024-04-12T05:55:34.870 INFO:tasks.workunit.client.0.smithi008.stdout:7/606: dread - d1/d15/fc8 zero size 2024-04-12T05:55:34.873 INFO:tasks.workunit.client.0.smithi008.stdout:4/496: creat d1/db/f8f x:0 0 0 2024-04-12T05:55:34.874 INFO:tasks.workunit.client.0.smithi008.stdout:0/507: mkdir d2/d11/d36/d42/d86/d9b 0 2024-04-12T05:55:34.876 INFO:tasks.workunit.client.0.smithi008.stdout:6/575: mkdir df/dc1 0 2024-04-12T05:55:34.879 INFO:tasks.workunit.client.0.smithi008.stdout:2/567: dread d6/f9 [0,4194304] 0 2024-04-12T05:55:34.880 INFO:tasks.workunit.client.0.smithi008.stdout:6/576: symlink df/lc2 0 2024-04-12T05:55:34.886 INFO:tasks.workunit.client.0.smithi008.stdout:4/497: creat d1/d22/d29/d30/f90 x:0 0 0 2024-04-12T05:55:34.887 INFO:tasks.workunit.client.0.smithi008.stdout:7/607: rename d1/d15/d43/d54/d58/db7/fcf to d1/fd5 0 2024-04-12T05:55:34.890 INFO:tasks.workunit.client.0.smithi008.stdout:2/568: dwrite d6/db/d38/f5b [0,4194304] 0 2024-04-12T05:55:34.892 INFO:tasks.workunit.client.0.smithi008.stdout:7/608: truncate d1/d5/da/d3c/fc1 982538 0 2024-04-12T05:55:34.892 INFO:tasks.workunit.client.0.smithi008.stdout:0/508: rename d2/c5e to d2/d11/c9c 0 2024-04-12T05:55:34.893 INFO:tasks.workunit.client.0.smithi008.stdout:2/569: dread - d6/db/d38/d41/fa5 zero size 2024-04-12T05:55:34.896 INFO:tasks.workunit.client.0.smithi008.stdout:7/609: mkdir d1/d5/da/d78/dd6 0 2024-04-12T05:55:34.898 INFO:tasks.workunit.client.0.smithi008.stdout:2/570: creat d6/db/d38/d41/fbe x:0 0 0 2024-04-12T05:55:34.900 INFO:tasks.workunit.client.0.smithi008.stdout:4/498: creat d1/d22/d29/d30/f91 x:0 0 0 2024-04-12T05:55:34.904 INFO:tasks.workunit.client.0.smithi008.stdout:2/571: mkdir d6/db/d60/d4f/dbf 0 2024-04-12T05:55:34.906 INFO:tasks.workunit.client.0.smithi008.stdout:4/499: creat d1/d22/d29/d46/d56/f92 x:0 0 0 2024-04-12T05:55:34.907 INFO:tasks.workunit.client.0.smithi008.stdout:6/577: dwrite df/d10/d15/f3f [0,4194304] 0 2024-04-12T05:55:34.908 INFO:tasks.workunit.client.0.smithi008.stdout:2/572: write d6/db/d29/f6c [356897,96264] 0 2024-04-12T05:55:34.910 INFO:tasks.workunit.client.0.smithi008.stdout:7/610: dread d1/d49/f40 [4194304,4194304] 0 2024-04-12T05:55:34.910 INFO:tasks.workunit.client.0.smithi008.stdout:4/500: fdatasync d1/d22/d29/d46/d56/f92 0 2024-04-12T05:55:34.910 INFO:tasks.workunit.client.0.smithi008.stdout:2/573: chown d6/db/d10/dbc 61 1 2024-04-12T05:55:34.911 INFO:tasks.workunit.client.0.smithi008.stdout:0/509: write d2/d11/d36/f7c [859456,6193] 0 2024-04-12T05:55:34.915 INFO:tasks.workunit.client.0.smithi008.stdout:2/574: truncate d6/db/f22 5032542 0 2024-04-12T05:55:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:4/501: getdents d1/db 0 2024-04-12T05:55:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:7/611: creat d1/d5/da/d60/fd7 x:0 0 0 2024-04-12T05:55:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:2/575: symlink d6/db/d60/d84/lc0 0 2024-04-12T05:55:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:6/578: rename df/d10/d15/d75/c87 to df/d10/cc3 0 2024-04-12T05:55:34.926 INFO:tasks.workunit.client.0.smithi008.stdout:6/579: readlink df/d21/l45 0 2024-04-12T05:55:34.927 INFO:tasks.workunit.client.0.smithi008.stdout:7/612: chown d1/d15/d43/d54/d58/db7/fa5 2 1 2024-04-12T05:55:34.927 INFO:tasks.workunit.client.0.smithi008.stdout:0/510: creat d2/d11/d2a/f9d x:0 0 0 2024-04-12T05:55:34.929 INFO:tasks.workunit.client.0.smithi008.stdout:0/511: dread - d2/d11/d36/d47/f91 zero size 2024-04-12T05:55:34.929 INFO:tasks.workunit.client.0.smithi008.stdout:2/576: mknod d6/db/d38/cc1 0 2024-04-12T05:55:34.929 INFO:tasks.workunit.client.0.smithi008.stdout:7/613: fdatasync d1/d5/da/d3c/d44/d8a/f3b 0 2024-04-12T05:55:34.930 INFO:tasks.workunit.client.0.smithi008.stdout:4/502: dread d1/d22/d29/d30/d47/f26 [0,4194304] 0 2024-04-12T05:55:34.931 INFO:tasks.workunit.client.0.smithi008.stdout:7/614: fdatasync d1/d5/f9d 0 2024-04-12T05:55:34.932 INFO:tasks.workunit.client.0.smithi008.stdout:2/577: mkdir d6/db/d29/dc2 0 2024-04-12T05:55:34.934 INFO:tasks.workunit.client.0.smithi008.stdout:4/503: rmdir d1/d8 39 2024-04-12T05:55:34.941 INFO:tasks.workunit.client.0.smithi008.stdout:2/578: creat d6/db/d6f/da6/fc3 x:0 0 0 2024-04-12T05:55:34.942 INFO:tasks.workunit.client.0.smithi008.stdout:0/512: getdents d2/d4b 0 2024-04-12T05:55:34.944 INFO:tasks.workunit.client.0.smithi008.stdout:4/504: getdents d1/d22/d29/d46/d56 0 2024-04-12T05:55:34.945 INFO:tasks.workunit.client.0.smithi008.stdout:0/513: rmdir d2/d11/d36/d82/d95 39 2024-04-12T05:55:34.949 INFO:tasks.workunit.client.0.smithi008.stdout:0/514: mknod d2/d11/c9e 0 2024-04-12T05:55:34.949 INFO:tasks.workunit.client.0.smithi008.stdout:4/505: dread - d1/d8/f84 zero size 2024-04-12T05:55:34.949 INFO:tasks.workunit.client.0.smithi008.stdout:0/515: read - d2/d11/f64 zero size 2024-04-12T05:55:34.950 INFO:tasks.workunit.client.0.smithi008.stdout:4/506: chown d1/d4f/f5b 1 1 2024-04-12T05:55:34.950 INFO:tasks.workunit.client.0.smithi008.stdout:0/516: dread - d2/d11/d2a/f9d zero size 2024-04-12T05:55:34.951 INFO:tasks.workunit.client.0.smithi008.stdout:7/615: dread d1/d5/da/d11/f69 [0,4194304] 0 2024-04-12T05:55:34.952 INFO:tasks.workunit.client.0.smithi008.stdout:4/507: chown d1/d22/d29/d30/d47/d28/f5c 111 1 2024-04-12T05:55:34.953 INFO:tasks.workunit.client.0.smithi008.stdout:0/517: write d2/d24/f32 [476513,102055] 0 2024-04-12T05:55:34.956 INFO:tasks.workunit.client.0.smithi008.stdout:4/508: symlink d1/d8/l93 0 2024-04-12T05:55:34.957 INFO:tasks.workunit.client.0.smithi008.stdout:0/518: creat d2/d24/d61/f9f x:0 0 0 2024-04-12T05:55:34.957 INFO:tasks.workunit.client.0.smithi008.stdout:7/616: mknod d1/d5/da/d3c/cd8 0 2024-04-12T05:55:34.961 INFO:tasks.workunit.client.0.smithi008.stdout:7/617: creat d1/d5/da/d1d/dc3/fd9 x:0 0 0 2024-04-12T05:55:34.965 INFO:tasks.workunit.client.0.smithi008.stdout:4/509: truncate d1/f36 1071655 0 2024-04-12T05:55:34.969 INFO:tasks.workunit.client.0.smithi008.stdout:4/510: unlink d1/db/f1a 0 2024-04-12T05:55:34.970 INFO:tasks.workunit.client.0.smithi008.stdout:7/618: dwrite d1/d5/f29 [4194304,4194304] 0 2024-04-12T05:55:34.971 INFO:tasks.workunit.client.0.smithi008.stdout:4/511: write d1/d22/d29/d30/f90 [282528,87150] 0 2024-04-12T05:55:34.973 INFO:tasks.workunit.client.0.smithi008.stdout:4/512: stat d1/f40 0 2024-04-12T05:55:34.977 INFO:tasks.workunit.client.0.smithi008.stdout:4/513: rename d1/d22/d29/f41 to d1/d22/d29/f94 0 2024-04-12T05:55:34.978 INFO:tasks.workunit.client.0.smithi008.stdout:0/519: dread d2/f66 [0,4194304] 0 2024-04-12T05:55:34.978 INFO:tasks.workunit.client.0.smithi008.stdout:0/520: chown d2/d11/d36/d47/f6d 49180 1 2024-04-12T05:55:34.979 INFO:tasks.workunit.client.0.smithi008.stdout:4/514: creat d1/d22/d29/d30/d47/d28/d4a/f95 x:0 0 0 2024-04-12T05:55:34.980 INFO:tasks.workunit.client.0.smithi008.stdout:0/521: dread - d2/d11/d36/f48 zero size 2024-04-12T05:55:34.980 INFO:tasks.workunit.client.0.smithi008.stdout:4/515: chown d1/d22/d29/d46/d56/f92 0 1 2024-04-12T05:55:35.026 INFO:tasks.workunit.client.0.smithi008.stdout:9/585: sync 2024-04-12T05:55:35.026 INFO:tasks.workunit.client.0.smithi008.stdout:8/509: sync 2024-04-12T05:55:35.026 INFO:tasks.workunit.client.0.smithi008.stdout:4/516: symlink d1/d22/d29/d46/d56/l96 0 2024-04-12T05:55:35.028 INFO:tasks.workunit.client.0.smithi008.stdout:4/517: fdatasync d1/d22/d29/d30/f52 0 2024-04-12T05:55:35.030 INFO:tasks.workunit.client.0.smithi008.stdout:9/586: creat d3/d22/d2b/d58/d5d/fcd x:0 0 0 2024-04-12T05:55:35.031 INFO:tasks.workunit.client.0.smithi008.stdout:4/518: dread - d1/d22/d29/d46/d56/f80 zero size 2024-04-12T05:55:35.033 INFO:tasks.workunit.client.0.smithi008.stdout:0/522: dwrite d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:35.035 INFO:tasks.workunit.client.0.smithi008.stdout:0/523: write d2/d11/d36/f7c [255296,48161] 0 2024-04-12T05:55:35.035 INFO:tasks.workunit.client.0.smithi008.stdout:8/510: rename dd/de/d4c/d66/d6d/d8e to dd/de/d16/db1 0 2024-04-12T05:55:35.036 INFO:tasks.workunit.client.0.smithi008.stdout:4/519: dread - d1/d22/d29/d30/d47/d28/d4a/f95 zero size 2024-04-12T05:55:35.038 INFO:tasks.workunit.client.0.smithi008.stdout:0/524: readlink d2/d11/d36/d47/l1c 0 2024-04-12T05:55:35.038 INFO:tasks.workunit.client.0.smithi008.stdout:8/511: mkdir dd/dab/db2 0 2024-04-12T05:55:35.041 INFO:tasks.workunit.client.0.smithi008.stdout:9/587: read d3/df/d13/d21/d2d/d30/f4d [1719177,95104] 0 2024-04-12T05:55:35.043 INFO:tasks.workunit.client.0.smithi008.stdout:0/525: unlink d2/d11/d36/d47/c84 0 2024-04-12T05:55:35.047 INFO:tasks.workunit.client.0.smithi008.stdout:9/588: creat d3/fce x:0 0 0 2024-04-12T05:55:35.051 INFO:tasks.workunit.client.0.smithi008.stdout:9/589: symlink d3/df/d13/d21/d2d/d32/d52/lcf 0 2024-04-12T05:55:35.054 INFO:tasks.workunit.client.0.smithi008.stdout:0/526: dwrite d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:35.054 INFO:tasks.workunit.client.0.smithi008.stdout:9/590: chown d3/df/d13/d21/d5c/d8f/dab/fc3 402530997 1 2024-04-12T05:55:35.057 INFO:tasks.workunit.client.0.smithi008.stdout:0/527: read d2/d11/d2a/f5c [15580,119674] 0 2024-04-12T05:55:35.058 INFO:tasks.workunit.client.0.smithi008.stdout:0/528: fdatasync d2/d24/d61/f9f 0 2024-04-12T05:55:35.059 INFO:tasks.workunit.client.0.smithi008.stdout:0/529: dread - d2/d11/d36/f48 zero size 2024-04-12T05:55:35.060 INFO:tasks.workunit.client.0.smithi008.stdout:8/512: dread dd/f1a [0,4194304] 0 2024-04-12T05:55:35.061 INFO:tasks.workunit.client.0.smithi008.stdout:9/591: readlink d3/df/d13/d21/d2d/d32/lbf 0 2024-04-12T05:55:35.062 INFO:tasks.workunit.client.0.smithi008.stdout:4/520: getdents d1/d22/d29/d46/d56 0 2024-04-12T05:55:35.071 INFO:tasks.workunit.client.0.smithi008.stdout:0/530: creat d2/d24/fa0 x:0 0 0 2024-04-12T05:55:35.073 INFO:tasks.workunit.client.0.smithi008.stdout:9/592: dwrite d3/fa [0,4194304] 0 2024-04-12T05:55:35.075 INFO:tasks.workunit.client.0.smithi008.stdout:8/513: dwrite dd/de/d16/d50/d56/f5d [0,4194304] 0 2024-04-12T05:55:35.077 INFO:tasks.workunit.client.0.smithi008.stdout:0/531: fdatasync d2/d11/d36/d47/f2c 0 2024-04-12T05:55:35.087 INFO:tasks.workunit.client.0.smithi008.stdout:8/514: link dd/de/d16/d50/d56/d61/d9c/cb0 dd/cb3 0 2024-04-12T05:55:35.089 INFO:tasks.workunit.client.0.smithi008.stdout:4/521: dread d1/f36 [0,4194304] 0 2024-04-12T05:55:35.089 INFO:tasks.workunit.client.0.smithi008.stdout:0/532: write d2/d4b/f5f [621978,57279] 0 2024-04-12T05:55:35.091 INFO:tasks.workunit.client.0.smithi008.stdout:8/515: symlink dd/de/d4c/d66/lb4 0 2024-04-12T05:55:35.094 INFO:tasks.workunit.client.0.smithi008.stdout:8/516: rmdir dd/d30/d32 39 2024-04-12T05:55:35.100 INFO:tasks.workunit.client.0.smithi008.stdout:0/533: write d2/d11/f2e [2985653,2045] 0 2024-04-12T05:55:35.105 INFO:tasks.workunit.client.0.smithi008.stdout:0/534: symlink d2/d11/d2a/la1 0 2024-04-12T05:55:35.106 INFO:tasks.workunit.client.0.smithi008.stdout:8/517: dwrite dd/de/d16/d50/d5e/f92 [0,4194304] 0 2024-04-12T05:55:35.106 INFO:tasks.workunit.client.0.smithi008.stdout:9/593: dread d3/f43 [4194304,4194304] 0 2024-04-12T05:55:35.109 INFO:tasks.workunit.client.0.smithi008.stdout:8/518: mkdir dd/d30/d3b/db5 0 2024-04-12T05:55:35.112 INFO:tasks.workunit.client.0.smithi008.stdout:9/594: rename d3/d22/d2b/d4e/l88 to d3/df/d13/d21/d2d/d32/ld0 0 2024-04-12T05:55:35.113 INFO:tasks.workunit.client.0.smithi008.stdout:8/519: creat dd/d30/d3b/fb6 x:0 0 0 2024-04-12T05:55:35.114 INFO:tasks.workunit.client.0.smithi008.stdout:8/520: readlink dd/de/d16/d50/d5e/l67 0 2024-04-12T05:55:35.116 INFO:tasks.workunit.client.0.smithi008.stdout:9/595: link d3/la0 d3/df/d13/d14/d95/ld1 0 2024-04-12T05:55:35.117 INFO:tasks.workunit.client.0.smithi008.stdout:8/521: creat dd/d30/d32/fb7 x:0 0 0 2024-04-12T05:55:35.120 INFO:tasks.workunit.client.0.smithi008.stdout:8/522: unlink dd/de/f1c 0 2024-04-12T05:55:35.121 INFO:tasks.workunit.client.0.smithi008.stdout:8/523: dread - dd/d30/d3b/fb6 zero size 2024-04-12T05:55:35.125 INFO:tasks.workunit.client.0.smithi008.stdout:8/524: read dd/de/d16/f6e [405692,117740] 0 2024-04-12T05:55:35.127 INFO:tasks.workunit.client.0.smithi008.stdout:9/596: truncate d3/df/d13/d21/d2d/d30/faa 644761 0 2024-04-12T05:55:35.128 INFO:tasks.workunit.client.0.smithi008.stdout:8/525: creat dd/de/d16/d50/d56/d61/d77/fb8 x:0 0 0 2024-04-12T05:55:35.129 INFO:tasks.workunit.client.0.smithi008.stdout:9/597: unlink d3/d22/d2b/d58/d38/c85 0 2024-04-12T05:55:35.131 INFO:tasks.workunit.client.0.smithi008.stdout:8/526: chown dd/c86 834921 1 2024-04-12T05:55:35.131 INFO:tasks.workunit.client.0.smithi008.stdout:9/598: chown d3/df/d13/d21/d5c/da4/fb4 4 1 2024-04-12T05:55:35.132 INFO:tasks.workunit.client.0.smithi008.stdout:8/527: creat dd/d4f/d52/fb9 x:0 0 0 2024-04-12T05:55:35.138 INFO:tasks.workunit.client.0.smithi008.stdout:9/599: dwrite d3/d22/d2b/d58/da8/db6/fcb [0,4194304] 0 2024-04-12T05:55:35.141 INFO:tasks.workunit.client.0.smithi008.stdout:8/528: rmdir dd 39 2024-04-12T05:55:35.146 INFO:tasks.workunit.client.0.smithi008.stdout:8/529: chown dd/c44 27 1 2024-04-12T05:55:35.148 INFO:tasks.workunit.client.0.smithi008.stdout:9/600: write d3/d22/d2b/d77/f78 [2908302,47172] 0 2024-04-12T05:55:35.152 INFO:tasks.workunit.client.0.smithi008.stdout:9/601: mknod d3/df/d13/d21/d2d/d30/d74/cd2 0 2024-04-12T05:55:35.153 INFO:tasks.workunit.client.0.smithi008.stdout:8/530: dwrite dd/de/d16/db1/d97/fa3 [0,4194304] 0 2024-04-12T05:55:35.154 INFO:tasks.workunit.client.0.smithi008.stdout:9/602: write d3/df/d13/d21/d2d/d75/f94 [1627486,27121] 0 2024-04-12T05:55:35.158 INFO:tasks.workunit.client.0.smithi008.stdout:9/603: creat d3/df/d13/d21/d2d/d75/fd3 x:0 0 0 2024-04-12T05:55:35.158 INFO:tasks.workunit.client.0.smithi008.stdout:9/604: read - d3/fce zero size 2024-04-12T05:55:35.160 INFO:tasks.workunit.client.0.smithi008.stdout:8/531: getdents dd/d4f/d52 0 2024-04-12T05:55:35.164 INFO:tasks.workunit.client.0.smithi008.stdout:8/532: unlink dd/de/f12 0 2024-04-12T05:55:35.167 INFO:tasks.workunit.client.0.smithi008.stdout:9/605: dwrite d3/fce [0,4194304] 0 2024-04-12T05:55:35.168 INFO:tasks.workunit.client.0.smithi008.stdout:8/533: truncate dd/de/d16/d50/d5e/f8c 265718 0 2024-04-12T05:55:35.170 INFO:tasks.workunit.client.0.smithi008.stdout:9/606: creat d3/df/d13/d21/d5c/da4/fd4 x:0 0 0 2024-04-12T05:55:35.172 INFO:tasks.workunit.client.0.smithi008.stdout:9/607: creat d3/df/d13/d21/d2d/d75/fd5 x:0 0 0 2024-04-12T05:55:35.173 INFO:tasks.workunit.client.0.smithi008.stdout:9/608: chown d3/df/d13/d21/d5c/l5e 242861 1 2024-04-12T05:55:35.174 INFO:tasks.workunit.client.0.smithi008.stdout:8/534: creat dd/de/d16/fba x:0 0 0 2024-04-12T05:55:35.175 INFO:tasks.workunit.client.0.smithi008.stdout:8/535: write dd/f99 [278476,76186] 0 2024-04-12T05:55:35.176 INFO:tasks.workunit.client.0.smithi008.stdout:9/609: creat d3/d22/d2b/d4e/d60/fd6 x:0 0 0 2024-04-12T05:55:35.179 INFO:tasks.workunit.client.0.smithi008.stdout:8/536: creat dd/de/d16/d50/d56/d61/fbb x:0 0 0 2024-04-12T05:55:35.183 INFO:tasks.workunit.client.0.smithi008.stdout:8/537: write dd/de/d4c/d66/da9/f6f [1309429,13452] 0 2024-04-12T05:55:35.184 INFO:tasks.workunit.client.0.smithi008.stdout:9/610: dwrite d3/df/d13/d14/fa3 [0,4194304] 0 2024-04-12T05:55:35.188 INFO:tasks.workunit.client.0.smithi008.stdout:8/538: symlink dd/de/d16/d50/d56/d61/lbc 0 2024-04-12T05:55:35.192 INFO:tasks.workunit.client.0.smithi008.stdout:8/539: creat dd/d30/d32/fbd x:0 0 0 2024-04-12T05:55:35.192 INFO:tasks.workunit.client.0.smithi008.stdout:8/540: chown dd/f36 554 1 2024-04-12T05:55:35.194 INFO:tasks.workunit.client.0.smithi008.stdout:9/611: rename d3/df/d13/d21/d2d/d75/d7c to d3/df/d13/d21/d5c/dd7 0 2024-04-12T05:55:35.195 INFO:tasks.workunit.client.0.smithi008.stdout:8/541: mkdir dd/de/d16/d50/d5e/dbe 0 2024-04-12T05:55:35.195 INFO:tasks.workunit.client.0.smithi008.stdout:9/612: fdatasync d3/d22/d2b/f72 0 2024-04-12T05:55:35.195 INFO:tasks.workunit.client.0.smithi008.stdout:8/542: readlink dd/de/d16/l21 0 2024-04-12T05:55:35.198 INFO:tasks.workunit.client.0.smithi008.stdout:9/613: fdatasync d3/df/d13/f5a 0 2024-04-12T05:55:35.200 INFO:tasks.workunit.client.0.smithi008.stdout:6/580: sync 2024-04-12T05:55:35.200 INFO:tasks.workunit.client.0.smithi008.stdout:5/904: sync 2024-04-12T05:55:35.200 INFO:tasks.workunit.client.0.smithi008.stdout:3/588: sync 2024-04-12T05:55:35.200 INFO:tasks.workunit.client.0.smithi008.stdout:1/518: sync 2024-04-12T05:55:35.201 INFO:tasks.workunit.client.0.smithi008.stdout:5/905: chown d0/dc/d106 49791 1 2024-04-12T05:55:35.204 INFO:tasks.workunit.client.0.smithi008.stdout:9/614: symlink d3/df/d13/d21/d2d/d32/ld8 0 2024-04-12T05:55:35.207 INFO:tasks.workunit.client.0.smithi008.stdout:8/543: dwrite dd/de/d16/d50/d5e/f92 [0,4194304] 0 2024-04-12T05:55:35.208 INFO:tasks.workunit.client.0.smithi008.stdout:1/519: dread d4/d6/d90/f8d [0,4194304] 0 2024-04-12T05:55:35.209 INFO:tasks.workunit.client.0.smithi008.stdout:5/906: truncate d0/dd/d11/d38/fdb 797681 0 2024-04-12T05:55:35.210 INFO:tasks.workunit.client.0.smithi008.stdout:3/589: mkdir d9/d10/d19/d22/da8/dc4 0 2024-04-12T05:55:35.211 INFO:tasks.workunit.client.0.smithi008.stdout:8/544: chown dd/d4f/d52/f8d 18769 1 2024-04-12T05:55:35.212 INFO:tasks.workunit.client.0.smithi008.stdout:9/615: write d3/df/d13/d21/d5c/dd7/fbe [980058,95675] 0 2024-04-12T05:55:35.213 INFO:tasks.workunit.client.0.smithi008.stdout:3/590: symlink d9/d11/lc5 0 2024-04-12T05:55:35.213 INFO:tasks.workunit.client.0.smithi008.stdout:1/520: rmdir d4/d33/d74 39 2024-04-12T05:55:35.215 INFO:tasks.workunit.client.0.smithi008.stdout:6/581: dread df/d10/d15/f3f [0,4194304] 0 2024-04-12T05:55:35.216 INFO:tasks.workunit.client.0.smithi008.stdout:5/907: creat d0/d40/f117 x:0 0 0 2024-04-12T05:55:35.216 INFO:tasks.workunit.client.0.smithi008.stdout:8/545: write dd/de/d16/d50/d56/d61/fbb [50645,56057] 0 2024-04-12T05:55:35.222 INFO:tasks.workunit.client.0.smithi008.stdout:1/521: truncate d4/d1a/f5f 897692 0 2024-04-12T05:55:35.224 INFO:tasks.workunit.client.0.smithi008.stdout:6/582: dread df/f23 [0,4194304] 0 2024-04-12T05:55:35.225 INFO:tasks.workunit.client.0.smithi008.stdout:8/546: symlink dd/de/d16/d50/d5e/lbf 0 2024-04-12T05:55:35.230 INFO:tasks.workunit.client.0.smithi008.stdout:5/908: rename d0/dd/d11/d38/l62 to d0/dd/l118 0 2024-04-12T05:55:35.230 INFO:tasks.workunit.client.0.smithi008.stdout:8/547: write dd/f99 [171489,95097] 0 2024-04-12T05:55:35.231 INFO:tasks.workunit.client.0.smithi008.stdout:9/616: dwrite d3/d22/d2b/d77/d82/f8a [0,4194304] 0 2024-04-12T05:55:35.235 INFO:tasks.workunit.client.0.smithi008.stdout:3/591: dwrite d9/d10/d19/d22/f5f [0,4194304] 0 2024-04-12T05:55:35.237 INFO:tasks.workunit.client.0.smithi008.stdout:1/522: creat d4/d1c/fb3 x:0 0 0 2024-04-12T05:55:35.239 INFO:tasks.workunit.client.0.smithi008.stdout:5/909: dread - d0/dc/d13/d1a/daf/fc3 zero size 2024-04-12T05:55:35.242 INFO:tasks.workunit.client.0.smithi008.stdout:6/583: rename df/d10/d3a/fa9 to df/dc1/fc4 0 2024-04-12T05:55:35.246 INFO:tasks.workunit.client.0.smithi008.stdout:3/592: creat d9/d2e/d3d/fc6 x:0 0 0 2024-04-12T05:55:35.248 INFO:tasks.workunit.client.0.smithi008.stdout:5/910: chown d0/d29/l46 1 1 2024-04-12T05:55:35.250 INFO:tasks.workunit.client.0.smithi008.stdout:8/548: dwrite dd/de/d16/d50/d56/d61/d77/fb8 [0,4194304] 0 2024-04-12T05:55:35.251 INFO:tasks.workunit.client.0.smithi008.stdout:6/584: creat df/d8c/da4/fc5 x:0 0 0 2024-04-12T05:55:35.254 INFO:tasks.workunit.client.0.smithi008.stdout:5/911: chown d0/dd/d11/d38/c33 29978410 1 2024-04-12T05:55:35.256 INFO:tasks.workunit.client.0.smithi008.stdout:1/523: link d4/d1a/f51 d4/d33/d74/d7a/fb4 0 2024-04-12T05:55:35.258 INFO:tasks.workunit.client.0.smithi008.stdout:3/593: mknod d9/d10/d19/cc7 0 2024-04-12T05:55:35.258 INFO:tasks.workunit.client.0.smithi008.stdout:1/524: fsync d4/dd/d17/fb2 0 2024-04-12T05:55:35.262 INFO:tasks.workunit.client.0.smithi008.stdout:8/549: mkdir dd/de/d4c/d66/d6c/dc0 0 2024-04-12T05:55:35.263 INFO:tasks.workunit.client.0.smithi008.stdout:3/594: fdatasync d9/d37/d69/f91 0 2024-04-12T05:55:35.263 INFO:tasks.workunit.client.0.smithi008.stdout:9/617: link d3/d22/d2b/d58/d38/l8b d3/d22/d2b/d58/d5d/ld9 0 2024-04-12T05:55:35.268 INFO:tasks.workunit.client.0.smithi008.stdout:1/525: mknod d4/dd/d21/d98/cb5 0 2024-04-12T05:55:35.269 INFO:tasks.workunit.client.0.smithi008.stdout:6/585: creat df/d8c/da4/dad/fc6 x:0 0 0 2024-04-12T05:55:35.269 INFO:tasks.workunit.client.0.smithi008.stdout:5/912: symlink d0/dc/d106/l119 0 2024-04-12T05:55:35.269 INFO:tasks.workunit.client.0.smithi008.stdout:6/586: dread - df/d21/f77 zero size 2024-04-12T05:55:35.269 INFO:tasks.workunit.client.0.smithi008.stdout:5/913: readlink d0/dd/d11/l105 0 2024-04-12T05:55:35.269 INFO:tasks.workunit.client.0.smithi008.stdout:8/550: rename dd/de/l20 to dd/de/d16/d50/d5e/lc1 0 2024-04-12T05:55:35.270 INFO:tasks.workunit.client.0.smithi008.stdout:3/595: symlink d9/d1e/lc8 0 2024-04-12T05:55:35.271 INFO:tasks.workunit.client.0.smithi008.stdout:9/618: truncate d3/d22/d2b/d77/f84 361222 0 2024-04-12T05:55:35.272 INFO:tasks.workunit.client.0.smithi008.stdout:9/619: chown d3/d22/d2b/d58/d89/lc1 143 1 2024-04-12T05:55:35.273 INFO:tasks.workunit.client.0.smithi008.stdout:1/526: mkdir d4/d33/d74/d7a/db6 0 2024-04-12T05:55:35.274 INFO:tasks.workunit.client.0.smithi008.stdout:6/587: creat df/d21/d4c/fc7 x:0 0 0 2024-04-12T05:55:35.275 INFO:tasks.workunit.client.0.smithi008.stdout:5/914: mknod d0/dd/d11/d38/d55/c11a 0 2024-04-12T05:55:35.277 INFO:tasks.workunit.client.0.smithi008.stdout:8/551: creat dd/de/d16/d73/fc2 x:0 0 0 2024-04-12T05:55:35.277 INFO:tasks.workunit.client.0.smithi008.stdout:8/552: truncate dd/d30/d3b/f74 1120923 0 2024-04-12T05:55:35.278 INFO:tasks.workunit.client.0.smithi008.stdout:8/553: stat dd/de/d16/c4d 0 2024-04-12T05:55:35.278 INFO:tasks.workunit.client.0.smithi008.stdout:8/554: readlink dd/la8 0 2024-04-12T05:55:35.278 INFO:tasks.workunit.client.0.smithi008.stdout:3/596: unlink d9/lb8 0 2024-04-12T05:55:35.281 INFO:tasks.workunit.client.0.smithi008.stdout:3/597: fdatasync d9/d10/d19/d22/f4d 0 2024-04-12T05:55:35.283 INFO:tasks.workunit.client.0.smithi008.stdout:6/588: unlink df/d10/d20/c28 0 2024-04-12T05:55:35.286 INFO:tasks.workunit.client.0.smithi008.stdout:1/527: getdents d4/d33/d74/da2 0 2024-04-12T05:55:35.289 INFO:tasks.workunit.client.0.smithi008.stdout:8/555: link dd/de/d4c/d66/da9/f7d dd/de/d16/d73/fc3 0 2024-04-12T05:55:35.290 INFO:tasks.workunit.client.0.smithi008.stdout:3/598: creat d9/d10/d19/d3a/dad/fc9 x:0 0 0 2024-04-12T05:55:35.292 INFO:tasks.workunit.client.0.smithi008.stdout:8/556: write dd/de/d16/f45 [94100,115460] 0 2024-04-12T05:55:35.292 INFO:tasks.workunit.client.0.smithi008.stdout:6/589: dwrite df/d10/d15/f9d [0,4194304] 0 2024-04-12T05:55:35.295 INFO:tasks.workunit.client.0.smithi008.stdout:3/599: mknod d9/d11/cca 0 2024-04-12T05:55:35.298 INFO:tasks.workunit.client.0.smithi008.stdout:6/590: chown df/d10/l35 272059 1 2024-04-12T05:55:35.299 INFO:tasks.workunit.client.0.smithi008.stdout:9/620: dread d3/d22/d2b/d58/f1e [0,4194304] 0 2024-04-12T05:55:35.302 INFO:tasks.workunit.client.0.smithi008.stdout:1/528: link d4/d33/l9d d4/dd/d17/d91/lb7 0 2024-04-12T05:55:35.302 INFO:tasks.workunit.client.0.smithi008.stdout:9/621: write d3/d22/d2b/d4e/d60/fd6 [160526,90005] 0 2024-04-12T05:55:35.304 INFO:tasks.workunit.client.0.smithi008.stdout:3/600: write d9/f1c [7352382,95858] 0 2024-04-12T05:55:35.307 INFO:tasks.workunit.client.0.smithi008.stdout:6/591: creat df/d10/d20/d3e/db4/fc8 x:0 0 0 2024-04-12T05:55:35.310 INFO:tasks.workunit.client.0.smithi008.stdout:9/622: mkdir d3/df/d13/d21/d5c/d8f/dab/dda 0 2024-04-12T05:55:35.313 INFO:tasks.workunit.client.0.smithi008.stdout:8/557: dread dd/d30/d3b/f55 [0,4194304] 0 2024-04-12T05:55:35.314 INFO:tasks.workunit.client.0.smithi008.stdout:5/915: truncate d0/dd/d11/d38/f43 1931030 0 2024-04-12T05:55:35.315 INFO:tasks.workunit.client.0.smithi008.stdout:3/601: mknod d9/d10/d19/d22/ccb 0 2024-04-12T05:55:35.322 INFO:tasks.workunit.client.0.smithi008.stdout:6/592: rename df/d10/d15/c5d to df/d8c/da4/dad/cc9 0 2024-04-12T05:55:35.323 INFO:tasks.workunit.client.0.smithi008.stdout:6/593: dread - df/d10/d3a/d78/f91 zero size 2024-04-12T05:55:35.324 INFO:tasks.workunit.client.0.smithi008.stdout:6/594: dread - df/d10/d15/d4f/d50/fb7 zero size 2024-04-12T05:55:35.325 INFO:tasks.workunit.client.0.smithi008.stdout:3/602: unlink d9/d10/c88 0 2024-04-12T05:55:35.326 INFO:tasks.workunit.client.0.smithi008.stdout:1/529: link d4/d24/d62/c93 d4/d33/cb8 0 2024-04-12T05:55:35.329 INFO:tasks.workunit.client.0.smithi008.stdout:5/916: mkdir d0/dab/d109/d11b 0 2024-04-12T05:55:35.329 INFO:tasks.workunit.client.0.smithi008.stdout:5/917: chown d0/dc/d57/d9d 56768109 1 2024-04-12T05:55:35.330 INFO:tasks.workunit.client.0.smithi008.stdout:3/603: rmdir d9/dd 39 2024-04-12T05:55:35.332 INFO:tasks.workunit.client.0.smithi008.stdout:3/604: write d9/f7a [478622,1198] 0 2024-04-12T05:55:35.333 INFO:tasks.workunit.client.0.smithi008.stdout:6/595: truncate df/f23 1225819 0 2024-04-12T05:55:35.334 INFO:tasks.workunit.client.0.smithi008.stdout:6/596: fsync df/d10/d15/f13 0 2024-04-12T05:55:35.335 INFO:tasks.workunit.client.0.smithi008.stdout:1/530: unlink d4/d7/d18/l50 0 2024-04-12T05:55:35.336 INFO:tasks.workunit.client.0.smithi008.stdout:9/623: write d3/d22/d2b/d58/d89/f8d [1197962,81111] 0 2024-04-12T05:55:35.336 INFO:tasks.workunit.client.0.smithi008.stdout:6/597: stat df/d21/c4d 0 2024-04-12T05:55:35.338 INFO:tasks.workunit.client.0.smithi008.stdout:8/558: link dd/f1f dd/d30/d32/fc4 0 2024-04-12T05:55:35.350 INFO:tasks.workunit.client.0.smithi008.stdout:1/531: dread d4/d33/d74/fab [0,4194304] 0 2024-04-12T05:55:35.351 INFO:tasks.workunit.client.0.smithi008.stdout:5/918: dread d0/dd/d3d/f4b [0,4194304] 0 2024-04-12T05:55:35.352 INFO:tasks.workunit.client.0.smithi008.stdout:3/605: dwrite d9/d10/d19/d3a/d67/f8a [0,4194304] 0 2024-04-12T05:55:35.358 INFO:tasks.workunit.client.0.smithi008.stdout:3/606: truncate d9/d2e/d3d/f6b 1409646 0 2024-04-12T05:55:35.362 INFO:tasks.workunit.client.0.smithi008.stdout:5/919: dwrite d0/dc/d13/d1a/f22 [4194304,4194304] 0 2024-04-12T05:55:35.372 INFO:tasks.workunit.client.0.smithi008.stdout:3/607: mknod d9/d10/d19/d3a/d7c/ccc 0 2024-04-12T05:55:35.373 INFO:tasks.workunit.client.0.smithi008.stdout:3/608: write d9/d2e/d3d/f51 [1766645,106319] 0 2024-04-12T05:55:35.374 INFO:tasks.workunit.client.0.smithi008.stdout:8/559: rmdir dd/dab/db2 0 2024-04-12T05:55:35.377 INFO:tasks.workunit.client.0.smithi008.stdout:9/624: rename d3/d22/d2b/d4a/f97 to d3/d22/d2b/d58/fdb 0 2024-04-12T05:55:35.377 INFO:tasks.workunit.client.0.smithi008.stdout:6/598: link df/d10/d15/d4f/d50/l9e df/d21/d4c/lca 0 2024-04-12T05:55:35.378 INFO:tasks.workunit.client.0.smithi008.stdout:5/920: symlink d0/dd/d5c/l11c 0 2024-04-12T05:55:35.379 INFO:tasks.workunit.client.0.smithi008.stdout:6/599: chown df/d10/d15/d4f/d50/f6e 56696851 1 2024-04-12T05:55:35.379 INFO:tasks.workunit.client.0.smithi008.stdout:3/609: creat d9/d11/d5b/fcd x:0 0 0 2024-04-12T05:55:35.379 INFO:tasks.workunit.client.0.smithi008.stdout:2/579: sync 2024-04-12T05:55:35.379 INFO:tasks.workunit.client.0.smithi008.stdout:7/619: sync 2024-04-12T05:55:35.380 INFO:tasks.workunit.client.0.smithi008.stdout:5/921: fdatasync d0/dd/d11/d38/fdf 0 2024-04-12T05:55:35.382 INFO:tasks.workunit.client.0.smithi008.stdout:6/600: chown df/d10/d3a/l93 50145 1 2024-04-12T05:55:35.382 INFO:tasks.workunit.client.0.smithi008.stdout:2/580: read d6/db/f22 [975097,111944] 0 2024-04-12T05:55:35.384 INFO:tasks.workunit.client.0.smithi008.stdout:6/601: write df/d10/d15/d4f/d50/f8d [1042857,38893] 0 2024-04-12T05:55:35.386 INFO:tasks.workunit.client.0.smithi008.stdout:5/922: creat d0/dc/d57/d9d/db5/dbc/f11d x:0 0 0 2024-04-12T05:55:35.389 INFO:tasks.workunit.client.0.smithi008.stdout:1/532: rename d4/dd/d8b to d4/d1c/db9 0 2024-04-12T05:55:35.394 INFO:tasks.workunit.client.0.smithi008.stdout:6/602: dread df/d10/d15/d4f/d50/f6c [0,4194304] 0 2024-04-12T05:55:35.394 INFO:tasks.workunit.client.0.smithi008.stdout:1/533: write d4/dd/d17/d26/f9b [459793,82545] 0 2024-04-12T05:55:35.394 INFO:tasks.workunit.client.0.smithi008.stdout:1/534: write d4/d1a/f5f [713800,91936] 0 2024-04-12T05:55:35.396 INFO:tasks.workunit.client.0.smithi008.stdout:1/535: dread - d4/d33/d74/d46/f7e zero size 2024-04-12T05:55:35.399 INFO:tasks.workunit.client.0.smithi008.stdout:8/560: dread dd/de/d16/f2e [0,4194304] 0 2024-04-12T05:55:35.399 INFO:tasks.workunit.client.0.smithi008.stdout:6/603: dwrite df/d21/f22 [0,4194304] 0 2024-04-12T05:55:35.402 INFO:tasks.workunit.client.0.smithi008.stdout:6/604: write df/d21/d4c/fc7 [334814,52618] 0 2024-04-12T05:55:35.404 INFO:tasks.workunit.client.0.smithi008.stdout:1/536: mkdir d4/dd/dba 0 2024-04-12T05:55:35.406 INFO:tasks.workunit.client.0.smithi008.stdout:8/561: rmdir dd/d4f 39 2024-04-12T05:55:35.407 INFO:tasks.workunit.client.0.smithi008.stdout:3/610: link d9/d44/f63 d9/d10/d19/d22/fce 0 2024-04-12T05:55:35.409 INFO:tasks.workunit.client.0.smithi008.stdout:5/923: creat d0/dc/d13/d1a/daf/f11e x:0 0 0 2024-04-12T05:55:35.410 INFO:tasks.workunit.client.0.smithi008.stdout:2/581: getdents d6/db/d60/d84 0 2024-04-12T05:55:35.414 INFO:tasks.workunit.client.0.smithi008.stdout:8/562: rename dd/d30/d32/f71 to dd/de/d16/d50/d5e/fc5 0 2024-04-12T05:55:35.416 INFO:tasks.workunit.client.0.smithi008.stdout:5/924: mkdir d0/dc/d57/d9d/d11f 0 2024-04-12T05:55:35.420 INFO:tasks.workunit.client.0.smithi008.stdout:6/605: link df/d21/f57 df/d10/d3a/d9b/fcb 0 2024-04-12T05:55:35.423 INFO:tasks.workunit.client.0.smithi008.stdout:8/563: dwrite dd/d30/d3b/fb6 [0,4194304] 0 2024-04-12T05:55:35.431 INFO:tasks.workunit.client.0.smithi008.stdout:9/625: truncate d3/fb 717202 0 2024-04-12T05:55:35.431 INFO:tasks.workunit.client.0.smithi008.stdout:8/564: truncate dd/d30/d32/fbd 958591 0 2024-04-12T05:55:35.433 INFO:tasks.workunit.client.0.smithi008.stdout:9/626: chown d3/df/d13/d21/d5c/d8f/la7 835 1 2024-04-12T05:55:35.435 INFO:tasks.workunit.client.0.smithi008.stdout:7/620: dwrite d1/d15/d43/d54/d58/f67 [0,4194304] 0 2024-04-12T05:55:35.435 INFO:tasks.workunit.client.0.smithi008.stdout:6/606: mkdir df/d8c/dcc 0 2024-04-12T05:55:35.437 INFO:tasks.workunit.client.0.smithi008.stdout:2/582: link d6/l44 d6/db/d60/d4f/dbf/lc4 0 2024-04-12T05:55:35.442 INFO:tasks.workunit.client.0.smithi008.stdout:1/537: link d4/dd/d17/d91/lb7 d4/d33/d37/lbb 0 2024-04-12T05:55:35.443 INFO:tasks.workunit.client.0.smithi008.stdout:8/565: chown dd/d4f/f96 557 1 2024-04-12T05:55:35.445 INFO:tasks.workunit.client.0.smithi008.stdout:1/538: readlink d4/d24/l3a 0 2024-04-12T05:55:35.448 INFO:tasks.workunit.client.0.smithi008.stdout:6/607: creat df/d8c/da4/dad/fcd x:0 0 0 2024-04-12T05:55:35.448 INFO:tasks.workunit.client.0.smithi008.stdout:2/583: truncate d6/db/d10/d23/f4d 4057510 0 2024-04-12T05:55:35.448 INFO:tasks.workunit.client.0.smithi008.stdout:7/621: dwrite d1/f2d [0,4194304] 0 2024-04-12T05:55:35.449 INFO:tasks.workunit.client.0.smithi008.stdout:8/566: mknod dd/d4f/d52/da5/cc6 0 2024-04-12T05:55:35.456 INFO:tasks.workunit.client.0.smithi008.stdout:9/627: truncate d3/d22/d2b/d58/d36/f3c 443246 0 2024-04-12T05:55:35.456 INFO:tasks.workunit.client.0.smithi008.stdout:5/925: truncate d0/dc/d13/d1a/daf/f5 8969149 0 2024-04-12T05:55:35.460 INFO:tasks.workunit.client.0.smithi008.stdout:1/539: symlink d4/dd/d17/lbc 0 2024-04-12T05:55:35.463 INFO:tasks.workunit.client.0.smithi008.stdout:9/628: truncate d3/d22/d2b/d77/f84 832048 0 2024-04-12T05:55:35.464 INFO:tasks.workunit.client.0.smithi008.stdout:2/584: creat d6/db/d10/d23/d35/fc5 x:0 0 0 2024-04-12T05:55:35.465 INFO:tasks.workunit.client.0.smithi008.stdout:6/608: rename df/d21/l3b to df/d10/lce 0 2024-04-12T05:55:35.467 INFO:tasks.workunit.client.0.smithi008.stdout:1/540: write d4/d24/faa [4898436,86062] 0 2024-04-12T05:55:35.470 INFO:tasks.workunit.client.0.smithi008.stdout:2/585: fsync d6/db/d10/d23/f5c 0 2024-04-12T05:55:35.472 INFO:tasks.workunit.client.0.smithi008.stdout:8/567: rename dd/de/d16/lae to dd/de/d4c/d66/lc7 0 2024-04-12T05:55:35.476 INFO:tasks.workunit.client.0.smithi008.stdout:1/541: creat d4/dd/d17/d66/fbd x:0 0 0 2024-04-12T05:55:35.477 INFO:tasks.workunit.client.0.smithi008.stdout:6/609: dread df/d21/f43 [0,4194304] 0 2024-04-12T05:55:35.478 INFO:tasks.workunit.client.0.smithi008.stdout:2/586: dread - d6/db/d60/f78 zero size 2024-04-12T05:55:35.482 INFO:tasks.workunit.client.0.smithi008.stdout:9/629: rename d3/d22/d2b/d77/f78 to d3/d22/d2b/d58/da8/db6/fdc 0 2024-04-12T05:55:35.484 INFO:tasks.workunit.client.0.smithi008.stdout:5/926: dread d0/d29/fbf [0,4194304] 0 2024-04-12T05:55:35.486 INFO:tasks.workunit.client.0.smithi008.stdout:6/610: stat fd 0 2024-04-12T05:55:35.488 INFO:tasks.workunit.client.0.smithi008.stdout:1/542: creat d4/dd/d17/d91/fbe x:0 0 0 2024-04-12T05:55:35.488 INFO:tasks.workunit.client.0.smithi008.stdout:7/622: dread d1/d5/da/f62 [4194304,4194304] 0 2024-04-12T05:55:35.491 INFO:tasks.workunit.client.0.smithi008.stdout:2/587: creat d6/db/d38/d41/fc6 x:0 0 0 2024-04-12T05:55:35.492 INFO:tasks.workunit.client.0.smithi008.stdout:8/568: mkdir dd/de/d16/dc8 0 2024-04-12T05:55:35.495 INFO:tasks.workunit.client.0.smithi008.stdout:2/588: write d6/db/fb8 [979141,4701] 0 2024-04-12T05:55:35.496 INFO:tasks.workunit.client.0.smithi008.stdout:6/611: dwrite df/d10/d15/d75/fba [0,4194304] 0 2024-04-12T05:55:35.497 INFO:tasks.workunit.client.0.smithi008.stdout:1/543: mknod d4/dd/d21/cbf 0 2024-04-12T05:55:35.501 INFO:tasks.workunit.client.0.smithi008.stdout:1/544: fsync d4/d1c/f8f 0 2024-04-12T05:55:35.503 INFO:tasks.workunit.client.0.smithi008.stdout:7/623: rename d1/d15/d43/d54/l7e to d1/d15/d43/d54/d58/dc2/lda 0 2024-04-12T05:55:35.507 INFO:tasks.workunit.client.0.smithi008.stdout:5/927: symlink d0/dc/d13/d1a/d3a/de6/l120 0 2024-04-12T05:55:35.508 INFO:tasks.workunit.client.0.smithi008.stdout:2/589: creat d6/db/d10/d23/d35/d3c/fc7 x:0 0 0 2024-04-12T05:55:35.509 INFO:tasks.workunit.client.0.smithi008.stdout:6/612: dwrite df/d10/d15/fbb [0,4194304] 0 2024-04-12T05:55:35.513 INFO:tasks.workunit.client.0.smithi008.stdout:8/569: rename dd/d65/l93 to dd/de/d16/d50/d5e/lc9 0 2024-04-12T05:55:35.514 INFO:tasks.workunit.client.0.smithi008.stdout:1/545: write d4/d6/d90/f8d [1424928,9253] 0 2024-04-12T05:55:35.516 INFO:tasks.workunit.client.0.smithi008.stdout:5/928: dread - d0/d29/d96/f9b zero size 2024-04-12T05:55:35.517 INFO:tasks.workunit.client.0.smithi008.stdout:9/630: getdents d3/d22/d2b/d58/d90/db1 0 2024-04-12T05:55:35.522 INFO:tasks.workunit.client.0.smithi008.stdout:6/613: symlink df/d10/d15/lcf 0 2024-04-12T05:55:35.524 INFO:tasks.workunit.client.0.smithi008.stdout:8/570: mkdir dd/de/d4c/d66/d6d/dca 0 2024-04-12T05:55:35.527 INFO:tasks.workunit.client.0.smithi008.stdout:6/614: rmdir df/d8c 39 2024-04-12T05:55:35.533 INFO:tasks.workunit.client.0.smithi008.stdout:1/546: creat d4/d33/fc0 x:0 0 0 2024-04-12T05:55:35.533 INFO:tasks.workunit.client.0.smithi008.stdout:7/624: write d1/f55 [198183,92808] 0 2024-04-12T05:55:35.535 INFO:tasks.workunit.client.0.smithi008.stdout:8/571: dwrite dd/d30/d3b/f74 [0,4194304] 0 2024-04-12T05:55:35.538 INFO:tasks.workunit.client.0.smithi008.stdout:5/929: rename d0/dc/d13/d1a/d35/f4c to d0/dc/d13/d1a/daf/f121 0 2024-04-12T05:55:35.543 INFO:tasks.workunit.client.0.smithi008.stdout:2/590: write d6/db/d10/d23/f5c [1029253,56741] 0 2024-04-12T05:55:35.543 INFO:tasks.workunit.client.0.smithi008.stdout:8/572: creat dd/de/d4c/fcb x:0 0 0 2024-04-12T05:55:35.543 INFO:tasks.workunit.client.0.smithi008.stdout:8/573: readlink dd/la8 0 2024-04-12T05:55:35.544 INFO:tasks.workunit.client.0.smithi008.stdout:1/547: mkdir d4/d1c/db9/dc1 0 2024-04-12T05:55:35.544 INFO:tasks.workunit.client.0.smithi008.stdout:9/631: getdents d3/df/d13/d21/d2d/d30/d74 0 2024-04-12T05:55:35.546 INFO:tasks.workunit.client.0.smithi008.stdout:5/930: fdatasync d0/dc/fc1 0 2024-04-12T05:55:35.547 INFO:tasks.workunit.client.0.smithi008.stdout:1/548: write d4/d1c/d4f/f6a [5218634,40935] 0 2024-04-12T05:55:35.547 INFO:tasks.workunit.client.0.smithi008.stdout:7/625: getdents d1/d5/da/d3c/dd3 0 2024-04-12T05:55:35.551 INFO:tasks.workunit.client.0.smithi008.stdout:2/591: mkdir d6/db/d10/d23/d35/d3c/db5/dc8 0 2024-04-12T05:55:35.552 INFO:tasks.workunit.client.0.smithi008.stdout:5/931: creat d0/dd/d11/d38/d55/d89/f122 x:0 0 0 2024-04-12T05:55:35.553 INFO:tasks.workunit.client.0.smithi008.stdout:7/626: fsync d1/d5/da/d3c/d44/d8a/fa7 0 2024-04-12T05:55:35.555 INFO:tasks.workunit.client.0.smithi008.stdout:1/549: dread - d4/d33/d74/d30/d72/d7c/f80 zero size 2024-04-12T05:55:35.555 INFO:tasks.workunit.client.0.smithi008.stdout:2/592: chown d6/db/d29/c67 278459376 1 2024-04-12T05:55:35.559 INFO:tasks.workunit.client.0.smithi008.stdout:1/550: write d4/d1c/f8f [427500,92179] 0 2024-04-12T05:55:35.565 INFO:tasks.workunit.client.0.smithi008.stdout:9/632: dwrite d3/d22/d2b/d58/d89/f8d [0,4194304] 0 2024-04-12T05:55:35.570 INFO:tasks.workunit.client.0.smithi008.stdout:8/574: dread dd/de/d16/f28 [0,4194304] 0 2024-04-12T05:55:35.570 INFO:tasks.workunit.client.0.smithi008.stdout:8/575: chown dd/d30/f35 92615 1 2024-04-12T05:55:35.570 INFO:tasks.workunit.client.0.smithi008.stdout:2/593: rename d6/db/d38/f5b to d6/db/d10/d23/d35/d3c/fc9 0 2024-04-12T05:55:35.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:35 smithi194 ceph-mon[49771]: pgmap v13: 65 pgs: 65 active+clean; 840 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 110 MiB/s wr, 383 op/s 2024-04-12T05:55:35.572 INFO:tasks.workunit.client.0.smithi008.stdout:9/633: fdatasync d3/d22/d2b/d4e/f6b 0 2024-04-12T05:55:35.573 INFO:tasks.workunit.client.0.smithi008.stdout:4/522: sync 2024-04-12T05:55:35.573 INFO:tasks.workunit.client.0.smithi008.stdout:0/535: sync 2024-04-12T05:55:35.573 INFO:tasks.workunit.client.0.smithi008.stdout:8/576: write dd/de/f29 [2552827,44922] 0 2024-04-12T05:55:35.574 INFO:tasks.workunit.client.0.smithi008.stdout:2/594: chown d6/db/d38/d41/fc6 354 1 2024-04-12T05:55:35.578 INFO:tasks.workunit.client.0.smithi008.stdout:2/595: read d6/f16 [1405116,39154] 0 2024-04-12T05:55:35.582 INFO:tasks.workunit.client.0.smithi008.stdout:0/536: rmdir d2/d11 39 2024-04-12T05:55:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:5/932: dread d0/f97 [0,4194304] 0 2024-04-12T05:55:35.584 INFO:tasks.workunit.client.0.smithi008.stdout:8/577: mknod dd/d4f/d52/da5/ccc 0 2024-04-12T05:55:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:9/634: creat d3/df/d13/d21/fdd x:0 0 0 2024-04-12T05:55:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:1/551: dwrite d4/d7/f48 [0,4194304] 0 2024-04-12T05:55:35.586 INFO:tasks.workunit.client.0.smithi008.stdout:4/523: write d1/d8/f25 [249848,14165] 0 2024-04-12T05:55:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:2/596: creat d6/db/d10/d23/d35/fca x:0 0 0 2024-04-12T05:55:35.590 INFO:tasks.workunit.client.0.smithi008.stdout:8/578: symlink dd/d4f/d52/lcd 0 2024-04-12T05:55:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:1/552: mknod d4/d7/d18/cc2 0 2024-04-12T05:55:35.594 INFO:tasks.workunit.client.0.smithi008.stdout:4/524: write d1/d8/f23 [4282887,95930] 0 2024-04-12T05:55:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:2/597: read d6/db/f4b [454962,72800] 0 2024-04-12T05:55:35.597 INFO:tasks.workunit.client.0.smithi008.stdout:5/933: dread d0/dd/fd9 [0,4194304] 0 2024-04-12T05:55:35.599 INFO:tasks.workunit.client.0.smithi008.stdout:4/525: fdatasync d1/d8/f84 0 2024-04-12T05:55:35.600 INFO:tasks.workunit.client.0.smithi008.stdout:1/553: mknod d4/dd/d17/cc3 0 2024-04-12T05:55:35.601 INFO:tasks.workunit.client.0.smithi008.stdout:9/635: dread d3/d22/d2b/d4e/f6b [0,4194304] 0 2024-04-12T05:55:35.604 INFO:tasks.workunit.client.0.smithi008.stdout:2/598: readlink d6/db/d38/d41/l9a 0 2024-04-12T05:55:35.605 INFO:tasks.workunit.client.0.smithi008.stdout:5/934: creat d0/d29/de9/f123 x:0 0 0 2024-04-12T05:55:35.606 INFO:tasks.workunit.client.0.smithi008.stdout:0/537: dwrite d2/d11/d36/d47/f6d [0,4194304] 0 2024-04-12T05:55:35.608 INFO:tasks.workunit.client.0.smithi008.stdout:0/538: stat d2/l4e 0 2024-04-12T05:55:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:1/554: mknod d4/d1c/db9/cc4 0 2024-04-12T05:55:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:2/599: readlink d6/db/d38/d41/l34 0 2024-04-12T05:55:35.609 INFO:tasks.workunit.client.0.smithi008.stdout:8/579: dwrite dd/f99 [0,4194304] 0 2024-04-12T05:55:35.611 INFO:tasks.workunit.client.0.smithi008.stdout:5/935: rmdir d0/dab/d109 39 2024-04-12T05:55:35.613 INFO:tasks.workunit.client.0.smithi008.stdout:4/526: chown d1/d22/d29/d30/d47/d28/f79 2030730 1 2024-04-12T05:55:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:9/636: mknod d3/df/d13/cde 0 2024-04-12T05:55:35.615 INFO:tasks.workunit.client.0.smithi008.stdout:4/527: write d1/d22/d29/d30/f34 [67096,102819] 0 2024-04-12T05:55:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:2/600: symlink d6/db/d60/d4f/lcb 0 2024-04-12T05:55:35.616 INFO:tasks.workunit.client.0.smithi008.stdout:0/539: truncate d2/d11/d2a/f5c 778156 0 2024-04-12T05:55:35.618 INFO:tasks.workunit.client.0.smithi008.stdout:2/601: read d6/db/d60/f59 [243294,51934] 0 2024-04-12T05:55:35.619 INFO:tasks.workunit.client.0.smithi008.stdout:1/555: fdatasync d4/d7/f4e 0 2024-04-12T05:55:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:0/540: fsync d2/d11/d36/d47/f91 0 2024-04-12T05:55:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:2/602: readlink d6/db/d10/d4c/l7f 0 2024-04-12T05:55:35.621 INFO:tasks.workunit.client.0.smithi008.stdout:9/637: truncate d3/d22/d2b/d58/fdb 615243 0 2024-04-12T05:55:35.624 INFO:tasks.workunit.client.0.smithi008.stdout:0/541: truncate d2/d11/d36/d47/f87 1462323 0 2024-04-12T05:55:35.626 INFO:tasks.workunit.client.0.smithi008.stdout:0/542: dread - d2/d24/d61/f9f zero size 2024-04-12T05:55:35.630 INFO:tasks.workunit.client.0.smithi008.stdout:8/580: getdents dd/de/d16/d50/d56/d61/d9c 0 2024-04-12T05:55:35.632 INFO:tasks.workunit.client.0.smithi008.stdout:1/556: mkdir d4/d33/dc5 0 2024-04-12T05:55:35.633 INFO:tasks.workunit.client.0.smithi008.stdout:5/936: getdents d0/dd/d11/d38/d55/d89/df3 0 2024-04-12T05:55:35.633 INFO:tasks.workunit.client.0.smithi008.stdout:0/543: write d2/d24/d39/f53 [3930470,96321] 0 2024-04-12T05:55:35.636 INFO:tasks.workunit.client.0.smithi008.stdout:2/603: dwrite d6/db/d38/d41/f6b [0,4194304] 0 2024-04-12T05:55:35.636 INFO:tasks.workunit.client.0.smithi008.stdout:8/581: write dd/de/d4c/fcb [227525,49175] 0 2024-04-12T05:55:35.639 INFO:tasks.workunit.client.0.smithi008.stdout:0/544: mkdir d2/d4b/da2 0 2024-04-12T05:55:35.641 INFO:tasks.workunit.client.0.smithi008.stdout:8/582: write dd/d4f/d52/fb9 [59258,69741] 0 2024-04-12T05:55:35.641 INFO:tasks.workunit.client.0.smithi008.stdout:2/604: write d6/db/d60/faf [74712,98109] 0 2024-04-12T05:55:35.642 INFO:tasks.workunit.client.0.smithi008.stdout:9/638: getdents d3/d22 0 2024-04-12T05:55:35.643 INFO:tasks.workunit.client.0.smithi008.stdout:4/528: truncate d1/d4f/f5a 742080 0 2024-04-12T05:55:35.645 INFO:tasks.workunit.client.0.smithi008.stdout:1/557: symlink d4/d1c/lc6 0 2024-04-12T05:55:35.646 INFO:tasks.workunit.client.0.smithi008.stdout:5/937: mknod d0/dc/c124 0 2024-04-12T05:55:35.647 INFO:tasks.workunit.client.0.smithi008.stdout:0/545: read - d2/d11/d36/d47/f91 zero size 2024-04-12T05:55:35.647 INFO:tasks.workunit.client.0.smithi008.stdout:8/583: dread - dd/de/d16/d50/f7f zero size 2024-04-12T05:55:35.651 INFO:tasks.workunit.client.0.smithi008.stdout:2/605: mkdir d6/db/d10/dcc 0 2024-04-12T05:55:35.651 INFO:tasks.workunit.client.0.smithi008.stdout:4/529: creat d1/d22/f97 x:0 0 0 2024-04-12T05:55:35.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:35 smithi008 ceph-mon[39722]: pgmap v13: 65 pgs: 65 active+clean; 840 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 110 MiB/s wr, 383 op/s 2024-04-12T05:55:35.653 INFO:tasks.workunit.client.0.smithi008.stdout:2/606: write d6/db/fb8 [1826014,98849] 0 2024-04-12T05:55:35.654 INFO:tasks.workunit.client.0.smithi008.stdout:5/938: creat d0/dc/d13/d1a/daf/d37/f125 x:0 0 0 2024-04-12T05:55:35.656 INFO:tasks.workunit.client.0.smithi008.stdout:0/546: creat d2/d4b/da2/fa3 x:0 0 0 2024-04-12T05:55:35.658 INFO:tasks.workunit.client.0.smithi008.stdout:1/558: creat d4/d33/d74/d7a/db6/fc7 x:0 0 0 2024-04-12T05:55:35.660 INFO:tasks.workunit.client.0.smithi008.stdout:5/939: creat d0/dc/d13/d1a/daf/de3/f126 x:0 0 0 2024-04-12T05:55:35.661 INFO:tasks.workunit.client.0.smithi008.stdout:2/607: creat d6/db/d60/d84/fcd x:0 0 0 2024-04-12T05:55:35.663 INFO:tasks.workunit.client.0.smithi008.stdout:4/530: symlink d1/d22/d29/d46/l98 0 2024-04-12T05:55:35.663 INFO:tasks.workunit.client.0.smithi008.stdout:0/547: creat d2/d11/fa4 x:0 0 0 2024-04-12T05:55:35.665 INFO:tasks.workunit.client.0.smithi008.stdout:0/548: chown d2/d4b/f99 54 1 2024-04-12T05:55:35.666 INFO:tasks.workunit.client.0.smithi008.stdout:1/559: rename d4/dd/d17/l5a to d4/d33/d74/d7a/lc8 0 2024-04-12T05:55:35.667 INFO:tasks.workunit.client.0.smithi008.stdout:0/549: write d2/d11/f64 [631435,124992] 0 2024-04-12T05:55:35.667 INFO:tasks.workunit.client.0.smithi008.stdout:9/639: getdents d3/d22/d2b/d58/d38 0 2024-04-12T05:55:35.669 INFO:tasks.workunit.client.0.smithi008.stdout:0/550: write d2/d11/d36/d47/f91 [604998,7004] 0 2024-04-12T05:55:35.671 INFO:tasks.workunit.client.0.smithi008.stdout:0/551: chown d2/d11/d36/d42/c52 5221791 1 2024-04-12T05:55:35.673 INFO:tasks.workunit.client.0.smithi008.stdout:9/640: rmdir d3/df/d13/d21/d2d 39 2024-04-12T05:55:35.674 INFO:tasks.workunit.client.0.smithi008.stdout:9/641: chown d3/d22/d2b/d77/fbd 855 1 2024-04-12T05:55:35.675 INFO:tasks.workunit.client.0.smithi008.stdout:0/552: symlink d2/d11/d36/d42/la5 0 2024-04-12T05:55:35.678 INFO:tasks.workunit.client.0.smithi008.stdout:9/642: write d3/df/d13/d21/d2d/d75/f94 [522298,31527] 0 2024-04-12T05:55:35.678 INFO:tasks.workunit.client.0.smithi008.stdout:0/553: mkdir d2/d11/d36/d42/da6 0 2024-04-12T05:55:35.682 INFO:tasks.workunit.client.0.smithi008.stdout:2/608: dread d6/db/d6f/f88 [0,4194304] 0 2024-04-12T05:55:35.682 INFO:tasks.workunit.client.0.smithi008.stdout:0/554: creat d2/d11/d36/d42/da6/fa7 x:0 0 0 2024-04-12T05:55:35.684 INFO:tasks.workunit.client.0.smithi008.stdout:9/643: creat d3/d22/d2b/d58/fdf x:0 0 0 2024-04-12T05:55:35.685 INFO:tasks.workunit.client.0.smithi008.stdout:9/644: stat d3/df/d13/d21/d5c/d8f/dab 0 2024-04-12T05:55:35.686 INFO:tasks.workunit.client.0.smithi008.stdout:2/609: truncate d6/db/d38/d41/f90 1344182 0 2024-04-12T05:55:35.690 INFO:tasks.workunit.client.0.smithi008.stdout:5/940: write d0/d29/de9/f101 [533759,5456] 0 2024-04-12T05:55:35.691 INFO:tasks.workunit.client.0.smithi008.stdout:2/610: mknod d6/db/d60/d4f/dbf/cce 0 2024-04-12T05:55:35.692 INFO:tasks.workunit.client.0.smithi008.stdout:8/584: dwrite dd/de/d16/f33 [0,4194304] 0 2024-04-12T05:55:35.693 INFO:tasks.workunit.client.0.smithi008.stdout:8/585: fsync dd/f22 0 2024-04-12T05:55:35.694 INFO:tasks.workunit.client.0.smithi008.stdout:1/560: write d4/d24/f32 [447569,34164] 0 2024-04-12T05:55:35.698 INFO:tasks.workunit.client.0.smithi008.stdout:2/611: chown d6/db/d38/d41/caa 132186 1 2024-04-12T05:55:35.699 INFO:tasks.workunit.client.0.smithi008.stdout:5/941: mkdir d0/dc/d13/d1a/daf/d37/d127 0 2024-04-12T05:55:35.700 INFO:tasks.workunit.client.0.smithi008.stdout:5/942: chown d0/dc/d13/d1a/d3a/de6/l120 5559141 1 2024-04-12T05:55:35.701 INFO:tasks.workunit.client.0.smithi008.stdout:8/586: symlink dd/de/d16/d50/d56/lce 0 2024-04-12T05:55:35.702 INFO:tasks.workunit.client.0.smithi008.stdout:1/561: mkdir d4/d33/d74/d7a/db6/dc9 0 2024-04-12T05:55:35.703 INFO:tasks.workunit.client.0.smithi008.stdout:8/587: write dd/d4f/f9a [361087,65205] 0 2024-04-12T05:55:35.705 INFO:tasks.workunit.client.0.smithi008.stdout:5/943: creat d0/dc/d13/d1a/d3a/f128 x:0 0 0 2024-04-12T05:55:35.706 INFO:tasks.workunit.client.0.smithi008.stdout:2/612: mkdir d6/db/d38/dcf 0 2024-04-12T05:55:35.709 INFO:tasks.workunit.client.0.smithi008.stdout:2/613: chown d6/cac 3816 1 2024-04-12T05:55:35.709 INFO:tasks.workunit.client.0.smithi008.stdout:0/555: dwrite d2/d11/d2a/f5d [0,4194304] 0 2024-04-12T05:55:35.710 INFO:tasks.workunit.client.0.smithi008.stdout:5/944: write d0/dc/d13/d1a/daf/fac [104535,4989] 0 2024-04-12T05:55:35.711 INFO:tasks.workunit.client.0.smithi008.stdout:2/614: dread - d6/db/d38/d41/fad zero size 2024-04-12T05:55:35.713 INFO:tasks.workunit.client.0.smithi008.stdout:2/615: chown d6/db/d38/d41 233 1 2024-04-12T05:55:35.715 INFO:tasks.workunit.client.0.smithi008.stdout:0/556: creat d2/d24/d85/fa8 x:0 0 0 2024-04-12T05:55:35.716 INFO:tasks.workunit.client.0.smithi008.stdout:8/588: rename dd/de/d4c/d66/d6c/d89 to dd/de/d16/db1/d97/dcf 0 2024-04-12T05:55:35.716 INFO:tasks.workunit.client.0.smithi008.stdout:2/616: unlink d6/db/d60/d4f/c6a 0 2024-04-12T05:55:35.718 INFO:tasks.workunit.client.0.smithi008.stdout:0/557: rmdir d2/d11/d36/d82 39 2024-04-12T05:55:35.721 INFO:tasks.workunit.client.0.smithi008.stdout:5/945: dread d0/dc/d13/d1a/daf/d37/f45 [4194304,4194304] 0 2024-04-12T05:55:35.722 INFO:tasks.workunit.client.0.smithi008.stdout:0/558: mknod d2/d11/d2a/ca9 0 2024-04-12T05:55:35.725 INFO:tasks.workunit.client.0.smithi008.stdout:2/617: link d6/db/d6f/cb6 d6/db/d29/dc2/cd0 0 2024-04-12T05:55:35.727 INFO:tasks.workunit.client.0.smithi008.stdout:2/618: chown d6/db/d29/f6c 0 1 2024-04-12T05:55:35.728 INFO:tasks.workunit.client.0.smithi008.stdout:5/946: creat d0/dc/d57/d9d/db5/dbc/f129 x:0 0 0 2024-04-12T05:55:35.729 INFO:tasks.workunit.client.0.smithi008.stdout:8/589: dwrite dd/de/f2f [0,4194304] 0 2024-04-12T05:55:35.732 INFO:tasks.workunit.client.0.smithi008.stdout:2/619: creat d6/db/d10/d4c/fd1 x:0 0 0 2024-04-12T05:55:35.734 INFO:tasks.workunit.client.0.smithi008.stdout:8/590: mknod dd/de/d4c/cd0 0 2024-04-12T05:55:35.734 INFO:tasks.workunit.client.0.smithi008.stdout:5/947: chown d0/dc/d13/d1a/daf/cb 182 1 2024-04-12T05:55:35.736 INFO:tasks.workunit.client.0.smithi008.stdout:0/559: dwrite d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:35.738 INFO:tasks.workunit.client.0.smithi008.stdout:2/620: symlink d6/db/d10/ld2 0 2024-04-12T05:55:35.738 INFO:tasks.workunit.client.0.smithi008.stdout:8/591: creat dd/de/d16/d50/fd1 x:0 0 0 2024-04-12T05:55:35.739 INFO:tasks.workunit.client.0.smithi008.stdout:2/621: chown d6/db/d10/d23 675503341 1 2024-04-12T05:55:35.740 INFO:tasks.workunit.client.0.smithi008.stdout:5/948: creat d0/dd/d11/d38/d55/d89/f12a x:0 0 0 2024-04-12T05:55:35.742 INFO:tasks.workunit.client.0.smithi008.stdout:5/949: symlink d0/d29/de9/l12b 0 2024-04-12T05:55:35.744 INFO:tasks.workunit.client.0.smithi008.stdout:8/592: creat dd/de/d16/d50/d56/d61/d77/fd2 x:0 0 0 2024-04-12T05:55:35.745 INFO:tasks.workunit.client.0.smithi008.stdout:2/622: creat d6/db/d38/dcf/fd3 x:0 0 0 2024-04-12T05:55:35.745 INFO:tasks.workunit.client.0.smithi008.stdout:0/560: dread d2/d11/d36/d47/f4f [0,4194304] 0 2024-04-12T05:55:35.746 INFO:tasks.workunit.client.0.smithi008.stdout:8/593: write dd/f99 [4465947,109359] 0 2024-04-12T05:55:35.750 INFO:tasks.workunit.client.0.smithi008.stdout:0/561: chown d2/d24/c6e 3572053 1 2024-04-12T05:55:35.753 INFO:tasks.workunit.client.0.smithi008.stdout:5/950: link d0/dc/d13/d1a/daf/de3/ceb d0/dd/d11/d38/c12c 0 2024-04-12T05:55:35.756 INFO:tasks.workunit.client.0.smithi008.stdout:8/594: getdents dd/d30/d32 0 2024-04-12T05:55:35.756 INFO:tasks.workunit.client.0.smithi008.stdout:8/595: dread - dd/d30/d32/f7e zero size 2024-04-12T05:55:35.757 INFO:tasks.workunit.client.0.smithi008.stdout:2/623: dread d6/db/fd [0,4194304] 0 2024-04-12T05:55:35.762 INFO:tasks.workunit.client.0.smithi008.stdout:8/596: link dd/d4f/l59 dd/de/d16/d50/d56/ld3 0 2024-04-12T05:55:35.766 INFO:tasks.workunit.client.0.smithi008.stdout:8/597: creat dd/d65/fd4 x:0 0 0 2024-04-12T05:55:35.769 INFO:tasks.workunit.client.0.smithi008.stdout:8/598: rename dd/de/c57 to dd/cd5 0 2024-04-12T05:55:35.770 INFO:tasks.workunit.client.0.smithi008.stdout:5/951: rmdir d0/d29/de9 39 2024-04-12T05:55:35.775 INFO:tasks.workunit.client.0.smithi008.stdout:0/562: truncate d2/d11/d36/d47/f8b 2059761 0 2024-04-12T05:55:35.776 INFO:tasks.workunit.client.0.smithi008.stdout:0/563: chown d2/d4b/c50 550148 1 2024-04-12T05:55:35.776 INFO:tasks.workunit.client.0.smithi008.stdout:2/624: truncate d6/db/d10/d23/f5c 148600 0 2024-04-12T05:55:35.779 INFO:tasks.workunit.client.0.smithi008.stdout:0/564: rename d2/l4e to d2/d24/d39/laa 0 2024-04-12T05:55:35.853 INFO:tasks.workunit.client.0.smithi008.stdout:5/952: truncate d0/f12 2651267 0 2024-04-12T05:55:35.856 INFO:tasks.workunit.client.0.smithi008.stdout:2/625: getdents d6/db/d60/d84 0 2024-04-12T05:55:35.856 INFO:tasks.workunit.client.0.smithi008.stdout:5/953: dread - d0/dd/d11/d38/d55/ff4 zero size 2024-04-12T05:55:35.857 INFO:tasks.workunit.client.0.smithi008.stdout:5/954: chown d0/l6f 41 1 2024-04-12T05:55:35.858 INFO:tasks.workunit.client.0.smithi008.stdout:8/599: dwrite dd/f1f [4194304,4194304] 0 2024-04-12T05:55:35.861 INFO:tasks.workunit.client.0.smithi008.stdout:2/626: dread d6/db/d29/f6c [0,4194304] 0 2024-04-12T05:55:35.861 INFO:tasks.workunit.client.0.smithi008.stdout:8/600: mkdir dd/de/d16/d50/d5e/dbe/dd6 0 2024-04-12T05:55:35.870 INFO:tasks.workunit.client.0.smithi008.stdout:2/627: dwrite d6/db/d38/d41/f6b [0,4194304] 0 2024-04-12T05:55:35.872 INFO:tasks.workunit.client.0.smithi008.stdout:2/628: readlink d6/db/lbd 0 2024-04-12T05:55:35.875 INFO:tasks.workunit.client.0.smithi008.stdout:2/629: write d6/db/d6f/f9b [416205,115656] 0 2024-04-12T05:55:35.876 INFO:tasks.workunit.client.0.smithi008.stdout:2/630: dread - d6/db/d38/d41/fbe zero size 2024-04-12T05:55:35.876 INFO:tasks.workunit.client.0.smithi008.stdout:2/631: stat d6/db/d38/d41/l9a 0 2024-04-12T05:55:35.878 INFO:tasks.workunit.client.0.smithi008.stdout:2/632: mknod d6/db/d10/d23/d35/d3c/db5/dc8/cd4 0 2024-04-12T05:55:35.880 INFO:tasks.workunit.client.0.smithi008.stdout:2/633: chown d6/l44 20079 1 2024-04-12T05:55:35.881 INFO:tasks.workunit.client.0.smithi008.stdout:2/634: dread - d6/db/d38/d41/f93 zero size 2024-04-12T05:55:35.883 INFO:tasks.workunit.client.0.smithi008.stdout:2/635: symlink d6/db/d38/ld5 0 2024-04-12T05:55:35.885 INFO:tasks.workunit.client.0.smithi008.stdout:2/636: fsync d6/db/d10/d4c/f5a 0 2024-04-12T05:55:35.886 INFO:tasks.workunit.client.0.smithi008.stdout:2/637: dread - d6/db/d38/dcf/fd3 zero size 2024-04-12T05:55:35.891 INFO:tasks.workunit.client.0.smithi008.stdout:2/638: dwrite d6/db/d38/d41/f77 [0,4194304] 0 2024-04-12T05:55:35.893 INFO:tasks.workunit.client.0.smithi008.stdout:8/601: dread dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:35.953 INFO:tasks.workunit.client.0.smithi008.stdout:2/639: dwrite d6/db/d10/d23/d35/fc5 [0,4194304] 0 2024-04-12T05:55:35.953 INFO:tasks.workunit.client.0.smithi008.stdout:8/602: dwrite dd/f36 [0,4194304] 0 2024-04-12T05:55:35.959 INFO:tasks.workunit.client.0.smithi008.stdout:8/603: truncate dd/de/d16/f28 3266142 0 2024-04-12T05:55:35.963 INFO:tasks.workunit.client.0.smithi008.stdout:8/604: creat dd/de/d4c/d66/d6c/dc0/fd7 x:0 0 0 2024-04-12T05:55:35.966 INFO:tasks.workunit.client.0.smithi008.stdout:8/605: getdents dd/de/d16/db1/d97/dcf 0 2024-04-12T05:55:35.967 INFO:tasks.workunit.client.0.smithi008.stdout:8/606: chown dd/d30/d3b/f9b 45 1 2024-04-12T05:55:35.969 INFO:tasks.workunit.client.0.smithi008.stdout:8/607: mknod dd/d4f/d52/da5/cd8 0 2024-04-12T05:55:35.971 INFO:tasks.workunit.client.0.smithi008.stdout:8/608: stat dd/d30/d32/f3a 0 2024-04-12T05:55:35.971 INFO:tasks.workunit.client.0.smithi008.stdout:2/640: dread d6/db/d10/d23/f4d [0,4194304] 0 2024-04-12T05:55:35.973 INFO:tasks.workunit.client.0.smithi008.stdout:2/641: readlink d6/db/d10/d23/d35/l47 0 2024-04-12T05:55:35.973 INFO:tasks.workunit.client.0.smithi008.stdout:8/609: mknod dd/de/d16/d73/cd9 0 2024-04-12T05:55:35.978 INFO:tasks.workunit.client.0.smithi008.stdout:2/642: rename d6/db/d38/d41/caa to d6/db/d10/cd6 0 2024-04-12T05:55:35.979 INFO:tasks.workunit.client.0.smithi008.stdout:8/610: dread dd/d30/d3b/fb6 [0,4194304] 0 2024-04-12T05:55:35.980 INFO:tasks.workunit.client.0.smithi008.stdout:2/643: write d6/db/d6f/f9b [1104353,11981] 0 2024-04-12T05:55:35.981 INFO:tasks.workunit.client.0.smithi008.stdout:2/644: write d6/db/d10/d4c/f82 [1397004,60792] 0 2024-04-12T05:55:35.989 INFO:tasks.workunit.client.0.smithi008.stdout:2/645: truncate d6/db/f89 2726292 0 2024-04-12T05:55:35.991 INFO:tasks.workunit.client.0.smithi008.stdout:2/646: unlink d6/db/d10/d23/d35/d3c/c54 0 2024-04-12T05:55:35.993 INFO:tasks.workunit.client.0.smithi008.stdout:2/647: unlink d6/f74 0 2024-04-12T05:55:35.995 INFO:tasks.workunit.client.0.smithi008.stdout:2/648: unlink d6/lc 0 2024-04-12T05:55:35.997 INFO:tasks.workunit.client.0.smithi008.stdout:2/649: creat d6/db/d6f/da6/fd7 x:0 0 0 2024-04-12T05:55:35.998 INFO:tasks.workunit.client.0.smithi008.stdout:2/650: write d6/db/d10/d4c/fa2 [492106,77658] 0 2024-04-12T05:55:35.999 INFO:tasks.workunit.client.0.smithi008.stdout:2/651: write d6/db/f69 [3329382,67430] 0 2024-04-12T05:55:36.007 INFO:tasks.workunit.client.0.smithi008.stdout:2/652: write d6/db/d29/f48 [5034943,54999] 0 2024-04-12T05:55:36.010 INFO:tasks.workunit.client.0.smithi008.stdout:2/653: creat d6/db/d29/dc2/fd8 x:0 0 0 2024-04-12T05:55:36.011 INFO:tasks.workunit.client.0.smithi008.stdout:2/654: read - d6/db/d38/d41/fa5 zero size 2024-04-12T05:55:36.012 INFO:tasks.workunit.client.0.smithi008.stdout:2/655: read - d6/db/d6f/da6/fc3 zero size 2024-04-12T05:55:36.014 INFO:tasks.workunit.client.0.smithi008.stdout:2/656: mkdir d6/db/d60/d57/dd9 0 2024-04-12T05:55:36.016 INFO:tasks.workunit.client.0.smithi008.stdout:2/657: truncate d6/db/d38/f25 1583382 0 2024-04-12T05:55:36.019 INFO:tasks.workunit.client.0.smithi008.stdout:2/658: link d6/db/d38/f7e d6/db/d10/fda 0 2024-04-12T05:55:36.024 INFO:tasks.workunit.client.0.smithi008.stdout:2/659: dread d6/db/d6f/f9b [0,4194304] 0 2024-04-12T05:55:36.025 INFO:tasks.workunit.client.0.smithi008.stdout:2/660: write d6/db/d10/d4c/f82 [1528850,73462] 0 2024-04-12T05:55:36.026 INFO:tasks.workunit.client.0.smithi008.stdout:2/661: fdatasync d6/db/d10/d4c/fd1 0 2024-04-12T05:55:36.028 INFO:tasks.workunit.client.0.smithi008.stdout:2/662: truncate d6/db/d29/dc2/fd8 215995 0 2024-04-12T05:55:36.029 INFO:tasks.workunit.client.0.smithi008.stdout:2/663: stat d6/db/d10/d23/f5c 0 2024-04-12T05:55:36.029 INFO:tasks.workunit.client.0.smithi008.stdout:2/664: chown d6/db/d10/d23/c3d 202 1 2024-04-12T05:55:36.030 INFO:tasks.workunit.client.0.smithi008.stdout:2/665: truncate d6/db/d29/f6c 1467056 0 2024-04-12T05:55:36.031 INFO:tasks.workunit.client.0.smithi008.stdout:3/611: sync 2024-04-12T05:55:36.031 INFO:tasks.workunit.client.0.smithi008.stdout:6/615: sync 2024-04-12T05:55:36.032 INFO:tasks.workunit.client.0.smithi008.stdout:2/666: mkdir d6/db/d60/d84/ddb 0 2024-04-12T05:55:36.035 INFO:tasks.workunit.client.0.smithi008.stdout:3/612: write d9/dd/f2f [3488595,36827] 0 2024-04-12T05:55:36.037 INFO:tasks.workunit.client.0.smithi008.stdout:3/613: rmdir d9/d37 39 2024-04-12T05:55:36.040 INFO:tasks.workunit.client.0.smithi008.stdout:2/667: rename d6/db/d10/d23/d35/d3c/db5/dc8 to d6/db/d10/d7b/ddc 0 2024-04-12T05:55:36.041 INFO:tasks.workunit.client.0.smithi008.stdout:2/668: write d6/db/d10/d4c/fa2 [17971,102137] 0 2024-04-12T05:55:36.041 INFO:tasks.workunit.client.0.smithi008.stdout:3/614: mkdir d9/d10/d19/d3a/dad/dcf 0 2024-04-12T05:55:36.043 INFO:tasks.workunit.client.0.smithi008.stdout:2/669: write d6/db/f9e [529414,82725] 0 2024-04-12T05:55:36.050 INFO:tasks.workunit.client.0.smithi008.stdout:6/616: truncate df/d21/d4c/f52 477721 0 2024-04-12T05:55:36.055 INFO:tasks.workunit.client.0.smithi008.stdout:3/615: dwrite d9/d37/f89 [0,4194304] 0 2024-04-12T05:55:36.058 INFO:tasks.workunit.client.0.smithi008.stdout:6/617: chown df/d10/d20/c76 18 1 2024-04-12T05:55:36.059 INFO:tasks.workunit.client.0.smithi008.stdout:2/670: dwrite d6/f15 [0,4194304] 0 2024-04-12T05:55:36.061 INFO:tasks.workunit.client.0.smithi008.stdout:3/616: unlink d9/d10/c9e 0 2024-04-12T05:55:36.066 INFO:tasks.workunit.client.0.smithi008.stdout:2/671: creat d6/db/d10/d23/d35/fdd x:0 0 0 2024-04-12T05:55:36.067 INFO:tasks.workunit.client.0.smithi008.stdout:6/618: mkdir df/d10/d15/dd0 0 2024-04-12T05:55:36.070 INFO:tasks.workunit.client.0.smithi008.stdout:6/619: stat df/d10/d15/d4f/d50/f5b 0 2024-04-12T05:55:36.071 INFO:tasks.workunit.client.0.smithi008.stdout:2/672: truncate d6/db/d38/d41/fad 616159 0 2024-04-12T05:55:36.072 INFO:tasks.workunit.client.0.smithi008.stdout:3/617: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:36.072 INFO:tasks.workunit.client.0.smithi008.stdout:2/673: readlink d6/lb9 0 2024-04-12T05:55:36.076 INFO:tasks.workunit.client.0.smithi008.stdout:2/674: creat d6/db/d10/d4c/fde x:0 0 0 2024-04-12T05:55:36.079 INFO:tasks.workunit.client.0.smithi008.stdout:2/675: link d6/l1a d6/db/d10/d23/ldf 0 2024-04-12T05:55:36.081 INFO:tasks.workunit.client.0.smithi008.stdout:3/618: dwrite d9/d2e/f3f [4194304,4194304] 0 2024-04-12T05:55:36.085 INFO:tasks.workunit.client.0.smithi008.stdout:2/676: creat d6/db/d60/d57/dd9/fe0 x:0 0 0 2024-04-12T05:55:36.091 INFO:tasks.workunit.client.0.smithi008.stdout:2/677: creat d6/db/d38/fe1 x:0 0 0 2024-04-12T05:55:36.092 INFO:tasks.workunit.client.0.smithi008.stdout:2/678: write d6/db/d10/d23/d35/d3c/f8f [5398568,59118] 0 2024-04-12T05:55:36.094 INFO:tasks.workunit.client.0.smithi008.stdout:6/620: dread df/d21/d4c/f52 [0,4194304] 0 2024-04-12T05:55:36.095 INFO:tasks.workunit.client.0.smithi008.stdout:2/679: mkdir d6/db/d6f/de2 0 2024-04-12T05:55:36.096 INFO:tasks.workunit.client.0.smithi008.stdout:2/680: chown d6/db/d6f 4964 1 2024-04-12T05:55:36.097 INFO:tasks.workunit.client.0.smithi008.stdout:6/621: rmdir df/d10/d15/d75 39 2024-04-12T05:55:36.098 INFO:tasks.workunit.client.0.smithi008.stdout:6/622: fdatasync df/d10/d15/d4f/f65 0 2024-04-12T05:55:36.099 INFO:tasks.workunit.client.0.smithi008.stdout:6/623: stat df/c26 0 2024-04-12T05:55:36.100 INFO:tasks.workunit.client.0.smithi008.stdout:3/619: dread d9/d10/d19/d22/d2c/f5e [0,4194304] 0 2024-04-12T05:55:36.101 INFO:tasks.workunit.client.0.smithi008.stdout:3/620: rmdir d9/d10/d19 39 2024-04-12T05:55:36.104 INFO:tasks.workunit.client.0.smithi008.stdout:6/624: rename df/d10/d15/d4f/d50 to df/d10/d15/dd0/dd1 0 2024-04-12T05:55:36.104 INFO:tasks.workunit.client.0.smithi008.stdout:2/681: dread d6/db/d10/f80 [0,4194304] 0 2024-04-12T05:55:36.107 INFO:tasks.workunit.client.0.smithi008.stdout:6/625: symlink df/d8c/da4/ld2 0 2024-04-12T05:55:36.111 INFO:tasks.workunit.client.0.smithi008.stdout:3/621: rename d9/d1e/f57 to d9/d10/d19/fd0 0 2024-04-12T05:55:36.113 INFO:tasks.workunit.client.0.smithi008.stdout:2/682: rename d6/db/d38/l19 to d6/db/d10/d4c/le3 0 2024-04-12T05:55:36.115 INFO:tasks.workunit.client.0.smithi008.stdout:6/626: dwrite df/d10/d15/f9d [0,4194304] 0 2024-04-12T05:55:36.118 INFO:tasks.workunit.client.0.smithi008.stdout:2/683: creat d6/db/d60/d84/fe4 x:0 0 0 2024-04-12T05:55:36.120 INFO:tasks.workunit.client.0.smithi008.stdout:2/684: dread - d6/db/d10/d4c/fd1 zero size 2024-04-12T05:55:36.122 INFO:tasks.workunit.client.0.smithi008.stdout:2/685: dread - d6/db/d38/fe1 zero size 2024-04-12T05:55:36.123 INFO:tasks.workunit.client.0.smithi008.stdout:6/627: dread df/f2e [0,4194304] 0 2024-04-12T05:55:36.123 INFO:tasks.workunit.client.0.smithi008.stdout:3/622: dwrite d9/d10/d19/d65/f90 [0,4194304] 0 2024-04-12T05:55:36.124 INFO:tasks.workunit.client.0.smithi008.stdout:2/686: readlink d6/db/d60/d84/l94 0 2024-04-12T05:55:36.125 INFO:tasks.workunit.client.0.smithi008.stdout:6/628: write df/d10/d3a/f7f [271639,16272] 0 2024-04-12T05:55:36.126 INFO:tasks.workunit.client.0.smithi008.stdout:3/623: symlink d9/d11/ld1 0 2024-04-12T05:55:36.127 INFO:tasks.workunit.client.0.smithi008.stdout:2/687: mknod d6/db/d6f/da6/ce5 0 2024-04-12T05:55:36.128 INFO:tasks.workunit.client.0.smithi008.stdout:3/624: dread - d9/f92 zero size 2024-04-12T05:55:36.130 INFO:tasks.workunit.client.0.smithi008.stdout:2/688: chown d6/db/d10/d4c 58971930 1 2024-04-12T05:55:36.131 INFO:tasks.workunit.client.0.smithi008.stdout:3/625: symlink d9/d2e/d3d/ld2 0 2024-04-12T05:55:36.136 INFO:tasks.workunit.client.0.smithi008.stdout:3/626: rename d9/d1e/c4a to d9/dd/cd3 0 2024-04-12T05:55:36.138 INFO:tasks.workunit.client.0.smithi008.stdout:2/689: dwrite d6/db/f4b [0,4194304] 0 2024-04-12T05:55:36.139 INFO:tasks.workunit.client.0.smithi008.stdout:3/627: chown d9/d10/d19/d22/da8/fb2 22 1 2024-04-12T05:55:36.140 INFO:tasks.workunit.client.0.smithi008.stdout:2/690: chown d6/db/d29/dc2 228 1 2024-04-12T05:55:36.140 INFO:tasks.workunit.client.0.smithi008.stdout:3/628: chown d9/d1e/lc8 1 1 2024-04-12T05:55:36.145 INFO:tasks.workunit.client.0.smithi008.stdout:2/691: getdents d6/db/d10/d7b 0 2024-04-12T05:55:36.147 INFO:tasks.workunit.client.0.smithi008.stdout:6/629: dwrite f8 [0,4194304] 0 2024-04-12T05:55:36.149 INFO:tasks.workunit.client.0.smithi008.stdout:3/629: dread d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:36.150 INFO:tasks.workunit.client.0.smithi008.stdout:2/692: creat d6/db/d10/db2/fe6 x:0 0 0 2024-04-12T05:55:36.152 INFO:tasks.workunit.client.0.smithi008.stdout:6/630: mkdir df/d10/d15/d75/dd3 0 2024-04-12T05:55:36.152 INFO:tasks.workunit.client.0.smithi008.stdout:2/693: write d6/db/d29/f6c [1910503,30894] 0 2024-04-12T05:55:36.152 INFO:tasks.workunit.client.0.smithi008.stdout:6/631: stat df/d10/d15/fbb 0 2024-04-12T05:55:36.153 INFO:tasks.workunit.client.0.smithi008.stdout:2/694: chown d6/db/dae 4285924 1 2024-04-12T05:55:36.154 INFO:tasks.workunit.client.0.smithi008.stdout:6/632: write df/d8c/f9a [694180,66629] 0 2024-04-12T05:55:36.155 INFO:tasks.workunit.client.0.smithi008.stdout:2/695: read - d6/db/d60/d57/dd9/fe0 zero size 2024-04-12T05:55:36.162 INFO:tasks.workunit.client.0.smithi008.stdout:6/633: dwrite df/d21/d4c/f6b [0,4194304] 0 2024-04-12T05:55:36.166 INFO:tasks.workunit.client.0.smithi008.stdout:6/634: truncate df/d10/d20/f8a 1022799 0 2024-04-12T05:55:36.167 INFO:tasks.workunit.client.0.smithi008.stdout:6/635: readlink df/d10/d15/d79/lac 0 2024-04-12T05:55:36.169 INFO:tasks.workunit.client.0.smithi008.stdout:6/636: symlink df/d10/ld4 0 2024-04-12T05:55:36.172 INFO:tasks.workunit.client.0.smithi008.stdout:6/637: rename df/d21/d4c/f80 to df/d8c/da4/dad/fd5 0 2024-04-12T05:55:36.182 INFO:tasks.workunit.client.0.smithi008.stdout:6/638: write df/d21/f43 [861752,117409] 0 2024-04-12T05:55:36.185 INFO:tasks.workunit.client.0.smithi008.stdout:4/531: sync 2024-04-12T05:55:36.185 INFO:tasks.workunit.client.0.smithi008.stdout:7/627: sync 2024-04-12T05:55:36.186 INFO:tasks.workunit.client.0.smithi008.stdout:2/696: dread d6/db/d38/f63 [0,4194304] 0 2024-04-12T05:55:36.189 INFO:tasks.workunit.client.0.smithi008.stdout:7/628: mkdir d1/d15/d43/ddb 0 2024-04-12T05:55:36.190 INFO:tasks.workunit.client.0.smithi008.stdout:4/532: truncate d1/d22/d29/f7c 758285 0 2024-04-12T05:55:36.193 INFO:tasks.workunit.client.0.smithi008.stdout:6/639: dwrite df/d10/fb5 [0,4194304] 0 2024-04-12T05:55:36.194 INFO:tasks.workunit.client.0.smithi008.stdout:4/533: write d1/d22/d29/d30/f88 [943427,111790] 0 2024-04-12T05:55:36.194 INFO:tasks.workunit.client.0.smithi008.stdout:2/697: dread d6/db/d38/f25 [0,4194304] 0 2024-04-12T05:55:36.195 INFO:tasks.workunit.client.0.smithi008.stdout:6/640: write f4 [2109291,102282] 0 2024-04-12T05:55:36.197 INFO:tasks.workunit.client.0.smithi008.stdout:2/698: symlink d6/db/d10/d4c/le7 0 2024-04-12T05:55:36.198 INFO:tasks.workunit.client.0.smithi008.stdout:4/534: mknod d1/d22/d29/d30/d60/c99 0 2024-04-12T05:55:36.200 INFO:tasks.workunit.client.0.smithi008.stdout:6/641: mkdir df/d10/d15/d75/dd6 0 2024-04-12T05:55:36.201 INFO:tasks.workunit.client.0.smithi008.stdout:2/699: symlink d6/db/d38/d41/le8 0 2024-04-12T05:55:36.203 INFO:tasks.workunit.client.0.smithi008.stdout:4/535: unlink d1/d22/d29/d46/c6d 0 2024-04-12T05:55:36.204 INFO:tasks.workunit.client.0.smithi008.stdout:4/536: truncate d1/d8/f25 2498988 0 2024-04-12T05:55:36.205 INFO:tasks.workunit.client.0.smithi008.stdout:6/642: symlink df/d10/d20/d3e/ld7 0 2024-04-12T05:55:36.209 INFO:tasks.workunit.client.0.smithi008.stdout:4/537: write d1/d22/d29/d30/f90 [1088709,107928] 0 2024-04-12T05:55:36.211 INFO:tasks.workunit.client.0.smithi008.stdout:6/643: rmdir df/d10/d15/d79 39 2024-04-12T05:55:36.211 INFO:tasks.workunit.client.0.smithi008.stdout:6/644: read - df/d21/f77 zero size 2024-04-12T05:55:36.212 INFO:tasks.workunit.client.0.smithi008.stdout:2/700: dread d6/db/fb8 [0,4194304] 0 2024-04-12T05:55:36.214 INFO:tasks.workunit.client.0.smithi008.stdout:6/645: symlink df/d21/ld8 0 2024-04-12T05:55:36.216 INFO:tasks.workunit.client.0.smithi008.stdout:2/701: creat d6/db/d60/d57/dd9/fe9 x:0 0 0 2024-04-12T05:55:36.216 INFO:tasks.workunit.client.0.smithi008.stdout:6/646: write df/f31 [5014863,108089] 0 2024-04-12T05:55:36.221 INFO:tasks.workunit.client.0.smithi008.stdout:2/702: mkdir d6/db/d6f/dea 0 2024-04-12T05:55:36.223 INFO:tasks.workunit.client.0.smithi008.stdout:2/703: readlink d6/db/d60/l45 0 2024-04-12T05:55:36.224 INFO:tasks.workunit.client.0.smithi008.stdout:4/538: dread d1/d8/f23 [0,4194304] 0 2024-04-12T05:55:36.225 INFO:tasks.workunit.client.0.smithi008.stdout:6/647: dwrite f4 [0,4194304] 0 2024-04-12T05:55:36.226 INFO:tasks.workunit.client.0.smithi008.stdout:6/648: dread - df/d21/fb0 zero size 2024-04-12T05:55:36.227 INFO:tasks.workunit.client.0.smithi008.stdout:2/704: mknod d6/db/d10/d23/d35/ceb 0 2024-04-12T05:55:36.229 INFO:tasks.workunit.client.0.smithi008.stdout:6/649: fdatasync df/d10/d15/dd0/dd1/f51 0 2024-04-12T05:55:36.229 INFO:tasks.workunit.client.0.smithi008.stdout:4/539: truncate d1/d22/d29/d30/d47/d28/f3f 6068085 0 2024-04-12T05:55:36.231 INFO:tasks.workunit.client.0.smithi008.stdout:2/705: rmdir d6/db/d38/d41 39 2024-04-12T05:55:36.234 INFO:tasks.workunit.client.0.smithi008.stdout:6/650: mknod df/d21/cd9 0 2024-04-12T05:55:36.235 INFO:tasks.workunit.client.0.smithi008.stdout:4/540: read d1/d22/d29/d30/d47/d28/f73 [2344609,4498] 0 2024-04-12T05:55:36.236 INFO:tasks.workunit.client.0.smithi008.stdout:2/706: mkdir d6/db/d10/d23/d35/d3c/dec 0 2024-04-12T05:55:36.239 INFO:tasks.workunit.client.0.smithi008.stdout:4/541: mknod d1/d4f/c9a 0 2024-04-12T05:55:36.240 INFO:tasks.workunit.client.0.smithi008.stdout:4/542: write d1/d22/d29/d30/f91 [248639,46551] 0 2024-04-12T05:55:36.242 INFO:tasks.workunit.client.0.smithi008.stdout:2/707: rename d6/db/d10/d23/d35/d3c/f70 to d6/db/d6f/fed 0 2024-04-12T05:55:36.243 INFO:tasks.workunit.client.0.smithi008.stdout:4/543: write d1/d4f/f5b [5093227,50309] 0 2024-04-12T05:55:36.246 INFO:tasks.workunit.client.0.smithi008.stdout:6/651: dread df/d10/d3a/f7f [0,4194304] 0 2024-04-12T05:55:36.247 INFO:tasks.workunit.client.0.smithi008.stdout:2/708: unlink d6/db/lbd 0 2024-04-12T05:55:36.249 INFO:tasks.workunit.client.0.smithi008.stdout:6/652: write df/d10/d15/d4f/f65 [4472275,8107] 0 2024-04-12T05:55:36.251 INFO:tasks.workunit.client.0.smithi008.stdout:2/709: creat d6/db/d38/dcf/fee x:0 0 0 2024-04-12T05:55:36.253 INFO:tasks.workunit.client.0.smithi008.stdout:2/710: readlink d6/db/d10/d7b/d7d/la0 0 2024-04-12T05:55:36.255 INFO:tasks.workunit.client.0.smithi008.stdout:6/653: creat df/d10/d15/fda x:0 0 0 2024-04-12T05:55:36.256 INFO:tasks.workunit.client.0.smithi008.stdout:2/711: truncate d6/db/d60/d57/dd9/fe0 959196 0 2024-04-12T05:55:36.259 INFO:tasks.workunit.client.0.smithi008.stdout:2/712: creat d6/db/d60/d84/fef x:0 0 0 2024-04-12T05:55:36.261 INFO:tasks.workunit.client.0.smithi008.stdout:6/654: rename df/d10/d15/fbb to df/d10/d3a/d78/d8b/fdb 0 2024-04-12T05:55:36.263 INFO:tasks.workunit.client.0.smithi008.stdout:4/544: dwrite d1/f36 [0,4194304] 0 2024-04-12T05:55:36.265 INFO:tasks.workunit.client.0.smithi008.stdout:4/545: truncate d1/d22/d29/d46/d56/f80 1037833 0 2024-04-12T05:55:36.269 INFO:tasks.workunit.client.0.smithi008.stdout:6/655: rename df/d10/d3a to df/d8c/db1/ddc 0 2024-04-12T05:55:36.272 INFO:tasks.workunit.client.0.smithi008.stdout:6/656: mkdir df/d8c/dcc/ddd 0 2024-04-12T05:55:36.296 INFO:tasks.workunit.client.0.smithi008.stdout:6/657: dread df/d10/f62 [0,4194304] 0 2024-04-12T05:55:36.298 INFO:tasks.workunit.client.0.smithi008.stdout:6/658: write df/d10/d20/d3e/db4/fc8 [224640,101864] 0 2024-04-12T05:55:36.399 INFO:tasks.workunit.client.0.smithi008.stdout:9/645: sync 2024-04-12T05:55:36.399 INFO:tasks.workunit.client.0.smithi008.stdout:1/562: sync 2024-04-12T05:55:36.400 INFO:tasks.workunit.client.0.smithi008.stdout:9/646: chown d3/d22/d2b/d77/d82/f8a 128995 1 2024-04-12T05:55:36.401 INFO:tasks.workunit.client.0.smithi008.stdout:9/647: fdatasync d3/df/d13/d21/d2d/d30/d74/f83 0 2024-04-12T05:55:36.413 INFO:tasks.workunit.client.0.smithi008.stdout:9/648: dread d3/f43 [4194304,4194304] 0 2024-04-12T05:55:36.429 INFO:tasks.workunit.client.0.smithi008.stdout:1/563: unlink d4/d1c/lc6 0 2024-04-12T05:55:36.431 INFO:tasks.workunit.client.0.smithi008.stdout:9/649: fdatasync d3/df/d10/f17 0 2024-04-12T05:55:36.433 INFO:tasks.workunit.client.0.smithi008.stdout:9/650: write d3/df/d13/d21/d2d/fa6 [1888939,87882] 0 2024-04-12T05:55:36.434 INFO:tasks.workunit.client.0.smithi008.stdout:1/564: mknod d4/d33/cca 0 2024-04-12T05:55:36.439 INFO:tasks.workunit.client.0.smithi008.stdout:1/565: creat d4/d1a/d57/fcb x:0 0 0 2024-04-12T05:55:36.441 INFO:tasks.workunit.client.0.smithi008.stdout:9/651: dread d3/df/d13/d21/d2d/d32/f7d [0,4194304] 0 2024-04-12T05:55:36.441 INFO:tasks.workunit.client.0.smithi008.stdout:1/566: symlink d4/d24/lcc 0 2024-04-12T05:55:36.443 INFO:tasks.workunit.client.0.smithi008.stdout:1/567: write d4/d33/fa5 [4457767,59264] 0 2024-04-12T05:55:36.448 INFO:tasks.workunit.client.0.smithi008.stdout:9/652: creat d3/d22/d2b/fe0 x:0 0 0 2024-04-12T05:55:36.449 INFO:tasks.workunit.client.0.smithi008.stdout:1/568: link d4/d6/f53 d4/d7/fcd 0 2024-04-12T05:55:36.450 INFO:tasks.workunit.client.0.smithi008.stdout:1/569: readlink d4/dd/d17/d66/l77 0 2024-04-12T05:55:36.455 INFO:tasks.workunit.client.0.smithi008.stdout:9/653: dread d3/df/d13/d21/f76 [0,4194304] 0 2024-04-12T05:55:36.459 INFO:tasks.workunit.client.0.smithi008.stdout:1/570: dwrite d4/dd/d17/d91/fbe [0,4194304] 0 2024-04-12T05:55:36.460 INFO:tasks.workunit.client.0.smithi008.stdout:9/654: mknod d3/df/d13/d21/d2d/d32/ce1 0 2024-04-12T05:55:36.461 INFO:tasks.workunit.client.0.smithi008.stdout:1/571: dread - d4/dd/d21/d9f/dac/fb0 zero size 2024-04-12T05:55:36.464 INFO:tasks.workunit.client.0.smithi008.stdout:9/655: truncate d3/d22/d2b/d77/f9c 410238 0 2024-04-12T05:55:36.465 INFO:tasks.workunit.client.0.smithi008.stdout:1/572: mknod d4/d33/d74/d7a/db6/cce 0 2024-04-12T05:55:36.469 INFO:tasks.workunit.client.0.smithi008.stdout:9/656: mkdir d3/df/d13/d21/d2d/d30/de2 0 2024-04-12T05:55:36.479 INFO:tasks.workunit.client.0.smithi008.stdout:9/657: write d3/d22/d2b/d77/f9c [1168477,1501] 0 2024-04-12T05:55:36.480 INFO:tasks.workunit.client.0.smithi008.stdout:1/573: dread d4/d6/d11/f42 [0,4194304] 0 2024-04-12T05:55:36.482 INFO:tasks.workunit.client.0.smithi008.stdout:9/658: dread - d3/df/d13/d21/d2d/d32/d52/f9f zero size 2024-04-12T05:55:36.482 INFO:tasks.workunit.client.0.smithi008.stdout:1/574: read d4/dd/d17/d26/f35 [456622,100208] 0 2024-04-12T05:55:36.484 INFO:tasks.workunit.client.0.smithi008.stdout:1/575: mknod d4/d33/d74/d30/ccf 0 2024-04-12T05:55:36.486 INFO:tasks.workunit.client.0.smithi008.stdout:9/659: creat d3/df/d10/fe3 x:0 0 0 2024-04-12T05:55:36.487 INFO:tasks.workunit.client.0.smithi008.stdout:1/576: symlink d4/dd/d21/d98/ld0 0 2024-04-12T05:55:36.487 INFO:tasks.workunit.client.0.smithi008.stdout:9/660: stat d3/df/d13/d21/d2d/d30/d74 0 2024-04-12T05:55:36.488 INFO:tasks.workunit.client.0.smithi008.stdout:1/577: write d4/dd/d17/f64 [103184,9676] 0 2024-04-12T05:55:36.489 INFO:tasks.workunit.client.0.smithi008.stdout:9/661: readlink d3/d22/d2b/d58/d38/lca 0 2024-04-12T05:55:36.490 INFO:tasks.workunit.client.0.smithi008.stdout:1/578: write d4/d1a/f9c [708457,92909] 0 2024-04-12T05:55:36.490 INFO:tasks.workunit.client.0.smithi008.stdout:9/662: fdatasync d3/df/d13/d21/d2d/f8e 0 2024-04-12T05:55:36.493 INFO:tasks.workunit.client.0.smithi008.stdout:1/579: symlink d4/dd/d17/d26/ld1 0 2024-04-12T05:55:36.495 INFO:tasks.workunit.client.0.smithi008.stdout:9/663: rename d3/d22/d2b/f41 to d3/d22/d2b/d4a/dc8/fe4 0 2024-04-12T05:55:36.499 INFO:tasks.workunit.client.0.smithi008.stdout:9/664: mknod d3/dae/dba/ce5 0 2024-04-12T05:55:36.501 INFO:tasks.workunit.client.0.smithi008.stdout:1/580: dwrite d4/d33/d74/d30/d72/f96 [0,4194304] 0 2024-04-12T05:55:36.504 INFO:tasks.workunit.client.0.smithi008.stdout:9/665: mknod d3/d22/d2b/d77/d82/ce6 0 2024-04-12T05:55:36.506 INFO:tasks.workunit.client.0.smithi008.stdout:1/581: rmdir d4/d33/d74/d30/d72/d7c 39 2024-04-12T05:55:36.506 INFO:tasks.workunit.client.0.smithi008.stdout:1/582: stat d4/dd/f9a 0 2024-04-12T05:55:36.507 INFO:tasks.workunit.client.0.smithi008.stdout:1/583: dread - d4/d1a/d57/fcb zero size 2024-04-12T05:55:36.508 INFO:tasks.workunit.client.0.smithi008.stdout:9/666: write d3/d22/d2b/d58/da8/fa9 [1455745,80692] 0 2024-04-12T05:55:36.509 INFO:tasks.workunit.client.0.smithi008.stdout:9/667: chown d3/d22/d2b/d58/d89 1 1 2024-04-12T05:55:36.510 INFO:tasks.workunit.client.0.smithi008.stdout:8/611: sync 2024-04-12T05:55:36.510 INFO:tasks.workunit.client.0.smithi008.stdout:0/565: sync 2024-04-12T05:55:36.510 INFO:tasks.workunit.client.0.smithi008.stdout:5/955: sync 2024-04-12T05:55:36.511 INFO:tasks.workunit.client.0.smithi008.stdout:9/668: rmdir d3/d22/d2b/d4a/dc8 39 2024-04-12T05:55:36.514 INFO:tasks.workunit.client.0.smithi008.stdout:5/956: fdatasync d0/dd/d11/d38/d55/d89/f122 0 2024-04-12T05:55:36.514 INFO:tasks.workunit.client.0.smithi008.stdout:9/669: fsync d3/df/d13/d21/d2d/d75/f94 0 2024-04-12T05:55:36.516 INFO:tasks.workunit.client.0.smithi008.stdout:0/566: creat d2/d24/fab x:0 0 0 2024-04-12T05:55:36.517 INFO:tasks.workunit.client.0.smithi008.stdout:0/567: chown d2 72510 1 2024-04-12T05:55:36.518 INFO:tasks.workunit.client.0.smithi008.stdout:8/612: dread dd/d30/f35 [0,4194304] 0 2024-04-12T05:55:36.519 INFO:tasks.workunit.client.0.smithi008.stdout:9/670: truncate d3/f35 884382 0 2024-04-12T05:55:36.521 INFO:tasks.workunit.client.0.smithi008.stdout:1/584: dwrite d4/d6/d90/fa7 [0,4194304] 0 2024-04-12T05:55:36.521 INFO:tasks.workunit.client.0.smithi008.stdout:0/568: fdatasync d2/d11/d2a/f37 0 2024-04-12T05:55:36.523 INFO:tasks.workunit.client.0.smithi008.stdout:0/569: chown d2 0 1 2024-04-12T05:55:36.523 INFO:tasks.workunit.client.0.smithi008.stdout:8/613: fdatasync dd/de/d16/d50/d56/f5d 0 2024-04-12T05:55:36.523 INFO:tasks.workunit.client.0.smithi008.stdout:9/671: chown d3/d22/d2b/d4e/f6b 56482 1 2024-04-12T05:55:36.529 INFO:tasks.workunit.client.0.smithi008.stdout:0/570: mknod d2/d11/d36/cac 0 2024-04-12T05:55:36.529 INFO:tasks.workunit.client.0.smithi008.stdout:9/672: read d3/df/d13/d21/f91 [651621,45745] 0 2024-04-12T05:55:36.530 INFO:tasks.workunit.client.0.smithi008.stdout:0/571: chown d2/c3f 12717408 1 2024-04-12T05:55:36.531 INFO:tasks.workunit.client.0.smithi008.stdout:8/614: write dd/de/d4c/d66/fa2 [350408,27761] 0 2024-04-12T05:55:36.535 INFO:tasks.workunit.client.0.smithi008.stdout:1/585: dwrite d4/d6/d11/f22 [0,4194304] 0 2024-04-12T05:55:36.539 INFO:tasks.workunit.client.0.smithi008.stdout:0/572: link d2/d11/d2a/f5d d2/d24/d61/fad 0 2024-04-12T05:55:36.539 INFO:tasks.workunit.client.0.smithi008.stdout:0/573: chown d2/f5 171173 1 2024-04-12T05:55:36.540 INFO:tasks.workunit.client.0.smithi008.stdout:1/586: unlink d4/d24/f45 0 2024-04-12T05:55:36.543 INFO:tasks.workunit.client.0.smithi008.stdout:1/587: read - d4/dd/d17/d66/fa8 zero size 2024-04-12T05:55:36.545 INFO:tasks.workunit.client.0.smithi008.stdout:9/673: rename d3/d22/l24 to d3/df/d13/d14/le7 0 2024-04-12T05:55:36.547 INFO:tasks.workunit.client.0.smithi008.stdout:1/588: creat d4/d33/d37/fd2 x:0 0 0 2024-04-12T05:55:36.548 INFO:tasks.workunit.client.0.smithi008.stdout:0/574: dwrite d2/d11/d36/d47/f94 [0,4194304] 0 2024-04-12T05:55:36.552 INFO:tasks.workunit.client.0.smithi008.stdout:9/674: write d3/d22/d2b/d77/d82/f87 [1345981,30804] 0 2024-04-12T05:55:36.555 INFO:tasks.workunit.client.0.smithi008.stdout:1/589: getdents d4/dd/d21 0 2024-04-12T05:55:36.556 INFO:tasks.workunit.client.0.smithi008.stdout:9/675: mknod d3/d22/d2b/d4e/d60/ce8 0 2024-04-12T05:55:36.558 INFO:tasks.workunit.client.0.smithi008.stdout:1/590: read - d4/dd/d17/d26/f76 zero size 2024-04-12T05:55:36.560 INFO:tasks.workunit.client.0.smithi008.stdout:1/591: write d4/d6/d90/fa7 [690142,34400] 0 2024-04-12T05:55:36.563 INFO:tasks.workunit.client.0.smithi008.stdout:1/592: creat d4/d6/d90/fd3 x:0 0 0 2024-04-12T05:55:36.564 INFO:tasks.workunit.client.0.smithi008.stdout:1/593: write d4/d33/d74/d7a/f92 [149869,87172] 0 2024-04-12T05:55:36.566 INFO:tasks.workunit.client.0.smithi008.stdout:1/594: chown d4/dd/d17/d26/f76 267766606 1 2024-04-12T05:55:36.567 INFO:tasks.workunit.client.0.smithi008.stdout:9/676: dread d3/d22/d2b/d58/da8/fa9 [0,4194304] 0 2024-04-12T05:55:36.571 INFO:tasks.workunit.client.0.smithi008.stdout:1/595: unlink d4/d6/l4b 0 2024-04-12T05:55:36.574 INFO:tasks.workunit.client.0.smithi008.stdout:9/677: getdents d3/df/d13/d21/d5c 0 2024-04-12T05:55:36.576 INFO:tasks.workunit.client.0.smithi008.stdout:9/678: mkdir d3/df/d13/d21/d2d/d30/d74/de9 0 2024-04-12T05:55:36.582 INFO:tasks.workunit.client.0.smithi008.stdout:9/679: rename d3/d22/d2b/d77/fbd to d3/d22/d2b/fea 0 2024-04-12T05:55:36.583 INFO:tasks.workunit.client.0.smithi008.stdout:1/596: dwrite d4/d1a/f51 [0,4194304] 0 2024-04-12T05:55:36.583 INFO:tasks.workunit.client.0.smithi008.stdout:9/680: fdatasync d3/df/f11 0 2024-04-12T05:55:36.592 INFO:tasks.workunit.client.0.smithi008.stdout:1/597: mkdir d4/d33/d74/d30/d72/d7c/dd4 0 2024-04-12T05:55:36.595 INFO:tasks.workunit.client.0.smithi008.stdout:1/598: fsync d4/dd/d17/d91/fbe 0 2024-04-12T05:55:36.601 INFO:tasks.workunit.client.0.smithi008.stdout:9/681: dwrite d3/d22/d2b/d4e/d60/f6d [0,4194304] 0 2024-04-12T05:55:36.604 INFO:tasks.workunit.client.0.smithi008.stdout:1/599: mkdir d4/d33/d74/d30/d72/d7c/dd4/dd5 0 2024-04-12T05:55:36.605 INFO:tasks.workunit.client.0.smithi008.stdout:9/682: chown d3/fa 155 1 2024-04-12T05:55:36.609 INFO:tasks.workunit.client.0.smithi008.stdout:1/600: write d4/d33/d74/d30/d72/d7c/f8c [162519,59092] 0 2024-04-12T05:55:36.611 INFO:tasks.workunit.client.0.smithi008.stdout:9/683: creat d3/df/d13/d21/d5c/d8f/feb x:0 0 0 2024-04-12T05:55:36.615 INFO:tasks.workunit.client.0.smithi008.stdout:9/684: fsync d3/d22/d2b/d4e/d60/f6d 0 2024-04-12T05:55:36.615 INFO:tasks.workunit.client.0.smithi008.stdout:1/601: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:36.618 INFO:tasks.workunit.client.0.smithi008.stdout:1/602: chown d4/d1a/la0 209 1 2024-04-12T05:55:36.622 INFO:tasks.workunit.client.0.smithi008.stdout:1/603: rename d4/d7/d18 to d4/d1a/d57/dd6 0 2024-04-12T05:55:36.624 INFO:tasks.workunit.client.0.smithi008.stdout:9/685: mknod d3/df/d10/cec 0 2024-04-12T05:55:36.625 INFO:tasks.workunit.client.0.smithi008.stdout:1/604: truncate d4/dd/d17/d26/f9b 988244 0 2024-04-12T05:55:36.625 INFO:tasks.workunit.client.0.smithi008.stdout:1/605: chown d4/d33/d37 1128521 1 2024-04-12T05:55:36.626 INFO:tasks.workunit.client.0.smithi008.stdout:9/686: mkdir d3/d22/d2b/d58/d90/ded 0 2024-04-12T05:55:36.627 INFO:tasks.workunit.client.0.smithi008.stdout:1/606: truncate d4/d33/d74/d30/d72/d7c/f80 922773 0 2024-04-12T05:55:36.629 INFO:tasks.workunit.client.0.smithi008.stdout:9/687: read d3/df/d13/d21/d2d/d30/f4d [648995,14273] 0 2024-04-12T05:55:36.629 INFO:tasks.workunit.client.0.smithi008.stdout:1/607: write d4/d1c/f8f [1399903,93420] 0 2024-04-12T05:55:36.637 INFO:tasks.workunit.client.0.smithi008.stdout:1/608: dwrite d4/d33/d74/d46/f86 [0,4194304] 0 2024-04-12T05:55:36.638 INFO:tasks.workunit.client.0.smithi008.stdout:1/609: dread - d4/dd/d17/d26/fa9 zero size 2024-04-12T05:55:36.639 INFO:tasks.workunit.client.0.smithi008.stdout:1/610: stat d4/f47 0 2024-04-12T05:55:36.646 INFO:tasks.workunit.client.0.smithi008.stdout:9/688: truncate d3/d22/d2b/d58/f55 1586637 0 2024-04-12T05:55:36.647 INFO:tasks.workunit.client.0.smithi008.stdout:1/611: write d4/f47 [308050,10532] 0 2024-04-12T05:55:36.649 INFO:tasks.workunit.client.0.smithi008.stdout:1/612: truncate d4/dd/f9a 998798 0 2024-04-12T05:55:36.657 INFO:tasks.workunit.client.0.smithi008.stdout:1/613: getdents d4/d24 0 2024-04-12T05:55:36.661 INFO:tasks.workunit.client.0.smithi008.stdout:1/614: rename d4/l16 to d4/d33/d74/ld7 0 2024-04-12T05:55:36.663 INFO:tasks.workunit.client.0.smithi008.stdout:1/615: chown d4/dd/d17/f63 6497301 1 2024-04-12T05:55:36.669 INFO:tasks.workunit.client.0.smithi008.stdout:1/616: dwrite d4/d33/fa5 [0,4194304] 0 2024-04-12T05:55:36.671 INFO:tasks.workunit.client.0.smithi008.stdout:1/617: stat d4/d1c/f8f 0 2024-04-12T05:55:36.671 INFO:tasks.workunit.client.0.smithi008.stdout:9/689: dread d3/df/d13/d21/d2d/f8e [0,4194304] 0 2024-04-12T05:55:36.674 INFO:tasks.workunit.client.0.smithi008.stdout:9/690: mknod d3/df/d13/d21/d5c/dd7/cee 0 2024-04-12T05:55:36.679 INFO:tasks.workunit.client.0.smithi008.stdout:1/618: dwrite d4/d33/d74/d7a/fb4 [0,4194304] 0 2024-04-12T05:55:36.681 INFO:tasks.workunit.client.0.smithi008.stdout:9/691: truncate d3/df/d13/d21/d2d/f8e 1974388 0 2024-04-12T05:55:36.684 INFO:tasks.workunit.client.0.smithi008.stdout:1/619: dread d4/d33/d74/d46/f86 [0,4194304] 0 2024-04-12T05:55:36.686 INFO:tasks.workunit.client.0.smithi008.stdout:9/692: rename d3/df/d13/d21/d2d/d32/ld8 to d3/d22/d2b/d58/d90/db1/lef 0 2024-04-12T05:55:36.687 INFO:tasks.workunit.client.0.smithi008.stdout:9/693: truncate d3/df/d13/d21/fdd 1019438 0 2024-04-12T05:55:36.689 INFO:tasks.workunit.client.0.smithi008.stdout:1/620: rename d4/d7/fcd to d4/d1c/fd8 0 2024-04-12T05:55:36.689 INFO:tasks.workunit.client.0.smithi008.stdout:9/694: getdents d3/df/d13/d21/d2d/d30/de2 0 2024-04-12T05:55:36.690 INFO:tasks.workunit.client.0.smithi008.stdout:1/621: chown d4/dd/f99 0 1 2024-04-12T05:55:36.693 INFO:tasks.workunit.client.0.smithi008.stdout:1/622: creat d4/d33/fd9 x:0 0 0 2024-04-12T05:55:36.695 INFO:tasks.workunit.client.0.smithi008.stdout:1/623: creat d4/d33/d74/d7a/db6/dc9/fda x:0 0 0 2024-04-12T05:55:36.706 INFO:tasks.workunit.client.0.smithi008.stdout:3/630: sync 2024-04-12T05:55:36.708 INFO:tasks.workunit.client.0.smithi008.stdout:3/631: unlink d9/d10/d19/d22/f5f 0 2024-04-12T05:55:36.709 INFO:tasks.workunit.client.0.smithi008.stdout:3/632: chown d9/d10/d19/d22/d2c/faf 245078985 1 2024-04-12T05:55:36.710 INFO:tasks.workunit.client.0.smithi008.stdout:3/633: mknod d9/d11/cd4 0 2024-04-12T05:55:36.711 INFO:tasks.workunit.client.0.smithi008.stdout:3/634: rmdir d9/d44 39 2024-04-12T05:55:36.728 INFO:tasks.workunit.client.0.smithi008.stdout:1/624: truncate d4/d6/d11/f22 605586 0 2024-04-12T05:55:36.729 INFO:tasks.workunit.client.0.smithi008.stdout:9/695: write d3/d22/d2b/d4a/dc8/fe4 [794596,94804] 0 2024-04-12T05:55:36.730 INFO:tasks.workunit.client.0.smithi008.stdout:3/635: write f1 [3645692,9614] 0 2024-04-12T05:55:36.732 INFO:tasks.workunit.client.0.smithi008.stdout:1/625: mkdir d4/d6/d11/ddb 0 2024-04-12T05:55:36.733 INFO:tasks.workunit.client.0.smithi008.stdout:9/696: mkdir d3/df0 0 2024-04-12T05:55:36.733 INFO:tasks.workunit.client.0.smithi008.stdout:3/636: rmdir d9/d37/d69 39 2024-04-12T05:55:36.733 INFO:tasks.workunit.client.0.smithi008.stdout:1/626: chown d4/d33/d74/d7a/fb4 1010350183 1 2024-04-12T05:55:36.737 INFO:tasks.workunit.client.0.smithi008.stdout:1/627: unlink d4/d33/fa5 0 2024-04-12T05:55:36.739 INFO:tasks.workunit.client.0.smithi008.stdout:9/697: truncate d3/df/d13/d21/d2d/f8e 1479433 0 2024-04-12T05:55:36.739 INFO:tasks.workunit.client.0.smithi008.stdout:3/637: symlink d9/d10/d19/d3a/dad/dcf/ld5 0 2024-04-12T05:55:36.741 INFO:tasks.workunit.client.0.smithi008.stdout:9/698: write d3/df/d13/d21/d2d/d30/d74/f83 [235405,43074] 0 2024-04-12T05:55:36.743 INFO:tasks.workunit.client.0.smithi008.stdout:1/628: rename d4/d33/d74/d30/d72/la3 to d4/d1c/ldc 0 2024-04-12T05:55:36.743 INFO:tasks.workunit.client.0.smithi008.stdout:3/638: mkdir d9/d10/d19/d3a/dad/dcf/dd6 0 2024-04-12T05:55:36.745 INFO:tasks.workunit.client.0.smithi008.stdout:3/639: fsync d9/d2e/d3d/f6b 0 2024-04-12T05:55:36.747 INFO:tasks.workunit.client.0.smithi008.stdout:3/640: fdatasync d9/d44/f9f 0 2024-04-12T05:55:36.748 INFO:tasks.workunit.client.0.smithi008.stdout:9/699: getdents d3/dae/dba 0 2024-04-12T05:55:36.749 INFO:tasks.workunit.client.0.smithi008.stdout:3/641: truncate d9/d10/d19/fd0 4950668 0 2024-04-12T05:55:36.750 INFO:tasks.workunit.client.0.smithi008.stdout:3/642: write d9/d11/d5b/f60 [841712,74655] 0 2024-04-12T05:55:36.755 INFO:tasks.workunit.client.0.smithi008.stdout:3/643: rename d9/d44/fc3 to d9/d10/d19/d22/da8/fd7 0 2024-04-12T05:55:36.759 INFO:tasks.workunit.client.0.smithi008.stdout:1/629: write d4/d33/f71 [946113,34172] 0 2024-04-12T05:55:36.760 INFO:tasks.workunit.client.0.smithi008.stdout:3/644: mkdir d9/d37/dd8 0 2024-04-12T05:55:36.761 INFO:tasks.workunit.client.0.smithi008.stdout:1/630: chown d4/d1c/fd8 1031821 1 2024-04-12T05:55:36.762 INFO:tasks.workunit.client.0.smithi008.stdout:1/631: fdatasync d4/d6/f88 0 2024-04-12T05:55:36.768 INFO:tasks.workunit.client.0.smithi008.stdout:3/645: dread d9/d10/d19/d22/d2c/f5e [0,4194304] 0 2024-04-12T05:55:36.770 INFO:tasks.workunit.client.0.smithi008.stdout:9/700: dwrite d3/d22/d2b/d4e/f6b [0,4194304] 0 2024-04-12T05:55:36.777 INFO:tasks.workunit.client.0.smithi008.stdout:3/646: dread d9/d2e/d3d/f6b [0,4194304] 0 2024-04-12T05:55:36.778 INFO:tasks.workunit.client.0.smithi008.stdout:9/701: creat d3/df/d13/d21/d2d/d30/de2/ff1 x:0 0 0 2024-04-12T05:55:36.779 INFO:tasks.workunit.client.0.smithi008.stdout:1/632: dwrite d4/d6/f53 [0,4194304] 0 2024-04-12T05:55:36.786 INFO:tasks.workunit.client.0.smithi008.stdout:3/647: mknod d9/d10/d19/d3a/d67/cd9 0 2024-04-12T05:55:36.786 INFO:tasks.workunit.client.0.smithi008.stdout:9/702: mkdir d3/df/d13/d21/d2d/d30/d74/de9/df2 0 2024-04-12T05:55:36.787 INFO:tasks.workunit.client.0.smithi008.stdout:1/633: link d4/d7/f56 d4/d33/d74/d7a/fdd 0 2024-04-12T05:55:36.789 INFO:tasks.workunit.client.0.smithi008.stdout:3/648: write d9/d10/d19/d22/d2c/faf [901154,89292] 0 2024-04-12T05:55:36.793 INFO:tasks.workunit.client.0.smithi008.stdout:1/634: fdatasync d4/dd/d17/d26/f36 0 2024-04-12T05:55:36.794 INFO:tasks.workunit.client.0.smithi008.stdout:3/649: unlink d9/d2e/d3d/ld2 0 2024-04-12T05:55:36.795 INFO:tasks.workunit.client.0.smithi008.stdout:1/635: fsync d4/dd/d17/d66/fa8 0 2024-04-12T05:55:36.798 INFO:tasks.workunit.client.0.smithi008.stdout:9/703: dwrite d3/d22/d2b/d58/da8/db6/fdc [0,4194304] 0 2024-04-12T05:55:36.803 INFO:tasks.workunit.client.0.smithi008.stdout:1/636: getdents d4/d33/d74/d7a 0 2024-04-12T05:55:36.803 INFO:tasks.workunit.client.0.smithi008.stdout:3/650: write d9/db7/fc1 [869636,5319] 0 2024-04-12T05:55:36.806 INFO:tasks.workunit.client.0.smithi008.stdout:3/651: truncate d9/d10/d19/d3a/dad/fc9 60986 0 2024-04-12T05:55:36.806 INFO:tasks.workunit.client.0.smithi008.stdout:1/637: creat d4/d33/d74/da2/fde x:0 0 0 2024-04-12T05:55:36.807 INFO:tasks.workunit.client.0.smithi008.stdout:9/704: truncate d3/df/d13/d21/d2d/d30/f4d 687543 0 2024-04-12T05:55:36.810 INFO:tasks.workunit.client.0.smithi008.stdout:1/638: creat d4/dd/d17/d26/fdf x:0 0 0 2024-04-12T05:55:36.812 INFO:tasks.workunit.client.0.smithi008.stdout:1/639: rmdir d4/d1a/d57/dd6 39 2024-04-12T05:55:36.813 INFO:tasks.workunit.client.0.smithi008.stdout:1/640: fdatasync d4/dd/d17/d26/fa9 0 2024-04-12T05:55:36.815 INFO:tasks.workunit.client.0.smithi008.stdout:9/705: write d3/f35 [142127,106008] 0 2024-04-12T05:55:36.818 INFO:tasks.workunit.client.0.smithi008.stdout:1/641: rename d4/dd/d21/d9f/fa6 to d4/d33/d74/d46/d4c/fe0 0 2024-04-12T05:55:36.819 INFO:tasks.workunit.client.0.smithi008.stdout:9/706: symlink d3/df/d13/d21/d5c/d8f/lf3 0 2024-04-12T05:55:36.820 INFO:tasks.workunit.client.0.smithi008.stdout:1/642: stat d4/d33/d74/d7a/db6/cce 0 2024-04-12T05:55:36.820 INFO:tasks.workunit.client.0.smithi008.stdout:9/707: fdatasync d3/d22/d2b/fe0 0 2024-04-12T05:55:36.823 INFO:tasks.workunit.client.0.smithi008.stdout:9/708: mknod d3/df/d13/d21/d5c/dd7/cf4 0 2024-04-12T05:55:36.825 INFO:tasks.workunit.client.0.smithi008.stdout:1/643: dread d4/dd/f39 [0,4194304] 0 2024-04-12T05:55:36.825 INFO:tasks.workunit.client.0.smithi008.stdout:9/709: read d3/fc5 [2605355,102691] 0 2024-04-12T05:55:36.827 INFO:tasks.workunit.client.0.smithi008.stdout:9/710: fdatasync d3/df/f86 0 2024-04-12T05:55:36.833 INFO:tasks.workunit.client.0.smithi008.stdout:1/644: dwrite d4/d1a/d57/fcb [0,4194304] 0 2024-04-12T05:55:36.845 INFO:tasks.workunit.client.0.smithi008.stdout:1/645: dread d4/d7/fc [0,4194304] 0 2024-04-12T05:55:36.850 INFO:tasks.workunit.client.0.smithi008.stdout:1/646: truncate d4/d6/d90/f8d 118125 0 2024-04-12T05:55:36.855 INFO:tasks.workunit.client.0.smithi008.stdout:9/711: dread d3/df/d13/d21/d2d/d32/f34 [0,4194304] 0 2024-04-12T05:55:36.856 INFO:tasks.workunit.client.0.smithi008.stdout:1/647: dwrite d4/d6/f59 [0,4194304] 0 2024-04-12T05:55:36.857 INFO:tasks.workunit.client.0.smithi008.stdout:9/712: dread - d3/d22/d2b/d58/d5d/fcd zero size 2024-04-12T05:55:36.868 INFO:tasks.workunit.client.0.smithi008.stdout:9/713: dwrite d3/d22/d2b/d58/fdb [0,4194304] 0 2024-04-12T05:55:36.870 INFO:tasks.workunit.client.0.smithi008.stdout:9/714: stat d3/df/d13/d21/d2d/d30/faa 0 2024-04-12T05:55:36.872 INFO:tasks.workunit.client.0.smithi008.stdout:9/715: chown d3/df/d13/d21/d2d/ca1 57432 1 2024-04-12T05:55:36.894 INFO:tasks.workunit.client.0.smithi008.stdout:9/716: dread d3/d22/d2b/d58/d38/f54 [0,4194304] 0 2024-04-12T05:55:36.896 INFO:tasks.workunit.client.0.smithi008.stdout:9/717: symlink d3/d22/d2b/d58/d90/ded/lf5 0 2024-04-12T05:55:36.899 INFO:tasks.workunit.client.0.smithi008.stdout:9/718: unlink d3/df/d13/d21/d2d/dbb/f9a 0 2024-04-12T05:55:36.901 INFO:tasks.workunit.client.0.smithi008.stdout:9/719: rename d3/df/f20 to d3/d22/d2b/d77/ff6 0 2024-04-12T05:55:36.903 INFO:tasks.workunit.client.0.smithi008.stdout:9/720: write d3/d22/d2b/d58/da8/db6/fc0 [832863,32883] 0 2024-04-12T05:55:36.905 INFO:tasks.workunit.client.0.smithi008.stdout:9/721: creat d3/d22/d2b/d58/d89/ff7 x:0 0 0 2024-04-12T05:55:36.908 INFO:tasks.workunit.client.0.smithi008.stdout:2/713: sync 2024-04-12T05:55:36.908 INFO:tasks.workunit.client.0.smithi008.stdout:6/659: sync 2024-04-12T05:55:36.908 INFO:tasks.workunit.client.0.smithi008.stdout:4/546: sync 2024-04-12T05:55:36.908 INFO:tasks.workunit.client.0.smithi008.stdout:7/629: sync 2024-04-12T05:55:36.911 INFO:tasks.workunit.client.0.smithi008.stdout:6/660: symlink df/d10/d15/d75/lde 0 2024-04-12T05:55:36.916 INFO:tasks.workunit.client.0.smithi008.stdout:6/661: dread - df/d8c/db1/ddc/d95/fa7 zero size 2024-04-12T05:55:36.917 INFO:tasks.workunit.client.0.smithi008.stdout:2/714: rename d6/db/d60/d57/c64 to d6/db/d10/d23/d35/d3c/db5/cf0 0 2024-04-12T05:55:36.918 INFO:tasks.workunit.client.0.smithi008.stdout:9/722: dread d3/d22/d2b/d58/f27 [0,4194304] 0 2024-04-12T05:55:36.922 INFO:tasks.workunit.client.0.smithi008.stdout:4/547: dwrite d1/d22/d29/d30/d47/f48 [4194304,4194304] 0 2024-04-12T05:55:36.923 INFO:tasks.workunit.client.0.smithi008.stdout:6/662: creat df/d10/d15/dd0/dd1/fdf x:0 0 0 2024-04-12T05:55:36.924 INFO:tasks.workunit.client.0.smithi008.stdout:4/548: fsync d1/d22/f8b 0 2024-04-12T05:55:36.926 INFO:tasks.workunit.client.0.smithi008.stdout:2/715: dread d6/db/d10/d23/f5c [0,4194304] 0 2024-04-12T05:55:36.927 INFO:tasks.workunit.client.0.smithi008.stdout:7/630: dwrite d1/d5/da/d3c/d44/f51 [12582912,4194304] 0 2024-04-12T05:55:36.928 INFO:tasks.workunit.client.0.smithi008.stdout:6/663: write df/d21/d4c/fbe [245743,35677] 0 2024-04-12T05:55:36.929 INFO:tasks.workunit.client.0.smithi008.stdout:9/723: truncate d3/df/d13/d21/d2d/fa6 2669389 0 2024-04-12T05:55:36.932 INFO:tasks.workunit.client.0.smithi008.stdout:9/724: chown d3/df/d13 390290 1 2024-04-12T05:55:36.934 INFO:tasks.workunit.client.0.smithi008.stdout:2/716: mknod d6/db/d60/d57/cf1 0 2024-04-12T05:55:36.938 INFO:tasks.workunit.client.0.smithi008.stdout:6/664: link df/d10/d15/f9d df/d10/d15/d79/fe0 0 2024-04-12T05:55:36.941 INFO:tasks.workunit.client.0.smithi008.stdout:9/725: link d3/df/d13/d21/d5c/d8f/dab/fc3 d3/df/d13/d21/d2d/d30/d74/ff8 0 2024-04-12T05:55:36.942 INFO:tasks.workunit.client.0.smithi008.stdout:2/717: rmdir d6/db/d60/d84/ddb 0 2024-04-12T05:55:36.943 INFO:tasks.workunit.client.0.smithi008.stdout:4/549: dwrite d1/d22/d29/f7c [0,4194304] 0 2024-04-12T05:55:36.943 INFO:tasks.workunit.client.0.smithi008.stdout:6/665: mknod df/d8c/da4/ce1 0 2024-04-12T05:55:36.944 INFO:tasks.workunit.client.0.smithi008.stdout:9/726: write d3/df/d13/d21/fdd [1087228,78791] 0 2024-04-12T05:55:36.948 INFO:tasks.workunit.client.0.smithi008.stdout:6/666: dread df/d10/d20/fae [0,4194304] 0 2024-04-12T05:55:36.949 INFO:tasks.workunit.client.0.smithi008.stdout:4/550: creat d1/d8/f9b x:0 0 0 2024-04-12T05:55:36.951 INFO:tasks.workunit.client.0.smithi008.stdout:2/718: rename d6/db/d6f/cb6 to d6/db/d6f/dea/cf2 0 2024-04-12T05:55:36.957 INFO:tasks.workunit.client.0.smithi008.stdout:9/727: dread d3/d22/d2b/d58/fdb [0,4194304] 0 2024-04-12T05:55:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:4/551: creat d1/d22/d29/d30/d47/d28/d74/f9c x:0 0 0 2024-04-12T05:55:36.961 INFO:tasks.workunit.client.0.smithi008.stdout:9/728: mkdir d3/d22/d2b/d58/d90/df9 0 2024-04-12T05:55:36.962 INFO:tasks.workunit.client.0.smithi008.stdout:4/552: chown d1/d22/d29/d30/f52 6851 1 2024-04-12T05:55:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:9/729: stat d3/df/d13/d21/d2d/d30/d74/f83 0 2024-04-12T05:55:36.963 INFO:tasks.workunit.client.0.smithi008.stdout:2/719: creat d6/db/d10/d23/d35/ff3 x:0 0 0 2024-04-12T05:55:36.964 INFO:tasks.workunit.client.0.smithi008.stdout:9/730: stat d3/df/d13/d14/d95 0 2024-04-12T05:55:36.965 INFO:tasks.workunit.client.0.smithi008.stdout:2/720: dread - d6/db/d10/d23/fa4 zero size 2024-04-12T05:55:36.967 INFO:tasks.workunit.client.0.smithi008.stdout:9/731: readlink d3/df/d13/d21/d2d/d32/d52/l57 0 2024-04-12T05:55:36.969 INFO:tasks.workunit.client.0.smithi008.stdout:4/553: getdents d1/db 0 2024-04-12T05:55:36.971 INFO:tasks.workunit.client.0.smithi008.stdout:9/732: mkdir d3/d22/d2b/d58/d90/df9/dfa 0 2024-04-12T05:55:36.975 INFO:tasks.workunit.client.0.smithi008.stdout:2/721: dwrite d6/db/d10/d7b/fb4 [0,4194304] 0 2024-04-12T05:55:36.977 INFO:tasks.workunit.client.0.smithi008.stdout:2/722: chown d6/db/d10/d23/l81 20 1 2024-04-12T05:55:36.980 INFO:tasks.workunit.client.0.smithi008.stdout:9/733: rmdir d3 39 2024-04-12T05:55:36.981 INFO:tasks.workunit.client.0.smithi008.stdout:4/554: getdents d1 0 2024-04-12T05:55:36.982 INFO:tasks.workunit.client.0.smithi008.stdout:2/723: mknod d6/cf4 0 2024-04-12T05:55:36.984 INFO:tasks.workunit.client.0.smithi008.stdout:4/555: write d1/d22/d29/f94 [1185848,62942] 0 2024-04-12T05:55:36.985 INFO:tasks.workunit.client.0.smithi008.stdout:2/724: write d6/db/f22 [4625970,79016] 0 2024-04-12T05:55:36.986 INFO:tasks.workunit.client.0.smithi008.stdout:2/725: chown d6/db/d6f/l95 0 1 2024-04-12T05:55:36.988 INFO:tasks.workunit.client.0.smithi008.stdout:9/734: mkdir d3/df/d13/d21/d2d/d32/dfb 0 2024-04-12T05:55:36.992 INFO:tasks.workunit.client.0.smithi008.stdout:2/726: getdents d6/db/d10 0 2024-04-12T05:55:36.993 INFO:tasks.workunit.client.0.smithi008.stdout:4/556: dwrite d1/d8/f25 [0,4194304] 0 2024-04-12T05:55:36.994 INFO:tasks.workunit.client.0.smithi008.stdout:9/735: write d3/df/d13/d21/d2d/fa6 [1347229,107220] 0 2024-04-12T05:55:36.996 INFO:tasks.workunit.client.0.smithi008.stdout:4/557: write d1/d4f/f5b [1078726,110738] 0 2024-04-12T05:55:36.997 INFO:tasks.workunit.client.0.smithi008.stdout:2/727: symlink d6/db/d10/dcc/lf5 0 2024-04-12T05:55:37.000 INFO:tasks.workunit.client.0.smithi008.stdout:2/728: symlink d6/db/d60/d4f/dbf/lf6 0 2024-04-12T05:55:37.002 INFO:tasks.workunit.client.0.smithi008.stdout:4/558: creat d1/d22/d29/d30/d47/d28/d4a/d62/f9d x:0 0 0 2024-04-12T05:55:37.003 INFO:tasks.workunit.client.0.smithi008.stdout:9/736: getdents d3/df/d13/d21/d5c/dd7 0 2024-04-12T05:55:37.005 INFO:tasks.workunit.client.0.smithi008.stdout:4/559: fdatasync d1/d22/d29/d30/d47/d28/f5c 0 2024-04-12T05:55:37.007 INFO:tasks.workunit.client.0.smithi008.stdout:2/729: rename d6/db/f22 to d6/db/d38/d41/ff7 0 2024-04-12T05:55:37.009 INFO:tasks.workunit.client.0.smithi008.stdout:4/560: unlink d1/d8/l8e 0 2024-04-12T05:55:37.010 INFO:tasks.workunit.client.0.smithi008.stdout:9/737: mkdir d3/df/d13/d21/d2d/d32/d52/db0/dfc 0 2024-04-12T05:55:37.012 INFO:tasks.workunit.client.0.smithi008.stdout:2/730: rename d6/db/d60/d57/c8d to d6/db/d10/db2/cf8 0 2024-04-12T05:55:37.013 INFO:tasks.workunit.client.0.smithi008.stdout:2/731: read - d6/db/d6f/da6/fc3 zero size 2024-04-12T05:55:37.013 INFO:tasks.workunit.client.0.smithi008.stdout:9/738: creat d3/d22/d2b/d4a/dc8/ffd x:0 0 0 2024-04-12T05:55:37.015 INFO:tasks.workunit.client.0.smithi008.stdout:2/732: chown d6/db/d10/d4c/le3 0 1 2024-04-12T05:55:37.016 INFO:tasks.workunit.client.0.smithi008.stdout:4/561: getdents d1/d4f 0 2024-04-12T05:55:37.018 INFO:tasks.workunit.client.0.smithi008.stdout:9/739: symlink d3/df/d13/d21/d2d/lfe 0 2024-04-12T05:55:37.020 INFO:tasks.workunit.client.0.smithi008.stdout:2/733: truncate d6/db/d10/d23/d35/ff3 977562 0 2024-04-12T05:55:37.021 INFO:tasks.workunit.client.0.smithi008.stdout:9/740: chown d3/df/d13/d21 10642819 1 2024-04-12T05:55:37.026 INFO:tasks.workunit.client.0.smithi008.stdout:9/741: stat d3/df/d13/d21/d2d/d32/ld0 0 2024-04-12T05:55:37.026 INFO:tasks.workunit.client.0.smithi008.stdout:4/562: dwrite d1/d22/f97 [0,4194304] 0 2024-04-12T05:55:37.030 INFO:tasks.workunit.client.0.smithi008.stdout:9/742: write d3/f6 [7315329,10098] 0 2024-04-12T05:55:37.031 INFO:tasks.workunit.client.0.smithi008.stdout:4/563: creat d1/d8/f9e x:0 0 0 2024-04-12T05:55:37.033 INFO:tasks.workunit.client.0.smithi008.stdout:4/564: stat d1/d8/f23 0 2024-04-12T05:55:37.033 INFO:tasks.workunit.client.0.smithi008.stdout:9/743: write d3/d6a/f9e [1038431,124222] 0 2024-04-12T05:55:37.033 INFO:tasks.workunit.client.0.smithi008.stdout:2/734: dwrite d6/db/d60/d84/fe4 [0,4194304] 0 2024-04-12T05:55:37.036 INFO:tasks.workunit.client.0.smithi008.stdout:2/735: read - d6/db/d38/d41/fc6 zero size 2024-04-12T05:55:37.043 INFO:tasks.workunit.client.0.smithi008.stdout:4/565: dwrite d1/d8/f84 [0,4194304] 0 2024-04-12T05:55:37.046 INFO:tasks.workunit.client.0.smithi008.stdout:2/736: link d6/db/d38/f75 d6/ff9 0 2024-04-12T05:55:37.046 INFO:tasks.workunit.client.0.smithi008.stdout:4/566: dread - d1/d22/d29/d46/f76 zero size 2024-04-12T05:55:37.048 INFO:tasks.workunit.client.0.smithi008.stdout:4/567: read - d1/d22/d29/d46/d56/f92 zero size 2024-04-12T05:55:37.050 INFO:tasks.workunit.client.0.smithi008.stdout:2/737: rename d6/db/d60/d57/l86 to d6/db/d38/dcf/lfa 0 2024-04-12T05:55:37.051 INFO:tasks.workunit.client.0.smithi008.stdout:2/738: write d6/db/f69 [1403497,81918] 0 2024-04-12T05:55:37.053 INFO:tasks.workunit.client.0.smithi008.stdout:4/568: creat d1/d4f/f9f x:0 0 0 2024-04-12T05:55:37.056 INFO:tasks.workunit.client.0.smithi008.stdout:9/744: write d3/df/f86 [1811639,67891] 0 2024-04-12T05:55:37.057 INFO:tasks.workunit.client.0.smithi008.stdout:4/569: fdatasync d1/d22/d29/d46/d56/f92 0 2024-04-12T05:55:37.059 INFO:tasks.workunit.client.0.smithi008.stdout:2/739: creat d6/db/ffb x:0 0 0 2024-04-12T05:55:37.059 INFO:tasks.workunit.client.0.smithi008.stdout:4/570: write d1/d8/f15 [2508934,19843] 0 2024-04-12T05:55:37.060 INFO:tasks.workunit.client.0.smithi008.stdout:2/740: fsync d6/db/d60/d57/f76 0 2024-04-12T05:55:37.061 INFO:tasks.workunit.client.0.smithi008.stdout:9/745: truncate d3/df/d13/d21/d2d/d30/d74/ff8 1004844 0 2024-04-12T05:55:37.063 INFO:tasks.workunit.client.0.smithi008.stdout:9/746: fsync d3/f25 0 2024-04-12T05:55:37.065 INFO:tasks.workunit.client.0.smithi008.stdout:2/741: mkdir d6/db/d10/d23/d35/dfc 0 2024-04-12T05:55:37.066 INFO:tasks.workunit.client.0.smithi008.stdout:2/742: chown d6/db/d10/d23/d35/fdd 2008 1 2024-04-12T05:55:37.068 INFO:tasks.workunit.client.0.smithi008.stdout:2/743: chown d6/db/d6f/da6/fd7 2707911 1 2024-04-12T05:55:37.069 INFO:tasks.workunit.client.0.smithi008.stdout:2/744: write f0 [3648077,129426] 0 2024-04-12T05:55:37.069 INFO:tasks.workunit.client.0.smithi008.stdout:9/747: creat d3/df/d13/fff x:0 0 0 2024-04-12T05:55:37.078 INFO:tasks.workunit.client.0.smithi008.stdout:9/748: dwrite d3/df/d13/d21/d2d/d30/d74/f83 [0,4194304] 0 2024-04-12T05:55:37.080 INFO:tasks.workunit.client.0.smithi008.stdout:2/745: dwrite d6/db/d38/d41/ff7 [4194304,4194304] 0 2024-04-12T05:55:37.082 INFO:tasks.workunit.client.0.smithi008.stdout:9/749: mknod d3/d22/d2b/d77/c100 0 2024-04-12T05:55:37.083 INFO:tasks.workunit.client.0.smithi008.stdout:2/746: write d6/db/d10/d7b/d7d/f98 [4680253,49370] 0 2024-04-12T05:55:37.084 INFO:tasks.workunit.client.0.smithi008.stdout:9/750: stat d3/df/d13/d14/d95/cc2 0 2024-04-12T05:55:37.085 INFO:tasks.workunit.client.0.smithi008.stdout:2/747: fdatasync d6/db/d10/d23/d35/d3c/f8f 0 2024-04-12T05:55:37.087 INFO:tasks.workunit.client.0.smithi008.stdout:2/748: chown d6/db/d10/d23/d35/d3c/ca7 393977589 1 2024-04-12T05:55:37.088 INFO:tasks.workunit.client.0.smithi008.stdout:9/751: mknod d3/df/d13/d21/d2d/d75/c101 0 2024-04-12T05:55:37.088 INFO:tasks.workunit.client.0.smithi008.stdout:2/749: write d6/db/d29/f48 [170356,111173] 0 2024-04-12T05:55:37.090 INFO:tasks.workunit.client.0.smithi008.stdout:8/615: sync 2024-04-12T05:55:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:5/957: sync 2024-04-12T05:55:37.091 INFO:tasks.workunit.client.0.smithi008.stdout:0/575: sync 2024-04-12T05:55:37.097 INFO:tasks.workunit.client.0.smithi008.stdout:5/958: write d0/dc/d57/d9d/db5/dbc/f129 [784301,119472] 0 2024-04-12T05:55:37.099 INFO:tasks.workunit.client.0.smithi008.stdout:5/959: fsync d0/d29/f10c 0 2024-04-12T05:55:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:0/576: dread d2/d11/d36/f80 [0,4194304] 0 2024-04-12T05:55:37.100 INFO:tasks.workunit.client.0.smithi008.stdout:9/752: dwrite d3/d22/d2b/d77/d82/fbc [0,4194304] 0 2024-04-12T05:55:37.103 INFO:tasks.workunit.client.0.smithi008.stdout:0/577: read d2/d11/d2a/f7b [158937,39051] 0 2024-04-12T05:55:37.104 INFO:tasks.workunit.client.0.smithi008.stdout:9/753: read - d3/df/d13/fff zero size 2024-04-12T05:55:37.107 INFO:tasks.workunit.client.0.smithi008.stdout:5/960: symlink d0/l12d 0 2024-04-12T05:55:37.109 INFO:tasks.workunit.client.0.smithi008.stdout:5/961: dread - d0/dc/d57/d9d/db5/dbc/f11d zero size 2024-04-12T05:55:37.112 INFO:tasks.workunit.client.0.smithi008.stdout:5/962: mknod d0/dd/dc7/c12e 0 2024-04-12T05:55:37.115 INFO:tasks.workunit.client.0.smithi008.stdout:9/754: symlink d3/df/d13/d21/d2d/l102 0 2024-04-12T05:55:37.118 INFO:tasks.workunit.client.0.smithi008.stdout:0/578: getdents d2/d11/d36 0 2024-04-12T05:55:37.118 INFO:tasks.workunit.client.0.smithi008.stdout:0/579: stat d2/d24/f32 0 2024-04-12T05:55:37.118 INFO:tasks.workunit.client.0.smithi008.stdout:0/580: chown d2/d24/d39 3836 1 2024-04-12T05:55:37.120 INFO:tasks.workunit.client.0.smithi008.stdout:9/755: creat d3/df/d13/d21/d2d/d30/d74/de9/df2/f103 x:0 0 0 2024-04-12T05:55:37.121 INFO:tasks.workunit.client.0.smithi008.stdout:8/616: truncate dd/de/d16/d50/d56/d61/d77/fb8 2402592 0 2024-04-12T05:55:37.122 INFO:tasks.workunit.client.0.smithi008.stdout:5/963: dread d0/dc/d13/d1a/daf/f1b [0,4194304] 0 2024-04-12T05:55:37.125 INFO:tasks.workunit.client.0.smithi008.stdout:9/756: read d3/d22/d2b/d58/da8/db6/fdc [3936991,4236] 0 2024-04-12T05:55:37.126 INFO:tasks.workunit.client.0.smithi008.stdout:8/617: fdatasync dd/d30/d32/f3a 0 2024-04-12T05:55:37.128 INFO:tasks.workunit.client.0.smithi008.stdout:5/964: fsync d0/dd/d11/d38/f59 0 2024-04-12T05:55:37.130 INFO:tasks.workunit.client.0.smithi008.stdout:2/750: dwrite d6/db/d60/d84/f8a [0,4194304] 0 2024-04-12T05:55:37.131 INFO:tasks.workunit.client.0.smithi008.stdout:0/581: dread d2/f66 [0,4194304] 0 2024-04-12T05:55:37.134 INFO:tasks.workunit.client.0.smithi008.stdout:8/618: write dd/de/d16/d50/d56/d61/d77/fd2 [385456,75507] 0 2024-04-12T05:55:37.135 INFO:tasks.workunit.client.0.smithi008.stdout:0/582: readlink d2/l78 0 2024-04-12T05:55:37.137 INFO:tasks.workunit.client.0.smithi008.stdout:2/751: mkdir d6/db/d29/dfd 0 2024-04-12T05:55:37.140 INFO:tasks.workunit.client.0.smithi008.stdout:9/757: creat d3/d22/d2b/d4a/f104 x:0 0 0 2024-04-12T05:55:37.140 INFO:tasks.workunit.client.0.smithi008.stdout:0/583: creat d2/d24/d39/fae x:0 0 0 2024-04-12T05:55:37.142 INFO:tasks.workunit.client.0.smithi008.stdout:9/758: write d3/df/d13/d21/d2d/d75/fd5 [279629,105618] 0 2024-04-12T05:55:37.143 INFO:tasks.workunit.client.0.smithi008.stdout:2/752: unlink d6/db/d10/d23/d35/d3c/c40 0 2024-04-12T05:55:37.145 INFO:tasks.workunit.client.0.smithi008.stdout:2/753: chown d6/db/d10/ld2 595467716 1 2024-04-12T05:55:37.145 INFO:tasks.workunit.client.0.smithi008.stdout:2/754: readlink d6/db/d10/d23/lb7 0 2024-04-12T05:55:37.146 INFO:tasks.workunit.client.0.smithi008.stdout:8/619: creat dd/de/fda x:0 0 0 2024-04-12T05:55:37.148 INFO:tasks.workunit.client.0.smithi008.stdout:0/584: rename d2/d24/d85/f8d to d2/d11/d2a/faf 0 2024-04-12T05:55:37.149 INFO:tasks.workunit.client.0.smithi008.stdout:5/965: truncate d0/dd/fd5 6457174 0 2024-04-12T05:55:37.152 INFO:tasks.workunit.client.0.smithi008.stdout:0/585: rename d2/d24/d61/f89 to d2/d11/fb0 0 2024-04-12T05:55:37.155 INFO:tasks.workunit.client.0.smithi008.stdout:8/620: link dd/de/fda dd/d4f/d52/da5/fdb 0 2024-04-12T05:55:37.156 INFO:tasks.workunit.client.0.smithi008.stdout:8/621: fsync dd/de/d16/d73/fc2 0 2024-04-12T05:55:37.158 INFO:tasks.workunit.client.0.smithi008.stdout:0/586: mknod d2/d11/d36/d82/cb1 0 2024-04-12T05:55:37.160 INFO:tasks.workunit.client.0.smithi008.stdout:8/622: symlink dd/de/d16/db1/d97/dcf/ldc 0 2024-04-12T05:55:37.161 INFO:tasks.workunit.client.0.smithi008.stdout:2/755: write d6/db/d60/f78 [339280,122957] 0 2024-04-12T05:55:37.162 INFO:tasks.workunit.client.0.smithi008.stdout:0/587: rmdir d2/d11/d36/d42/d86 39 2024-04-12T05:55:37.165 INFO:tasks.workunit.client.0.smithi008.stdout:2/756: symlink d6/db/d38/dcf/lfe 0 2024-04-12T05:55:37.166 INFO:tasks.workunit.client.0.smithi008.stdout:8/623: unlink dd/de/d16/db1/cac 0 2024-04-12T05:55:37.169 INFO:tasks.workunit.client.0.smithi008.stdout:0/588: rename d2/d11/fa4 to d2/d11/d36/d47/fb2 0 2024-04-12T05:55:37.174 INFO:tasks.workunit.client.0.smithi008.stdout:0/589: dread d2/d11/d36/d47/f91 [0,4194304] 0 2024-04-12T05:55:37.175 INFO:tasks.workunit.client.0.smithi008.stdout:8/624: mknod dd/d30/cdd 0 2024-04-12T05:55:37.181 INFO:tasks.workunit.client.0.smithi008.stdout:0/590: rename d2/d24/d61/f9f to d2/d11/d36/d82/d95/fb3 0 2024-04-12T05:55:37.182 INFO:tasks.workunit.client.0.smithi008.stdout:0/591: write d2/d24/fa0 [996225,64389] 0 2024-04-12T05:55:37.183 INFO:tasks.workunit.client.0.smithi008.stdout:2/757: dread d6/f43 [0,4194304] 0 2024-04-12T05:55:37.184 INFO:tasks.workunit.client.0.smithi008.stdout:0/592: rmdir d2/d11/d36 39 2024-04-12T05:55:37.185 INFO:tasks.workunit.client.0.smithi008.stdout:8/625: dread dd/d30/d3b/f55 [0,4194304] 0 2024-04-12T05:55:37.187 INFO:tasks.workunit.client.0.smithi008.stdout:2/758: dread d6/db/d38/d41/fad [0,4194304] 0 2024-04-12T05:55:37.188 INFO:tasks.workunit.client.0.smithi008.stdout:2/759: dread - d6/db/d38/d41/fbe zero size 2024-04-12T05:55:37.190 INFO:tasks.workunit.client.0.smithi008.stdout:0/593: creat d2/d11/d36/d82/fb4 x:0 0 0 2024-04-12T05:55:37.192 INFO:tasks.workunit.client.0.smithi008.stdout:8/626: rename dd/d4f/f9a to dd/de/d4c/d66/d6d/fde 0 2024-04-12T05:55:37.194 INFO:tasks.workunit.client.0.smithi008.stdout:2/760: unlink d6/db/d10/d4c/c52 0 2024-04-12T05:55:37.194 INFO:tasks.workunit.client.0.smithi008.stdout:8/627: write dd/de/d16/d50/d5e/f85 [876138,27250] 0 2024-04-12T05:55:37.195 INFO:tasks.workunit.client.0.smithi008.stdout:8/628: chown dd/de/d16/l21 1 1 2024-04-12T05:55:37.196 INFO:tasks.workunit.client.0.smithi008.stdout:2/761: creat d6/db/d29/fff x:0 0 0 2024-04-12T05:55:37.197 INFO:tasks.workunit.client.0.smithi008.stdout:0/594: mkdir d2/d11/d2a/db5 0 2024-04-12T05:55:37.199 INFO:tasks.workunit.client.0.smithi008.stdout:2/762: mknod d6/db/d60/d57/dd9/c100 0 2024-04-12T05:55:37.201 INFO:tasks.workunit.client.0.smithi008.stdout:0/595: creat d2/d11/d36/d82/d95/fb6 x:0 0 0 2024-04-12T05:55:37.202 INFO:tasks.workunit.client.0.smithi008.stdout:2/763: rmdir d6/db/d6f 39 2024-04-12T05:55:37.203 INFO:tasks.workunit.client.0.smithi008.stdout:8/629: dwrite dd/d4f/f7b [0,4194304] 0 2024-04-12T05:55:37.205 INFO:tasks.workunit.client.0.smithi008.stdout:0/596: creat d2/d11/d36/d42/da6/fb7 x:0 0 0 2024-04-12T05:55:37.205 INFO:tasks.workunit.client.0.smithi008.stdout:2/764: write d6/db/d38/d41/f93 [308899,25591] 0 2024-04-12T05:55:37.209 INFO:tasks.workunit.client.0.smithi008.stdout:8/630: unlink dd/de/c34 0 2024-04-12T05:55:37.212 INFO:tasks.workunit.client.0.smithi008.stdout:2/765: symlink d6/db/d10/d7b/l101 0 2024-04-12T05:55:37.215 INFO:tasks.workunit.client.0.smithi008.stdout:0/597: dread d2/d11/f18 [0,4194304] 0 2024-04-12T05:55:37.216 INFO:tasks.workunit.client.0.smithi008.stdout:0/598: dread - d2/d11/d2a/f7e zero size 2024-04-12T05:55:37.217 INFO:tasks.workunit.client.0.smithi008.stdout:8/631: dwrite dd/f36 [0,4194304] 0 2024-04-12T05:55:37.217 INFO:tasks.workunit.client.0.smithi008.stdout:2/766: mkdir d6/db/d60/d57/d102 0 2024-04-12T05:55:37.218 INFO:tasks.workunit.client.0.smithi008.stdout:8/632: readlink dd/d30/d32/l53 0 2024-04-12T05:55:37.220 INFO:tasks.workunit.client.0.smithi008.stdout:8/633: write dd/d30/d32/f7e [706342,120252] 0 2024-04-12T05:55:37.228 INFO:tasks.workunit.client.0.smithi008.stdout:8/634: link l1 dd/de/d16/d50/d56/d61/ldf 0 2024-04-12T05:55:37.229 INFO:tasks.workunit.client.0.smithi008.stdout:0/599: write d2/d24/d39/f60 [442685,60151] 0 2024-04-12T05:55:37.230 INFO:tasks.workunit.client.0.smithi008.stdout:2/767: truncate d6/db/d10/d7b/d7d/f98 3261713 0 2024-04-12T05:55:37.231 INFO:tasks.workunit.client.0.smithi008.stdout:0/600: write d2/d24/f76 [1233521,92061] 0 2024-04-12T05:55:37.233 INFO:tasks.workunit.client.0.smithi008.stdout:8/635: creat dd/de/d16/d50/d56/d61/fe0 x:0 0 0 2024-04-12T05:55:37.234 INFO:tasks.workunit.client.0.smithi008.stdout:0/601: creat d2/d4b/da2/fb8 x:0 0 0 2024-04-12T05:55:37.239 INFO:tasks.workunit.client.0.smithi008.stdout:2/768: dwrite d6/db/fd [0,4194304] 0 2024-04-12T05:55:37.239 INFO:tasks.workunit.client.0.smithi008.stdout:0/602: creat d2/d11/fb9 x:0 0 0 2024-04-12T05:55:37.240 INFO:tasks.workunit.client.0.smithi008.stdout:8/636: getdents dd/de/d4c/d66/d6d 0 2024-04-12T05:55:37.242 INFO:tasks.workunit.client.0.smithi008.stdout:2/769: chown d6/db/d6f/l95 13 1 2024-04-12T05:55:37.246 INFO:tasks.workunit.client.0.smithi008.stdout:0/603: rename d2/d11/d2a/f7b to d2/d11/d36/d82/d95/fba 0 2024-04-12T05:55:37.246 INFO:tasks.workunit.client.0.smithi008.stdout:8/637: read dd/d30/d32/f3a [1264126,96983] 0 2024-04-12T05:55:37.247 INFO:tasks.workunit.client.0.smithi008.stdout:2/770: getdents d6/db/d10/d23/d35/d3c/dec 0 2024-04-12T05:55:37.248 INFO:tasks.workunit.client.0.smithi008.stdout:8/638: fdatasync dd/f4e 0 2024-04-12T05:55:37.252 INFO:tasks.workunit.client.0.smithi008.stdout:2/771: creat d6/db/d10/d23/d35/d3c/dec/f103 x:0 0 0 2024-04-12T05:55:37.254 INFO:tasks.workunit.client.0.smithi008.stdout:0/604: creat d2/d11/d36/d42/fbb x:0 0 0 2024-04-12T05:55:37.255 INFO:tasks.workunit.client.0.smithi008.stdout:2/772: dread - d6/db/d6f/fed zero size 2024-04-12T05:55:37.256 INFO:tasks.workunit.client.0.smithi008.stdout:2/773: readlink l3 0 2024-04-12T05:55:37.258 INFO:tasks.workunit.client.0.smithi008.stdout:2/774: write d6/db/d10/f80 [1707197,121824] 0 2024-04-12T05:55:37.258 INFO:tasks.workunit.client.0.smithi008.stdout:2/775: readlink d6/l92 0 2024-04-12T05:55:37.262 INFO:tasks.workunit.client.0.smithi008.stdout:0/605: write d2/d11/fb0 [4109691,46105] 0 2024-04-12T05:55:37.265 INFO:tasks.workunit.client.0.smithi008.stdout:2/776: dread d6/db/d60/d84/f8a [0,4194304] 0 2024-04-12T05:55:37.266 INFO:tasks.workunit.client.0.smithi008.stdout:2/777: chown d6/db/d60/d4f/lcb 462490118 1 2024-04-12T05:55:37.270 INFO:tasks.workunit.client.0.smithi008.stdout:2/778: truncate d6/db/d10/d4c/fde 805122 0 2024-04-12T05:55:37.271 INFO:tasks.workunit.client.0.smithi008.stdout:0/606: dwrite d2/d4b/da2/fb8 [0,4194304] 0 2024-04-12T05:55:37.272 INFO:tasks.workunit.client.0.smithi008.stdout:8/639: dread dd/d4f/d52/f8d [0,4194304] 0 2024-04-12T05:55:37.275 INFO:tasks.workunit.client.0.smithi008.stdout:0/607: unlink d2/d24/fab 0 2024-04-12T05:55:37.282 INFO:tasks.workunit.client.0.smithi008.stdout:2/779: dwrite d6/db/d60/d57/dd9/fe9 [0,4194304] 0 2024-04-12T05:55:37.285 INFO:tasks.workunit.client.0.smithi008.stdout:2/780: stat d6/db/d10/fda 0 2024-04-12T05:55:37.285 INFO:tasks.workunit.client.0.smithi008.stdout:8/640: dwrite dd/f36 [0,4194304] 0 2024-04-12T05:55:37.289 INFO:tasks.workunit.client.0.smithi008.stdout:2/781: chown d6/db/d60/d4f 927296 1 2024-04-12T05:55:37.290 INFO:tasks.workunit.client.0.smithi008.stdout:3/652: sync 2024-04-12T05:55:37.290 INFO:tasks.workunit.client.0.smithi008.stdout:1/648: sync 2024-04-12T05:55:37.291 INFO:tasks.workunit.client.0.smithi008.stdout:0/608: dwrite d2/d11/d36/f81 [0,4194304] 0 2024-04-12T05:55:37.291 INFO:tasks.workunit.client.0.smithi008.stdout:8/641: dread - dd/de/d16/d50/d56/d61/fe0 zero size 2024-04-12T05:55:37.294 INFO:tasks.workunit.client.0.smithi008.stdout:0/609: fsync d2/d11/d36/d47/f87 0 2024-04-12T05:55:37.295 INFO:tasks.workunit.client.0.smithi008.stdout:3/653: write d9/d10/d19/d22/da8/fb2 [201610,76400] 0 2024-04-12T05:55:37.298 INFO:tasks.workunit.client.0.smithi008.stdout:2/782: rename d6/db/d10/l50 to d6/db/d29/dc2/l104 0 2024-04-12T05:55:37.300 INFO:tasks.workunit.client.0.smithi008.stdout:3/654: mkdir d9/d10/d19/d3a/d67/dda 0 2024-04-12T05:55:37.304 INFO:tasks.workunit.client.0.smithi008.stdout:1/649: rename d4/d1a/d57/dd6/c29 to d4/dd/ce1 0 2024-04-12T05:55:37.309 INFO:tasks.workunit.client.0.smithi008.stdout:0/610: dwrite d2/d11/d36/d47/f6d [0,4194304] 0 2024-04-12T05:55:37.309 INFO:tasks.workunit.client.0.smithi008.stdout:3/655: creat d9/d37/fdb x:0 0 0 2024-04-12T05:55:37.309 INFO:tasks.workunit.client.0.smithi008.stdout:8/642: getdents dd/de 0 2024-04-12T05:55:37.309 INFO:tasks.workunit.client.0.smithi008.stdout:1/650: creat d4/dd/d21/d9f/dac/fe2 x:0 0 0 2024-04-12T05:55:37.312 INFO:tasks.workunit.client.0.smithi008.stdout:3/656: write d9/d44/f6e [832841,12114] 0 2024-04-12T05:55:37.314 INFO:tasks.workunit.client.0.smithi008.stdout:8/643: fdatasync dd/d30/d3b/f55 0 2024-04-12T05:55:37.317 INFO:tasks.workunit.client.0.smithi008.stdout:0/611: read d2/d11/f2e [1375847,111093] 0 2024-04-12T05:55:37.323 INFO:tasks.workunit.client.0.smithi008.stdout:8/644: rename dd/f39 to dd/d30/d3b/fe1 0 2024-04-12T05:55:37.325 INFO:tasks.workunit.client.0.smithi008.stdout:1/651: link d4/dd/d17/d66/fbd d4/dd/d17/d66/fe3 0 2024-04-12T05:55:37.325 INFO:tasks.workunit.client.0.smithi008.stdout:8/645: fdatasync dd/d30/d3b/f9b 0 2024-04-12T05:55:37.326 INFO:tasks.workunit.client.0.smithi008.stdout:3/657: dread d9/d44/fa3 [0,4194304] 0 2024-04-12T05:55:37.328 INFO:tasks.workunit.client.0.smithi008.stdout:0/612: link d2/d11/d36/d42/l7a d2/d24/d85/lbc 0 2024-04-12T05:55:37.330 INFO:tasks.workunit.client.0.smithi008.stdout:0/613: fdatasync d2/fe 0 2024-04-12T05:55:37.332 INFO:tasks.workunit.client.0.smithi008.stdout:3/658: creat d9/d10/d19/d22/fdc x:0 0 0 2024-04-12T05:55:37.334 INFO:tasks.workunit.client.0.smithi008.stdout:0/614: mkdir d2/d24/dbd 0 2024-04-12T05:55:37.335 INFO:tasks.workunit.client.0.smithi008.stdout:3/659: chown d9/d2e/d3d 218 1 2024-04-12T05:55:37.337 INFO:tasks.workunit.client.0.smithi008.stdout:8/646: dwrite dd/de/f29 [0,4194304] 0 2024-04-12T05:55:37.337 INFO:tasks.workunit.client.0.smithi008.stdout:1/652: dwrite d4/dd/d17/d26/fdf [0,4194304] 0 2024-04-12T05:55:37.340 INFO:tasks.workunit.client.0.smithi008.stdout:8/647: chown dd/c86 3436621 1 2024-04-12T05:55:37.340 INFO:tasks.workunit.client.0.smithi008.stdout:1/653: chown d4/d1c/fb3 441095 1 2024-04-12T05:55:37.341 INFO:tasks.workunit.client.0.smithi008.stdout:3/660: creat d9/d10/d19/d3a/dad/fdd x:0 0 0 2024-04-12T05:55:37.343 INFO:tasks.workunit.client.0.smithi008.stdout:0/615: creat d2/d11/d36/d42/da6/fbe x:0 0 0 2024-04-12T05:55:37.347 INFO:tasks.workunit.client.0.smithi008.stdout:1/654: creat d4/d1c/db9/fe4 x:0 0 0 2024-04-12T05:55:37.349 INFO:tasks.workunit.client.0.smithi008.stdout:2/783: dread d6/db/d10/d23/d35/d3c/fc9 [0,4194304] 0 2024-04-12T05:55:37.350 INFO:tasks.workunit.client.0.smithi008.stdout:3/661: unlink d9/d10/d19/d22/c3e 0 2024-04-12T05:55:37.354 INFO:tasks.workunit.client.0.smithi008.stdout:2/784: mknod d6/db/d6f/de2/c105 0 2024-04-12T05:55:37.354 INFO:tasks.workunit.client.0.smithi008.stdout:0/616: dwrite d2/d24/f76 [0,4194304] 0 2024-04-12T05:55:37.355 INFO:tasks.workunit.client.0.smithi008.stdout:3/662: truncate d9/d37/fdb 587456 0 2024-04-12T05:55:37.368 INFO:tasks.workunit.client.0.smithi008.stdout:1/655: dwrite d4/d33/fd9 [0,4194304] 0 2024-04-12T05:55:37.377 INFO:tasks.workunit.client.0.smithi008.stdout:8/648: dread dd/d4f/d52/f69 [0,4194304] 0 2024-04-12T05:55:37.378 INFO:tasks.workunit.client.0.smithi008.stdout:2/785: creat d6/db/d29/dc2/f106 x:0 0 0 2024-04-12T05:55:37.379 INFO:tasks.workunit.client.0.smithi008.stdout:3/663: symlink d9/d10/d19/d3a/dad/dcf/lde 0 2024-04-12T05:55:37.379 INFO:tasks.workunit.client.0.smithi008.stdout:2/786: readlink d6/l44 0 2024-04-12T05:55:37.381 INFO:tasks.workunit.client.0.smithi008.stdout:2/787: dread - d6/db/d60/d84/fcd zero size 2024-04-12T05:55:37.381 INFO:tasks.workunit.client.0.smithi008.stdout:8/649: creat dd/de/d16/db1/fe2 x:0 0 0 2024-04-12T05:55:37.382 INFO:tasks.workunit.client.0.smithi008.stdout:3/664: write d9/d10/d19/d22/d2c/faf [1646087,92420] 0 2024-04-12T05:55:37.383 INFO:tasks.workunit.client.0.smithi008.stdout:1/656: creat d4/d1c/db9/dc1/fe5 x:0 0 0 2024-04-12T05:55:37.386 INFO:tasks.workunit.client.0.smithi008.stdout:0/617: rename d2/d11/c9c to d2/d11/d36/d42/d86/d9b/cbf 0 2024-04-12T05:55:37.386 INFO:tasks.workunit.client.0.smithi008.stdout:0/618: dread - d2/d24/d85/fa8 zero size 2024-04-12T05:55:37.387 INFO:tasks.workunit.client.0.smithi008.stdout:2/788: symlink d6/db/d60/d4f/l107 0 2024-04-12T05:55:37.392 INFO:tasks.workunit.client.0.smithi008.stdout:8/650: creat dd/de/d16/dc8/fe3 x:0 0 0 2024-04-12T05:55:37.393 INFO:tasks.workunit.client.0.smithi008.stdout:8/651: write dd/d4f/f5f [152838,127003] 0 2024-04-12T05:55:37.394 INFO:tasks.workunit.client.0.smithi008.stdout:0/619: creat d2/d4b/da2/fc0 x:0 0 0 2024-04-12T05:55:37.395 INFO:tasks.workunit.client.0.smithi008.stdout:2/789: creat d6/db/d10/d23/f108 x:0 0 0 2024-04-12T05:55:37.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:37 smithi008 ceph-mon[39722]: pgmap v14: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 50 MiB/s rd, 186 MiB/s wr, 636 op/s 2024-04-12T05:55:37.407 INFO:tasks.workunit.client.0.smithi008.stdout:8/652: getdents dd/de/d16/d50/d56 0 2024-04-12T05:55:37.409 INFO:tasks.workunit.client.0.smithi008.stdout:8/653: mkdir dd/d30/d3b/de4 0 2024-04-12T05:55:37.414 INFO:tasks.workunit.client.0.smithi008.stdout:8/654: link dd/de/d16/d50/d5e/f92 dd/de/d16/db1/d97/fe5 0 2024-04-12T05:55:37.416 INFO:tasks.workunit.client.0.smithi008.stdout:8/655: creat dd/de/d16/d73/fe6 x:0 0 0 2024-04-12T05:55:37.416 INFO:tasks.workunit.client.0.smithi008.stdout:1/657: write d4/d1a/f6c [841484,80807] 0 2024-04-12T05:55:37.422 INFO:tasks.workunit.client.0.smithi008.stdout:1/658: creat d4/dd/d17/d26/fe6 x:0 0 0 2024-04-12T05:55:37.422 INFO:tasks.workunit.client.0.smithi008.stdout:0/620: write d2/d11/d36/f67 [76368,223] 0 2024-04-12T05:55:37.423 INFO:tasks.workunit.client.0.smithi008.stdout:8/656: symlink dd/de/d16/le7 0 2024-04-12T05:55:37.423 INFO:tasks.workunit.client.0.smithi008.stdout:2/790: truncate d6/db/d10/d23/d35/fc5 1036788 0 2024-04-12T05:55:37.426 INFO:tasks.workunit.client.0.smithi008.stdout:2/791: fsync d6/db/d38/dcf/fee 0 2024-04-12T05:55:37.430 INFO:tasks.workunit.client.0.smithi008.stdout:0/621: rename d2/d11/d36/f81 to d2/fc1 0 2024-04-12T05:55:37.432 INFO:tasks.workunit.client.0.smithi008.stdout:0/622: write d2/d4b/da2/fb8 [2494371,5646] 0 2024-04-12T05:55:37.432 INFO:tasks.workunit.client.0.smithi008.stdout:8/657: symlink dd/d30/d3b/de4/le8 0 2024-04-12T05:55:37.433 INFO:tasks.workunit.client.0.smithi008.stdout:0/623: write d2/d24/fa0 [1787469,67874] 0 2024-04-12T05:55:37.434 INFO:tasks.workunit.client.0.smithi008.stdout:1/659: creat d4/dd/d17/fe7 x:0 0 0 2024-04-12T05:55:37.434 INFO:tasks.workunit.client.0.smithi008.stdout:2/792: symlink d6/db/d10/d23/d35/l109 0 2024-04-12T05:55:37.435 INFO:tasks.workunit.client.0.smithi008.stdout:0/624: readlink d2/d11/l71 0 2024-04-12T05:55:37.438 INFO:tasks.workunit.client.0.smithi008.stdout:8/658: unlink dd/de/d4c/d66/fa2 0 2024-04-12T05:55:37.438 INFO:tasks.workunit.client.0.smithi008.stdout:0/625: chown d2/d11/c38 0 1 2024-04-12T05:55:37.441 INFO:tasks.workunit.client.0.smithi008.stdout:2/793: rename d6/db/ffb to d6/db/d29/dc2/f10a 0 2024-04-12T05:55:37.441 INFO:tasks.workunit.client.0.smithi008.stdout:8/659: read dd/de/d16/db1/d97/fa3 [1367691,575] 0 2024-04-12T05:55:37.443 INFO:tasks.workunit.client.0.smithi008.stdout:2/794: fdatasync d6/db/d38/d41/f99 0 2024-04-12T05:55:37.445 INFO:tasks.workunit.client.0.smithi008.stdout:1/660: dwrite d4/dd/f99 [0,4194304] 0 2024-04-12T05:55:37.453 INFO:tasks.workunit.client.0.smithi008.stdout:1/661: rmdir d4/d24/d62 39 2024-04-12T05:55:37.456 INFO:tasks.workunit.client.0.smithi008.stdout:2/795: read d6/db/d10/f80 [1286291,46019] 0 2024-04-12T05:55:37.457 INFO:tasks.workunit.client.0.smithi008.stdout:1/662: read d4/d33/d74/d30/d72/d7c/f8c [95775,69119] 0 2024-04-12T05:55:37.463 INFO:tasks.workunit.client.0.smithi008.stdout:0/626: rename d2/d11/d36/d42/f65 to d2/d11/d2a/db5/fc2 0 2024-04-12T05:55:37.464 INFO:tasks.workunit.client.0.smithi008.stdout:8/660: creat dd/fe9 x:0 0 0 2024-04-12T05:55:37.465 INFO:tasks.workunit.client.0.smithi008.stdout:2/796: stat d6/db/d6f/dea/cf2 0 2024-04-12T05:55:37.468 INFO:tasks.workunit.client.0.smithi008.stdout:6/667: sync 2024-04-12T05:55:37.468 INFO:tasks.workunit.client.0.smithi008.stdout:7/631: sync 2024-04-12T05:55:37.468 INFO:tasks.workunit.client.0.smithi008.stdout:2/797: readlink d6/db/d60/d84/l9f 0 2024-04-12T05:55:37.468 INFO:tasks.workunit.client.0.smithi008.stdout:1/663: dwrite d4/d1c/f8f [0,4194304] 0 2024-04-12T05:55:37.470 INFO:tasks.workunit.client.0.smithi008.stdout:8/661: symlink dd/d65/lea 0 2024-04-12T05:55:37.471 INFO:tasks.workunit.client.0.smithi008.stdout:8/662: readlink dd/l14 0 2024-04-12T05:55:37.478 INFO:tasks.workunit.client.0.smithi008.stdout:2/798: mknod d6/db/d60/d57/dd9/c10b 0 2024-04-12T05:55:37.478 INFO:tasks.workunit.client.0.smithi008.stdout:6/668: dread df/d10/d15/dd0/dd1/f8d [0,4194304] 0 2024-04-12T05:55:37.478 INFO:tasks.workunit.client.0.smithi008.stdout:0/627: rename d2/d11/d2a/la1 to d2/d11/lc3 0 2024-04-12T05:55:37.479 INFO:tasks.workunit.client.0.smithi008.stdout:1/664: symlink d4/d33/d74/le8 0 2024-04-12T05:55:37.480 INFO:tasks.workunit.client.0.smithi008.stdout:7/632: chown d1/d5/da/d1d/d82/fb9 762 1 2024-04-12T05:55:37.483 INFO:tasks.workunit.client.0.smithi008.stdout:8/663: rename dd/de/d16/d50/f7f to dd/de/d16/d73/feb 0 2024-04-12T05:55:37.484 INFO:tasks.workunit.client.0.smithi008.stdout:6/669: symlink df/d10/d15/d4f/le2 0 2024-04-12T05:55:37.485 INFO:tasks.workunit.client.0.smithi008.stdout:7/633: dread - d1/d15/d43/d54/d58/db7/fb6 zero size 2024-04-12T05:55:37.486 INFO:tasks.workunit.client.0.smithi008.stdout:6/670: stat df/d10/d20/d3e 0 2024-04-12T05:55:37.488 INFO:tasks.workunit.client.0.smithi008.stdout:0/628: dread d2/d11/d36/d47/f87 [0,4194304] 0 2024-04-12T05:55:37.491 INFO:tasks.workunit.client.0.smithi008.stdout:8/664: dread - dd/de/d16/d50/d56/d61/f63 zero size 2024-04-12T05:55:37.492 INFO:tasks.workunit.client.0.smithi008.stdout:1/665: rename d4/d24/d62/c93 to d4/d33/dc5/ce9 0 2024-04-12T05:55:37.492 INFO:tasks.workunit.client.0.smithi008.stdout:7/634: creat d1/d5/da/d60/da1/fdc x:0 0 0 2024-04-12T05:55:37.493 INFO:tasks.workunit.client.0.smithi008.stdout:0/629: mknod d2/d11/d36/d47/cc4 0 2024-04-12T05:55:37.494 INFO:tasks.workunit.client.0.smithi008.stdout:6/671: creat df/fe3 x:0 0 0 2024-04-12T05:55:37.495 INFO:tasks.workunit.client.0.smithi008.stdout:8/665: rmdir dd/d30/d3b 39 2024-04-12T05:55:37.496 INFO:tasks.workunit.client.0.smithi008.stdout:0/630: chown d2/d11/d36/f46 328 1 2024-04-12T05:55:37.498 INFO:tasks.workunit.client.0.smithi008.stdout:6/672: symlink df/d8c/le4 0 2024-04-12T05:55:37.500 INFO:tasks.workunit.client.0.smithi008.stdout:1/666: dread d4/d1a/d57/dd6/f81 [0,4194304] 0 2024-04-12T05:55:37.500 INFO:tasks.workunit.client.0.smithi008.stdout:8/666: rename dd/d4f/d52/fb9 to dd/de/d4c/fec 0 2024-04-12T05:55:37.501 INFO:tasks.workunit.client.0.smithi008.stdout:8/667: chown dd/de/d16/d50/d56 127 1 2024-04-12T05:55:37.502 INFO:tasks.workunit.client.0.smithi008.stdout:6/673: mkdir df/d10/d15/dd0/dd1/de5 0 2024-04-12T05:55:37.503 INFO:tasks.workunit.client.0.smithi008.stdout:1/667: readlink d4/dd/d17/d91/lb7 0 2024-04-12T05:55:37.503 INFO:tasks.workunit.client.0.smithi008.stdout:6/674: chown df/d10/d15/d4f/le2 7 1 2024-04-12T05:55:37.505 INFO:tasks.workunit.client.0.smithi008.stdout:7/635: getdents d1/d5/da/d11 0 2024-04-12T05:55:37.506 INFO:tasks.workunit.client.0.smithi008.stdout:6/675: fdatasync df/d10/d15/dd0/dd1/fdf 0 2024-04-12T05:55:37.507 INFO:tasks.workunit.client.0.smithi008.stdout:8/668: creat dd/d30/d3b/fed x:0 0 0 2024-04-12T05:55:37.509 INFO:tasks.workunit.client.0.smithi008.stdout:7/636: write d1/d5/da/d78/fbd [3730892,57385] 0 2024-04-12T05:55:37.511 INFO:tasks.workunit.client.0.smithi008.stdout:6/676: symlink df/d10/d15/dd0/dd1/le6 0 2024-04-12T05:55:37.512 INFO:tasks.workunit.client.0.smithi008.stdout:6/677: dread - df/d21/f77 zero size 2024-04-12T05:55:37.512 INFO:tasks.workunit.client.0.smithi008.stdout:7/637: write d1/d5/da/f2a [347315,14724] 0 2024-04-12T05:55:37.516 INFO:tasks.workunit.client.0.smithi008.stdout:7/638: chown d1/d15/d43/d54/d58/f67 710131784 1 2024-04-12T05:55:37.516 INFO:tasks.workunit.client.0.smithi008.stdout:1/668: dread d4/d6/d11/f42 [0,4194304] 0 2024-04-12T05:55:37.520 INFO:tasks.workunit.client.0.smithi008.stdout:6/678: rename df/d8c/db1/ddc/c99 to df/d8c/da4/dad/ce7 0 2024-04-12T05:55:37.521 INFO:tasks.workunit.client.0.smithi008.stdout:7/639: symlink d1/d49/ldd 0 2024-04-12T05:55:37.522 INFO:tasks.workunit.client.0.smithi008.stdout:0/631: dread d2/d11/d2a/f37 [0,4194304] 0 2024-04-12T05:55:37.523 INFO:tasks.workunit.client.0.smithi008.stdout:7/640: fdatasync d1/d5/da/d3c/d44/d8a/fa7 0 2024-04-12T05:55:37.524 INFO:tasks.workunit.client.0.smithi008.stdout:8/669: dread dd/ff [0,4194304] 0 2024-04-12T05:55:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:6/679: rename df/d10/d20/f89 to df/d21/d4c/fe8 0 2024-04-12T05:55:37.525 INFO:tasks.workunit.client.0.smithi008.stdout:7/641: chown d1/d49/c6d 0 1 2024-04-12T05:55:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:0/632: readlink d2/d24/l54 0 2024-04-12T05:55:37.527 INFO:tasks.workunit.client.0.smithi008.stdout:1/669: getdents d4/d33/d74/d7a/db6/dc9 0 2024-04-12T05:55:37.531 INFO:tasks.workunit.client.0.smithi008.stdout:7/642: creat d1/d5/da/fde x:0 0 0 2024-04-12T05:55:37.534 INFO:tasks.workunit.client.0.smithi008.stdout:1/670: write d4/dd/f85 [528501,63552] 0 2024-04-12T05:55:37.536 INFO:tasks.workunit.client.0.smithi008.stdout:7/643: rmdir d1/d15/d43/d54/d58/dc2 39 2024-04-12T05:55:37.539 INFO:tasks.workunit.client.0.smithi008.stdout:8/670: getdents dd/d30/d3b/de4 0 2024-04-12T05:55:37.540 INFO:tasks.workunit.client.0.smithi008.stdout:0/633: dwrite d2/d4b/da2/fa3 [0,4194304] 0 2024-04-12T05:55:37.543 INFO:tasks.workunit.client.0.smithi008.stdout:6/680: dread df/d10/d15/d4f/f65 [0,4194304] 0 2024-04-12T05:55:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:7/644: creat d1/d5/da/d1d/d68/fdf x:0 0 0 2024-04-12T05:55:37.544 INFO:tasks.workunit.client.0.smithi008.stdout:7/645: stat d1/fb8 0 2024-04-12T05:55:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:0/634: fsync d2/d11/d36/d82/fb4 0 2024-04-12T05:55:37.546 INFO:tasks.workunit.client.0.smithi008.stdout:6/681: read df/d10/d20/d3e/f72 [1161804,2620] 0 2024-04-12T05:55:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:6/682: chown df/d8c/da4/fc5 47944121 1 2024-04-12T05:55:37.548 INFO:tasks.workunit.client.0.smithi008.stdout:1/671: rename d4/dd/d17/fe7 to d4/dd/d17/d91/da4/fea 0 2024-04-12T05:55:37.549 INFO:tasks.workunit.client.0.smithi008.stdout:8/671: mknod dd/de/d16/db1/cee 0 2024-04-12T05:55:37.555 INFO:tasks.workunit.client.0.smithi008.stdout:0/635: unlink d2/f6a 0 2024-04-12T05:55:37.557 INFO:tasks.workunit.client.0.smithi008.stdout:1/672: mknod d4/d1c/d4f/ceb 0 2024-04-12T05:55:37.557 INFO:tasks.workunit.client.0.smithi008.stdout:6/683: dwrite df/f23 [0,4194304] 0 2024-04-12T05:55:37.558 INFO:tasks.workunit.client.0.smithi008.stdout:8/672: mknod dd/dab/cef 0 2024-04-12T05:55:37.560 INFO:tasks.workunit.client.0.smithi008.stdout:7/646: unlink d1/l27 0 2024-04-12T05:55:37.565 INFO:tasks.workunit.client.0.smithi008.stdout:6/684: chown df/d10/d15/dd0/dd1/f82 201924257 1 2024-04-12T05:55:37.567 INFO:tasks.workunit.client.0.smithi008.stdout:1/673: symlink d4/d33/d74/d30/d72/d7c/dd4/lec 0 2024-04-12T05:55:37.568 INFO:tasks.workunit.client.0.smithi008.stdout:1/674: chown d4/d33/c8e 10 1 2024-04-12T05:55:37.569 INFO:tasks.workunit.client.0.smithi008.stdout:1/675: write d4/dd/f9a [568193,41246] 0 2024-04-12T05:55:37.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:37 smithi194 ceph-mon[49771]: pgmap v14: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 50 MiB/s rd, 186 MiB/s wr, 636 op/s 2024-04-12T05:55:37.571 INFO:tasks.workunit.client.0.smithi008.stdout:1/676: write d4/d1c/db9/fe4 [1009841,101903] 0 2024-04-12T05:55:37.576 INFO:tasks.workunit.client.0.smithi008.stdout:0/636: creat d2/d11/fc5 x:0 0 0 2024-04-12T05:55:37.582 INFO:tasks.workunit.client.0.smithi008.stdout:0/637: creat d2/d4b/fc6 x:0 0 0 2024-04-12T05:55:37.585 INFO:tasks.workunit.client.0.smithi008.stdout:8/673: getdents dd/de/d16/d50/d5e 0 2024-04-12T05:55:37.588 INFO:tasks.workunit.client.0.smithi008.stdout:1/677: creat d4/d24/fed x:0 0 0 2024-04-12T05:55:37.589 INFO:tasks.workunit.client.0.smithi008.stdout:7/647: dwrite d1/f6 [0,4194304] 0 2024-04-12T05:55:37.592 INFO:tasks.workunit.client.0.smithi008.stdout:7/648: truncate d1/d15/fc8 726683 0 2024-04-12T05:55:37.593 INFO:tasks.workunit.client.0.smithi008.stdout:7/649: write d1/d5/da/d78/f74 [6542640,57997] 0 2024-04-12T05:55:37.595 INFO:tasks.workunit.client.0.smithi008.stdout:6/685: getdents df/d8c/db1/ddc/d78 0 2024-04-12T05:55:37.598 INFO:tasks.workunit.client.0.smithi008.stdout:8/674: rename dd/de/d16/d73 to dd/d30/d3b/db5/df0 0 2024-04-12T05:55:37.600 INFO:tasks.workunit.client.0.smithi008.stdout:1/678: creat d4/d24/d62/fee x:0 0 0 2024-04-12T05:55:37.601 INFO:tasks.workunit.client.0.smithi008.stdout:1/679: readlink d4/d33/d74/d7a/lc8 0 2024-04-12T05:55:37.603 INFO:tasks.workunit.client.0.smithi008.stdout:6/686: mkdir df/d10/d15/d4f/de9 0 2024-04-12T05:55:37.606 INFO:tasks.workunit.client.0.smithi008.stdout:1/680: symlink d4/d24/d62/lef 0 2024-04-12T05:55:37.608 INFO:tasks.workunit.client.0.smithi008.stdout:6/687: stat df/d10/d15/l54 0 2024-04-12T05:55:37.609 INFO:tasks.workunit.client.0.smithi008.stdout:1/681: fsync d4/d24/f32 0 2024-04-12T05:55:37.610 INFO:tasks.workunit.client.0.smithi008.stdout:0/638: link d2/d11/d36/d82/cb1 d2/d24/d85/cc7 0 2024-04-12T05:55:37.613 INFO:tasks.workunit.client.0.smithi008.stdout:1/682: getdents d4/dd/dba 0 2024-04-12T05:55:37.615 INFO:tasks.workunit.client.0.smithi008.stdout:0/639: symlink d2/d11/d36/d82/lc8 0 2024-04-12T05:55:37.616 INFO:tasks.workunit.client.0.smithi008.stdout:0/640: stat d2/c15 0 2024-04-12T05:55:37.620 INFO:tasks.workunit.client.0.smithi008.stdout:6/688: getdents df/d21 0 2024-04-12T05:55:37.621 INFO:tasks.workunit.client.0.smithi008.stdout:6/689: chown df/d21/f81 367 1 2024-04-12T05:55:37.624 INFO:tasks.workunit.client.0.smithi008.stdout:1/683: rename d4/d33/cb8 to d4/dd/d17/d91/da4/cf0 0 2024-04-12T05:55:37.625 INFO:tasks.workunit.client.0.smithi008.stdout:0/641: creat d2/d11/d36/d42/da6/fc9 x:0 0 0 2024-04-12T05:55:37.630 INFO:tasks.workunit.client.0.smithi008.stdout:7/650: dread d1/d15/f20 [8388608,4194304] 0 2024-04-12T05:55:37.631 INFO:tasks.workunit.client.0.smithi008.stdout:8/675: dwrite dd/de/d4c/d66/da9/f7d [4194304,4194304] 0 2024-04-12T05:55:37.636 INFO:tasks.workunit.client.0.smithi008.stdout:6/690: truncate df/d8c/f9a 147484 0 2024-04-12T05:55:37.637 INFO:tasks.workunit.client.0.smithi008.stdout:8/676: stat dd/de/d16/d50 0 2024-04-12T05:55:37.639 INFO:tasks.workunit.client.0.smithi008.stdout:1/684: write d4/d33/d74/d7a/fdd [832068,115886] 0 2024-04-12T05:55:37.639 INFO:tasks.workunit.client.0.smithi008.stdout:0/642: write d2/d4b/f5f [923899,46778] 0 2024-04-12T05:55:37.639 INFO:tasks.workunit.client.0.smithi008.stdout:6/691: dread - df/d8c/da4/dad/fc6 zero size 2024-04-12T05:55:37.644 INFO:tasks.workunit.client.0.smithi008.stdout:6/692: chown df/d8c/da4/dad/fcd 253582 1 2024-04-12T05:55:37.647 INFO:tasks.workunit.client.0.smithi008.stdout:0/643: symlink d2/lca 0 2024-04-12T05:55:37.650 INFO:tasks.workunit.client.0.smithi008.stdout:7/651: dwrite d1/d49/f53 [0,4194304] 0 2024-04-12T05:55:37.653 INFO:tasks.workunit.client.0.smithi008.stdout:6/693: symlink df/d10/d15/dd0/dd1/de5/lea 0 2024-04-12T05:55:37.659 INFO:tasks.workunit.client.0.smithi008.stdout:6/694: dread - df/d10/f56 zero size 2024-04-12T05:55:37.661 INFO:tasks.workunit.client.0.smithi008.stdout:1/685: dwrite d4/dd/d17/d26/f76 [4194304,4194304] 0 2024-04-12T05:55:37.663 INFO:tasks.workunit.client.0.smithi008.stdout:0/644: creat d2/d4b/da2/fcb x:0 0 0 2024-04-12T05:55:37.664 INFO:tasks.workunit.client.0.smithi008.stdout:0/645: write d2/d11/d2a/f7e [596884,39504] 0 2024-04-12T05:55:37.669 INFO:tasks.workunit.client.0.smithi008.stdout:6/695: unlink df/d10/d15/f4b 0 2024-04-12T05:55:37.669 INFO:tasks.workunit.client.0.smithi008.stdout:1/686: creat d4/d33/d74/d7a/ff1 x:0 0 0 2024-04-12T05:55:37.672 INFO:tasks.workunit.client.0.smithi008.stdout:6/696: fdatasync df/f1f 0 2024-04-12T05:55:37.672 INFO:tasks.workunit.client.0.smithi008.stdout:0/646: symlink d2/d11/d2a/lcc 0 2024-04-12T05:55:37.676 INFO:tasks.workunit.client.0.smithi008.stdout:7/652: dwrite d1/d5/da/d78/fbd [0,4194304] 0 2024-04-12T05:55:37.679 INFO:tasks.workunit.client.0.smithi008.stdout:0/647: chown d2/d24/d85 27364 1 2024-04-12T05:55:37.693 INFO:tasks.workunit.client.0.smithi008.stdout:8/677: dread dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:37.693 INFO:tasks.workunit.client.0.smithi008.stdout:6/697: dwrite df/d10/d15/d75/fba [0,4194304] 0 2024-04-12T05:55:37.695 INFO:tasks.workunit.client.0.smithi008.stdout:7/653: write d1/d15/d43/d54/d77/f89 [551305,45245] 0 2024-04-12T05:55:37.699 INFO:tasks.workunit.client.0.smithi008.stdout:1/687: dwrite d4/d1a/f51 [0,4194304] 0 2024-04-12T05:55:37.701 INFO:tasks.workunit.client.0.smithi008.stdout:4/571: sync 2024-04-12T05:55:37.705 INFO:tasks.workunit.client.0.smithi008.stdout:9/759: sync 2024-04-12T05:55:37.705 INFO:tasks.workunit.client.0.smithi008.stdout:1/688: readlink d4/d33/d37/lbb 0 2024-04-12T05:55:37.706 INFO:tasks.workunit.client.0.smithi008.stdout:0/648: dread d2/d11/d2a/f88 [0,4194304] 0 2024-04-12T05:55:37.706 INFO:tasks.workunit.client.0.smithi008.stdout:4/572: chown d1/d22/d29/d30/f90 25488042 1 2024-04-12T05:55:37.708 INFO:tasks.workunit.client.0.smithi008.stdout:7/654: write d1/d5/da/d78/f74 [7416862,68964] 0 2024-04-12T05:55:37.709 INFO:tasks.workunit.client.0.smithi008.stdout:4/573: readlink d1/d22/d29/l33 0 2024-04-12T05:55:37.711 INFO:tasks.workunit.client.0.smithi008.stdout:4/574: chown d1/d4f/f7f 373 1 2024-04-12T05:55:37.712 INFO:tasks.workunit.client.0.smithi008.stdout:6/698: symlink df/d10/d15/d79/leb 0 2024-04-12T05:55:37.712 INFO:tasks.workunit.client.0.smithi008.stdout:1/689: rmdir d4/dd/d17/d66 39 2024-04-12T05:55:37.715 INFO:tasks.workunit.client.0.smithi008.stdout:4/575: creat d1/fa0 x:0 0 0 2024-04-12T05:55:37.717 INFO:tasks.workunit.client.0.smithi008.stdout:6/699: mknod df/d10/d15/d4f/cec 0 2024-04-12T05:55:37.719 INFO:tasks.workunit.client.0.smithi008.stdout:1/690: symlink d4/d6/lf2 0 2024-04-12T05:55:37.719 INFO:tasks.workunit.client.0.smithi008.stdout:6/700: readlink df/d10/d15/d79/l85 0 2024-04-12T05:55:37.722 INFO:tasks.workunit.client.0.smithi008.stdout:1/691: write d4/dd/d17/d26/fa9 [353503,81458] 0 2024-04-12T05:55:37.722 INFO:tasks.workunit.client.0.smithi008.stdout:6/701: chown df/d8c/db1/ddc/c4a 5072852 1 2024-04-12T05:55:37.722 INFO:tasks.workunit.client.0.smithi008.stdout:0/649: rename d2/d24/c30 to d2/ccd 0 2024-04-12T05:55:37.725 INFO:tasks.workunit.client.0.smithi008.stdout:0/650: chown d2/d4b/f99 9683 1 2024-04-12T05:55:37.726 INFO:tasks.workunit.client.0.smithi008.stdout:4/576: fsync d1/fa0 0 2024-04-12T05:55:37.727 INFO:tasks.workunit.client.0.smithi008.stdout:9/760: truncate d3/df/d13/d21/d2d/fa6 1436931 0 2024-04-12T05:55:37.728 INFO:tasks.workunit.client.0.smithi008.stdout:0/651: dread d2/d11/f18 [0,4194304] 0 2024-04-12T05:55:37.730 INFO:tasks.workunit.client.0.smithi008.stdout:4/577: write d1/d8/f55 [3504584,90271] 0 2024-04-12T05:55:37.733 INFO:tasks.workunit.client.0.smithi008.stdout:6/702: creat df/d10/d20/fed x:0 0 0 2024-04-12T05:55:37.735 INFO:tasks.workunit.client.0.smithi008.stdout:4/578: dread - d1/d4f/f89 zero size 2024-04-12T05:55:37.737 INFO:tasks.workunit.client.0.smithi008.stdout:9/761: creat d3/df/d13/d21/d5c/da4/db5/f105 x:0 0 0 2024-04-12T05:55:37.740 INFO:tasks.workunit.client.0.smithi008.stdout:8/678: dread dd/de/d4c/d66/da9/f6f [0,4194304] 0 2024-04-12T05:55:37.742 INFO:tasks.workunit.client.0.smithi008.stdout:1/692: dwrite d4/dd/f39 [0,4194304] 0 2024-04-12T05:55:37.744 INFO:tasks.workunit.client.0.smithi008.stdout:6/703: chown df/d10/d20/d3e/fb2 0 1 2024-04-12T05:55:37.746 INFO:tasks.workunit.client.0.smithi008.stdout:9/762: creat d3/d22/d2b/d77/d82/f106 x:0 0 0 2024-04-12T05:55:37.746 INFO:tasks.workunit.client.0.smithi008.stdout:8/679: chown dd/de/d16/c4d 5044104 1 2024-04-12T05:55:37.748 INFO:tasks.workunit.client.0.smithi008.stdout:7/655: dwrite d1/d15/d43/d54/d58/db7/fac [4194304,4194304] 0 2024-04-12T05:55:37.748 INFO:tasks.workunit.client.0.smithi008.stdout:6/704: chown df/d10/d15/fda 433203255 1 2024-04-12T05:55:37.750 INFO:tasks.workunit.client.0.smithi008.stdout:4/579: getdents d1/d8 0 2024-04-12T05:55:37.754 INFO:tasks.workunit.client.0.smithi008.stdout:1/693: mknod d4/d33/d37/cf3 0 2024-04-12T05:55:37.754 INFO:tasks.workunit.client.0.smithi008.stdout:8/680: write dd/d30/d3b/fed [122527,73889] 0 2024-04-12T05:55:37.754 INFO:tasks.workunit.client.0.smithi008.stdout:5/966: sync 2024-04-12T05:55:37.755 INFO:tasks.workunit.client.0.smithi008.stdout:6/705: mkdir df/d8c/dcc/dee 0 2024-04-12T05:55:37.757 INFO:tasks.workunit.client.0.smithi008.stdout:5/967: fdatasync d0/dc/d13/d1a/daf/f84 0 2024-04-12T05:55:37.761 INFO:tasks.workunit.client.0.smithi008.stdout:1/694: mknod d4/d33/d37/cf4 0 2024-04-12T05:55:37.762 INFO:tasks.workunit.client.0.smithi008.stdout:0/652: dread d2/d11/d36/d47/f1d [0,4194304] 0 2024-04-12T05:55:37.765 INFO:tasks.workunit.client.0.smithi008.stdout:8/681: unlink dd/d30/d32/l4b 0 2024-04-12T05:55:37.769 INFO:tasks.workunit.client.0.smithi008.stdout:4/580: creat d1/d22/d29/d30/d47/d28/fa1 x:0 0 0 2024-04-12T05:55:37.769 INFO:tasks.workunit.client.0.smithi008.stdout:5/968: read d0/dc/f6a [3220003,69269] 0 2024-04-12T05:55:37.769 INFO:tasks.workunit.client.0.smithi008.stdout:0/653: read d2/d11/d2a/f37 [3930920,56917] 0 2024-04-12T05:55:37.774 INFO:tasks.workunit.client.0.smithi008.stdout:4/581: read d1/d22/f4c [1950819,82954] 0 2024-04-12T05:55:37.777 INFO:tasks.workunit.client.0.smithi008.stdout:9/763: dwrite d3/df/d13/d21/d2d/d30/de2/ff1 [0,4194304] 0 2024-04-12T05:55:37.777 INFO:tasks.workunit.client.0.smithi008.stdout:6/706: truncate df/d8c/f9a 1029327 0 2024-04-12T05:55:37.779 INFO:tasks.workunit.client.0.smithi008.stdout:4/582: truncate d1/d22/d29/d30/f52 719009 0 2024-04-12T05:55:37.781 INFO:tasks.workunit.client.0.smithi008.stdout:4/583: read - d1/d8/f5f zero size 2024-04-12T05:55:37.781 INFO:tasks.workunit.client.0.smithi008.stdout:1/695: truncate d4/d7/ff 122567 0 2024-04-12T05:55:37.783 INFO:tasks.workunit.client.0.smithi008.stdout:6/707: chown df/d8c/da4/ld2 56488 1 2024-04-12T05:55:37.783 INFO:tasks.workunit.client.0.smithi008.stdout:8/682: getdents dd/de/d4c/d66/d6d/dca 0 2024-04-12T05:55:37.785 INFO:tasks.workunit.client.0.smithi008.stdout:1/696: fdatasync d4/d33/fd9 0 2024-04-12T05:55:37.785 INFO:tasks.workunit.client.0.smithi008.stdout:7/656: dread d1/d5/da/d3c/d44/f51 [4194304,4194304] 0 2024-04-12T05:55:37.785 INFO:tasks.workunit.client.0.smithi008.stdout:5/969: fsync d0/f12 0 2024-04-12T05:55:37.789 INFO:tasks.workunit.client.0.smithi008.stdout:1/697: chown d4/d1c/db9/dc1/fe5 125 1 2024-04-12T05:55:37.793 INFO:tasks.workunit.client.0.smithi008.stdout:8/683: dread dd/de/f29 [0,4194304] 0 2024-04-12T05:55:37.796 INFO:tasks.workunit.client.0.smithi008.stdout:6/708: fsync df/d10/d15/f2a 0 2024-04-12T05:55:37.800 INFO:tasks.workunit.client.0.smithi008.stdout:7/657: rename d1/f4f to d1/d5/da/d1d/d68/fe0 0 2024-04-12T05:55:37.804 INFO:tasks.workunit.client.0.smithi008.stdout:8/684: mkdir dd/de/d16/d50/df1 0 2024-04-12T05:55:37.805 INFO:tasks.workunit.client.0.smithi008.stdout:1/698: link d4/dd/d17/d91/lb7 d4/d33/d74/da2/lf5 0 2024-04-12T05:55:37.808 INFO:tasks.workunit.client.0.smithi008.stdout:5/970: dread d0/dc/d57/f66 [0,4194304] 0 2024-04-12T05:55:37.808 INFO:tasks.workunit.client.0.smithi008.stdout:8/685: write dd/d30/d32/fc4 [418804,15312] 0 2024-04-12T05:55:37.810 INFO:tasks.workunit.client.0.smithi008.stdout:6/709: mkdir df/d10/d15/dd0/dd1/def 0 2024-04-12T05:55:37.812 INFO:tasks.workunit.client.0.smithi008.stdout:8/686: fsync dd/de/d4c/d66/d6c/dc0/fd7 0 2024-04-12T05:55:37.815 INFO:tasks.workunit.client.0.smithi008.stdout:0/654: dread d2/d24/d61/fad [0,4194304] 0 2024-04-12T05:55:37.816 INFO:tasks.workunit.client.0.smithi008.stdout:9/764: write d3/df/d13/d21/d2d/d75/f94 [1251420,55608] 0 2024-04-12T05:55:37.817 INFO:tasks.workunit.client.0.smithi008.stdout:5/971: creat d0/dd/d3d/d5d/f12f x:0 0 0 2024-04-12T05:55:37.820 INFO:tasks.workunit.client.0.smithi008.stdout:1/699: symlink d4/d6/d11/ddb/lf6 0 2024-04-12T05:55:37.822 INFO:tasks.workunit.client.0.smithi008.stdout:4/584: dread d1/d22/d29/f7c [0,4194304] 0 2024-04-12T05:55:37.824 INFO:tasks.workunit.client.0.smithi008.stdout:9/765: rmdir d3/df/d13/d21/d5c 39 2024-04-12T05:55:37.827 INFO:tasks.workunit.client.0.smithi008.stdout:8/687: rename dd/f54 to dd/d65/ff2 0 2024-04-12T05:55:37.830 INFO:tasks.workunit.client.0.smithi008.stdout:1/700: fdatasync d4/dd/d17/d66/fa8 0 2024-04-12T05:55:37.830 INFO:tasks.workunit.client.0.smithi008.stdout:7/658: getdents d1/d49 0 2024-04-12T05:55:37.830 INFO:tasks.workunit.client.0.smithi008.stdout:9/766: dread d3/d22/d2b/d58/d90/db1/fb3 [0,4194304] 0 2024-04-12T05:55:37.831 INFO:tasks.workunit.client.0.smithi008.stdout:3/665: sync 2024-04-12T05:55:37.832 INFO:tasks.workunit.client.0.smithi008.stdout:5/972: creat d0/dd/d11/d38/d55/d89/df3/f130 x:0 0 0 2024-04-12T05:55:37.833 INFO:tasks.workunit.client.0.smithi008.stdout:4/585: symlink d1/d22/d29/d30/d47/d28/d4a/d62/la2 0 2024-04-12T05:55:37.834 INFO:tasks.workunit.client.0.smithi008.stdout:9/767: write d3/d6a/f9e [3425087,59013] 0 2024-04-12T05:55:37.835 INFO:tasks.workunit.client.0.smithi008.stdout:8/688: unlink dd/d4f/d52/f8d 0 2024-04-12T05:55:37.836 INFO:tasks.workunit.client.0.smithi008.stdout:0/655: dread d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:37.840 INFO:tasks.workunit.client.0.smithi008.stdout:7/659: fdatasync d1/d5/da/d3c/fbe 0 2024-04-12T05:55:37.840 INFO:tasks.workunit.client.0.smithi008.stdout:6/710: dread df/d8c/da4/dad/fd5 [0,4194304] 0 2024-04-12T05:55:37.843 INFO:tasks.workunit.client.0.smithi008.stdout:1/701: rename d4/l14 to d4/d24/d62/lf7 0 2024-04-12T05:55:37.843 INFO:tasks.workunit.client.0.smithi008.stdout:5/973: symlink d0/dc/d13/d1a/daf/de3/l131 0 2024-04-12T05:55:37.845 INFO:tasks.workunit.client.0.smithi008.stdout:4/586: creat d1/d22/d29/d30/d60/fa3 x:0 0 0 2024-04-12T05:55:37.846 INFO:tasks.workunit.client.0.smithi008.stdout:7/660: symlink d1/d15/d43/d54/d77/le1 0 2024-04-12T05:55:37.846 INFO:tasks.workunit.client.0.smithi008.stdout:6/711: mkdir df/d10/d20/d3e/db4/df0 0 2024-04-12T05:55:37.848 INFO:tasks.workunit.client.0.smithi008.stdout:3/666: link d9/d37/fdb d9/d10/d19/d22/da8/fdf 0 2024-04-12T05:55:37.849 INFO:tasks.workunit.client.0.smithi008.stdout:6/712: truncate df/d10/d15/dd0/dd1/fb7 39774 0 2024-04-12T05:55:37.854 INFO:tasks.workunit.client.0.smithi008.stdout:5/974: creat d0/dd/d11/d38/d55/d89/df3/f132 x:0 0 0 2024-04-12T05:55:37.855 INFO:tasks.workunit.client.0.smithi008.stdout:4/587: rename d1/db/c64 to d1/db/ca4 0 2024-04-12T05:55:37.857 INFO:tasks.workunit.client.0.smithi008.stdout:9/768: creat d3/df/d13/f107 x:0 0 0 2024-04-12T05:55:37.857 INFO:tasks.workunit.client.0.smithi008.stdout:3/667: fdatasync d9/d10/d19/d22/da8/fdf 0 2024-04-12T05:55:37.858 INFO:tasks.workunit.client.0.smithi008.stdout:1/702: dwrite d4/dd/d21/d9f/dac/fe2 [0,4194304] 0 2024-04-12T05:55:37.865 INFO:tasks.workunit.client.0.smithi008.stdout:8/689: rename dd/d4f/d52/c94 to dd/de/d16/cf3 0 2024-04-12T05:55:37.870 INFO:tasks.workunit.client.0.smithi008.stdout:1/703: mkdir d4/d1c/db9/df8 0 2024-04-12T05:55:37.872 INFO:tasks.workunit.client.0.smithi008.stdout:7/661: rename d1/d5/f33 to d1/d5/da/d1d/d82/d9f/fe2 0 2024-04-12T05:55:37.873 INFO:tasks.workunit.client.0.smithi008.stdout:9/769: write d3/df/d13/d21/d5c/dd7/fbe [1190197,8141] 0 2024-04-12T05:55:37.873 INFO:tasks.workunit.client.0.smithi008.stdout:3/668: symlink d9/d2e/d3d/le0 0 2024-04-12T05:55:37.876 INFO:tasks.workunit.client.0.smithi008.stdout:6/713: dwrite df/d10/d15/dd0/dd1/f51 [0,4194304] 0 2024-04-12T05:55:37.876 INFO:tasks.workunit.client.0.smithi008.stdout:5/975: creat d0/dc/d13/d1a/daf/de3/f133 x:0 0 0 2024-04-12T05:55:37.877 INFO:tasks.workunit.client.0.smithi008.stdout:4/588: dwrite d1/d22/d29/d30/f57 [0,4194304] 0 2024-04-12T05:55:37.881 INFO:tasks.workunit.client.0.smithi008.stdout:9/770: write d3/d22/d2b/d4a/f104 [161369,56867] 0 2024-04-12T05:55:37.886 INFO:tasks.workunit.client.0.smithi008.stdout:7/662: unlink d1/d5/f9d 0 2024-04-12T05:55:37.889 INFO:tasks.workunit.client.0.smithi008.stdout:7/663: fdatasync d1/d5/da/d78/fbd 0 2024-04-12T05:55:37.889 INFO:tasks.workunit.client.0.smithi008.stdout:6/714: unlink df/d21/f55 0 2024-04-12T05:55:37.895 INFO:tasks.workunit.client.0.smithi008.stdout:9/771: readlink d3/d22/d2b/d58/d38/l8b 0 2024-04-12T05:55:37.900 INFO:tasks.workunit.client.0.smithi008.stdout:3/669: rename d9/dd/l46 to d9/d10/d4b/le1 0 2024-04-12T05:55:37.903 INFO:tasks.workunit.client.0.smithi008.stdout:5/976: readlink d0/dd/d3d/d5d/l90 0 2024-04-12T05:55:37.907 INFO:tasks.workunit.client.0.smithi008.stdout:1/704: getdents d4/d1c/db9/dc1 0 2024-04-12T05:55:37.910 INFO:tasks.workunit.client.0.smithi008.stdout:6/715: symlink df/d8c/db1/ddc/lf1 0 2024-04-12T05:55:37.914 INFO:tasks.workunit.client.0.smithi008.stdout:8/690: dwrite dd/de/d16/d50/d56/d61/d77/fb8 [0,4194304] 0 2024-04-12T05:55:37.915 INFO:tasks.workunit.client.0.smithi008.stdout:6/716: dread - df/d8c/db1/ddc/d9b/fbc zero size 2024-04-12T05:55:37.915 INFO:tasks.workunit.client.0.smithi008.stdout:7/664: truncate d1/d5/da/d78/fbd 3529992 0 2024-04-12T05:55:37.916 INFO:tasks.workunit.client.0.smithi008.stdout:5/977: creat d0/dd/d11/d38/dbd/f134 x:0 0 0 2024-04-12T05:55:37.917 INFO:tasks.workunit.client.0.smithi008.stdout:4/589: dread d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:37.917 INFO:tasks.workunit.client.0.smithi008.stdout:1/705: rmdir d4/dd/d17/d66 39 2024-04-12T05:55:37.919 INFO:tasks.workunit.client.0.smithi008.stdout:5/978: write d0/dc/f65 [1315342,65760] 0 2024-04-12T05:55:37.923 INFO:tasks.workunit.client.0.smithi008.stdout:7/665: unlink d1/c9e 0 2024-04-12T05:55:37.926 INFO:tasks.workunit.client.0.smithi008.stdout:1/706: fsync d4/d33/d74/fab 0 2024-04-12T05:55:37.928 INFO:tasks.workunit.client.0.smithi008.stdout:1/707: fsync d4/f7f 0 2024-04-12T05:55:37.930 INFO:tasks.workunit.client.0.smithi008.stdout:5/979: mknod d0/dd/d3d/c135 0 2024-04-12T05:55:37.932 INFO:tasks.workunit.client.0.smithi008.stdout:7/666: rmdir d1/d5/da/d78 39 2024-04-12T05:55:37.933 INFO:tasks.workunit.client.0.smithi008.stdout:3/670: dwrite d9/d10/d19/d22/d2c/f5e [0,4194304] 0 2024-04-12T05:55:37.937 INFO:tasks.workunit.client.0.smithi008.stdout:6/717: write df/d10/d15/f24 [254081,31719] 0 2024-04-12T05:55:37.939 INFO:tasks.workunit.client.0.smithi008.stdout:1/708: rmdir d4/dd/d21/d9f/dac 39 2024-04-12T05:55:37.939 INFO:tasks.workunit.client.0.smithi008.stdout:3/671: chown d9/d10/d19/d22/f28 2 1 2024-04-12T05:55:37.942 INFO:tasks.workunit.client.0.smithi008.stdout:7/667: mkdir d1/d15/d43/d54/d58/db7/de3 0 2024-04-12T05:55:37.943 INFO:tasks.workunit.client.0.smithi008.stdout:8/691: dread dd/d30/d3b/fb6 [0,4194304] 0 2024-04-12T05:55:37.944 INFO:tasks.workunit.client.0.smithi008.stdout:3/672: symlink d9/d11/d5b/le2 0 2024-04-12T05:55:37.946 INFO:tasks.workunit.client.0.smithi008.stdout:8/692: write dd/d4f/f96 [2137323,103269] 0 2024-04-12T05:55:37.949 INFO:tasks.workunit.client.0.smithi008.stdout:6/718: creat df/d10/ff2 x:0 0 0 2024-04-12T05:55:37.953 INFO:tasks.workunit.client.0.smithi008.stdout:5/980: write d0/d40/f5e [5296683,62541] 0 2024-04-12T05:55:37.954 INFO:tasks.workunit.client.0.smithi008.stdout:4/590: dread d1/d22/d29/d30/d47/f48 [4194304,4194304] 0 2024-04-12T05:55:37.954 INFO:tasks.workunit.client.0.smithi008.stdout:7/668: creat d1/d5/da/d60/fe4 x:0 0 0 2024-04-12T05:55:37.956 INFO:tasks.workunit.client.0.smithi008.stdout:3/673: mkdir d9/d10/d19/d3a/dad/dcf/de3 0 2024-04-12T05:55:37.956 INFO:tasks.workunit.client.0.smithi008.stdout:9/772: dwrite d3/df/d13/d21/d2d/d32/f7d [0,4194304] 0 2024-04-12T05:55:37.964 INFO:tasks.workunit.client.0.smithi008.stdout:8/693: truncate dd/d4f/d52/f79 848293 0 2024-04-12T05:55:37.965 INFO:tasks.workunit.client.0.smithi008.stdout:6/719: rename df/d10/f2c to df/d10/d15/dd0/dd1/def/ff3 0 2024-04-12T05:55:37.968 INFO:tasks.workunit.client.0.smithi008.stdout:7/669: rmdir d1/d5/da/d3c/d44/d8a 39 2024-04-12T05:55:37.971 INFO:tasks.workunit.client.0.smithi008.stdout:4/591: rename d1/d22/d29/f94 to d1/d4f/fa5 0 2024-04-12T05:55:37.971 INFO:tasks.workunit.client.0.smithi008.stdout:8/694: chown dd/d4f/l59 10771 1 2024-04-12T05:55:37.972 INFO:tasks.workunit.client.0.smithi008.stdout:1/709: dwrite d4/d33/d74/d46/f86 [0,4194304] 0 2024-04-12T05:55:37.974 INFO:tasks.workunit.client.0.smithi008.stdout:7/670: read d1/d15/d43/d54/d77/f89 [563026,80046] 0 2024-04-12T05:55:37.975 INFO:tasks.workunit.client.0.smithi008.stdout:3/674: creat d9/d44/fe4 x:0 0 0 2024-04-12T05:55:37.980 INFO:tasks.workunit.client.0.smithi008.stdout:7/671: fdatasync d1/fb8 0 2024-04-12T05:55:37.982 INFO:tasks.workunit.client.0.smithi008.stdout:5/981: dwrite d0/dd/fd5 [0,4194304] 0 2024-04-12T05:55:37.984 INFO:tasks.workunit.client.0.smithi008.stdout:1/710: symlink d4/d33/lf9 0 2024-04-12T05:55:37.990 INFO:tasks.workunit.client.0.smithi008.stdout:6/720: dwrite df/d10/d20/d3e/db4/fc8 [0,4194304] 0 2024-04-12T05:55:37.991 INFO:tasks.workunit.client.0.smithi008.stdout:9/773: dread d3/f43 [4194304,4194304] 0 2024-04-12T05:55:37.994 INFO:tasks.workunit.client.0.smithi008.stdout:9/774: write d3/d22/d2b/d77/f9c [285989,9311] 0 2024-04-12T05:55:37.996 INFO:tasks.workunit.client.0.smithi008.stdout:9/775: write d3/df/d13/d21/d2d/f8e [2488823,38034] 0 2024-04-12T05:55:38.058 INFO:tasks.workunit.client.0.smithi008.stdout:2/799: sync 2024-04-12T05:55:38.062 INFO:tasks.workunit.client.0.smithi008.stdout:1/711: creat d4/d33/d74/d46/ffa x:0 0 0 2024-04-12T05:55:38.063 INFO:tasks.workunit.client.0.smithi008.stdout:3/675: creat d9/d37/d69/fe5 x:0 0 0 2024-04-12T05:55:38.065 INFO:tasks.workunit.client.0.smithi008.stdout:5/982: mknod d0/dc/d57/d9d/db5/dbc/c136 0 2024-04-12T05:55:38.065 INFO:tasks.workunit.client.0.smithi008.stdout:4/592: truncate d1/d22/d29/d30/f52 602617 0 2024-04-12T05:55:38.067 INFO:tasks.workunit.client.0.smithi008.stdout:5/983: write d0/dd/fa1 [158107,115145] 0 2024-04-12T05:55:38.069 INFO:tasks.workunit.client.0.smithi008.stdout:6/721: rmdir df/d10/d20/d3e 39 2024-04-12T05:55:38.069 INFO:tasks.workunit.client.0.smithi008.stdout:5/984: chown d0/d29/dd6 6135549 1 2024-04-12T05:55:38.071 INFO:tasks.workunit.client.0.smithi008.stdout:9/776: mknod d3/df/d13/d21/d5c/dd7/c108 0 2024-04-12T05:55:38.071 INFO:tasks.workunit.client.0.smithi008.stdout:5/985: write d0/dd/d11/d38/d55/d89/f12a [780604,17733] 0 2024-04-12T05:55:38.074 INFO:tasks.workunit.client.0.smithi008.stdout:5/986: chown d0/dd/d11/d38/d55/d89 3 1 2024-04-12T05:55:38.075 INFO:tasks.workunit.client.0.smithi008.stdout:3/676: unlink d9/d10/d19/d22/fa5 0 2024-04-12T05:55:38.082 INFO:tasks.workunit.client.0.smithi008.stdout:1/712: mknod d4/cfb 0 2024-04-12T05:55:38.084 INFO:tasks.workunit.client.0.smithi008.stdout:3/677: creat d9/d10/d19/d3a/fe6 x:0 0 0 2024-04-12T05:55:38.086 INFO:tasks.workunit.client.0.smithi008.stdout:4/593: creat d1/d22/d29/d30/d47/d28/fa6 x:0 0 0 2024-04-12T05:55:38.087 INFO:tasks.workunit.client.0.smithi008.stdout:3/678: readlink d9/d37/l99 0 2024-04-12T05:55:38.089 INFO:tasks.workunit.client.0.smithi008.stdout:5/987: dwrite d0/dc/d57/d9d/db5/dbc/f11d [0,4194304] 0 2024-04-12T05:55:38.090 INFO:tasks.workunit.client.0.smithi008.stdout:3/679: write d9/d10/f6a [143674,59354] 0 2024-04-12T05:55:38.091 INFO:tasks.workunit.client.0.smithi008.stdout:6/722: write df/d10/d15/d79/fe0 [3746925,62931] 0 2024-04-12T05:55:38.096 INFO:tasks.workunit.client.0.smithi008.stdout:7/672: truncate d1/d5/da/f62 4295935 0 2024-04-12T05:55:38.097 INFO:tasks.workunit.client.0.smithi008.stdout:4/594: mkdir d1/d22/d29/d46/da7 0 2024-04-12T05:55:38.097 INFO:tasks.workunit.client.0.smithi008.stdout:2/800: link d6/db/d10/d7b/ddc/cd4 d6/db/d10/d23/c10c 0 2024-04-12T05:55:38.100 INFO:tasks.workunit.client.0.smithi008.stdout:4/595: fdatasync d1/d4f/f5b 0 2024-04-12T05:55:38.100 INFO:tasks.workunit.client.0.smithi008.stdout:4/596: chown d1/d22 7700 1 2024-04-12T05:55:38.102 INFO:tasks.workunit.client.0.smithi008.stdout:3/680: read d9/d10/f33 [3137558,94638] 0 2024-04-12T05:55:38.102 INFO:tasks.workunit.client.0.smithi008.stdout:5/988: mkdir d0/d40/d56/d137 0 2024-04-12T05:55:38.103 INFO:tasks.workunit.client.0.smithi008.stdout:9/777: link d3/d22/d2b/c92 d3/df/d10/c109 0 2024-04-12T05:55:38.105 INFO:tasks.workunit.client.0.smithi008.stdout:9/778: write d3/f25 [4499776,16318] 0 2024-04-12T05:55:38.106 INFO:tasks.workunit.client.0.smithi008.stdout:2/801: creat d6/f10d x:0 0 0 2024-04-12T05:55:38.111 INFO:tasks.workunit.client.0.smithi008.stdout:4/597: rename d1/d22/d29/d46/l7a to d1/d22/la8 0 2024-04-12T05:55:38.112 INFO:tasks.workunit.client.0.smithi008.stdout:6/723: write df/d21/d4c/fbe [139529,116917] 0 2024-04-12T05:55:38.113 INFO:tasks.workunit.client.0.smithi008.stdout:1/713: getdents d4/d6/d90 0 2024-04-12T05:55:38.117 INFO:tasks.workunit.client.0.smithi008.stdout:3/681: dread - d9/d11/f98 zero size 2024-04-12T05:55:38.118 INFO:tasks.workunit.client.0.smithi008.stdout:9/779: creat d3/df/d13/d21/d2d/d30/de2/f10a x:0 0 0 2024-04-12T05:55:38.119 INFO:tasks.workunit.client.0.smithi008.stdout:2/802: symlink d6/db/d10/d23/d35/d3c/l10e 0 2024-04-12T05:55:38.120 INFO:tasks.workunit.client.0.smithi008.stdout:7/673: dwrite d1/d5/da/d3c/d44/f51 [8388608,4194304] 0 2024-04-12T05:55:38.122 INFO:tasks.workunit.client.0.smithi008.stdout:1/714: unlink d4/d33/d74/da2/fde 0 2024-04-12T05:55:38.123 INFO:tasks.workunit.client.0.smithi008.stdout:4/598: creat d1/d22/d29/d30/d47/d28/d74/fa9 x:0 0 0 2024-04-12T05:55:38.125 INFO:tasks.workunit.client.0.smithi008.stdout:7/674: readlink d1/d5/da/d3c/d44/l46 0 2024-04-12T05:55:38.126 INFO:tasks.workunit.client.0.smithi008.stdout:2/803: unlink d6/db/d60/d4f/la9 0 2024-04-12T05:55:38.128 INFO:tasks.workunit.client.0.smithi008.stdout:9/780: write d3/d22/d2b/d77/d82/fbc [3489730,49530] 0 2024-04-12T05:55:38.133 INFO:tasks.workunit.client.0.smithi008.stdout:1/715: unlink d4/d33/d74/ld7 0 2024-04-12T05:55:38.134 INFO:tasks.workunit.client.0.smithi008.stdout:9/781: write d3/d22/d2b/d4e/d60/fd6 [576243,105553] 0 2024-04-12T05:55:38.140 INFO:tasks.workunit.client.0.smithi008.stdout:1/716: write d4/dd/d17/d26/fdf [1702432,76781] 0 2024-04-12T05:55:38.140 INFO:tasks.workunit.client.0.smithi008.stdout:3/682: getdents d9 0 2024-04-12T05:55:38.143 INFO:tasks.workunit.client.0.smithi008.stdout:5/989: dread d0/dc/d13/f15 [0,4194304] 0 2024-04-12T05:55:38.147 INFO:tasks.workunit.client.0.smithi008.stdout:2/804: rename d6/db/d29/c2a to d6/db/d10/d23/d35/c10f 0 2024-04-12T05:55:38.150 INFO:tasks.workunit.client.0.smithi008.stdout:4/599: dwrite d1/d22/d29/d46/d56/f63 [0,4194304] 0 2024-04-12T05:55:38.150 INFO:tasks.workunit.client.0.smithi008.stdout:6/724: dread df/d8c/db1/ddc/d78/d8b/fdb [0,4194304] 0 2024-04-12T05:55:38.158 INFO:tasks.workunit.client.0.smithi008.stdout:6/725: chown df/d10/d15/l46 177 1 2024-04-12T05:55:38.158 INFO:tasks.workunit.client.0.smithi008.stdout:4/600: mknod d1/d22/d29/d46/caa 0 2024-04-12T05:55:38.159 INFO:tasks.workunit.client.0.smithi008.stdout:9/782: link d3/df/c3b d3/df/d13/d21/d2d/d32/c10b 0 2024-04-12T05:55:38.160 INFO:tasks.workunit.client.0.smithi008.stdout:2/805: dwrite d6/db/d38/d41/f6b [0,4194304] 0 2024-04-12T05:55:38.161 INFO:tasks.workunit.client.0.smithi008.stdout:4/601: readlink d1/d8/l5e 0 2024-04-12T05:55:38.162 INFO:tasks.workunit.client.0.smithi008.stdout:5/990: getdents d0/dab/d109/d11b 0 2024-04-12T05:55:38.164 INFO:tasks.workunit.client.0.smithi008.stdout:3/683: link d9/d10/d19/d3a/d67/la0 d9/d10/d19/d3a/d67/dda/le7 0 2024-04-12T05:55:38.164 INFO:tasks.workunit.client.0.smithi008.stdout:9/783: stat d3/df/d13/d21/d5c/lc6 0 2024-04-12T05:55:38.165 INFO:tasks.workunit.client.0.smithi008.stdout:7/675: dwrite d1/d5/da/f62 [0,4194304] 0 2024-04-12T05:55:38.165 INFO:tasks.workunit.client.0.smithi008.stdout:6/726: symlink df/d8c/da4/dad/lf4 0 2024-04-12T05:55:38.165 INFO:tasks.workunit.client.0.smithi008.stdout:1/717: link d4/d24/d62/lef d4/dd/d21/d9f/lfc 0 2024-04-12T05:55:38.167 INFO:tasks.workunit.client.0.smithi008.stdout:3/684: fsync d9/dd/fa6 0 2024-04-12T05:55:38.172 INFO:tasks.workunit.client.0.smithi008.stdout:3/685: creat d9/fe8 x:0 0 0 2024-04-12T05:55:38.173 INFO:tasks.workunit.client.0.smithi008.stdout:9/784: creat d3/d22/d2b/d58/d89/f10c x:0 0 0 2024-04-12T05:55:38.175 INFO:tasks.workunit.client.0.smithi008.stdout:7/676: symlink d1/d15/d43/d54/d58/d5e/d7f/le5 0 2024-04-12T05:55:38.176 INFO:tasks.workunit.client.0.smithi008.stdout:9/785: truncate d3/df/d13/fff 817543 0 2024-04-12T05:55:38.181 INFO:tasks.workunit.client.0.smithi008.stdout:7/677: creat d1/d15/fe6 x:0 0 0 2024-04-12T05:55:38.182 INFO:tasks.workunit.client.0.smithi008.stdout:7/678: chown d1/d15/d43/ddb 36506 1 2024-04-12T05:55:38.186 INFO:tasks.workunit.client.0.smithi008.stdout:7/679: fdatasync d1/d5/da/d78/f88 0 2024-04-12T05:55:38.188 INFO:tasks.workunit.client.0.smithi008.stdout:9/786: dwrite d3/d22/d2b/d77/d82/f106 [0,4194304] 0 2024-04-12T05:55:38.197 INFO:tasks.workunit.client.0.smithi008.stdout:1/718: dread d4/d6/f59 [0,4194304] 0 2024-04-12T05:55:38.197 INFO:tasks.workunit.client.0.smithi008.stdout:9/787: rmdir d3/d22/d2b/d58/d89 39 2024-04-12T05:55:38.200 INFO:tasks.workunit.client.0.smithi008.stdout:1/719: mkdir d4/d24/d62/dfd 0 2024-04-12T05:55:38.201 INFO:tasks.workunit.client.0.smithi008.stdout:5/991: write d0/dd/dad/fdc [313964,15814] 0 2024-04-12T05:55:38.203 INFO:tasks.workunit.client.0.smithi008.stdout:4/602: truncate d1/d22/d29/d30/f34 1589045 0 2024-04-12T05:55:38.204 INFO:tasks.workunit.client.0.smithi008.stdout:9/788: mknod d3/df/d13/d21/d2d/d30/de2/c10d 0 2024-04-12T05:55:38.205 INFO:tasks.workunit.client.0.smithi008.stdout:2/806: dread d6/db/d38/f75 [0,4194304] 0 2024-04-12T05:55:38.206 INFO:tasks.workunit.client.0.smithi008.stdout:1/720: rmdir d4/d1c/db9 39 2024-04-12T05:55:38.208 INFO:tasks.workunit.client.0.smithi008.stdout:9/789: rmdir d3/df/d13/d21/d5c/dd7 39 2024-04-12T05:55:38.208 INFO:tasks.workunit.client.0.smithi008.stdout:1/721: fdatasync d4/d24/fed 0 2024-04-12T05:55:38.211 INFO:tasks.workunit.client.0.smithi008.stdout:2/807: write d6/db/d38/dcf/fee [521743,48195] 0 2024-04-12T05:55:38.214 INFO:tasks.workunit.client.0.smithi008.stdout:5/992: getdents d0/d29 0 2024-04-12T05:55:38.216 INFO:tasks.workunit.client.0.smithi008.stdout:4/603: rename d1/d22/d29/d30/d47/d28/f73 to d1/d22/d29/d30/d47/d28/d74/fab 0 2024-04-12T05:55:38.219 INFO:tasks.workunit.client.0.smithi008.stdout:9/790: write d3/d22/d2b/fea [1025191,119663] 0 2024-04-12T05:55:38.219 INFO:tasks.workunit.client.0.smithi008.stdout:5/993: truncate d0/dd/dad/fdc 1292978 0 2024-04-12T05:55:38.221 INFO:tasks.workunit.client.0.smithi008.stdout:1/722: rename d4/dd/d17/d26/f2e to d4/d33/d74/d7a/db6/dc9/ffe 0 2024-04-12T05:55:38.222 INFO:tasks.workunit.client.0.smithi008.stdout:7/680: dwrite d1/d5/fd1 [4194304,4194304] 0 2024-04-12T05:55:38.223 INFO:tasks.workunit.client.0.smithi008.stdout:5/994: fdatasync d0/d40/f77 0 2024-04-12T05:55:38.224 INFO:tasks.workunit.client.0.smithi008.stdout:4/604: symlink d1/d22/d29/d30/d47/d28/d4a/lac 0 2024-04-12T05:55:38.224 INFO:tasks.workunit.client.0.smithi008.stdout:1/723: readlink d4/d7/l13 0 2024-04-12T05:55:38.231 INFO:tasks.workunit.client.0.smithi008.stdout:5/995: readlink d0/dd/d11/d38/dbd/lf2 0 2024-04-12T05:55:38.231 INFO:tasks.workunit.client.0.smithi008.stdout:4/605: unlink d1/d22/d29/d30/c7e 0 2024-04-12T05:55:38.232 INFO:tasks.workunit.client.0.smithi008.stdout:3/686: dread d9/d10/d19/f50 [0,4194304] 0 2024-04-12T05:55:38.232 INFO:tasks.workunit.client.0.smithi008.stdout:1/724: truncate d4/d6/d90/fd3 639391 0 2024-04-12T05:55:38.233 INFO:tasks.workunit.client.0.smithi008.stdout:2/808: dwrite d6/db/d6f/f88 [0,4194304] 0 2024-04-12T05:55:38.335 INFO:tasks.workunit.client.0.smithi008.stdout:1/725: symlink d4/d1a/d57/dd6/lff 0 2024-04-12T05:55:38.336 INFO:tasks.workunit.client.0.smithi008.stdout:0/656: sync 2024-04-12T05:55:38.337 INFO:tasks.workunit.client.0.smithi008.stdout:2/809: mknod d6/db/d6f/da6/c110 0 2024-04-12T05:55:38.340 INFO:tasks.workunit.client.0.smithi008.stdout:4/606: rename d1/d22/f8b to d1/d22/d29/d46/d56/fad 0 2024-04-12T05:55:38.342 INFO:tasks.workunit.client.0.smithi008.stdout:3/687: symlink d9/d10/d19/d65/le9 0 2024-04-12T05:55:38.343 INFO:tasks.workunit.client.0.smithi008.stdout:0/657: dwrite d2/d11/d2a/f9d [0,4194304] 0 2024-04-12T05:55:38.343 INFO:tasks.workunit.client.0.smithi008.stdout:4/607: dread - d1/d22/d29/d30/d47/d28/d74/f9c zero size 2024-04-12T05:55:38.344 INFO:tasks.workunit.client.0.smithi008.stdout:5/996: mknod d0/c138 0 2024-04-12T05:55:38.345 INFO:tasks.workunit.client.0.smithi008.stdout:9/791: write d3/df/d13/d21/d5c/d8f/dab/fc3 [17888,13306] 0 2024-04-12T05:55:38.347 INFO:tasks.workunit.client.0.smithi008.stdout:5/997: chown d0/l12d 2 1 2024-04-12T05:55:38.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:38 smithi008 ceph-mon[39722]: pgmap v15: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 135 MiB/s wr, 459 op/s 2024-04-12T05:55:38.427 INFO:tasks.workunit.client.0.smithi008.stdout:3/688: fdatasync d9/d10/d19/d65/f81 0 2024-04-12T05:55:38.436 INFO:tasks.workunit.client.0.smithi008.stdout:0/658: truncate d2/d11/d36/d47/fb2 381664 0 2024-04-12T05:55:38.440 INFO:tasks.workunit.client.0.smithi008.stdout:7/681: dwrite d1/d15/d43/d54/fc4 [0,4194304] 0 2024-04-12T05:55:38.448 INFO:tasks.workunit.client.0.smithi008.stdout:0/659: symlink d2/d11/d36/d82/d95/lce 0 2024-04-12T05:55:38.449 INFO:tasks.workunit.client.0.smithi008.stdout:1/726: getdents d4/d33/d74/d30/d72/d7c/dd4 0 2024-04-12T05:55:38.454 INFO:tasks.workunit.client.0.smithi008.stdout:4/608: rename d1/d22/f97 to d1/d22/d29/d30/d47/d28/fae 0 2024-04-12T05:55:38.458 INFO:tasks.workunit.client.0.smithi008.stdout:3/689: symlink d9/d37/lea 0 2024-04-12T05:55:38.459 INFO:tasks.workunit.client.0.smithi008.stdout:4/609: stat d1/d22/d29/d30/d60/fa3 0 2024-04-12T05:55:38.459 INFO:tasks.workunit.client.0.smithi008.stdout:0/660: dwrite d2/d4b/f5f [0,4194304] 0 2024-04-12T05:55:38.461 INFO:tasks.workunit.client.0.smithi008.stdout:5/998: dread d0/dc/d13/d1a/daf/f121 [0,4194304] 0 2024-04-12T05:55:38.464 INFO:tasks.workunit.client.0.smithi008.stdout:1/727: getdents d4/d1a/d57 0 2024-04-12T05:55:38.465 INFO:tasks.workunit.client.0.smithi008.stdout:0/661: read d2/d11/d36/d47/f87 [659473,63717] 0 2024-04-12T05:55:38.469 INFO:tasks.workunit.client.0.smithi008.stdout:7/682: rename d1/d5/da/d1d/d68/ld0 to d1/d15/d43/d54/d58/d5e/le7 0 2024-04-12T05:55:38.469 INFO:tasks.workunit.client.0.smithi008.stdout:4/610: mkdir d1/d22/d29/d46/d56/daf 0 2024-04-12T05:55:38.470 INFO:tasks.workunit.client.0.smithi008.stdout:4/611: dread - d1/fa0 zero size 2024-04-12T05:55:38.471 INFO:tasks.workunit.client.0.smithi008.stdout:5/999: rmdir d0/dc/d57/d9d/db5/dbc 39 2024-04-12T05:55:38.473 INFO:tasks.workunit.client.0.smithi008.stdout:0/662: stat d2/d24/d85/cc7 0 2024-04-12T05:55:38.476 INFO:tasks.workunit.client.0.smithi008.stdout:3/690: rename d9/d10/d19/fd0 to d9/d10/d19/d3a/dad/feb 0 2024-04-12T05:55:38.479 INFO:tasks.workunit.client.0.smithi008.stdout:0/663: getdents d2/d24/dbd 0 2024-04-12T05:55:38.483 INFO:tasks.workunit.client.0.smithi008.stdout:1/728: dwrite d4/d6/d11/f42 [0,4194304] 0 2024-04-12T05:55:38.487 INFO:tasks.workunit.client.0.smithi008.stdout:7/683: dwrite d1/d15/d43/d54/d58/db7/fac [0,4194304] 0 2024-04-12T05:55:38.488 INFO:tasks.workunit.client.0.smithi008.stdout:4/612: dwrite d1/d22/d29/d30/d47/d28/f5c [0,4194304] 0 2024-04-12T05:55:38.489 INFO:tasks.workunit.client.0.smithi008.stdout:1/729: write d4/f9 [2846265,98233] 0 2024-04-12T05:55:38.491 INFO:tasks.workunit.client.0.smithi008.stdout:1/730: chown d4/d33/d74/d7a 41797 1 2024-04-12T05:55:38.492 INFO:tasks.workunit.client.0.smithi008.stdout:0/664: dread d2/d24/d39/f60 [0,4194304] 0 2024-04-12T05:55:38.492 INFO:tasks.workunit.client.0.smithi008.stdout:3/691: write d9/d10/d19/d22/d2c/f83 [59888,97038] 0 2024-04-12T05:55:38.494 INFO:tasks.workunit.client.0.smithi008.stdout:7/684: fsync d1/d15/d43/d54/d77/f89 0 2024-04-12T05:55:38.495 INFO:tasks.workunit.client.0.smithi008.stdout:4/613: truncate d1/d22/d29/d30/d47/d28/d4a/f95 780486 0 2024-04-12T05:55:38.496 INFO:tasks.workunit.client.0.smithi008.stdout:1/731: chown d4/dd/d17/f31 9672 1 2024-04-12T05:55:38.499 INFO:tasks.workunit.client.0.smithi008.stdout:8/695: sync 2024-04-12T05:55:38.499 INFO:tasks.workunit.client.0.smithi008.stdout:3/692: mkdir d9/d10/d19/d22/dec 0 2024-04-12T05:55:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:0/665: creat d2/d11/d36/d42/fcf x:0 0 0 2024-04-12T05:55:38.506 INFO:tasks.workunit.client.0.smithi008.stdout:8/696: chown dd/de/d16/d50/d56/ca1 84 1 2024-04-12T05:55:38.508 INFO:tasks.workunit.client.0.smithi008.stdout:4/614: mknod d1/d22/d29/d30/d47/d28/d4a/d62/cb0 0 2024-04-12T05:55:38.515 INFO:tasks.workunit.client.0.smithi008.stdout:7/685: creat d1/d5/da/d78/dab/fe8 x:0 0 0 2024-04-12T05:55:38.518 INFO:tasks.workunit.client.0.smithi008.stdout:4/615: symlink d1/d22/d29/d30/d47/d28/d74/lb1 0 2024-04-12T05:55:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:7/686: read d1/d15/f59 [629260,45826] 0 2024-04-12T05:55:38.521 INFO:tasks.workunit.client.0.smithi008.stdout:1/732: getdents d4/dd/d21/d98 0 2024-04-12T05:55:38.523 INFO:tasks.workunit.client.0.smithi008.stdout:8/697: getdents dd/de/d16/db1/d97 0 2024-04-12T05:55:38.524 INFO:tasks.workunit.client.0.smithi008.stdout:4/616: symlink d1/d22/d29/d30/d47/d28/d74/lb2 0 2024-04-12T05:55:38.525 INFO:tasks.workunit.client.0.smithi008.stdout:1/733: read d4/d33/d74/d7a/fdd [2613706,127511] 0 2024-04-12T05:55:38.531 INFO:tasks.workunit.client.0.smithi008.stdout:8/698: mkdir dd/de/d4c/d66/d6c/df4 0 2024-04-12T05:55:38.532 INFO:tasks.workunit.client.0.smithi008.stdout:4/617: creat d1/d22/d29/d30/d47/d28/d74/fb3 x:0 0 0 2024-04-12T05:55:38.536 INFO:tasks.workunit.client.0.smithi008.stdout:1/734: unlink d4/dd/d17/d26/f36 0 2024-04-12T05:55:38.537 INFO:tasks.workunit.client.0.smithi008.stdout:0/666: dwrite d2/d11/d2a/f73 [0,4194304] 0 2024-04-12T05:55:38.537 INFO:tasks.workunit.client.0.smithi008.stdout:4/618: read d1/d8/f23 [3643924,120437] 0 2024-04-12T05:55:38.538 INFO:tasks.workunit.client.0.smithi008.stdout:8/699: mkdir dd/d4f/df5 0 2024-04-12T05:55:38.540 INFO:tasks.workunit.client.0.smithi008.stdout:7/687: dread d1/d15/f59 [0,4194304] 0 2024-04-12T05:55:38.541 INFO:tasks.workunit.client.0.smithi008.stdout:3/693: dread d9/dd/f7f [0,4194304] 0 2024-04-12T05:55:38.541 INFO:tasks.workunit.client.0.smithi008.stdout:4/619: chown d1/db/c6f 45 1 2024-04-12T05:55:38.542 INFO:tasks.workunit.client.0.smithi008.stdout:8/700: chown dd/de/d16/d50/d5e/lc1 10 1 2024-04-12T05:55:38.543 INFO:tasks.workunit.client.0.smithi008.stdout:1/735: unlink d4/d33/d74/d7a/db6/dc9/ffe 0 2024-04-12T05:55:38.547 INFO:tasks.workunit.client.0.smithi008.stdout:1/736: read d4/d33/d37/f55 [2397261,19084] 0 2024-04-12T05:55:38.547 INFO:tasks.workunit.client.0.smithi008.stdout:3/694: rmdir d9/d10/d19/d3a/d67 39 2024-04-12T05:55:38.550 INFO:tasks.workunit.client.0.smithi008.stdout:8/701: write dd/d4f/d52/f69 [1918147,86591] 0 2024-04-12T05:55:38.553 INFO:tasks.workunit.client.0.smithi008.stdout:7/688: creat d1/fe9 x:0 0 0 2024-04-12T05:55:38.553 INFO:tasks.workunit.client.0.smithi008.stdout:0/667: dwrite d2/d11/d36/d82/d95/fb3 [0,4194304] 0 2024-04-12T05:55:38.557 INFO:tasks.workunit.client.0.smithi008.stdout:7/689: write d1/fe9 [164843,81921] 0 2024-04-12T05:55:38.558 INFO:tasks.workunit.client.0.smithi008.stdout:4/620: write d1/f20 [1425024,41182] 0 2024-04-12T05:55:38.560 INFO:tasks.workunit.client.0.smithi008.stdout:3/695: dread - d9/d2e/f59 zero size 2024-04-12T05:55:38.561 INFO:tasks.workunit.client.0.smithi008.stdout:0/668: creat d2/d4b/da2/fd0 x:0 0 0 2024-04-12T05:55:38.562 INFO:tasks.workunit.client.0.smithi008.stdout:8/702: mkdir dd/de/d16/d50/d5e/dbe/dd6/df6 0 2024-04-12T05:55:38.563 INFO:tasks.workunit.client.0.smithi008.stdout:4/621: mknod d1/d8/cb4 0 2024-04-12T05:55:38.564 INFO:tasks.workunit.client.0.smithi008.stdout:7/690: creat d1/fea x:0 0 0 2024-04-12T05:55:38.566 INFO:tasks.workunit.client.0.smithi008.stdout:1/737: dwrite d4/dd/d17/d26/fa9 [0,4194304] 0 2024-04-12T05:55:38.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:38 smithi194 ceph-mon[49771]: pgmap v15: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 48 MiB/s rd, 135 MiB/s wr, 459 op/s 2024-04-12T05:55:38.569 INFO:tasks.workunit.client.0.smithi008.stdout:3/696: dread d9/d10/d19/d22/d2c/f83 [0,4194304] 0 2024-04-12T05:55:38.572 INFO:tasks.workunit.client.0.smithi008.stdout:7/691: creat d1/d5/da/d1d/dc3/feb x:0 0 0 2024-04-12T05:55:38.576 INFO:tasks.workunit.client.0.smithi008.stdout:8/703: getdents dd/de/d16/db1 0 2024-04-12T05:55:38.580 INFO:tasks.workunit.client.0.smithi008.stdout:0/669: dwrite d2/d11/d36/d42/f62 [0,4194304] 0 2024-04-12T05:55:38.583 INFO:tasks.workunit.client.0.smithi008.stdout:7/692: rename d1/d5/da/d3c/d44/fa2 to d1/d49/fec 0 2024-04-12T05:55:38.585 INFO:tasks.workunit.client.0.smithi008.stdout:0/670: creat d2/d11/d2a/db5/fd1 x:0 0 0 2024-04-12T05:55:38.586 INFO:tasks.workunit.client.0.smithi008.stdout:0/671: chown d2 1052729471 1 2024-04-12T05:55:38.587 INFO:tasks.workunit.client.0.smithi008.stdout:1/738: rename d4/dd/d21/d9f/dac to d4/dd/d21/d9f/d100 0 2024-04-12T05:55:38.587 INFO:tasks.workunit.client.0.smithi008.stdout:3/697: dwrite d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:38.589 INFO:tasks.workunit.client.0.smithi008.stdout:8/704: mkdir dd/de/d16/df7 0 2024-04-12T05:55:38.592 INFO:tasks.workunit.client.0.smithi008.stdout:0/672: creat d2/d24/d85/fd2 x:0 0 0 2024-04-12T05:55:38.593 INFO:tasks.workunit.client.0.smithi008.stdout:1/739: dread d4/d24/f32 [0,4194304] 0 2024-04-12T05:55:38.595 INFO:tasks.workunit.client.0.smithi008.stdout:7/693: truncate d1/d5/da/d11/f65 4484535 0 2024-04-12T05:55:38.596 INFO:tasks.workunit.client.0.smithi008.stdout:6/727: sync 2024-04-12T05:55:38.598 INFO:tasks.workunit.client.0.smithi008.stdout:4/622: dwrite d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:38.598 INFO:tasks.workunit.client.0.smithi008.stdout:1/740: mknod d4/dd/dba/c101 0 2024-04-12T05:55:38.602 INFO:tasks.workunit.client.0.smithi008.stdout:0/673: mknod d2/d4b/cd3 0 2024-04-12T05:55:38.602 INFO:tasks.workunit.client.0.smithi008.stdout:3/698: creat d9/d10/d19/fed x:0 0 0 2024-04-12T05:55:38.604 INFO:tasks.workunit.client.0.smithi008.stdout:7/694: creat d1/d5/da/d60/d7b/fed x:0 0 0 2024-04-12T05:55:38.606 INFO:tasks.workunit.client.0.smithi008.stdout:0/674: dread d2/d11/d36/d47/fb2 [0,4194304] 0 2024-04-12T05:55:38.611 INFO:tasks.workunit.client.0.smithi008.stdout:1/741: truncate d4/d6/d90/f8d 275728 0 2024-04-12T05:55:38.612 INFO:tasks.workunit.client.0.smithi008.stdout:6/728: symlink df/d10/lf5 0 2024-04-12T05:55:38.612 INFO:tasks.workunit.client.0.smithi008.stdout:7/695: dread d1/d5/da/f62 [0,4194304] 0 2024-04-12T05:55:38.614 INFO:tasks.workunit.client.0.smithi008.stdout:1/742: chown d4/d6/f40 15 1 2024-04-12T05:55:38.614 INFO:tasks.workunit.client.0.smithi008.stdout:3/699: fsync d9/d11/d5b/fcd 0 2024-04-12T05:55:38.617 INFO:tasks.workunit.client.0.smithi008.stdout:3/700: read - d9/d2e/d3d/fc6 zero size 2024-04-12T05:55:38.624 INFO:tasks.workunit.client.0.smithi008.stdout:7/696: rmdir d1/d5/da/d60 39 2024-04-12T05:55:38.626 INFO:tasks.workunit.client.0.smithi008.stdout:0/675: creat d2/d11/d36/d82/fd4 x:0 0 0 2024-04-12T05:55:38.628 INFO:tasks.workunit.client.0.smithi008.stdout:8/705: dwrite dd/f49 [0,4194304] 0 2024-04-12T05:55:38.629 INFO:tasks.workunit.client.0.smithi008.stdout:4/623: truncate d1/f40 685493 0 2024-04-12T05:55:38.629 INFO:tasks.workunit.client.0.smithi008.stdout:1/743: dread d4/dd/f9a [0,4194304] 0 2024-04-12T05:55:38.634 INFO:tasks.workunit.client.0.smithi008.stdout:6/729: dwrite df/d10/d15/dd0/dd1/fb7 [0,4194304] 0 2024-04-12T05:55:38.634 INFO:tasks.workunit.client.0.smithi008.stdout:1/744: chown d4/f9 308195 1 2024-04-12T05:55:38.642 INFO:tasks.workunit.client.0.smithi008.stdout:7/697: rename d1/d5/da/fb3 to d1/d5/da/d78/dd6/fee 0 2024-04-12T05:55:38.643 INFO:tasks.workunit.client.0.smithi008.stdout:3/701: write d9/d2e/d3d/f56 [2388538,116693] 0 2024-04-12T05:55:38.644 INFO:tasks.workunit.client.0.smithi008.stdout:7/698: readlink d1/d5/l1a 0 2024-04-12T05:55:38.651 INFO:tasks.workunit.client.0.smithi008.stdout:6/730: link df/d10/d20/d3e/faf df/d21/ff6 0 2024-04-12T05:55:38.651 INFO:tasks.workunit.client.0.smithi008.stdout:4/624: dwrite d1/d22/d29/d30/f90 [0,4194304] 0 2024-04-12T05:55:38.652 INFO:tasks.workunit.client.0.smithi008.stdout:7/699: chown d1/d5/da/d78 11510 1 2024-04-12T05:55:38.654 INFO:tasks.workunit.client.0.smithi008.stdout:2/810: sync 2024-04-12T05:55:38.654 INFO:tasks.workunit.client.0.smithi008.stdout:9/792: sync 2024-04-12T05:55:38.655 INFO:tasks.workunit.client.0.smithi008.stdout:3/702: mkdir d9/d2e/dee 0 2024-04-12T05:55:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:2/811: chown d6/db/d10/d23/d35/d3c/l62 62567088 1 2024-04-12T05:55:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:6/731: creat df/d8c/db1/ddc/d78/ff7 x:0 0 0 2024-04-12T05:55:38.665 INFO:tasks.workunit.client.0.smithi008.stdout:1/745: write d4/d7/f4e [879360,7233] 0 2024-04-12T05:55:38.667 INFO:tasks.workunit.client.0.smithi008.stdout:9/793: unlink d3/df/d13/d21/d2d/d75/fd3 0 2024-04-12T05:55:38.676 INFO:tasks.workunit.client.0.smithi008.stdout:0/676: dwrite d2/d11/d36/d47/f8b [0,4194304] 0 2024-04-12T05:55:38.678 INFO:tasks.workunit.client.0.smithi008.stdout:9/794: creat d3/d22/f10e x:0 0 0 2024-04-12T05:55:38.678 INFO:tasks.workunit.client.0.smithi008.stdout:4/625: link d1/l87 d1/d22/d29/lb5 0 2024-04-12T05:55:38.682 INFO:tasks.workunit.client.0.smithi008.stdout:8/706: dwrite dd/de/d4c/d66/d6d/fde [0,4194304] 0 2024-04-12T05:55:38.688 INFO:tasks.workunit.client.0.smithi008.stdout:2/812: dwrite d6/db/f9e [0,4194304] 0 2024-04-12T05:55:38.690 INFO:tasks.workunit.client.0.smithi008.stdout:7/700: link d1/d49/fec d1/d15/d43/fef 0 2024-04-12T05:55:38.692 INFO:tasks.workunit.client.0.smithi008.stdout:0/677: readlink d2/d11/d2a/l43 0 2024-04-12T05:55:38.692 INFO:tasks.workunit.client.0.smithi008.stdout:4/626: creat d1/d22/d29/d46/fb6 x:0 0 0 2024-04-12T05:55:38.696 INFO:tasks.workunit.client.0.smithi008.stdout:9/795: read d3/df/d13/d21/f76 [326703,49225] 0 2024-04-12T05:55:38.696 INFO:tasks.workunit.client.0.smithi008.stdout:2/813: mknod d6/db/d38/dcf/c111 0 2024-04-12T05:55:38.699 INFO:tasks.workunit.client.0.smithi008.stdout:8/707: chown dd/de/d16/c3d 0 1 2024-04-12T05:55:38.699 INFO:tasks.workunit.client.0.smithi008.stdout:1/746: dwrite d4/d6/d90/fd3 [0,4194304] 0 2024-04-12T05:55:38.704 INFO:tasks.workunit.client.0.smithi008.stdout:0/678: symlink d2/d11/d36/d82/ld5 0 2024-04-12T05:55:38.712 INFO:tasks.workunit.client.0.smithi008.stdout:1/747: fdatasync d4/dd/d17/d26/fe6 0 2024-04-12T05:55:38.712 INFO:tasks.workunit.client.0.smithi008.stdout:2/814: creat d6/db/d60/d57/d102/f112 x:0 0 0 2024-04-12T05:55:38.716 INFO:tasks.workunit.client.0.smithi008.stdout:4/627: dwrite d1/fa0 [0,4194304] 0 2024-04-12T05:55:38.716 INFO:tasks.workunit.client.0.smithi008.stdout:3/703: dwrite d9/f25 [0,4194304] 0 2024-04-12T05:55:38.719 INFO:tasks.workunit.client.0.smithi008.stdout:0/679: mkdir d2/d11/d36/dd6 0 2024-04-12T05:55:38.721 INFO:tasks.workunit.client.0.smithi008.stdout:7/701: dread d1/d5/da/d1d/d68/fa6 [0,4194304] 0 2024-04-12T05:55:38.724 INFO:tasks.workunit.client.0.smithi008.stdout:2/815: unlink d6/db/d10/d23/f5c 0 2024-04-12T05:55:38.724 INFO:tasks.workunit.client.0.smithi008.stdout:4/628: mkdir d1/db/db7 0 2024-04-12T05:55:38.726 INFO:tasks.workunit.client.0.smithi008.stdout:1/748: readlink d4/d33/l9d 0 2024-04-12T05:55:38.726 INFO:tasks.workunit.client.0.smithi008.stdout:9/796: dread d3/df/d13/d21/d5c/d8f/dab/fc3 [0,4194304] 0 2024-04-12T05:55:38.728 INFO:tasks.workunit.client.0.smithi008.stdout:0/680: mkdir d2/d11/d36/d42/da6/dd7 0 2024-04-12T05:55:38.729 INFO:tasks.workunit.client.0.smithi008.stdout:4/629: write d1/d8/f9e [706299,103301] 0 2024-04-12T05:55:38.730 INFO:tasks.workunit.client.0.smithi008.stdout:1/749: readlink d4/dd/d17/d26/l79 0 2024-04-12T05:55:38.732 INFO:tasks.workunit.client.0.smithi008.stdout:1/750: stat d4/d33/c8e 0 2024-04-12T05:55:38.733 INFO:tasks.workunit.client.0.smithi008.stdout:9/797: chown d3/df/d13/d21/d2d/d75 498155 1 2024-04-12T05:55:38.736 INFO:tasks.workunit.client.0.smithi008.stdout:0/681: readlink d2/d11/d36/d82/ld5 0 2024-04-12T05:55:38.737 INFO:tasks.workunit.client.0.smithi008.stdout:2/816: rename d6/db/d10/d23/d35/d3c to d6/db/d6f/de2/d113 0 2024-04-12T05:55:38.737 INFO:tasks.workunit.client.0.smithi008.stdout:3/704: truncate d9/d1e/f64 3250787 0 2024-04-12T05:55:38.745 INFO:tasks.workunit.client.0.smithi008.stdout:4/630: mknod d1/d22/d29/d30/d47/d28/cb8 0 2024-04-12T05:55:38.748 INFO:tasks.workunit.client.0.smithi008.stdout:3/705: dread d9/d10/d19/d22/da8/fdf [0,4194304] 0 2024-04-12T05:55:38.751 INFO:tasks.workunit.client.0.smithi008.stdout:7/702: dwrite d1/d5/da/d60/da1/fdc [0,4194304] 0 2024-04-12T05:55:38.753 INFO:tasks.workunit.client.0.smithi008.stdout:8/708: dread dd/de/f47 [0,4194304] 0 2024-04-12T05:55:38.753 INFO:tasks.workunit.client.0.smithi008.stdout:9/798: write d3/df/d13/d21/f91 [1749642,8641] 0 2024-04-12T05:55:38.759 INFO:tasks.workunit.client.0.smithi008.stdout:2/817: creat d6/db/d38/f114 x:0 0 0 2024-04-12T05:55:38.760 INFO:tasks.workunit.client.0.smithi008.stdout:9/799: fdatasync d3/d22/d2b/d77/d82/fbc 0 2024-04-12T05:55:38.761 INFO:tasks.workunit.client.0.smithi008.stdout:7/703: write d1/d15/d43/d54/d58/db7/fac [1670625,72868] 0 2024-04-12T05:55:38.764 INFO:tasks.workunit.client.0.smithi008.stdout:0/682: truncate d2/d24/d39/f53 647087 0 2024-04-12T05:55:38.765 INFO:tasks.workunit.client.0.smithi008.stdout:0/683: chown d2/d11/d2a/f4a 316117 1 2024-04-12T05:55:38.768 INFO:tasks.workunit.client.0.smithi008.stdout:1/751: dwrite d4/d24/faa [0,4194304] 0 2024-04-12T05:55:38.774 INFO:tasks.workunit.client.0.smithi008.stdout:2/818: creat d6/db/d38/dcf/f115 x:0 0 0 2024-04-12T05:55:38.774 INFO:tasks.workunit.client.0.smithi008.stdout:9/800: creat d3/df/d13/d21/d5c/d8f/f10f x:0 0 0 2024-04-12T05:55:38.776 INFO:tasks.workunit.client.0.smithi008.stdout:2/819: chown d6/f6d 125 1 2024-04-12T05:55:38.777 INFO:tasks.workunit.client.0.smithi008.stdout:2/820: chown d6/db/d38/dcf/lfe 240525070 1 2024-04-12T05:55:38.779 INFO:tasks.workunit.client.0.smithi008.stdout:8/709: write dd/de/d16/f28 [2636430,33928] 0 2024-04-12T05:55:38.779 INFO:tasks.workunit.client.0.smithi008.stdout:0/684: readlink d2/d11/d36/d42/l7a 0 2024-04-12T05:55:38.781 INFO:tasks.workunit.client.0.smithi008.stdout:2/821: write d6/db/d38/dcf/fd3 [526438,94174] 0 2024-04-12T05:55:38.782 INFO:tasks.workunit.client.0.smithi008.stdout:9/801: fdatasync d3/d22/d2b/fea 0 2024-04-12T05:55:38.784 INFO:tasks.workunit.client.0.smithi008.stdout:9/802: fdatasync d3/df/d13/d21/d5c/da4/db5/fc9 0 2024-04-12T05:55:38.785 INFO:tasks.workunit.client.0.smithi008.stdout:7/704: getdents d1/d5/da/d1d/dc3 0 2024-04-12T05:55:38.788 INFO:tasks.workunit.client.0.smithi008.stdout:0/685: creat d2/d11/d36/d42/da6/fd8 x:0 0 0 2024-04-12T05:55:38.798 INFO:tasks.workunit.client.0.smithi008.stdout:8/710: dwrite dd/de/d4c/fcb [0,4194304] 0 2024-04-12T05:55:38.803 INFO:tasks.workunit.client.0.smithi008.stdout:3/706: dread d9/f82 [0,4194304] 0 2024-04-12T05:55:38.804 INFO:tasks.workunit.client.0.smithi008.stdout:7/705: dwrite d1/d5/da/d3c/fc1 [0,4194304] 0 2024-04-12T05:55:38.812 INFO:tasks.workunit.client.0.smithi008.stdout:8/711: chown dd/d30/d32/fb7 172113 1 2024-04-12T05:55:38.828 INFO:tasks.workunit.client.0.smithi008.stdout:7/706: dread d1/d5/da/d11/fbc [0,4194304] 0 2024-04-12T05:55:38.837 INFO:tasks.workunit.client.0.smithi008.stdout:2/822: mkdir d6/db/d60/d57/d102/d116 0 2024-04-12T05:55:38.839 INFO:tasks.workunit.client.0.smithi008.stdout:1/752: link d4/d1c/db9/cc4 d4/d33/d74/d7a/db6/c102 0 2024-04-12T05:55:38.839 INFO:tasks.workunit.client.0.smithi008.stdout:2/823: truncate d6/db/f9e 4784201 0 2024-04-12T05:55:38.848 INFO:tasks.workunit.client.0.smithi008.stdout:9/803: unlink d3/df/d13/d14/d95/cc2 0 2024-04-12T05:55:38.853 INFO:tasks.workunit.client.0.smithi008.stdout:8/712: mknod dd/d30/d3b/de4/cf8 0 2024-04-12T05:55:38.856 INFO:tasks.workunit.client.0.smithi008.stdout:8/713: write dd/de/d16/f4a [1451969,108328] 0 2024-04-12T05:55:38.863 INFO:tasks.workunit.client.0.smithi008.stdout:9/804: mknod d3/d22/d2b/d58/d36/c110 0 2024-04-12T05:55:38.865 INFO:tasks.workunit.client.0.smithi008.stdout:1/753: dread d4/d7/fe [0,4194304] 0 2024-04-12T05:55:38.869 INFO:tasks.workunit.client.0.smithi008.stdout:2/824: dread d6/db/d10/d23/f4d [4194304,4194304] 0 2024-04-12T05:55:38.871 INFO:tasks.workunit.client.0.smithi008.stdout:3/707: creat d9/dd/fef x:0 0 0 2024-04-12T05:55:38.876 INFO:tasks.workunit.client.0.smithi008.stdout:3/708: dread d9/f25 [0,4194304] 0 2024-04-12T05:55:38.877 INFO:tasks.workunit.client.0.smithi008.stdout:3/709: stat d9/dd/c31 0 2024-04-12T05:55:38.877 INFO:tasks.workunit.client.0.smithi008.stdout:9/805: symlink d3/df0/l111 0 2024-04-12T05:55:38.879 INFO:tasks.workunit.client.0.smithi008.stdout:2/825: unlink d6/db/d38/d41/c4a 0 2024-04-12T05:55:38.879 INFO:tasks.workunit.client.0.smithi008.stdout:9/806: dread - d3/d22/d2b/fe0 zero size 2024-04-12T05:55:38.882 INFO:tasks.workunit.client.0.smithi008.stdout:2/826: dread d6/db/d38/f7e [0,4194304] 0 2024-04-12T05:55:38.885 INFO:tasks.workunit.client.0.smithi008.stdout:9/807: symlink d3/df/d13/d21/d5c/da4/db5/l112 0 2024-04-12T05:55:38.887 INFO:tasks.workunit.client.0.smithi008.stdout:1/754: creat d4/dd/d17/f103 x:0 0 0 2024-04-12T05:55:38.897 INFO:tasks.workunit.client.0.smithi008.stdout:1/755: dwrite d4/d1c/d4f/f6a [4194304,4194304] 0 2024-04-12T05:55:38.920 INFO:tasks.workunit.client.0.smithi008.stdout:0/686: truncate d2/d11/fb0 388535 0 2024-04-12T05:55:38.927 INFO:tasks.workunit.client.0.smithi008.stdout:9/808: write d3/d22/d2b/d58/f55 [1606175,87810] 0 2024-04-12T05:55:38.928 INFO:tasks.workunit.client.0.smithi008.stdout:7/707: creat d1/d5/da/d1d/ff0 x:0 0 0 2024-04-12T05:55:38.929 INFO:tasks.workunit.client.0.smithi008.stdout:9/809: write d3/df/d13/d21/d5c/da4/fd4 [902570,91292] 0 2024-04-12T05:55:38.931 INFO:tasks.workunit.client.0.smithi008.stdout:7/708: read d1/f6 [3873199,119160] 0 2024-04-12T05:55:38.931 INFO:tasks.workunit.client.0.smithi008.stdout:2/827: symlink d6/db/d10/d23/l117 0 2024-04-12T05:55:38.934 INFO:tasks.workunit.client.0.smithi008.stdout:7/709: read d1/d15/fc8 [710759,27462] 0 2024-04-12T05:55:38.934 INFO:tasks.workunit.client.0.smithi008.stdout:2/828: chown d6/db/d60/d57/dd9/fe0 479616236 1 2024-04-12T05:55:38.934 INFO:tasks.workunit.client.0.smithi008.stdout:8/714: rename dd/f36 to dd/de/d4c/d66/ff9 0 2024-04-12T05:55:38.938 INFO:tasks.workunit.client.0.smithi008.stdout:3/710: rename d9/d2e/c6d to d9/d10/d19/d3a/d7c/cf0 0 2024-04-12T05:55:38.949 INFO:tasks.workunit.client.0.smithi008.stdout:1/756: rename d4/d7/f56 to d4/d33/d74/d30/d72/d7c/dd4/dd5/f104 0 2024-04-12T05:55:38.950 INFO:tasks.workunit.client.0.smithi008.stdout:9/810: getdents d3/df/d13/d21/d5c/da4 0 2024-04-12T05:55:38.952 INFO:tasks.workunit.client.0.smithi008.stdout:9/811: write d3/d22/d2b/d58/fdf [559685,7440] 0 2024-04-12T05:55:38.955 INFO:tasks.workunit.client.0.smithi008.stdout:3/711: symlink d9/db7/lf1 0 2024-04-12T05:55:38.959 INFO:tasks.workunit.client.0.smithi008.stdout:3/712: write d9/d2e/f3f [8616403,68863] 0 2024-04-12T05:55:38.963 INFO:tasks.workunit.client.0.smithi008.stdout:0/687: dwrite d2/d11/f23 [0,4194304] 0 2024-04-12T05:55:38.964 INFO:tasks.workunit.client.0.smithi008.stdout:8/715: dread dd/de/d16/f33 [0,4194304] 0 2024-04-12T05:55:38.964 INFO:tasks.workunit.client.0.smithi008.stdout:9/812: getdents d3/d22/d2b/d77 0 2024-04-12T05:55:38.969 INFO:tasks.workunit.client.0.smithi008.stdout:8/716: dread - dd/de/d16/fba zero size 2024-04-12T05:55:38.970 INFO:tasks.workunit.client.0.smithi008.stdout:2/829: dwrite d6/db/d38/d41/fad [0,4194304] 0 2024-04-12T05:55:38.972 INFO:tasks.workunit.client.0.smithi008.stdout:9/813: fsync d3/d22/d2b/d58/d38/f54 0 2024-04-12T05:55:38.976 INFO:tasks.workunit.client.0.smithi008.stdout:0/688: getdents d2/d4b/da2 0 2024-04-12T05:55:38.977 INFO:tasks.workunit.client.0.smithi008.stdout:7/710: dread d1/d5/da/d3c/d44/f6f [0,4194304] 0 2024-04-12T05:55:38.984 INFO:tasks.workunit.client.0.smithi008.stdout:0/689: creat d2/d11/d36/d47/fd9 x:0 0 0 2024-04-12T05:55:38.985 INFO:tasks.workunit.client.0.smithi008.stdout:7/711: mkdir d1/d5/da/d1d/d68/df1 0 2024-04-12T05:55:38.985 INFO:tasks.workunit.client.0.smithi008.stdout:0/690: dread - d2/d24/d39/fae zero size 2024-04-12T05:55:38.985 INFO:tasks.workunit.client.0.smithi008.stdout:8/717: unlink dd/de/d16/d50/d56/d61/ldf 0 2024-04-12T05:55:38.987 INFO:tasks.workunit.client.0.smithi008.stdout:7/712: creat d1/d5/da/d78/dab/ff2 x:0 0 0 2024-04-12T05:55:38.988 INFO:tasks.workunit.client.0.smithi008.stdout:8/718: dread - dd/d65/fd4 zero size 2024-04-12T05:55:38.989 INFO:tasks.workunit.client.0.smithi008.stdout:8/719: fdatasync dd/f22 0 2024-04-12T05:55:38.990 INFO:tasks.workunit.client.0.smithi008.stdout:2/830: dwrite d6/db/f69 [0,4194304] 0 2024-04-12T05:55:38.996 INFO:tasks.workunit.client.0.smithi008.stdout:2/831: dread - d6/db/d38/d41/fa5 zero size 2024-04-12T05:55:38.997 INFO:tasks.workunit.client.0.smithi008.stdout:9/814: dread d3/fa [0,4194304] 0 2024-04-12T05:55:38.998 INFO:tasks.workunit.client.0.smithi008.stdout:9/815: chown d3/df/d13/d14/l5f 854 1 2024-04-12T05:55:39.000 INFO:tasks.workunit.client.0.smithi008.stdout:2/832: creat d6/db/d10/d7b/f118 x:0 0 0 2024-04-12T05:55:39.002 INFO:tasks.workunit.client.0.smithi008.stdout:0/691: dread d2/fc1 [0,4194304] 0 2024-04-12T05:55:39.004 INFO:tasks.workunit.client.0.smithi008.stdout:3/713: dread d9/d2e/d3d/f73 [0,4194304] 0 2024-04-12T05:55:39.005 INFO:tasks.workunit.client.0.smithi008.stdout:9/816: dread d3/df/d13/d21/d2d/d30/faa [0,4194304] 0 2024-04-12T05:55:39.007 INFO:tasks.workunit.client.0.smithi008.stdout:3/714: rmdir d9/d11/d5b 39 2024-04-12T05:55:39.008 INFO:tasks.workunit.client.0.smithi008.stdout:0/692: creat d2/d11/d2a/fda x:0 0 0 2024-04-12T05:55:39.011 INFO:tasks.workunit.client.0.smithi008.stdout:7/713: dread d1/d5/da/d11/f35 [12582912,4194304] 0 2024-04-12T05:55:39.011 INFO:tasks.workunit.client.0.smithi008.stdout:3/715: truncate d9/f82 1960592 0 2024-04-12T05:55:39.013 INFO:tasks.workunit.client.0.smithi008.stdout:9/817: symlink d3/df/d13/d21/d2d/dbb/l113 0 2024-04-12T05:55:39.014 INFO:tasks.workunit.client.0.smithi008.stdout:7/714: truncate d1/d15/fe6 938777 0 2024-04-12T05:55:39.016 INFO:tasks.workunit.client.0.smithi008.stdout:9/818: mkdir d3/d6a/d114 0 2024-04-12T05:55:39.019 INFO:tasks.workunit.client.0.smithi008.stdout:3/716: creat d9/d10/d19/d22/da8/dc4/ff2 x:0 0 0 2024-04-12T05:55:39.020 INFO:tasks.workunit.client.0.smithi008.stdout:3/717: chown d9/d10/d19/d22/l77 103004 1 2024-04-12T05:55:39.022 INFO:tasks.workunit.client.0.smithi008.stdout:3/718: dread d9/d11/f49 [4194304,4194304] 0 2024-04-12T05:55:39.023 INFO:tasks.workunit.client.0.smithi008.stdout:9/819: mkdir d3/df/d13/d21/d2d/d30/d74/d115 0 2024-04-12T05:55:39.026 INFO:tasks.workunit.client.0.smithi008.stdout:8/720: dread dd/d30/d32/fc4 [4194304,4194304] 0 2024-04-12T05:55:39.028 INFO:tasks.workunit.client.0.smithi008.stdout:8/721: write dd/d30/d32/fbd [781855,106673] 0 2024-04-12T05:55:39.029 INFO:tasks.workunit.client.0.smithi008.stdout:3/719: creat d9/d10/d19/d3a/d67/ff3 x:0 0 0 2024-04-12T05:55:39.030 INFO:tasks.workunit.client.0.smithi008.stdout:0/693: truncate d2/d11/d36/d47/f94 2196144 0 2024-04-12T05:55:39.031 INFO:tasks.workunit.client.0.smithi008.stdout:7/715: write d1/d5/da/f62 [5253172,13516] 0 2024-04-12T05:55:39.035 INFO:tasks.workunit.client.0.smithi008.stdout:8/722: creat dd/de/d16/d50/d56/d61/d9c/ffa x:0 0 0 2024-04-12T05:55:39.036 INFO:tasks.workunit.client.0.smithi008.stdout:0/694: rmdir d2/d11/d36/d82 39 2024-04-12T05:55:39.041 INFO:tasks.workunit.client.0.smithi008.stdout:8/723: symlink dd/de/d4c/lfb 0 2024-04-12T05:55:39.041 INFO:tasks.workunit.client.0.smithi008.stdout:3/720: dwrite d9/d10/d19/d3a/dad/fdd [0,4194304] 0 2024-04-12T05:55:39.045 INFO:tasks.workunit.client.0.smithi008.stdout:9/820: write d3/d22/d2b/d58/d36/f3c [1122961,98463] 0 2024-04-12T05:55:39.046 INFO:tasks.workunit.client.0.smithi008.stdout:3/721: unlink d9/d2e/d3d/f73 0 2024-04-12T05:55:39.047 INFO:tasks.workunit.client.0.smithi008.stdout:8/724: creat dd/de/d16/d50/d56/d61/d9c/ffc x:0 0 0 2024-04-12T05:55:39.049 INFO:tasks.workunit.client.0.smithi008.stdout:3/722: write d9/d37/d69/fe5 [672948,59977] 0 2024-04-12T05:55:39.050 INFO:tasks.workunit.client.0.smithi008.stdout:9/821: symlink d3/d22/d2b/l116 0 2024-04-12T05:55:39.055 INFO:tasks.workunit.client.0.smithi008.stdout:8/725: rename dd/d4f/l5a to dd/d30/d3b/lfd 0 2024-04-12T05:55:39.059 INFO:tasks.workunit.client.0.smithi008.stdout:7/716: dwrite d1/d5/fc9 [0,4194304] 0 2024-04-12T05:55:39.061 INFO:tasks.workunit.client.0.smithi008.stdout:8/726: dread - dd/d30/d3b/db5/df0/fe6 zero size 2024-04-12T05:55:39.061 INFO:tasks.workunit.client.0.smithi008.stdout:9/822: unlink d3/df/d13/d14/le7 0 2024-04-12T05:55:39.062 INFO:tasks.workunit.client.0.smithi008.stdout:8/727: fsync dd/f4e 0 2024-04-12T05:55:39.065 INFO:tasks.workunit.client.0.smithi008.stdout:9/823: fdatasync d3/d22/d2b/d4e/d60/f6d 0 2024-04-12T05:55:39.070 INFO:tasks.workunit.client.0.smithi008.stdout:3/723: dwrite d9/d10/d19/d22/da8/fb2 [0,4194304] 0 2024-04-12T05:55:39.077 INFO:tasks.workunit.client.0.smithi008.stdout:7/717: write d1/fb8 [988362,85777] 0 2024-04-12T05:55:39.078 INFO:tasks.workunit.client.0.smithi008.stdout:3/724: chown d9/d44/f63 2472 1 2024-04-12T05:55:39.078 INFO:tasks.workunit.client.0.smithi008.stdout:8/728: truncate dd/f1f 528204 0 2024-04-12T05:55:39.079 INFO:tasks.workunit.client.0.smithi008.stdout:9/824: dwrite d3/df/d13/d21/d5c/dd7/fbe [0,4194304] 0 2024-04-12T05:55:39.079 INFO:tasks.workunit.client.0.smithi008.stdout:3/725: chown d9/d10/d19/d3a/d7c 3073 1 2024-04-12T05:55:39.084 INFO:tasks.workunit.client.0.smithi008.stdout:7/718: creat d1/d5/da/d3c/dd3/ff3 x:0 0 0 2024-04-12T05:55:39.093 INFO:tasks.workunit.client.0.smithi008.stdout:7/719: truncate d1/d5/da/d11/f65 4412562 0 2024-04-12T05:55:39.098 INFO:tasks.workunit.client.0.smithi008.stdout:3/726: dread d9/d10/f6a [0,4194304] 0 2024-04-12T05:55:39.101 INFO:tasks.workunit.client.0.smithi008.stdout:3/727: mknod d9/d10/d19/cf4 0 2024-04-12T05:55:39.103 INFO:tasks.workunit.client.0.smithi008.stdout:3/728: mknod d9/d44/cf5 0 2024-04-12T05:55:39.104 INFO:tasks.workunit.client.0.smithi008.stdout:3/729: stat d9/d10/d19/d22/lb0 0 2024-04-12T05:55:39.107 INFO:tasks.workunit.client.0.smithi008.stdout:7/720: dread d1/d5/da/d3c/d44/d8a/f9b [0,4194304] 0 2024-04-12T05:55:39.110 INFO:tasks.workunit.client.0.smithi008.stdout:3/730: write d9/d10/f9c [77129,10391] 0 2024-04-12T05:55:39.112 INFO:tasks.workunit.client.0.smithi008.stdout:3/731: chown d9/d10/d4b 1 1 2024-04-12T05:55:39.116 INFO:tasks.workunit.client.0.smithi008.stdout:7/721: dwrite d1/d5/da/d3c/d44/d8a/fa7 [0,4194304] 0 2024-04-12T05:55:39.118 INFO:tasks.workunit.client.0.smithi008.stdout:3/732: creat d9/dd/ff6 x:0 0 0 2024-04-12T05:55:39.122 INFO:tasks.workunit.client.0.smithi008.stdout:7/722: mknod d1/d15/d43/cf4 0 2024-04-12T05:55:39.126 INFO:tasks.workunit.client.0.smithi008.stdout:3/733: dwrite d9/d10/d19/d22/f28 [8388608,4194304] 0 2024-04-12T05:55:39.129 INFO:tasks.workunit.client.0.smithi008.stdout:7/723: rename d1/d5/da/d11/f25 to d1/d5/da/d3c/d44/ff5 0 2024-04-12T05:55:39.130 INFO:tasks.workunit.client.0.smithi008.stdout:7/724: chown d1/d5/da/d60/da1/lce 14083550 1 2024-04-12T05:55:39.132 INFO:tasks.workunit.client.0.smithi008.stdout:7/725: symlink d1/d15/d43/d5a/lf6 0 2024-04-12T05:55:39.134 INFO:tasks.workunit.client.0.smithi008.stdout:7/726: mkdir d1/d15/df7 0 2024-04-12T05:55:39.135 INFO:tasks.workunit.client.0.smithi008.stdout:7/727: creat d1/d15/d43/d54/d58/db7/ff8 x:0 0 0 2024-04-12T05:55:39.142 INFO:tasks.workunit.client.0.smithi008.stdout:7/728: dwrite d1/d5/da/d78/f88 [0,4194304] 0 2024-04-12T05:55:39.152 INFO:tasks.workunit.client.0.smithi008.stdout:7/729: read d1/f3 [453409,26980] 0 2024-04-12T05:55:39.154 INFO:tasks.workunit.client.0.smithi008.stdout:7/730: mknod d1/d5/da/d78/cf9 0 2024-04-12T05:55:39.155 INFO:tasks.workunit.client.0.smithi008.stdout:7/731: readlink d1/d5/da/d1d/l6b 0 2024-04-12T05:55:39.156 INFO:tasks.workunit.client.0.smithi008.stdout:7/732: read d1/d5/da/d78/f74 [5125209,109223] 0 2024-04-12T05:55:39.174 INFO:tasks.workunit.client.0.smithi008.stdout:7/733: dread d1/d5/da/d3c/d44/ff5 [0,4194304] 0 2024-04-12T05:55:39.181 INFO:tasks.workunit.client.0.smithi008.stdout:7/734: dwrite d1/d5/da/d1d/dc3/feb [0,4194304] 0 2024-04-12T05:55:39.265 INFO:tasks.workunit.client.0.smithi008.stdout:7/735: dwrite d1/d5/da/d3c/d44/f6f [0,4194304] 0 2024-04-12T05:55:39.269 INFO:tasks.workunit.client.0.smithi008.stdout:7/736: symlink d1/d15/d43/d54/d58/db7/de3/lfa 0 2024-04-12T05:55:39.271 INFO:tasks.workunit.client.0.smithi008.stdout:7/737: mkdir d1/d5/da/d1d/d82/dfb 0 2024-04-12T05:55:39.273 INFO:tasks.workunit.client.0.smithi008.stdout:7/738: write d1/d15/d43/d54/d58/f67 [1637809,35622] 0 2024-04-12T05:55:39.277 INFO:tasks.workunit.client.0.smithi008.stdout:7/739: mkdir d1/d15/d43/d54/d58/dfc 0 2024-04-12T05:55:39.281 INFO:tasks.workunit.client.0.smithi008.stdout:7/740: link d1/d5/da/d78/f74 d1/ffd 0 2024-04-12T05:55:39.284 INFO:tasks.workunit.client.0.smithi008.stdout:7/741: creat d1/d5/da/d1d/ffe x:0 0 0 2024-04-12T05:55:39.286 INFO:tasks.workunit.client.0.smithi008.stdout:7/742: mkdir d1/d5/da/d60/d7b/dff 0 2024-04-12T05:55:39.287 INFO:tasks.workunit.client.0.smithi008.stdout:7/743: dread - d1/d5/da/d60/fd7 zero size 2024-04-12T05:55:39.292 INFO:tasks.workunit.client.0.smithi008.stdout:7/744: rmdir d1/d5/da/d1d 39 2024-04-12T05:55:39.295 INFO:tasks.workunit.client.0.smithi008.stdout:7/745: creat d1/d15/d43/f100 x:0 0 0 2024-04-12T05:55:39.296 INFO:tasks.workunit.client.0.smithi008.stdout:7/746: dread - d1/d15/d43/d54/d77/f96 zero size 2024-04-12T05:55:39.301 INFO:tasks.workunit.client.0.smithi008.stdout:4/631: sync 2024-04-12T05:55:39.301 INFO:tasks.workunit.client.0.smithi008.stdout:6/732: sync 2024-04-12T05:55:39.303 INFO:tasks.workunit.client.0.smithi008.stdout:7/747: truncate d1/d5/da/d1d/d68/fe0 1062250 0 2024-04-12T05:55:39.304 INFO:tasks.workunit.client.0.smithi008.stdout:4/632: write d1/d4f/f89 [898215,107825] 0 2024-04-12T05:55:39.304 INFO:tasks.workunit.client.0.smithi008.stdout:7/748: chown d1/d15/d43/d54/d77/f96 1127079 1 2024-04-12T05:55:39.305 INFO:tasks.workunit.client.0.smithi008.stdout:6/733: creat df/d10/d15/dd0/ff8 x:0 0 0 2024-04-12T05:55:39.306 INFO:tasks.workunit.client.0.smithi008.stdout:6/734: chown df/d8c/db1/ddc/d9b 372743 1 2024-04-12T05:55:39.308 INFO:tasks.workunit.client.0.smithi008.stdout:6/735: readlink df/d8c/db1/ddc/d78/laa 0 2024-04-12T05:55:39.309 INFO:tasks.workunit.client.0.smithi008.stdout:7/749: rename d1/d5/da/d3c/dd3/ff3 to d1/d5/da/d3c/dd3/f101 0 2024-04-12T05:55:39.310 INFO:tasks.workunit.client.0.smithi008.stdout:4/633: rename d1/d22/d29/d46/d56/c86 to d1/d22/d29/d30/cb9 0 2024-04-12T05:55:39.311 INFO:tasks.workunit.client.0.smithi008.stdout:7/750: truncate d1/d5/da/d78/f74 9265229 0 2024-04-12T05:55:39.311 INFO:tasks.workunit.client.0.smithi008.stdout:6/736: symlink df/d10/d15/d75/lf9 0 2024-04-12T05:55:39.312 INFO:tasks.workunit.client.0.smithi008.stdout:4/634: chown d1/d22/d29/d30/d60/fa3 2 1 2024-04-12T05:55:39.314 INFO:tasks.workunit.client.0.smithi008.stdout:7/751: mknod d1/d5/da/d3c/dd3/c102 0 2024-04-12T05:55:39.315 INFO:tasks.workunit.client.0.smithi008.stdout:4/635: chown d1/d22/la8 13203 1 2024-04-12T05:55:39.316 INFO:tasks.workunit.client.0.smithi008.stdout:7/752: mkdir d1/d5/da/d11/d103 0 2024-04-12T05:55:39.316 INFO:tasks.workunit.client.0.smithi008.stdout:4/636: readlink d1/d22/d29/d46/d56/l81 0 2024-04-12T05:55:39.317 INFO:tasks.workunit.client.0.smithi008.stdout:6/737: mknod df/d10/d15/d4f/de9/cfa 0 2024-04-12T05:55:39.326 INFO:tasks.workunit.client.0.smithi008.stdout:7/753: write d1/d15/d43/d5a/f98 [309120,26925] 0 2024-04-12T05:55:39.327 INFO:tasks.workunit.client.0.smithi008.stdout:4/637: dwrite d1/d8/f15 [0,4194304] 0 2024-04-12T05:55:39.330 INFO:tasks.workunit.client.0.smithi008.stdout:6/738: dwrite df/d10/f62 [0,4194304] 0 2024-04-12T05:55:39.331 INFO:tasks.workunit.client.0.smithi008.stdout:4/638: write d1/d4f/fa5 [1561245,107552] 0 2024-04-12T05:55:39.333 INFO:tasks.workunit.client.0.smithi008.stdout:7/754: creat d1/d5/da/d60/d7b/f104 x:0 0 0 2024-04-12T05:55:39.337 INFO:tasks.workunit.client.0.smithi008.stdout:6/739: fsync df/d21/d4c/f53 0 2024-04-12T05:55:39.340 INFO:tasks.workunit.client.0.smithi008.stdout:6/740: unlink df/d10/d15/d4f/le2 0 2024-04-12T05:55:39.343 INFO:tasks.workunit.client.0.smithi008.stdout:6/741: mkdir df/d21/d4c/dfb 0 2024-04-12T05:55:39.349 INFO:tasks.workunit.client.0.smithi008.stdout:7/755: write d1/fb [1487343,35594] 0 2024-04-12T05:55:39.352 INFO:tasks.workunit.client.0.smithi008.stdout:4/639: dread d1/d4f/f5b [0,4194304] 0 2024-04-12T05:55:39.356 INFO:tasks.workunit.client.0.smithi008.stdout:4/640: rename d1/d8/f9e to d1/db/db7/fba 0 2024-04-12T05:55:39.356 INFO:tasks.workunit.client.0.smithi008.stdout:4/641: chown d1/d8/l5e 11 1 2024-04-12T05:55:39.358 INFO:tasks.workunit.client.0.smithi008.stdout:4/642: dread d1/d8/f42 [0,4194304] 0 2024-04-12T05:55:39.361 INFO:tasks.workunit.client.0.smithi008.stdout:4/643: creat d1/db/db7/fbb x:0 0 0 2024-04-12T05:55:39.363 INFO:tasks.workunit.client.0.smithi008.stdout:6/742: dwrite df/d8c/db1/ddc/d9b/fcb [0,4194304] 0 2024-04-12T05:55:39.364 INFO:tasks.workunit.client.0.smithi008.stdout:4/644: symlink d1/d22/d29/d30/d47/lbc 0 2024-04-12T05:55:39.365 INFO:tasks.workunit.client.0.smithi008.stdout:6/743: stat df/d8c/da4/dad/fd5 0 2024-04-12T05:55:39.369 INFO:tasks.workunit.client.0.smithi008.stdout:4/645: symlink d1/d22/d29/d30/d47/d28/d74/lbd 0 2024-04-12T05:55:39.371 INFO:tasks.workunit.client.0.smithi008.stdout:6/744: truncate df/d10/d20/d3e/f5c 488166 0 2024-04-12T05:55:39.371 INFO:tasks.workunit.client.0.smithi008.stdout:4/646: fsync d1/d22/d29/d30/d47/d28/f5c 0 2024-04-12T05:55:39.372 INFO:tasks.workunit.client.0.smithi008.stdout:4/647: stat d1/d4f/f6c 0 2024-04-12T05:55:39.373 INFO:tasks.workunit.client.0.smithi008.stdout:6/745: mkdir df/d10/d20/d3e/dfc 0 2024-04-12T05:55:39.374 INFO:tasks.workunit.client.0.smithi008.stdout:4/648: fdatasync d1/d22/d29/d30/d47/d28/d74/fb3 0 2024-04-12T05:55:39.377 INFO:tasks.workunit.client.0.smithi008.stdout:6/746: mknod df/d8c/db1/ddc/d78/cfd 0 2024-04-12T05:55:39.381 INFO:tasks.workunit.client.0.smithi008.stdout:4/649: dread d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:39.383 INFO:tasks.workunit.client.0.smithi008.stdout:4/650: mknod d1/d8/cbe 0 2024-04-12T05:55:39.383 INFO:tasks.workunit.client.0.smithi008.stdout:6/747: link df/d10/lf5 df/d10/d15/d79/lfe 0 2024-04-12T05:55:39.384 INFO:tasks.workunit.client.0.smithi008.stdout:4/651: fsync d1/f20 0 2024-04-12T05:55:39.385 INFO:tasks.workunit.client.0.smithi008.stdout:6/748: stat df/d21/ld8 0 2024-04-12T05:55:39.385 INFO:tasks.workunit.client.0.smithi008.stdout:4/652: write d1/d22/d29/d46/d56/f63 [183847,82221] 0 2024-04-12T05:55:39.386 INFO:tasks.workunit.client.0.smithi008.stdout:6/749: fdatasync df/d10/d15/dd0/dd1/fb7 0 2024-04-12T05:55:39.389 INFO:tasks.workunit.client.0.smithi008.stdout:6/750: fdatasync f5 0 2024-04-12T05:55:39.390 INFO:tasks.workunit.client.0.smithi008.stdout:6/751: chown df/d8c/dcc/ddd 118 1 2024-04-12T05:55:39.394 INFO:tasks.workunit.client.0.smithi008.stdout:4/653: rename d1/d22/d29/d30/d47/d28/d4a/d62/f9d to d1/d22/d29/d30/d47/d28/fbf 0 2024-04-12T05:55:39.399 INFO:tasks.workunit.client.0.smithi008.stdout:4/654: mkdir d1/d22/d29/d30/d60/dc0 0 2024-04-12T05:55:39.399 INFO:tasks.workunit.client.0.smithi008.stdout:6/752: dwrite df/d8c/db1/ddc/d9b/fbc [0,4194304] 0 2024-04-12T05:55:39.402 INFO:tasks.workunit.client.0.smithi008.stdout:4/655: rmdir d1/d22/d29/d46/d56 39 2024-04-12T05:55:39.404 INFO:tasks.workunit.client.0.smithi008.stdout:6/753: getdents df/dc1 0 2024-04-12T05:55:39.404 INFO:tasks.workunit.client.0.smithi008.stdout:6/754: chown df/d8c/db1 0 1 2024-04-12T05:55:39.405 INFO:tasks.workunit.client.0.smithi008.stdout:4/656: symlink d1/d22/d29/d30/d47/lc1 0 2024-04-12T05:55:39.406 INFO:tasks.workunit.client.0.smithi008.stdout:6/755: fsync df/d10/d20/d3e/fa1 0 2024-04-12T05:55:39.409 INFO:tasks.workunit.client.0.smithi008.stdout:6/756: mknod df/cff 0 2024-04-12T05:55:39.409 INFO:tasks.workunit.client.0.smithi008.stdout:6/757: chown df/d21/ff6 178482 1 2024-04-12T05:55:39.411 INFO:tasks.workunit.client.0.smithi008.stdout:4/657: rename d1/d22/d29/d30/d60/c83 to d1/d22/d29/d30/cc2 0 2024-04-12T05:55:39.412 INFO:tasks.workunit.client.0.smithi008.stdout:6/758: getdents df/d21/d4c/dfb 0 2024-04-12T05:55:39.415 INFO:tasks.workunit.client.0.smithi008.stdout:4/658: creat d1/d8/fc3 x:0 0 0 2024-04-12T05:55:39.418 INFO:tasks.workunit.client.0.smithi008.stdout:4/659: creat d1/d22/d29/d46/d56/fc4 x:0 0 0 2024-04-12T05:55:39.420 INFO:tasks.workunit.client.0.smithi008.stdout:4/660: mkdir d1/d8/dc5 0 2024-04-12T05:55:39.421 INFO:tasks.workunit.client.0.smithi008.stdout:6/759: write df/f67 [84211,33025] 0 2024-04-12T05:55:39.423 INFO:tasks.workunit.client.0.smithi008.stdout:6/760: chown df/d8c/db1/ddc/d9b/fbc 0 1 2024-04-12T05:55:39.424 INFO:tasks.workunit.client.0.smithi008.stdout:6/761: chown df/d8c/db1/ddc/c4a 0 1 2024-04-12T05:55:39.425 INFO:tasks.workunit.client.0.smithi008.stdout:4/661: link d1/db/c3d d1/d22/d29/d46/d56/cc6 0 2024-04-12T05:55:39.426 INFO:tasks.workunit.client.0.smithi008.stdout:4/662: chown d1/db/db7/fba 64794 1 2024-04-12T05:55:39.427 INFO:tasks.workunit.client.0.smithi008.stdout:6/762: getdents df/d8c/db1/ddc/d78/d8b 0 2024-04-12T05:55:39.431 INFO:tasks.workunit.client.0.smithi008.stdout:6/763: rename f5 to df/d10/d15/d79/f100 0 2024-04-12T05:55:39.433 INFO:tasks.workunit.client.0.smithi008.stdout:6/764: mknod df/d10/d15/dd0/dd1/c101 0 2024-04-12T05:55:39.433 INFO:tasks.workunit.client.0.smithi008.stdout:6/765: readlink df/l2d 0 2024-04-12T05:55:39.445 INFO:tasks.workunit.client.0.smithi008.stdout:6/766: write df/d10/d15/d79/f8f [1926594,123296] 0 2024-04-12T05:55:39.449 INFO:tasks.workunit.client.0.smithi008.stdout:4/663: dwrite d1/d22/d29/d30/f52 [0,4194304] 0 2024-04-12T05:55:39.449 INFO:tasks.workunit.client.0.smithi008.stdout:6/767: write df/d21/d4c/f6b [1858884,102636] 0 2024-04-12T05:55:39.465 INFO:tasks.workunit.client.0.smithi008.stdout:1/757: sync 2024-04-12T05:55:39.477 INFO:tasks.workunit.client.0.smithi008.stdout:1/758: creat d4/dd/d17/d91/f105 x:0 0 0 2024-04-12T05:55:39.478 INFO:tasks.workunit.client.0.smithi008.stdout:6/768: mkdir df/d21/d4c/dfb/d102 0 2024-04-12T05:55:39.482 INFO:tasks.workunit.client.0.smithi008.stdout:6/769: write df/d10/d15/d75/fba [3502345,15440] 0 2024-04-12T05:55:39.482 INFO:tasks.workunit.client.0.smithi008.stdout:6/770: read df/d21/d4c/fbe [180861,25725] 0 2024-04-12T05:55:39.482 INFO:tasks.workunit.client.0.smithi008.stdout:1/759: creat d4/d33/d74/d30/d72/d7c/dd4/dd5/f106 x:0 0 0 2024-04-12T05:55:39.482 INFO:tasks.workunit.client.0.smithi008.stdout:6/771: chown df/d8c/db1 1015 1 2024-04-12T05:55:39.485 INFO:tasks.workunit.client.0.smithi008.stdout:1/760: readlink d4/d33/d37/l41 0 2024-04-12T05:55:39.502 INFO:tasks.workunit.client.0.smithi008.stdout:6/772: dread fe [0,4194304] 0 2024-04-12T05:55:39.506 INFO:tasks.workunit.client.0.smithi008.stdout:6/773: symlink df/d8c/da4/l103 0 2024-04-12T05:55:39.507 INFO:tasks.workunit.client.0.smithi008.stdout:1/761: dwrite d4/d33/d74/d30/f5d [0,4194304] 0 2024-04-12T05:55:39.514 INFO:tasks.workunit.client.0.smithi008.stdout:6/774: link f9 df/d10/d15/d75/dd6/f104 0 2024-04-12T05:55:39.515 INFO:tasks.workunit.client.0.smithi008.stdout:1/762: getdents d4/d6 0 2024-04-12T05:55:39.517 INFO:tasks.workunit.client.0.smithi008.stdout:1/763: read d4/f47 [138285,35171] 0 2024-04-12T05:55:39.519 INFO:tasks.workunit.client.0.smithi008.stdout:1/764: read d4/d24/f32 [341265,58068] 0 2024-04-12T05:55:39.523 INFO:tasks.workunit.client.0.smithi008.stdout:6/775: dwrite df/d21/d4c/f6b [4194304,4194304] 0 2024-04-12T05:55:39.524 INFO:tasks.workunit.client.0.smithi008.stdout:6/776: dread - df/fe3 zero size 2024-04-12T05:55:39.526 INFO:tasks.workunit.client.0.smithi008.stdout:6/777: stat df/d10/d20/d3e/dfc 0 2024-04-12T05:55:39.528 INFO:tasks.workunit.client.0.smithi008.stdout:1/765: dwrite d4/d1a/d57/dd6/f81 [0,4194304] 0 2024-04-12T05:55:39.530 INFO:tasks.workunit.client.0.smithi008.stdout:6/778: mkdir df/d8c/db1/ddc/d105 0 2024-04-12T05:55:39.530 INFO:tasks.workunit.client.0.smithi008.stdout:1/766: fdatasync d4/dd/d17/d26/fe6 0 2024-04-12T05:55:39.533 INFO:tasks.workunit.client.0.smithi008.stdout:6/779: mkdir df/d8c/dcc/d106 0 2024-04-12T05:55:39.536 INFO:tasks.workunit.client.0.smithi008.stdout:1/767: rename d4/d1a/d57/laf to d4/dd/d21/l107 0 2024-04-12T05:55:39.537 INFO:tasks.workunit.client.0.smithi008.stdout:6/780: creat df/d8c/dcc/dee/f107 x:0 0 0 2024-04-12T05:55:39.539 INFO:tasks.workunit.client.0.smithi008.stdout:1/768: truncate d4/f7f 41883 0 2024-04-12T05:55:39.559 INFO:tasks.workunit.client.0.smithi008.stdout:6/781: write df/d10/d20/d3e/f72 [4280509,50491] 0 2024-04-12T05:55:39.564 INFO:tasks.workunit.client.0.smithi008.stdout:6/782: link df/d8c/da4/dad/ce7 df/d8c/db1/ddc/d105/c108 0 2024-04-12T05:55:39.566 INFO:tasks.workunit.client.0.smithi008.stdout:6/783: rmdir df/d10/d15/dd0 39 2024-04-12T05:55:39.570 INFO:tasks.workunit.client.0.smithi008.stdout:6/784: creat df/d21/f109 x:0 0 0 2024-04-12T05:55:39.570 INFO:tasks.workunit.client.0.smithi008.stdout:6/785: dread - df/d8c/db1/ddc/d78/ff7 zero size 2024-04-12T05:55:39.574 INFO:tasks.workunit.client.0.smithi008.stdout:6/786: getdents df/d10/d20/d3e/db4 0 2024-04-12T05:55:39.574 INFO:tasks.workunit.client.0.smithi008.stdout:6/787: write f8 [1048943,81125] 0 2024-04-12T05:55:39.578 INFO:tasks.workunit.client.0.smithi008.stdout:6/788: truncate df/d10/d20/d3e/f5c 906583 0 2024-04-12T05:55:39.579 INFO:tasks.workunit.client.0.smithi008.stdout:6/789: mkdir df/d21/d4c/dfb/d10a 0 2024-04-12T05:55:39.585 INFO:tasks.workunit.client.0.smithi008.stdout:6/790: dwrite df/d21/f43 [0,4194304] 0 2024-04-12T05:55:39.591 INFO:tasks.workunit.client.0.smithi008.stdout:6/791: rename df/d21/f57 to df/d8c/db1/ddc/d9b/f10b 0 2024-04-12T05:55:39.594 INFO:tasks.workunit.client.0.smithi008.stdout:6/792: write fd [333309,59800] 0 2024-04-12T05:55:39.644 INFO:tasks.workunit.client.0.smithi008.stdout:2/833: sync 2024-04-12T05:55:39.644 INFO:tasks.workunit.client.0.smithi008.stdout:8/729: sync 2024-04-12T05:55:39.644 INFO:tasks.workunit.client.0.smithi008.stdout:0/695: sync 2024-04-12T05:55:39.644 INFO:tasks.workunit.client.0.smithi008.stdout:0/696: stat d2/d24 0 2024-04-12T05:55:39.646 INFO:tasks.workunit.client.0.smithi008.stdout:0/697: mkdir d2/ddb 0 2024-04-12T05:55:39.647 INFO:tasks.workunit.client.0.smithi008.stdout:2/834: read d6/db/d10/d23/d35/fc5 [591714,61579] 0 2024-04-12T05:55:39.652 INFO:tasks.workunit.client.0.smithi008.stdout:2/835: creat d6/db/d60/d4f/f119 x:0 0 0 2024-04-12T05:55:39.653 INFO:tasks.workunit.client.0.smithi008.stdout:2/836: stat d6/db/d10/d7b 0 2024-04-12T05:55:39.653 INFO:tasks.workunit.client.0.smithi008.stdout:8/730: truncate dd/de/d4c/d66/da9/f7d 5086678 0 2024-04-12T05:55:39.656 INFO:tasks.workunit.client.0.smithi008.stdout:2/837: creat d6/db/d6f/da6/f11a x:0 0 0 2024-04-12T05:55:39.659 INFO:tasks.workunit.client.0.smithi008.stdout:8/731: mknod dd/de/d16/d50/d5e/dbe/dd6/cfe 0 2024-04-12T05:55:39.660 INFO:tasks.workunit.client.0.smithi008.stdout:2/838: readlink d6/db/d10/d23/l7a 0 2024-04-12T05:55:39.661 INFO:tasks.workunit.client.0.smithi008.stdout:2/839: write d6/f10d [182858,130730] 0 2024-04-12T05:55:39.662 INFO:tasks.workunit.client.0.smithi008.stdout:0/698: dread d2/f72 [0,4194304] 0 2024-04-12T05:55:39.663 INFO:tasks.workunit.client.0.smithi008.stdout:2/840: chown d6/db/d29/dc2/fd8 71958728 1 2024-04-12T05:55:39.665 INFO:tasks.workunit.client.0.smithi008.stdout:2/841: write d6/db/d6f/f88 [4230608,32058] 0 2024-04-12T05:55:39.669 INFO:tasks.workunit.client.0.smithi008.stdout:8/732: dwrite dd/de/d16/d50/d56/d61/fe0 [0,4194304] 0 2024-04-12T05:55:39.677 INFO:tasks.workunit.client.0.smithi008.stdout:2/842: dread d6/db/d10/f80 [0,4194304] 0 2024-04-12T05:55:39.682 INFO:tasks.workunit.client.0.smithi008.stdout:8/733: dwrite dd/d30/d3b/db5/df0/feb [0,4194304] 0 2024-04-12T05:55:39.682 INFO:tasks.workunit.client.0.smithi008.stdout:2/843: getdents d6/db/d6f/da6 0 2024-04-12T05:55:39.686 INFO:tasks.workunit.client.0.smithi008.stdout:8/734: mknod dd/d30/cff 0 2024-04-12T05:55:39.692 INFO:tasks.workunit.client.0.smithi008.stdout:2/844: dwrite d6/db/d29/dc2/f106 [0,4194304] 0 2024-04-12T05:55:39.695 INFO:tasks.workunit.client.0.smithi008.stdout:2/845: symlink d6/db/d10/d7b/ddc/l11b 0 2024-04-12T05:55:39.698 INFO:tasks.workunit.client.0.smithi008.stdout:2/846: creat d6/db/dae/f11c x:0 0 0 2024-04-12T05:55:39.701 INFO:tasks.workunit.client.0.smithi008.stdout:2/847: fdatasync d6/db/d10/d7b/fb4 0 2024-04-12T05:55:39.704 INFO:tasks.workunit.client.0.smithi008.stdout:8/735: dwrite dd/d4f/d52/f79 [0,4194304] 0 2024-04-12T05:55:39.709 INFO:tasks.workunit.client.0.smithi008.stdout:2/848: dwrite d6/db/d38/d41/f99 [0,4194304] 0 2024-04-12T05:55:39.718 INFO:tasks.workunit.client.0.smithi008.stdout:2/849: rename d6/l1a to d6/db/d10/d23/d35/dfc/l11d 0 2024-04-12T05:55:39.725 INFO:tasks.workunit.client.0.smithi008.stdout:2/850: rename d6/db/d10/d7b to d6/db/d60/d11e 0 2024-04-12T05:55:39.727 INFO:tasks.workunit.client.0.smithi008.stdout:2/851: write d6/db/d60/d84/fcd [1014385,2896] 0 2024-04-12T05:55:39.728 INFO:tasks.workunit.client.0.smithi008.stdout:8/736: dread dd/de/d16/db1/d97/fa3 [0,4194304] 0 2024-04-12T05:55:39.734 INFO:tasks.workunit.client.0.smithi008.stdout:8/737: write dd/de/d16/f72 [4524128,10960] 0 2024-04-12T05:55:39.737 INFO:tasks.workunit.client.0.smithi008.stdout:8/738: truncate dd/de/d4c/d66/d6c/dc0/fd7 380103 0 2024-04-12T05:55:39.738 INFO:tasks.workunit.client.0.smithi008.stdout:2/852: rename d6/db/d60/d11e/ddc to d6/db/d10/d23/d35/d11f 0 2024-04-12T05:55:39.738 INFO:tasks.workunit.client.0.smithi008.stdout:2/853: stat d6/db/d38/d41 0 2024-04-12T05:55:39.745 INFO:tasks.workunit.client.0.smithi008.stdout:8/739: truncate dd/de/d4c/fcb 3944068 0 2024-04-12T05:55:39.745 INFO:tasks.workunit.client.0.smithi008.stdout:2/854: dwrite d6/db/d6f/f9b [0,4194304] 0 2024-04-12T05:55:39.747 INFO:tasks.workunit.client.0.smithi008.stdout:2/855: read d6/db/d38/f7e [747407,41445] 0 2024-04-12T05:55:39.748 INFO:tasks.workunit.client.0.smithi008.stdout:8/740: read dd/de/d16/d50/d56/d61/fe0 [642213,34691] 0 2024-04-12T05:55:39.748 INFO:tasks.workunit.client.0.smithi008.stdout:2/856: fsync d6/db/d38/d41/f77 0 2024-04-12T05:55:39.756 INFO:tasks.workunit.client.0.smithi008.stdout:2/857: write d6/db/d10/db2/fe6 [235813,14567] 0 2024-04-12T05:55:39.756 INFO:tasks.workunit.client.0.smithi008.stdout:8/741: dwrite dd/d30/d3b/db5/df0/fc2 [0,4194304] 0 2024-04-12T05:55:39.758 INFO:tasks.workunit.client.0.smithi008.stdout:2/858: write d6/db/d6f/f9b [660043,35873] 0 2024-04-12T05:55:39.761 INFO:tasks.workunit.client.0.smithi008.stdout:8/742: symlink dd/de/d4c/d66/l100 0 2024-04-12T05:55:39.762 INFO:tasks.workunit.client.0.smithi008.stdout:8/743: chown dd/fe9 0 1 2024-04-12T05:55:39.763 INFO:tasks.workunit.client.0.smithi008.stdout:2/859: symlink d6/db/d29/dc2/l120 0 2024-04-12T05:55:39.764 INFO:tasks.workunit.client.0.smithi008.stdout:8/744: rmdir dd/de/d4c/d66 39 2024-04-12T05:55:39.767 INFO:tasks.workunit.client.0.smithi008.stdout:2/860: creat d6/db/d6f/dea/f121 x:0 0 0 2024-04-12T05:55:39.771 INFO:tasks.workunit.client.0.smithi008.stdout:8/745: dwrite dd/fe9 [0,4194304] 0 2024-04-12T05:55:39.771 INFO:tasks.workunit.client.0.smithi008.stdout:2/861: mkdir d6/db/d6f/de2/d113/db5/d122 0 2024-04-12T05:55:39.776 INFO:tasks.workunit.client.0.smithi008.stdout:8/746: symlink dd/de/d16/d50/d5e/dbe/dd6/l101 0 2024-04-12T05:55:39.781 INFO:tasks.workunit.client.0.smithi008.stdout:8/747: mknod dd/d30/c102 0 2024-04-12T05:55:39.781 INFO:tasks.workunit.client.0.smithi008.stdout:2/862: dwrite d6/db/d60/d84/fe4 [4194304,4194304] 0 2024-04-12T05:55:39.785 INFO:tasks.workunit.client.0.smithi008.stdout:8/748: creat dd/de/f103 x:0 0 0 2024-04-12T05:55:39.785 INFO:tasks.workunit.client.0.smithi008.stdout:9/825: sync 2024-04-12T05:55:39.785 INFO:tasks.workunit.client.0.smithi008.stdout:7/756: sync 2024-04-12T05:55:39.785 INFO:tasks.workunit.client.0.smithi008.stdout:3/734: sync 2024-04-12T05:55:39.789 INFO:tasks.workunit.client.0.smithi008.stdout:3/735: fdatasync d9/d1e/f42 0 2024-04-12T05:55:39.790 INFO:tasks.workunit.client.0.smithi008.stdout:9/826: chown d3/d22/d2b/d58/d38/lca 465 1 2024-04-12T05:55:39.790 INFO:tasks.workunit.client.0.smithi008.stdout:7/757: dread - d1/d15/d43/d54/d58/db7/fb6 zero size 2024-04-12T05:55:39.795 INFO:tasks.workunit.client.0.smithi008.stdout:3/736: symlink d9/d11/d5b/lf7 0 2024-04-12T05:55:39.796 INFO:tasks.workunit.client.0.smithi008.stdout:9/827: mknod d3/d22/d2b/d4a/c117 0 2024-04-12T05:55:39.802 INFO:tasks.workunit.client.0.smithi008.stdout:8/749: rename dd/d30/d3b/fed to dd/de/d16/db1/f104 0 2024-04-12T05:55:39.809 INFO:tasks.workunit.client.0.smithi008.stdout:9/828: mknod d3/df/d13/d21/d2d/d32/d52/db0/c118 0 2024-04-12T05:55:39.809 INFO:tasks.workunit.client.0.smithi008.stdout:2/863: dwrite d6/db/d10/d23/d35/f49 [0,4194304] 0 2024-04-12T05:55:39.813 INFO:tasks.workunit.client.0.smithi008.stdout:8/750: rename dd/de/d16/f6e to dd/de/d16/d50/d56/d61/f105 0 2024-04-12T05:55:39.814 INFO:tasks.workunit.client.0.smithi008.stdout:3/737: dwrite d9/d1e/f42 [4194304,4194304] 0 2024-04-12T05:55:39.815 INFO:tasks.workunit.client.0.smithi008.stdout:9/829: mkdir d3/df/d13/d21/d5c/da4/d119 0 2024-04-12T05:55:39.816 INFO:tasks.workunit.client.0.smithi008.stdout:7/758: dread d1/d5/da/d78/fbd [0,4194304] 0 2024-04-12T05:55:39.817 INFO:tasks.workunit.client.0.smithi008.stdout:3/738: read - d9/dd/ff6 zero size 2024-04-12T05:55:39.819 INFO:tasks.workunit.client.0.smithi008.stdout:9/830: truncate d3/d22/d2b/d58/f1d 2951251 0 2024-04-12T05:55:39.820 INFO:tasks.workunit.client.0.smithi008.stdout:2/864: creat d6/db/d10/d23/d35/f123 x:0 0 0 2024-04-12T05:55:39.825 INFO:tasks.workunit.client.0.smithi008.stdout:8/751: rename dd/d30/d3b/de4/cf8 to dd/d4f/df5/c106 0 2024-04-12T05:55:39.825 INFO:tasks.workunit.client.0.smithi008.stdout:3/739: symlink d9/d10/d4b/lf8 0 2024-04-12T05:55:39.828 INFO:tasks.workunit.client.0.smithi008.stdout:3/740: rmdir d9/db7 39 2024-04-12T05:55:39.829 INFO:tasks.workunit.client.0.smithi008.stdout:2/865: symlink d6/db/d10/dcc/l124 0 2024-04-12T05:55:39.829 INFO:tasks.workunit.client.0.smithi008.stdout:7/759: dread d1/d5/da/d1d/d68/fa6 [0,4194304] 0 2024-04-12T05:55:39.831 INFO:tasks.workunit.client.0.smithi008.stdout:0/699: sync 2024-04-12T05:55:39.831 INFO:tasks.workunit.client.0.smithi008.stdout:1/769: sync 2024-04-12T05:55:39.831 INFO:tasks.workunit.client.0.smithi008.stdout:6/793: sync 2024-04-12T05:55:39.831 INFO:tasks.workunit.client.0.smithi008.stdout:4/664: sync 2024-04-12T05:55:39.833 INFO:tasks.workunit.client.0.smithi008.stdout:9/831: rename d3/d22/d2b/d4a/l93 to d3/l11a 0 2024-04-12T05:55:39.835 INFO:tasks.workunit.client.0.smithi008.stdout:2/866: write d6/db/d38/d41/fad [3072954,17020] 0 2024-04-12T05:55:39.836 INFO:tasks.workunit.client.0.smithi008.stdout:0/700: mkdir d2/d24/d85/ddc 0 2024-04-12T05:55:39.840 INFO:tasks.workunit.client.0.smithi008.stdout:4/665: creat d1/d4f/fc7 x:0 0 0 2024-04-12T05:55:39.843 INFO:tasks.workunit.client.0.smithi008.stdout:1/770: creat d4/d33/d74/da2/f108 x:0 0 0 2024-04-12T05:55:39.843 INFO:tasks.workunit.client.0.smithi008.stdout:1/771: dread - d4/d6/f88 zero size 2024-04-12T05:55:39.847 INFO:tasks.workunit.client.0.smithi008.stdout:9/832: creat d3/d22/d2b/d4e/f11b x:0 0 0 2024-04-12T05:55:39.849 INFO:tasks.workunit.client.0.smithi008.stdout:2/867: symlink d6/db/dae/l125 0 2024-04-12T05:55:39.850 INFO:tasks.workunit.client.0.smithi008.stdout:8/752: write dd/de/d16/db1/f9f [450180,104979] 0 2024-04-12T05:55:39.850 INFO:tasks.workunit.client.0.smithi008.stdout:0/701: mknod d2/d11/d36/d42/d86/d9b/cdd 0 2024-04-12T05:55:39.851 INFO:tasks.workunit.client.0.smithi008.stdout:1/772: dread d4/d1c/d4f/f6a [4194304,4194304] 0 2024-04-12T05:55:39.852 INFO:tasks.workunit.client.0.smithi008.stdout:2/868: fsync d6/db/d6f/da6/fd7 0 2024-04-12T05:55:39.853 INFO:tasks.workunit.client.0.smithi008.stdout:4/666: mknod d1/cc8 0 2024-04-12T05:55:39.854 INFO:tasks.workunit.client.0.smithi008.stdout:4/667: readlink d1/d8/l5e 0 2024-04-12T05:55:39.855 INFO:tasks.workunit.client.0.smithi008.stdout:1/773: write d4/d24/d62/fee [4127,36235] 0 2024-04-12T05:55:39.856 INFO:tasks.workunit.client.0.smithi008.stdout:2/869: dread d6/db/d29/dc2/fd8 [0,4194304] 0 2024-04-12T05:55:39.857 INFO:tasks.workunit.client.0.smithi008.stdout:6/794: write df/d8c/fa3 [143324,125684] 0 2024-04-12T05:55:39.858 INFO:tasks.workunit.client.0.smithi008.stdout:0/702: mkdir d2/d24/d85/dde 0 2024-04-12T05:55:39.861 INFO:tasks.workunit.client.0.smithi008.stdout:3/741: link d9/d10/d19/d3a/dad/feb d9/d10/d19/d22/da8/dc4/ff9 0 2024-04-12T05:55:39.865 INFO:tasks.workunit.client.0.smithi008.stdout:1/774: unlink d4/f27 0 2024-04-12T05:55:39.868 INFO:tasks.workunit.client.0.smithi008.stdout:8/753: dread dd/de/d16/d50/d5e/fc5 [0,4194304] 0 2024-04-12T05:55:39.870 INFO:tasks.workunit.client.0.smithi008.stdout:4/668: write d1/d22/d29/d30/d47/d28/fae [5227737,121969] 0 2024-04-12T05:55:39.871 INFO:tasks.workunit.client.0.smithi008.stdout:2/870: dwrite d6/db/dae/f11c [0,4194304] 0 2024-04-12T05:55:39.873 INFO:tasks.workunit.client.0.smithi008.stdout:3/742: fsync f7 0 2024-04-12T05:55:39.873 INFO:tasks.workunit.client.0.smithi008.stdout:0/703: dread - d2/d11/d36/d82/d95/fb6 zero size 2024-04-12T05:55:39.875 INFO:tasks.workunit.client.0.smithi008.stdout:7/760: dwrite d1/d49/fec [0,4194304] 0 2024-04-12T05:55:39.876 INFO:tasks.workunit.client.0.smithi008.stdout:1/775: mknod d4/d33/d74/d7a/c109 0 2024-04-12T05:55:39.878 INFO:tasks.workunit.client.0.smithi008.stdout:9/833: truncate d3/d22/d2b/d58/fdb 1430491 0 2024-04-12T05:55:39.878 INFO:tasks.workunit.client.0.smithi008.stdout:2/871: fsync d6/db/d6f/dea/f121 0 2024-04-12T05:55:39.882 INFO:tasks.workunit.client.0.smithi008.stdout:1/776: write d4/d33/d74/d30/f5d [3278916,107241] 0 2024-04-12T05:55:39.883 INFO:tasks.workunit.client.0.smithi008.stdout:6/795: dread df/d10/d15/dd0/dd1/f51 [0,4194304] 0 2024-04-12T05:55:39.887 INFO:tasks.workunit.client.0.smithi008.stdout:4/669: symlink d1/db/db7/lc9 0 2024-04-12T05:55:39.887 INFO:tasks.workunit.client.0.smithi008.stdout:3/743: creat d9/d10/d19/d3a/d7c/ffa x:0 0 0 2024-04-12T05:55:39.887 INFO:tasks.workunit.client.0.smithi008.stdout:7/761: rmdir d1/d15/d43/d54/d58/db7/de3 39 2024-04-12T05:55:39.888 INFO:tasks.workunit.client.0.smithi008.stdout:6/796: symlink df/d10/d15/dd0/l10c 0 2024-04-12T05:55:39.889 INFO:tasks.workunit.client.0.smithi008.stdout:0/704: symlink d2/d24/d85/ddc/ldf 0 2024-04-12T05:55:39.890 INFO:tasks.workunit.client.0.smithi008.stdout:6/797: dread - df/d8c/db1/ddc/d95/fa7 zero size 2024-04-12T05:55:39.976 INFO:tasks.workunit.client.0.smithi008.stdout:4/670: creat d1/d22/d29/d46/d56/fca x:0 0 0 2024-04-12T05:55:39.977 INFO:tasks.workunit.client.0.smithi008.stdout:4/671: write d1/d8/f25 [3972130,62213] 0 2024-04-12T05:55:39.980 INFO:tasks.workunit.client.0.smithi008.stdout:9/834: mkdir d3/df/d13/d21/d2d/d30/d11c 0 2024-04-12T05:55:39.981 INFO:tasks.workunit.client.0.smithi008.stdout:3/744: readlink d9/d10/d19/d3a/d67/la0 0 2024-04-12T05:55:39.982 INFO:tasks.workunit.client.0.smithi008.stdout:9/835: truncate d3/d22/d2b/d4a/dc8/ffd 461797 0 2024-04-12T05:55:39.984 INFO:tasks.workunit.client.0.smithi008.stdout:2/872: rename d6/db/d60/d57/d102 to d6/db/d60/d4f/d126 0 2024-04-12T05:55:39.985 INFO:tasks.workunit.client.0.smithi008.stdout:9/836: truncate d3/df/d13/d21/d2d/d30/d74/f83 4844458 0 2024-04-12T05:55:39.986 INFO:tasks.workunit.client.0.smithi008.stdout:0/705: symlink d2/d11/d2a/db5/le0 0 2024-04-12T05:55:39.987 INFO:tasks.workunit.client.0.smithi008.stdout:6/798: write df/d8c/db1/ddc/d9b/fcb [3509806,32063] 0 2024-04-12T05:55:39.990 INFO:tasks.workunit.client.0.smithi008.stdout:8/754: write dd/de/f27 [1608940,125667] 0 2024-04-12T05:55:39.990 INFO:tasks.workunit.client.0.smithi008.stdout:7/762: truncate d1/f6c 1989740 0 2024-04-12T05:55:39.994 INFO:tasks.workunit.client.0.smithi008.stdout:2/873: write d6/f43 [791173,27166] 0 2024-04-12T05:55:39.995 INFO:tasks.workunit.client.0.smithi008.stdout:0/706: creat d2/d24/d61/fe1 x:0 0 0 2024-04-12T05:55:39.997 INFO:tasks.workunit.client.0.smithi008.stdout:9/837: rename d3/df/d13/d21/d5c/d8f/fc4 to d3/df/d13/d21/d2d/d32/d52/f11d 0 2024-04-12T05:55:39.999 INFO:tasks.workunit.client.0.smithi008.stdout:8/755: mknod dd/d4f/df5/c107 0 2024-04-12T05:55:40.000 INFO:tasks.workunit.client.0.smithi008.stdout:7/763: stat d1/d5/da/d11/f65 0 2024-04-12T05:55:40.002 INFO:tasks.workunit.client.0.smithi008.stdout:3/745: fsync d9/d44/f6e 0 2024-04-12T05:55:40.005 INFO:tasks.workunit.client.0.smithi008.stdout:9/838: fdatasync d3/d22/d2b/d58/d38/f9b 0 2024-04-12T05:55:40.006 INFO:tasks.workunit.client.0.smithi008.stdout:8/756: rmdir dd/d30/d32 39 2024-04-12T05:55:40.007 INFO:tasks.workunit.client.0.smithi008.stdout:9/839: write d3/df/d13/d21/d5c/da4/db5/fc9 [831986,24000] 0 2024-04-12T05:55:40.011 INFO:tasks.workunit.client.0.smithi008.stdout:3/746: rmdir d9/d10/d19/d3a/dad/dcf 39 2024-04-12T05:55:40.013 INFO:tasks.workunit.client.0.smithi008.stdout:0/707: write d2/d11/d36/d47/f68 [982173,34001] 0 2024-04-12T05:55:40.015 INFO:tasks.workunit.client.0.smithi008.stdout:4/672: getdents d1 0 2024-04-12T05:55:40.016 INFO:tasks.workunit.client.0.smithi008.stdout:6/799: link df/d10/d15/d4f/de9/cfa df/d8c/db1/ddc/d78/c10d 0 2024-04-12T05:55:40.017 INFO:tasks.workunit.client.0.smithi008.stdout:9/840: symlink d3/df/d13/d21/d2d/d30/l11e 0 2024-04-12T05:55:40.018 INFO:tasks.workunit.client.0.smithi008.stdout:7/764: rename d1/d5/da/d1d/lae to d1/d15/d43/d54/d77/l105 0 2024-04-12T05:55:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:2/874: link d6/db/d6f/de2/d113/ca7 d6/db/d60/d4f/c127 0 2024-04-12T05:55:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:3/747: creat d9/d10/d19/d3a/d67/ffb x:0 0 0 2024-04-12T05:55:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:4/673: read d1/d22/d29/d30/d47/d28/fae [2371426,14391] 0 2024-04-12T05:55:40.020 INFO:tasks.workunit.client.0.smithi008.stdout:0/708: mknod d2/d11/d36/d82/d95/ce2 0 2024-04-12T05:55:40.022 INFO:tasks.workunit.client.0.smithi008.stdout:6/800: mkdir df/d8c/db1/d10e 0 2024-04-12T05:55:40.025 INFO:tasks.workunit.client.0.smithi008.stdout:2/875: dread - d6/db/d10/d4c/fd1 zero size 2024-04-12T05:55:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:4/674: unlink d1/d4f/f6c 0 2024-04-12T05:55:40.027 INFO:tasks.workunit.client.0.smithi008.stdout:9/841: mknod d3/df/d13/d21/d2d/c11f 0 2024-04-12T05:55:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:8/757: dread dd/de/f47 [4194304,4194304] 0 2024-04-12T05:55:40.028 INFO:tasks.workunit.client.0.smithi008.stdout:2/876: chown d6/db/d38/d41/f93 10285 1 2024-04-12T05:55:40.030 INFO:tasks.workunit.client.0.smithi008.stdout:0/709: creat d2/d24/d85/ddc/fe3 x:0 0 0 2024-04-12T05:55:40.036 INFO:tasks.workunit.client.0.smithi008.stdout:4/675: mkdir d1/d4f/dcb 0 2024-04-12T05:55:40.037 INFO:tasks.workunit.client.0.smithi008.stdout:2/877: dwrite d6/db/d60/d57/f76 [0,4194304] 0 2024-04-12T05:55:40.038 INFO:tasks.workunit.client.0.smithi008.stdout:8/758: truncate fb 4833637 0 2024-04-12T05:55:40.039 INFO:tasks.workunit.client.0.smithi008.stdout:9/842: mkdir d3/df/d13/d21/d2d/d32/d52/d120 0 2024-04-12T05:55:40.041 INFO:tasks.workunit.client.0.smithi008.stdout:9/843: stat d3/df/d13/d21/d2d/d30/d74/d115 0 2024-04-12T05:55:40.042 INFO:tasks.workunit.client.0.smithi008.stdout:4/676: truncate d1/d22/d29/d30/d47/d28/d4a/f95 1821775 0 2024-04-12T05:55:40.046 INFO:tasks.workunit.client.0.smithi008.stdout:6/801: mkdir df/d10/d20/d3e/dfc/d10f 0 2024-04-12T05:55:40.048 INFO:tasks.workunit.client.0.smithi008.stdout:2/878: creat d6/db/d60/d11e/f128 x:0 0 0 2024-04-12T05:55:40.050 INFO:tasks.workunit.client.0.smithi008.stdout:0/710: mknod d2/d11/d2a/ce4 0 2024-04-12T05:55:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:9/844: readlink d3/l11a 0 2024-04-12T05:55:40.051 INFO:tasks.workunit.client.0.smithi008.stdout:9/845: stat d3/df/d13/d21/f91 0 2024-04-12T05:55:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:3/748: rmdir d9/d10/d19/d3a/d67/dc2 0 2024-04-12T05:55:40.053 INFO:tasks.workunit.client.0.smithi008.stdout:3/749: stat d9/d10/d19/d65 0 2024-04-12T05:55:40.054 INFO:tasks.workunit.client.0.smithi008.stdout:4/677: creat d1/d8/dc5/fcc x:0 0 0 2024-04-12T05:55:40.057 INFO:tasks.workunit.client.0.smithi008.stdout:0/711: mknod d2/ddb/ce5 0 2024-04-12T05:55:40.057 INFO:tasks.workunit.client.0.smithi008.stdout:3/750: write d9/d2e/d3d/f56 [1664099,49623] 0 2024-04-12T05:55:40.059 INFO:tasks.workunit.client.0.smithi008.stdout:7/765: link d1/d15/d43/d54/d77/l105 d1/d5/da/d3c/l106 0 2024-04-12T05:55:40.063 INFO:tasks.workunit.client.0.smithi008.stdout:6/802: getdents df/d8c/da4/dad 0 2024-04-12T05:55:40.067 INFO:tasks.workunit.client.0.smithi008.stdout:4/678: dwrite d1/d4f/f5b [4194304,4194304] 0 2024-04-12T05:55:40.068 INFO:tasks.workunit.client.0.smithi008.stdout:0/712: creat d2/d4b/da2/fe6 x:0 0 0 2024-04-12T05:55:40.068 INFO:tasks.workunit.client.0.smithi008.stdout:2/879: dread d6/db/d10/d23/d35/fc5 [0,4194304] 0 2024-04-12T05:55:40.070 INFO:tasks.workunit.client.0.smithi008.stdout:4/679: dread - d1/d22/d29/d46/f76 zero size 2024-04-12T05:55:40.081 INFO:tasks.workunit.client.0.smithi008.stdout:8/759: write dd/de/d16/f80 [130110,111662] 0 2024-04-12T05:55:40.082 INFO:tasks.workunit.client.0.smithi008.stdout:4/680: chown d1/d22/d29/d46/d56/cc6 53539139 1 2024-04-12T05:55:40.084 INFO:tasks.workunit.client.0.smithi008.stdout:7/766: rmdir d1/d15/d43/d54/d58/dfc 0 2024-04-12T05:55:40.084 INFO:tasks.workunit.client.0.smithi008.stdout:0/713: dwrite d2/d11/f64 [0,4194304] 0 2024-04-12T05:55:40.084 INFO:tasks.workunit.client.0.smithi008.stdout:6/803: dwrite df/d10/d15/d79/fe0 [0,4194304] 0 2024-04-12T05:55:40.086 INFO:tasks.workunit.client.0.smithi008.stdout:8/760: unlink dd/d30/l38 0 2024-04-12T05:55:40.089 INFO:tasks.workunit.client.0.smithi008.stdout:2/880: write d6/f6d [526827,111423] 0 2024-04-12T05:55:40.091 INFO:tasks.workunit.client.0.smithi008.stdout:4/681: mknod d1/d22/d29/d46/d56/ccd 0 2024-04-12T05:55:40.094 INFO:tasks.workunit.client.0.smithi008.stdout:4/682: read - d1/d22/d29/d30/d47/d28/d74/fa9 zero size 2024-04-12T05:55:40.095 INFO:tasks.workunit.client.0.smithi008.stdout:7/767: rename d1/d15/f20 to d1/d5/da/d60/d7b/f107 0 2024-04-12T05:55:40.098 INFO:tasks.workunit.client.0.smithi008.stdout:3/751: dwrite d9/d11/f98 [0,4194304] 0 2024-04-12T05:55:40.102 INFO:tasks.workunit.client.0.smithi008.stdout:8/761: write dd/de/d4c/d66/d6d/fde [93713,84258] 0 2024-04-12T05:55:40.102 INFO:tasks.workunit.client.0.smithi008.stdout:0/714: truncate d2/fe 2740259 0 2024-04-12T05:55:40.106 INFO:tasks.workunit.client.0.smithi008.stdout:6/804: mkdir df/d10/d15/d75/dd3/d110 0 2024-04-12T05:55:40.106 INFO:tasks.workunit.client.0.smithi008.stdout:4/683: mknod d1/db/db7/cce 0 2024-04-12T05:55:40.106 INFO:tasks.workunit.client.0.smithi008.stdout:7/768: creat d1/d5/da/d3c/dd3/f108 x:0 0 0 2024-04-12T05:55:40.106 INFO:tasks.workunit.client.0.smithi008.stdout:6/805: chown df/d10/d15 131871 1 2024-04-12T05:55:40.108 INFO:tasks.workunit.client.0.smithi008.stdout:0/715: rmdir d2/d11/d36/d42 39 2024-04-12T05:55:40.108 INFO:tasks.workunit.client.0.smithi008.stdout:6/806: chown df/d10/d20/fae 269 1 2024-04-12T05:55:40.110 INFO:tasks.workunit.client.0.smithi008.stdout:0/716: chown d2/d4b/da2/fd0 1 1 2024-04-12T05:55:40.110 INFO:tasks.workunit.client.0.smithi008.stdout:3/752: symlink d9/d10/d19/d3a/lfc 0 2024-04-12T05:55:40.112 INFO:tasks.workunit.client.0.smithi008.stdout:8/762: mkdir dd/de/d16/d50/d5e/dbe/dd6/d108 0 2024-04-12T05:55:40.114 INFO:tasks.workunit.client.0.smithi008.stdout:7/769: mkdir d1/d5/da/d11/d109 0 2024-04-12T05:55:40.117 INFO:tasks.workunit.client.0.smithi008.stdout:3/753: dread d9/d2e/d3d/f80 [0,4194304] 0 2024-04-12T05:55:40.118 INFO:tasks.workunit.client.0.smithi008.stdout:2/881: write d6/db/d6f/fed [792901,80671] 0 2024-04-12T05:55:40.121 INFO:tasks.workunit.client.0.smithi008.stdout:2/882: write d6/db/d38/d41/fc6 [136234,30643] 0 2024-04-12T05:55:40.123 INFO:tasks.workunit.client.0.smithi008.stdout:6/807: rename df/d10/d15/d4f/de9/cfa to df/d8c/c111 0 2024-04-12T05:55:40.126 INFO:tasks.workunit.client.0.smithi008.stdout:3/754: fdatasync d9/d2e/f74 0 2024-04-12T05:55:40.129 INFO:tasks.workunit.client.0.smithi008.stdout:2/883: dread d6/db/d60/d57/f76 [0,4194304] 0 2024-04-12T05:55:40.138 INFO:tasks.workunit.client.0.smithi008.stdout:3/755: chown d9/d10/d4b/fa7 5227924 1 2024-04-12T05:55:40.140 INFO:tasks.workunit.client.0.smithi008.stdout:4/684: dread d1/d8/f55 [0,4194304] 0 2024-04-12T05:55:40.141 INFO:tasks.workunit.client.0.smithi008.stdout:2/884: dwrite d6/db/d10/d23/d35/fca [0,4194304] 0 2024-04-12T05:55:40.144 INFO:tasks.workunit.client.0.smithi008.stdout:8/763: rename dd/de/d16/d50/d56/d61/l8a to dd/de/d4c/d66/d6d/dca/l109 0 2024-04-12T05:55:40.144 INFO:tasks.workunit.client.0.smithi008.stdout:7/770: write d1/f6c [2269982,87960] 0 2024-04-12T05:55:40.152 INFO:tasks.workunit.client.0.smithi008.stdout:0/717: getdents d2/d11/d36/d42/da6 0 2024-04-12T05:55:40.153 INFO:tasks.workunit.client.0.smithi008.stdout:4/685: mkdir d1/d22/d29/d46/d56/daf/dcf 0 2024-04-12T05:55:40.154 INFO:tasks.workunit.client.0.smithi008.stdout:7/771: creat d1/d15/df7/f10a x:0 0 0 2024-04-12T05:55:40.154 INFO:tasks.workunit.client.0.smithi008.stdout:3/756: symlink d9/d10/d19/d3a/lfd 0 2024-04-12T05:55:40.154 INFO:tasks.workunit.client.0.smithi008.stdout:2/885: dwrite d6/db/d29/f6c [0,4194304] 0 2024-04-12T05:55:40.157 INFO:tasks.workunit.client.0.smithi008.stdout:6/808: write df/d8c/f9a [59104,103480] 0 2024-04-12T05:55:40.159 INFO:tasks.workunit.client.0.smithi008.stdout:0/718: write d2/d11/d2a/f4a [3938500,3407] 0 2024-04-12T05:55:40.160 INFO:tasks.workunit.client.0.smithi008.stdout:8/764: link dd/d4f/d52/da5/cc6 dd/de/d4c/d66/d6d/c10a 0 2024-04-12T05:55:40.162 INFO:tasks.workunit.client.0.smithi008.stdout:3/757: creat d9/d2e/d3d/ffe x:0 0 0 2024-04-12T05:55:40.166 INFO:tasks.workunit.client.0.smithi008.stdout:7/772: getdents d1/d5/da/d11/d109 0 2024-04-12T05:55:40.167 INFO:tasks.workunit.client.0.smithi008.stdout:8/765: creat dd/de/d4c/d66/d6d/dca/f10b x:0 0 0 2024-04-12T05:55:40.168 INFO:tasks.workunit.client.0.smithi008.stdout:6/809: rename df/d10/d15/dd0/dd1/de5 to df/d10/d15/dd0/dd1/d112 0 2024-04-12T05:55:40.170 INFO:tasks.workunit.client.0.smithi008.stdout:2/886: getdents d6/db/d29/dc2 0 2024-04-12T05:55:40.171 INFO:tasks.workunit.client.0.smithi008.stdout:4/686: dwrite d1/d22/d29/d30/f90 [0,4194304] 0 2024-04-12T05:55:40.171 INFO:tasks.workunit.client.0.smithi008.stdout:6/810: chown df/d8c/dcc 25 1 2024-04-12T05:55:40.174 INFO:tasks.workunit.client.0.smithi008.stdout:0/719: getdents d2/d11/d36/d82 0 2024-04-12T05:55:40.174 INFO:tasks.workunit.client.0.smithi008.stdout:7/773: write d1/d15/d43/d54/d58/db7/fac [6794850,22538] 0 2024-04-12T05:55:40.176 INFO:tasks.workunit.client.0.smithi008.stdout:8/766: rmdir dd/de/d4c/d66/d6d 39 2024-04-12T05:55:40.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/774: write d1/d15/df7/f10a [611598,8486] 0 2024-04-12T05:55:40.179 INFO:tasks.workunit.client.0.smithi008.stdout:8/767: chown dd/de/d16/d50/d5e/dbe/dd6/cfe 1883 1 2024-04-12T05:55:40.180 INFO:tasks.workunit.client.0.smithi008.stdout:6/811: fsync df/d8c/db1/ddc/d78/fa6 0 2024-04-12T05:55:40.180 INFO:tasks.workunit.client.0.smithi008.stdout:7/775: write d1/d15/fe6 [35685,57237] 0 2024-04-12T05:55:40.182 INFO:tasks.workunit.client.0.smithi008.stdout:4/687: mkdir d1/d22/d29/d30/d60/dd0 0 2024-04-12T05:55:40.183 INFO:tasks.workunit.client.0.smithi008.stdout:2/887: symlink d6/db/d6f/de2/d113/db5/d122/l129 0 2024-04-12T05:55:40.186 INFO:tasks.workunit.client.0.smithi008.stdout:3/758: write d9/d10/d19/d22/da8/dc4/ff9 [497826,51255] 0 2024-04-12T05:55:40.187 INFO:tasks.workunit.client.0.smithi008.stdout:8/768: readlink dd/d30/d32/l53 0 2024-04-12T05:55:40.190 INFO:tasks.workunit.client.0.smithi008.stdout:7/776: fsync d1/d5/f2e 0 2024-04-12T05:55:40.193 INFO:tasks.workunit.client.0.smithi008.stdout:0/720: link d2/d11/d36/d42/l45 d2/d11/d36/d42/d86/d9b/le7 0 2024-04-12T05:55:40.193 INFO:tasks.workunit.client.0.smithi008.stdout:2/888: truncate d6/db/d10/db2/fba 416747 0 2024-04-12T05:55:40.194 INFO:tasks.workunit.client.0.smithi008.stdout:6/812: mknod df/d10/d20/c113 0 2024-04-12T05:55:40.195 INFO:tasks.workunit.client.0.smithi008.stdout:4/688: dwrite d1/d8/dc5/fcc [0,4194304] 0 2024-04-12T05:55:40.197 INFO:tasks.workunit.client.0.smithi008.stdout:3/759: mkdir d9/d10/d19/d3a/dad/dff 0 2024-04-12T05:55:40.197 INFO:tasks.workunit.client.0.smithi008.stdout:6/813: write df/d21/d4c/fe8 [7667802,17766] 0 2024-04-12T05:55:40.202 INFO:tasks.workunit.client.0.smithi008.stdout:8/769: mknod dd/de/d16/db1/d97/c10c 0 2024-04-12T05:55:40.203 INFO:tasks.workunit.client.0.smithi008.stdout:0/721: symlink d2/d24/d39/le8 0 2024-04-12T05:55:40.203 INFO:tasks.workunit.client.0.smithi008.stdout:7/777: mkdir d1/d5/d10b 0 2024-04-12T05:55:40.204 INFO:tasks.workunit.client.0.smithi008.stdout:4/689: unlink d1/d22/d29/d30/d47/d28/cb8 0 2024-04-12T05:55:40.207 INFO:tasks.workunit.client.0.smithi008.stdout:2/889: creat d6/db/f12a x:0 0 0 2024-04-12T05:55:40.207 INFO:tasks.workunit.client.0.smithi008.stdout:3/760: mknod d9/d10/d19/d22/dec/c100 0 2024-04-12T05:55:40.207 INFO:tasks.workunit.client.0.smithi008.stdout:4/690: stat d1/d22/d29/d30/d47/d28/d4a/d62/la2 0 2024-04-12T05:55:40.208 INFO:tasks.workunit.client.0.smithi008.stdout:2/890: chown d6/ff9 5865670 1 2024-04-12T05:55:40.208 INFO:tasks.workunit.client.0.smithi008.stdout:0/722: mkdir d2/d24/dbd/de9 0 2024-04-12T05:55:40.211 INFO:tasks.workunit.client.0.smithi008.stdout:7/778: link d1/fb d1/d15/d43/d54/d58/d5e/d7f/f10c 0 2024-04-12T05:55:40.213 INFO:tasks.workunit.client.0.smithi008.stdout:4/691: mknod d1/d22/d29/d46/d56/daf/cd1 0 2024-04-12T05:55:40.215 INFO:tasks.workunit.client.0.smithi008.stdout:3/761: rename d9/d11/cca to d9/d37/d69/c101 0 2024-04-12T05:55:40.216 INFO:tasks.workunit.client.0.smithi008.stdout:6/814: dread df/d10/d15/dd0/dd1/def/ff3 [0,4194304] 0 2024-04-12T05:55:40.217 INFO:tasks.workunit.client.0.smithi008.stdout:7/779: dread - d1/d5/da/d78/dd6/fee zero size 2024-04-12T05:55:40.220 INFO:tasks.workunit.client.0.smithi008.stdout:4/692: creat d1/d22/d29/fd2 x:0 0 0 2024-04-12T05:55:40.221 INFO:tasks.workunit.client.0.smithi008.stdout:3/762: creat d9/d10/d4b/f102 x:0 0 0 2024-04-12T05:55:40.222 INFO:tasks.workunit.client.0.smithi008.stdout:0/723: dwrite d2/d11/fb9 [0,4194304] 0 2024-04-12T05:55:40.222 INFO:tasks.workunit.client.0.smithi008.stdout:7/780: stat d1/d5/da/d1d/fc5 0 2024-04-12T05:55:40.225 INFO:tasks.workunit.client.0.smithi008.stdout:2/891: dwrite d6/db/d38/d41/ff7 [4194304,4194304] 0 2024-04-12T05:55:40.227 INFO:tasks.workunit.client.0.smithi008.stdout:0/724: chown d2/c15 47603 1 2024-04-12T05:55:40.231 INFO:tasks.workunit.client.0.smithi008.stdout:4/693: rename d1/d22/d29/d30/d47/d28/d74/lbd to d1/d22/d29/d46/da7/ld3 0 2024-04-12T05:55:40.232 INFO:tasks.workunit.client.0.smithi008.stdout:0/725: chown d2/d11/d36/d42/d86/d9b/cdd 96 1 2024-04-12T05:55:40.232 INFO:tasks.workunit.client.0.smithi008.stdout:3/763: truncate d9/d10/d19/d3a/fe6 487110 0 2024-04-12T05:55:40.234 INFO:tasks.workunit.client.0.smithi008.stdout:6/815: truncate df/d8c/db1/ddc/fa0 1007643 0 2024-04-12T05:55:40.238 INFO:tasks.workunit.client.0.smithi008.stdout:3/764: readlink d9/d10/d19/d22/d2c/la9 0 2024-04-12T05:55:40.238 INFO:tasks.workunit.client.0.smithi008.stdout:4/694: mknod d1/d22/d29/d46/d56/daf/dcf/cd4 0 2024-04-12T05:55:40.239 INFO:tasks.workunit.client.0.smithi008.stdout:2/892: rename d6/db/d60/d4f/dbf/lf6 to d6/db/d60/d4f/l12b 0 2024-04-12T05:55:40.241 INFO:tasks.workunit.client.0.smithi008.stdout:6/816: symlink df/d10/d15/d75/dd3/d110/l114 0 2024-04-12T05:55:40.242 INFO:tasks.workunit.client.0.smithi008.stdout:4/695: mkdir d1/d4f/dd5 0 2024-04-12T05:55:40.244 INFO:tasks.workunit.client.0.smithi008.stdout:7/781: dwrite d1/d5/da/d3c/d44/f51 [8388608,4194304] 0 2024-04-12T05:55:40.248 INFO:tasks.workunit.client.0.smithi008.stdout:0/726: dwrite d2/d11/d2a/db5/fd1 [0,4194304] 0 2024-04-12T05:55:40.250 INFO:tasks.workunit.client.0.smithi008.stdout:4/696: fsync d1/db/f18 0 2024-04-12T05:55:40.253 INFO:tasks.workunit.client.0.smithi008.stdout:0/727: dread - d2/d11/d36/d82/d95/fb6 zero size 2024-04-12T05:55:40.255 INFO:tasks.workunit.client.0.smithi008.stdout:3/765: dwrite d9/d2e/d3d/fc6 [0,4194304] 0 2024-04-12T05:55:40.264 INFO:tasks.workunit.client.0.smithi008.stdout:6/817: chown df/d10/d15/dd0/dd1/l70 6 1 2024-04-12T05:55:40.267 INFO:tasks.workunit.client.0.smithi008.stdout:4/697: mkdir d1/d22/d29/d30/d60/dd6 0 2024-04-12T05:55:40.268 INFO:tasks.workunit.client.0.smithi008.stdout:7/782: symlink d1/l10d 0 2024-04-12T05:55:40.270 INFO:tasks.workunit.client.0.smithi008.stdout:2/893: getdents d6/db/d60/d84 0 2024-04-12T05:55:40.270 INFO:tasks.workunit.client.0.smithi008.stdout:6/818: creat df/d8c/f115 x:0 0 0 2024-04-12T05:55:40.275 INFO:tasks.workunit.client.0.smithi008.stdout:0/728: write d2/d24/d61/fad [843794,127222] 0 2024-04-12T05:55:40.276 INFO:tasks.workunit.client.0.smithi008.stdout:7/783: write d1/d5/fc9 [257479,8079] 0 2024-04-12T05:55:40.277 INFO:tasks.workunit.client.0.smithi008.stdout:4/698: fdatasync d1/d22/d29/d30/d47/d28/f5c 0 2024-04-12T05:55:40.277 INFO:tasks.workunit.client.0.smithi008.stdout:3/766: creat d9/d37/f103 x:0 0 0 2024-04-12T05:55:40.280 INFO:tasks.workunit.client.0.smithi008.stdout:2/894: write d6/db/d38/f63 [1908855,21867] 0 2024-04-12T05:55:40.284 INFO:tasks.workunit.client.0.smithi008.stdout:7/784: write d1/d5/da/d3c/dd3/f108 [1032876,106416] 0 2024-04-12T05:55:40.284 INFO:tasks.workunit.client.0.smithi008.stdout:6/819: dread df/d21/d4c/fc7 [0,4194304] 0 2024-04-12T05:55:40.285 INFO:tasks.workunit.client.0.smithi008.stdout:3/767: mknod d9/d10/d19/d22/da8/dc4/c104 0 2024-04-12T05:55:40.287 INFO:tasks.workunit.client.0.smithi008.stdout:2/895: unlink d6/db/d10/d23/l7a 0 2024-04-12T05:55:40.288 INFO:tasks.workunit.client.0.smithi008.stdout:4/699: rmdir d1/d22/d29/d30/d60/dc0 0 2024-04-12T05:55:40.289 INFO:tasks.workunit.client.0.smithi008.stdout:7/785: unlink d1/d5/da/d60/c99 0 2024-04-12T05:55:40.293 INFO:tasks.workunit.client.0.smithi008.stdout:2/896: creat d6/db/d10/d23/d35/d11f/f12c x:0 0 0 2024-04-12T05:55:40.293 INFO:tasks.workunit.client.0.smithi008.stdout:4/700: mknod d1/d22/d29/d30/d47/d28/d4a/cd7 0 2024-04-12T05:55:40.294 INFO:tasks.workunit.client.0.smithi008.stdout:2/897: fdatasync d6/db/d10/d23/f4d 0 2024-04-12T05:55:40.295 INFO:tasks.workunit.client.0.smithi008.stdout:2/898: dread - d6/db/d38/fe1 zero size 2024-04-12T05:55:40.297 INFO:tasks.workunit.client.0.smithi008.stdout:3/768: mkdir d9/d10/d105 0 2024-04-12T05:55:40.300 INFO:tasks.workunit.client.0.smithi008.stdout:6/820: write df/d8c/db1/ddc/d78/fa6 [310654,2671] 0 2024-04-12T05:55:40.301 INFO:tasks.workunit.client.0.smithi008.stdout:7/786: mkdir d1/d15/d43/d10e 0 2024-04-12T05:55:40.302 INFO:tasks.workunit.client.0.smithi008.stdout:0/729: truncate d2/d11/d2a/f5d 4164041 0 2024-04-12T05:55:40.303 INFO:tasks.workunit.client.0.smithi008.stdout:6/821: stat df/d8c/db1/ddc/d78/fa6 0 2024-04-12T05:55:40.306 INFO:tasks.workunit.client.0.smithi008.stdout:4/701: fdatasync d1/d8/f37 0 2024-04-12T05:55:40.306 INFO:tasks.workunit.client.0.smithi008.stdout:2/899: dwrite d6/db/d60/d57/dd9/fe0 [0,4194304] 0 2024-04-12T05:55:40.308 INFO:tasks.workunit.client.0.smithi008.stdout:6/822: chown df/d8c/db1/ddc/d9b/fbc 337 1 2024-04-12T05:55:40.309 INFO:tasks.workunit.client.0.smithi008.stdout:0/730: fsync d2/d11/d36/d47/f91 0 2024-04-12T05:55:40.310 INFO:tasks.workunit.client.0.smithi008.stdout:7/787: creat d1/d5/da/d1d/dc3/f10f x:0 0 0 2024-04-12T05:55:40.312 INFO:tasks.workunit.client.0.smithi008.stdout:2/900: unlink d6/db/d60/d11e/f118 0 2024-04-12T05:55:40.314 INFO:tasks.workunit.client.0.smithi008.stdout:2/901: dread - d6/db/d38/f114 zero size 2024-04-12T05:55:40.318 INFO:tasks.workunit.client.0.smithi008.stdout:7/788: symlink d1/d15/d43/d54/d58/l110 0 2024-04-12T05:55:40.319 INFO:tasks.workunit.client.0.smithi008.stdout:2/902: chown d6/db/d38/d41/f93 108670740 1 2024-04-12T05:55:40.320 INFO:tasks.workunit.client.0.smithi008.stdout:0/731: dwrite d2/d4b/da2/fe6 [0,4194304] 0 2024-04-12T05:55:40.321 INFO:tasks.workunit.client.0.smithi008.stdout:4/702: creat d1/d22/d29/d30/fd8 x:0 0 0 2024-04-12T05:55:40.322 INFO:tasks.workunit.client.0.smithi008.stdout:4/703: dread - d1/d4f/fc7 zero size 2024-04-12T05:55:40.324 INFO:tasks.workunit.client.0.smithi008.stdout:7/789: write d1/d5/da/d11/f35 [16846813,83990] 0 2024-04-12T05:55:40.325 INFO:tasks.workunit.client.0.smithi008.stdout:4/704: write d1/db/f8f [1015327,95545] 0 2024-04-12T05:55:40.326 INFO:tasks.workunit.client.0.smithi008.stdout:7/790: fsync d1/d15/df7/f10a 0 2024-04-12T05:55:40.328 INFO:tasks.workunit.client.0.smithi008.stdout:2/903: read d6/db/d38/f63 [704416,92811] 0 2024-04-12T05:55:40.332 INFO:tasks.workunit.client.0.smithi008.stdout:6/823: write df/d10/d15/d75/f9f [853101,118478] 0 2024-04-12T05:55:40.338 INFO:tasks.workunit.client.0.smithi008.stdout:4/705: dwrite d1/d22/d29/d46/f76 [0,4194304] 0 2024-04-12T05:55:40.338 INFO:tasks.workunit.client.0.smithi008.stdout:2/904: write d6/db/d38/f63 [4418775,105541] 0 2024-04-12T05:55:40.341 INFO:tasks.workunit.client.0.smithi008.stdout:7/791: truncate d1/d5/da/d78/fbd 4403301 0 2024-04-12T05:55:40.345 INFO:tasks.workunit.client.0.smithi008.stdout:6/824: link fe df/d10/d15/d79/f116 0 2024-04-12T05:55:40.347 INFO:tasks.workunit.client.0.smithi008.stdout:2/905: mknod d6/db/d6f/de2/c12d 0 2024-04-12T05:55:40.348 INFO:tasks.workunit.client.0.smithi008.stdout:0/732: dwrite d2/fa [0,4194304] 0 2024-04-12T05:55:40.349 INFO:tasks.workunit.client.0.smithi008.stdout:7/792: unlink d1/d5/da/d60/d7b/f107 0 2024-04-12T05:55:40.351 INFO:tasks.workunit.client.0.smithi008.stdout:2/906: truncate d6/db/d6f/fed 1578043 0 2024-04-12T05:55:40.354 INFO:tasks.workunit.client.0.smithi008.stdout:4/706: write d1/f40 [130540,40420] 0 2024-04-12T05:55:40.357 INFO:tasks.workunit.client.0.smithi008.stdout:2/907: mknod d6/db/d60/d4f/d126/d116/c12e 0 2024-04-12T05:55:40.359 INFO:tasks.workunit.client.0.smithi008.stdout:6/825: getdents df/d8c/db1/ddc/d105 0 2024-04-12T05:55:40.360 INFO:tasks.workunit.client.0.smithi008.stdout:2/908: creat d6/db/d6f/de2/d113/db5/d122/f12f x:0 0 0 2024-04-12T05:55:40.363 INFO:tasks.workunit.client.0.smithi008.stdout:2/909: chown d6/db/d60/d84/fef 2476448 1 2024-04-12T05:55:40.364 INFO:tasks.workunit.client.0.smithi008.stdout:0/733: write d2/d11/d2a/db5/fc2 [948913,5504] 0 2024-04-12T05:55:40.365 INFO:tasks.workunit.client.0.smithi008.stdout:7/793: truncate d1/d5/da/d3c/d44/f6f 3945388 0 2024-04-12T05:55:40.367 INFO:tasks.workunit.client.0.smithi008.stdout:2/910: getdents d6/db/d6f/da6 0 2024-04-12T05:55:40.368 INFO:tasks.workunit.client.0.smithi008.stdout:6/826: dwrite df/d8c/db1/ddc/d9b/fbc [4194304,4194304] 0 2024-04-12T05:55:40.370 INFO:tasks.workunit.client.0.smithi008.stdout:0/734: mkdir d2/d11/d36/dd6/dea 0 2024-04-12T05:55:40.370 INFO:tasks.workunit.client.0.smithi008.stdout:7/794: dread - d1/d5/da/d78/faa zero size 2024-04-12T05:55:40.372 INFO:tasks.workunit.client.0.smithi008.stdout:0/735: write d2/d11/f2e [2428157,32583] 0 2024-04-12T05:55:40.373 INFO:tasks.workunit.client.0.smithi008.stdout:2/911: symlink d6/db/d60/d4f/l130 0 2024-04-12T05:55:40.379 INFO:tasks.workunit.client.0.smithi008.stdout:2/912: chown d6/db/d6f/de2/d113/db5/cf0 55 1 2024-04-12T05:55:40.379 INFO:tasks.workunit.client.0.smithi008.stdout:0/736: read d2/d24/f76 [1190673,51729] 0 2024-04-12T05:55:40.381 INFO:tasks.workunit.client.0.smithi008.stdout:2/913: chown d6/db/d10 144 1 2024-04-12T05:55:40.382 INFO:tasks.workunit.client.0.smithi008.stdout:7/795: dwrite d1/d5/da/d60/fe4 [0,4194304] 0 2024-04-12T05:55:40.385 INFO:tasks.workunit.client.0.smithi008.stdout:2/914: mkdir d6/db/d60/d11e/d131 0 2024-04-12T05:55:40.385 INFO:tasks.workunit.client.0.smithi008.stdout:7/796: fsync d1/d15/d43/f100 0 2024-04-12T05:55:40.385 INFO:tasks.workunit.client.0.smithi008.stdout:0/737: getdents d2/d11/d36/d42/da6/dd7 0 2024-04-12T05:55:40.391 INFO:tasks.workunit.client.0.smithi008.stdout:6/827: dwrite f9 [0,4194304] 0 2024-04-12T05:55:40.392 INFO:tasks.workunit.client.0.smithi008.stdout:0/738: creat d2/d11/d2a/db5/feb x:0 0 0 2024-04-12T05:55:40.393 INFO:tasks.workunit.client.0.smithi008.stdout:7/797: truncate d1/d49/f4b 5138479 0 2024-04-12T05:55:40.395 INFO:tasks.workunit.client.0.smithi008.stdout:0/739: dread - d2/d4b/f99 zero size 2024-04-12T05:55:40.395 INFO:tasks.workunit.client.0.smithi008.stdout:7/798: chown d1/d15/d43/d54/d58/d5e/d7f/le5 2891 1 2024-04-12T05:55:40.396 INFO:tasks.workunit.client.0.smithi008.stdout:6/828: write df/d8c/db1/ddc/d9b/f10b [2252108,12834] 0 2024-04-12T05:55:40.397 INFO:tasks.workunit.client.0.smithi008.stdout:2/915: dread d6/db/d60/d57/dd9/fe0 [0,4194304] 0 2024-04-12T05:55:40.400 INFO:tasks.workunit.client.0.smithi008.stdout:0/740: symlink d2/d24/d85/ddc/lec 0 2024-04-12T05:55:40.401 INFO:tasks.workunit.client.0.smithi008.stdout:2/916: write d6/db/d60/d84/fe4 [9115377,78511] 0 2024-04-12T05:55:40.401 INFO:tasks.workunit.client.0.smithi008.stdout:7/799: mknod d1/d15/c111 0 2024-04-12T05:55:40.403 INFO:tasks.workunit.client.0.smithi008.stdout:7/800: stat d1/d5/da/d78/cf9 0 2024-04-12T05:55:40.404 INFO:tasks.workunit.client.0.smithi008.stdout:0/741: creat d2/d24/d61/fed x:0 0 0 2024-04-12T05:55:40.404 INFO:tasks.workunit.client.0.smithi008.stdout:2/917: rename d6 to d6/db/d60/d4f/d132 22 2024-04-12T05:55:40.406 INFO:tasks.workunit.client.0.smithi008.stdout:7/801: mknod d1/d15/d43/c112 0 2024-04-12T05:55:40.408 INFO:tasks.workunit.client.0.smithi008.stdout:2/918: dread d6/f10d [0,4194304] 0 2024-04-12T05:55:40.413 INFO:tasks.workunit.client.0.smithi008.stdout:2/919: creat d6/db/d6f/de2/d113/f133 x:0 0 0 2024-04-12T05:55:40.418 INFO:tasks.workunit.client.0.smithi008.stdout:0/742: dwrite d2/d4b/fc6 [0,4194304] 0 2024-04-12T05:55:40.418 INFO:tasks.workunit.client.0.smithi008.stdout:2/920: readlink d6/db/d10/l8b 0 2024-04-12T05:55:40.424 INFO:tasks.workunit.client.0.smithi008.stdout:0/743: write d2/d11/d36/d82/fb4 [257429,113971] 0 2024-04-12T05:55:40.425 INFO:tasks.workunit.client.0.smithi008.stdout:6/829: dwrite df/d8c/db1/ddc/d95/fb3 [0,4194304] 0 2024-04-12T05:55:40.429 INFO:tasks.workunit.client.0.smithi008.stdout:7/802: dread d1/d5/da/d11/f35 [4194304,4194304] 0 2024-04-12T05:55:40.430 INFO:tasks.workunit.client.0.smithi008.stdout:7/803: chown d1/d15/fe6 36837 1 2024-04-12T05:55:40.431 INFO:tasks.workunit.client.0.smithi008.stdout:0/744: unlink d2/d11/d36/d42/d86/d9b/cdd 0 2024-04-12T05:55:40.431 INFO:tasks.workunit.client.0.smithi008.stdout:7/804: stat d1/d5/da/d3c/d44/d8a/f3b 0 2024-04-12T05:55:40.434 INFO:tasks.workunit.client.0.smithi008.stdout:7/805: fdatasync d1/d5/da/d60/d7b/f104 0 2024-04-12T05:55:40.436 INFO:tasks.workunit.client.0.smithi008.stdout:2/921: dread d6/f15 [0,4194304] 0 2024-04-12T05:55:40.455 INFO:tasks.workunit.client.0.smithi008.stdout:7/806: dread d1/d5/da/d3c/d44/fa0 [0,4194304] 0 2024-04-12T05:55:40.456 INFO:tasks.workunit.client.0.smithi008.stdout:6/830: write df/d10/d15/d79/f100 [378004,17333] 0 2024-04-12T05:55:40.466 INFO:tasks.workunit.client.0.smithi008.stdout:6/831: dwrite df/d8c/fa3 [0,4194304] 0 2024-04-12T05:55:40.468 INFO:tasks.workunit.client.0.smithi008.stdout:0/745: dread d2/d4b/da2/fb8 [0,4194304] 0 2024-04-12T05:55:40.519 INFO:tasks.workunit.client.0.smithi008.stdout:1/777: sync 2024-04-12T05:55:40.555 INFO:tasks.workunit.client.0.smithi008.stdout:0/746: symlink d2/d11/d36/d42/d86/d9b/lee 0 2024-04-12T05:55:40.556 INFO:tasks.workunit.client.0.smithi008.stdout:1/778: write d4/dd/f9a [1258846,90062] 0 2024-04-12T05:55:40.560 INFO:tasks.workunit.client.0.smithi008.stdout:1/779: symlink d4/dd/d21/d9f/d100/l10a 0 2024-04-12T05:55:40.560 INFO:tasks.workunit.client.0.smithi008.stdout:0/747: dread d2/d11/d36/d47/f91 [0,4194304] 0 2024-04-12T05:55:40.561 INFO:tasks.workunit.client.0.smithi008.stdout:6/832: link df/d10/d15/d75/fa2 df/d21/d4c/dfb/f117 0 2024-04-12T05:55:40.563 INFO:tasks.workunit.client.0.smithi008.stdout:1/780: write d4/d24/d62/fee [716572,84288] 0 2024-04-12T05:55:40.563 INFO:tasks.workunit.client.0.smithi008.stdout:0/748: read d2/d11/d36/d47/f68 [575253,37505] 0 2024-04-12T05:55:40.575 INFO:tasks.workunit.client.0.smithi008.stdout:0/749: dwrite d2/d11/d36/d42/fbb [0,4194304] 0 2024-04-12T05:55:40.580 INFO:tasks.workunit.client.0.smithi008.stdout:0/750: dread - d2/d11/d36/d82/d95/f98 zero size 2024-04-12T05:55:40.585 INFO:tasks.workunit.client.0.smithi008.stdout:7/807: dwrite d1/d15/d43/d54/d77/f89 [0,4194304] 0 2024-04-12T05:55:40.585 INFO:tasks.workunit.client.0.smithi008.stdout:0/751: write d2/d11/f23 [974166,124048] 0 2024-04-12T05:55:40.587 INFO:tasks.workunit.client.0.smithi008.stdout:1/781: dread d4/dd/d17/d26/fdf [0,4194304] 0 2024-04-12T05:55:40.592 INFO:tasks.workunit.client.0.smithi008.stdout:7/808: symlink d1/d5/da/d3c/l113 0 2024-04-12T05:55:40.593 INFO:tasks.workunit.client.0.smithi008.stdout:6/833: dwrite df/d10/d15/dd0/dd1/f6c [0,4194304] 0 2024-04-12T05:55:40.598 INFO:tasks.workunit.client.0.smithi008.stdout:0/752: rename d2/d11/d36/d47/l1c to d2/d24/d85/ddc/lef 0 2024-04-12T05:55:40.599 INFO:tasks.workunit.client.0.smithi008.stdout:6/834: symlink df/d10/d15/d79/l118 0 2024-04-12T05:55:40.603 INFO:tasks.workunit.client.0.smithi008.stdout:0/753: fsync d2/d11/fb0 0 2024-04-12T05:55:40.605 INFO:tasks.workunit.client.0.smithi008.stdout:6/835: rmdir df/d8c/db1/ddc/d78/d8b 39 2024-04-12T05:55:40.605 INFO:tasks.workunit.client.0.smithi008.stdout:7/809: link d1/d5/da/f62 d1/d5/da/d3c/f114 0 2024-04-12T05:55:40.607 INFO:tasks.workunit.client.0.smithi008.stdout:7/810: stat d1/d5/da/d3c/l113 0 2024-04-12T05:55:40.609 INFO:tasks.workunit.client.0.smithi008.stdout:6/836: truncate df/d10/d15/d75/dd6/f104 4590404 0 2024-04-12T05:55:40.611 INFO:tasks.workunit.client.0.smithi008.stdout:1/782: dwrite d4/d1a/f5f [0,4194304] 0 2024-04-12T05:55:40.615 INFO:tasks.workunit.client.0.smithi008.stdout:0/754: mknod d2/d11/d36/d42/da6/dd7/cf0 0 2024-04-12T05:55:40.616 INFO:tasks.workunit.client.0.smithi008.stdout:7/811: mknod d1/d5/da/d3c/d44/d8a/c115 0 2024-04-12T05:55:40.617 INFO:tasks.workunit.client.0.smithi008.stdout:1/783: mkdir d4/d1c/d4f/d10b 0 2024-04-12T05:55:40.618 INFO:tasks.workunit.client.0.smithi008.stdout:7/812: dread - d1/d15/d43/d54/d77/f96 zero size 2024-04-12T05:55:40.619 INFO:tasks.workunit.client.0.smithi008.stdout:6/837: write df/d21/f43 [2667229,49217] 0 2024-04-12T05:55:40.620 INFO:tasks.workunit.client.0.smithi008.stdout:7/813: chown d1/d5/da/d1d/d68 10501271 1 2024-04-12T05:55:40.622 INFO:tasks.workunit.client.0.smithi008.stdout:1/784: symlink d4/d33/d74/d30/d72/d7c/l10c 0 2024-04-12T05:55:40.624 INFO:tasks.workunit.client.0.smithi008.stdout:7/814: creat d1/d5/da/d78/dd6/f116 x:0 0 0 2024-04-12T05:55:40.631 INFO:tasks.workunit.client.0.smithi008.stdout:7/815: mkdir d1/d5/da/d3c/d117 0 2024-04-12T05:55:40.632 INFO:tasks.workunit.client.0.smithi008.stdout:1/785: truncate d4/d6/d11/f42 3873729 0 2024-04-12T05:55:40.640 INFO:tasks.workunit.client.0.smithi008.stdout:0/755: dwrite d2/d11/d36/d47/f94 [0,4194304] 0 2024-04-12T05:55:40.642 INFO:tasks.workunit.client.0.smithi008.stdout:7/816: write d1/d5/da/d3c/d44/d8a/f9b [678133,84973] 0 2024-04-12T05:55:40.647 INFO:tasks.workunit.client.0.smithi008.stdout:6/838: dread df/d8c/db1/ddc/d9b/fbc [0,4194304] 0 2024-04-12T05:55:40.648 INFO:tasks.workunit.client.0.smithi008.stdout:1/786: dwrite d4/d33/d74/d46/d4c/fe0 [0,4194304] 0 2024-04-12T05:55:40.654 INFO:tasks.workunit.client.0.smithi008.stdout:7/817: rename d1/d5/da/d60/d7b/cb5 to d1/d5/da/d11/d103/c118 0 2024-04-12T05:55:40.656 INFO:tasks.workunit.client.0.smithi008.stdout:7/818: symlink d1/d5/da/d60/da1/l119 0 2024-04-12T05:55:40.660 INFO:tasks.workunit.client.0.smithi008.stdout:6/839: dread df/d10/d15/d75/f9f [0,4194304] 0 2024-04-12T05:55:40.665 INFO:tasks.workunit.client.0.smithi008.stdout:1/787: dwrite d4/d1c/db9/fe4 [0,4194304] 0 2024-04-12T05:55:40.668 INFO:tasks.workunit.client.0.smithi008.stdout:1/788: write d4/dd/d17/d91/da4/fea [526966,94356] 0 2024-04-12T05:55:40.671 INFO:tasks.workunit.client.0.smithi008.stdout:1/789: symlink d4/d33/d74/l10d 0 2024-04-12T05:55:40.673 INFO:tasks.workunit.client.0.smithi008.stdout:1/790: mknod d4/d33/d74/da2/c10e 0 2024-04-12T05:55:40.680 INFO:tasks.workunit.client.0.smithi008.stdout:6/840: dread df/d10/d15/d79/f8f [0,4194304] 0 2024-04-12T05:55:40.682 INFO:tasks.workunit.client.0.smithi008.stdout:6/841: stat df/d8c/db1/d10e 0 2024-04-12T05:55:40.683 INFO:tasks.workunit.client.0.smithi008.stdout:1/791: dwrite d4/f47 [0,4194304] 0 2024-04-12T05:55:40.684 INFO:tasks.workunit.client.0.smithi008.stdout:0/756: dread d2/d11/d2a/f88 [0,4194304] 0 2024-04-12T05:55:40.688 INFO:tasks.workunit.client.0.smithi008.stdout:1/792: write d4/d6/d90/fa7 [3430341,38275] 0 2024-04-12T05:55:40.688 INFO:tasks.workunit.client.0.smithi008.stdout:6/842: creat df/d21/d4c/dfb/f119 x:0 0 0 2024-04-12T05:55:40.690 INFO:tasks.workunit.client.0.smithi008.stdout:9/846: sync 2024-04-12T05:55:40.690 INFO:tasks.workunit.client.0.smithi008.stdout:1/793: fsync d4/d6/f40 0 2024-04-12T05:55:40.692 INFO:tasks.workunit.client.0.smithi008.stdout:7/819: dread d1/d5/da/d1d/d68/fe0 [0,4194304] 0 2024-04-12T05:55:40.696 INFO:tasks.workunit.client.0.smithi008.stdout:1/794: creat d4/dd/f10f x:0 0 0 2024-04-12T05:55:40.701 INFO:tasks.workunit.client.0.smithi008.stdout:1/795: mknod d4/d33/d74/d7a/db6/dc9/c110 0 2024-04-12T05:55:40.704 INFO:tasks.workunit.client.0.smithi008.stdout:1/796: rmdir d4/d1c/db9 39 2024-04-12T05:55:40.705 INFO:tasks.workunit.client.0.smithi008.stdout:7/820: creat d1/d5/da/d1d/d82/f11a x:0 0 0 2024-04-12T05:55:40.708 INFO:tasks.workunit.client.0.smithi008.stdout:7/821: creat d1/d5/da/d78/f11b x:0 0 0 2024-04-12T05:55:40.709 INFO:tasks.workunit.client.0.smithi008.stdout:0/757: write d2/d11/f25 [4244556,101352] 0 2024-04-12T05:55:40.709 INFO:tasks.workunit.client.0.smithi008.stdout:1/797: symlink d4/dd/d17/d66/l111 0 2024-04-12T05:55:40.714 INFO:tasks.workunit.client.0.smithi008.stdout:7/822: fdatasync d1/d5/da/f62 0 2024-04-12T05:55:40.714 INFO:tasks.workunit.client.0.smithi008.stdout:1/798: stat d4/d24/d62/lef 0 2024-04-12T05:55:40.714 INFO:tasks.workunit.client.0.smithi008.stdout:7/823: readlink d1/d15/d43/l7a 0 2024-04-12T05:55:40.714 INFO:tasks.workunit.client.0.smithi008.stdout:0/758: mknod d2/d11/d36/d42/da6/cf1 0 2024-04-12T05:55:40.716 INFO:tasks.workunit.client.0.smithi008.stdout:2/922: sync 2024-04-12T05:55:40.716 INFO:tasks.workunit.client.0.smithi008.stdout:8/770: sync 2024-04-12T05:55:40.716 INFO:tasks.workunit.client.0.smithi008.stdout:3/769: sync 2024-04-12T05:55:40.716 INFO:tasks.workunit.client.0.smithi008.stdout:4/707: sync 2024-04-12T05:55:40.726 INFO:tasks.workunit.client.0.smithi008.stdout:6/843: dwrite df/d10/d20/d3e/f5c [0,4194304] 0 2024-04-12T05:55:40.726 INFO:tasks.workunit.client.0.smithi008.stdout:3/770: rmdir d9/d10/d19/d22/d2c 39 2024-04-12T05:55:40.728 INFO:tasks.workunit.client.0.smithi008.stdout:9/847: dwrite d3/df/f28 [4194304,4194304] 0 2024-04-12T05:55:40.733 INFO:tasks.workunit.client.0.smithi008.stdout:3/771: chown d9/f7a 0 1 2024-04-12T05:55:40.733 INFO:tasks.workunit.client.0.smithi008.stdout:0/759: creat d2/d11/d36/d47/ff2 x:0 0 0 2024-04-12T05:55:40.733 INFO:tasks.workunit.client.0.smithi008.stdout:1/799: mknod d4/d33/d74/d30/d72/c112 0 2024-04-12T05:55:40.736 INFO:tasks.workunit.client.0.smithi008.stdout:4/708: symlink d1/d22/d29/d30/d47/d28/d74/ld9 0 2024-04-12T05:55:40.741 INFO:tasks.workunit.client.0.smithi008.stdout:6/844: readlink df/d10/lce 0 2024-04-12T05:55:40.741 INFO:tasks.workunit.client.0.smithi008.stdout:8/771: getdents dd/de/d4c/d66/d6c/df4 0 2024-04-12T05:55:40.745 INFO:tasks.workunit.client.0.smithi008.stdout:8/772: unlink dd/de/d16/f33 0 2024-04-12T05:55:40.745 INFO:tasks.workunit.client.0.smithi008.stdout:4/709: chown d1/d22/d29/d46/l54 62949 1 2024-04-12T05:55:40.748 INFO:tasks.workunit.client.0.smithi008.stdout:0/760: symlink d2/d24/d85/dde/lf3 0 2024-04-12T05:55:40.750 INFO:tasks.workunit.client.0.smithi008.stdout:3/772: rename d9/d10/d19/d22/d2c/lb4 to d9/d10/d105/l106 0 2024-04-12T05:55:40.753 INFO:tasks.workunit.client.0.smithi008.stdout:2/923: dread d6/f43 [4194304,4194304] 0 2024-04-12T05:55:40.755 INFO:tasks.workunit.client.0.smithi008.stdout:8/773: unlink dd/d30/d3b/f55 0 2024-04-12T05:55:40.756 INFO:tasks.workunit.client.0.smithi008.stdout:4/710: rmdir d1/d22/d29/d30/d47/d28/d4a 39 2024-04-12T05:55:40.758 INFO:tasks.workunit.client.0.smithi008.stdout:0/761: readlink d2/d24/d39/laa 0 2024-04-12T05:55:40.761 INFO:tasks.workunit.client.0.smithi008.stdout:7/824: dread d1/d5/da/d1d/dc3/feb [0,4194304] 0 2024-04-12T05:55:40.761 INFO:tasks.workunit.client.0.smithi008.stdout:9/848: dwrite d3/df/d13/d21/d2d/d30/d74/ff8 [0,4194304] 0 2024-04-12T05:55:40.761 INFO:tasks.workunit.client.0.smithi008.stdout:3/773: write d9/d10/d19/d22/d2c/faf [544909,48192] 0 2024-04-12T05:55:40.761 INFO:tasks.workunit.client.0.smithi008.stdout:6/845: rename df/d10/d15/dd0/dd1/f51 to df/d8c/dcc/d106/f11a 0 2024-04-12T05:55:40.766 INFO:tasks.workunit.client.0.smithi008.stdout:4/711: write d1/d22/d29/d46/d56/f80 [1367753,18904] 0 2024-04-12T05:55:40.767 INFO:tasks.workunit.client.0.smithi008.stdout:8/774: fsync dd/de/f2f 0 2024-04-12T05:55:40.767 INFO:tasks.workunit.client.0.smithi008.stdout:2/924: symlink d6/db/d6f/l134 0 2024-04-12T05:55:40.768 INFO:tasks.workunit.client.0.smithi008.stdout:0/762: write d2/d24/fa0 [819311,78300] 0 2024-04-12T05:55:40.775 INFO:tasks.workunit.client.0.smithi008.stdout:6/846: dread - df/d8c/da4/fc5 zero size 2024-04-12T05:55:40.775 INFO:tasks.workunit.client.0.smithi008.stdout:1/800: dread d4/d1a/d57/fcb [0,4194304] 0 2024-04-12T05:55:40.776 INFO:tasks.workunit.client.0.smithi008.stdout:4/712: creat d1/d22/fda x:0 0 0 2024-04-12T05:55:40.777 INFO:tasks.workunit.client.0.smithi008.stdout:7/825: creat d1/d5/da/d60/f11c x:0 0 0 2024-04-12T05:55:40.778 INFO:tasks.workunit.client.0.smithi008.stdout:8/775: chown dd/de/d4c/fcb 1 1 2024-04-12T05:55:40.778 INFO:tasks.workunit.client.0.smithi008.stdout:3/774: mkdir d9/d44/d107 0 2024-04-12T05:55:40.780 INFO:tasks.workunit.client.0.smithi008.stdout:2/925: symlink d6/db/d10/d23/d35/dfc/l135 0 2024-04-12T05:55:40.784 INFO:tasks.workunit.client.0.smithi008.stdout:0/763: truncate d2/d11/d36/d82/d95/f98 701379 0 2024-04-12T05:55:40.787 INFO:tasks.workunit.client.0.smithi008.stdout:8/776: chown dd/de/d16/db1/d97/c10c 706982011 1 2024-04-12T05:55:40.789 INFO:tasks.workunit.client.0.smithi008.stdout:7/826: truncate d1/d5/da/d78/dab/fe8 956274 0 2024-04-12T05:55:40.789 INFO:tasks.workunit.client.0.smithi008.stdout:9/849: dwrite d3/df/d13/f107 [0,4194304] 0 2024-04-12T05:55:40.792 INFO:tasks.workunit.client.0.smithi008.stdout:3/775: unlink d9/c95 0 2024-04-12T05:55:40.793 INFO:tasks.workunit.client.0.smithi008.stdout:2/926: mknod d6/db/d10/d23/d35/d11f/c136 0 2024-04-12T05:55:40.794 INFO:tasks.workunit.client.0.smithi008.stdout:6/847: dwrite df/d10/d15/d79/f100 [0,4194304] 0 2024-04-12T05:55:40.795 INFO:tasks.workunit.client.0.smithi008.stdout:8/777: write dd/f49 [1041446,115746] 0 2024-04-12T05:55:40.796 INFO:tasks.workunit.client.0.smithi008.stdout:1/801: creat d4/d33/d74/d30/d72/f113 x:0 0 0 2024-04-12T05:55:40.797 INFO:tasks.workunit.client.0.smithi008.stdout:0/764: truncate d2/d11/d36/d47/f87 1520457 0 2024-04-12T05:55:40.798 INFO:tasks.workunit.client.0.smithi008.stdout:7/827: truncate d1/d5/da/d1d/ffe 34373 0 2024-04-12T05:55:40.800 INFO:tasks.workunit.client.0.smithi008.stdout:9/850: rename d3/fc5 to d3/d22/f121 0 2024-04-12T05:55:40.801 INFO:tasks.workunit.client.0.smithi008.stdout:4/713: getdents d1/d22/d29 0 2024-04-12T05:55:40.803 INFO:tasks.workunit.client.0.smithi008.stdout:1/802: mknod d4/dd/d21/d9f/d100/c114 0 2024-04-12T05:55:40.808 INFO:tasks.workunit.client.0.smithi008.stdout:6/848: write df/d10/d15/dd0/dd1/f6e [4375787,16437] 0 2024-04-12T05:55:40.811 INFO:tasks.workunit.client.0.smithi008.stdout:3/776: write d9/d11/f85 [433732,112163] 0 2024-04-12T05:55:40.811 INFO:tasks.workunit.client.0.smithi008.stdout:9/851: chown d3/df/d13/d21/d5c/dd7/cf4 60115246 1 2024-04-12T05:55:40.811 INFO:tasks.workunit.client.0.smithi008.stdout:8/778: rename dd/de/d4c/d66/da9 to dd/de/d16/d10d 0 2024-04-12T05:55:40.811 INFO:tasks.workunit.client.0.smithi008.stdout:7/828: mkdir d1/d5/da/d11/d56/d8f/d11d 0 2024-04-12T05:55:40.815 INFO:tasks.workunit.client.0.smithi008.stdout:6/849: stat ca 0 2024-04-12T05:55:40.816 INFO:tasks.workunit.client.0.smithi008.stdout:1/803: mknod d4/d6/d11/ddb/c115 0 2024-04-12T05:55:40.817 INFO:tasks.workunit.client.0.smithi008.stdout:0/765: write d2/d11/d36/d82/d95/fba [4501474,26178] 0 2024-04-12T05:55:40.818 INFO:tasks.workunit.client.0.smithi008.stdout:9/852: fsync d3/df/d13/d21/d5c/d8f/f10f 0 2024-04-12T05:55:40.818 INFO:tasks.workunit.client.0.smithi008.stdout:2/927: dwrite d6/db/d60/d84/fe4 [4194304,4194304] 0 2024-04-12T05:55:40.819 INFO:tasks.workunit.client.0.smithi008.stdout:4/714: mkdir d1/d22/d29/d30/d60/dd0/ddb 0 2024-04-12T05:55:40.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:40 smithi194 ceph-mon[49771]: pgmap v16: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 74 MiB/s rd, 191 MiB/s wr, 695 op/s 2024-04-12T05:55:40.821 INFO:tasks.workunit.client.0.smithi008.stdout:7/829: fdatasync d1/ffd 0 2024-04-12T05:55:40.822 INFO:tasks.workunit.client.0.smithi008.stdout:9/853: stat d3/f25 0 2024-04-12T05:55:40.826 INFO:tasks.workunit.client.0.smithi008.stdout:8/779: dread - dd/de/d4c/d66/d6d/dca/f10b zero size 2024-04-12T05:55:40.832 INFO:tasks.workunit.client.0.smithi008.stdout:2/928: unlink d6/db/d10/dcc/l124 0 2024-04-12T05:55:40.833 INFO:tasks.workunit.client.0.smithi008.stdout:3/777: rename d9/d44/f63 to d9/d10/d19/d3a/d7c/f108 0 2024-04-12T05:55:40.837 INFO:tasks.workunit.client.0.smithi008.stdout:6/850: creat df/d8c/db1/ddc/d105/f11b x:0 0 0 2024-04-12T05:55:40.841 INFO:tasks.workunit.client.0.smithi008.stdout:6/851: dread - df/d10/d15/fda zero size 2024-04-12T05:55:40.841 INFO:tasks.workunit.client.0.smithi008.stdout:0/766: dwrite d2/d11/d36/d82/d95/fb6 [0,4194304] 0 2024-04-12T05:55:40.842 INFO:tasks.workunit.client.0.smithi008.stdout:4/715: link d1/d22/d29/fd2 d1/d4f/dd5/fdc 0 2024-04-12T05:55:40.844 INFO:tasks.workunit.client.0.smithi008.stdout:6/852: chown df/d10/f56 27483678 1 2024-04-12T05:55:40.845 INFO:tasks.workunit.client.0.smithi008.stdout:8/780: dwrite dd/de/d16/f28 [0,4194304] 0 2024-04-12T05:55:40.848 INFO:tasks.workunit.client.0.smithi008.stdout:1/804: write d4/d1c/db9/fe4 [5216675,127005] 0 2024-04-12T05:55:40.851 INFO:tasks.workunit.client.0.smithi008.stdout:2/929: unlink d6/db/d10/d4c/fd1 0 2024-04-12T05:55:40.858 INFO:tasks.workunit.client.0.smithi008.stdout:9/854: dwrite d3/d22/d2b/d58/f3f [4194304,4194304] 0 2024-04-12T05:55:40.863 INFO:tasks.workunit.client.0.smithi008.stdout:8/781: chown dd/de/c10 127430489 1 2024-04-12T05:55:40.864 INFO:tasks.workunit.client.0.smithi008.stdout:7/830: creat d1/d15/d43/f11e x:0 0 0 2024-04-12T05:55:40.864 INFO:tasks.workunit.client.0.smithi008.stdout:8/782: readlink dd/la8 0 2024-04-12T05:55:40.868 INFO:tasks.workunit.client.0.smithi008.stdout:3/778: symlink d9/d10/d19/d3a/dad/dcf/de3/l109 0 2024-04-12T05:55:40.869 INFO:tasks.workunit.client.0.smithi008.stdout:0/767: dwrite d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:40.874 INFO:tasks.workunit.client.0.smithi008.stdout:7/831: creat d1/d5/da/d3c/dd3/f11f x:0 0 0 2024-04-12T05:55:40.874 INFO:tasks.workunit.client.0.smithi008.stdout:4/716: creat d1/d22/d29/d46/fdd x:0 0 0 2024-04-12T05:55:40.875 INFO:tasks.workunit.client.0.smithi008.stdout:7/832: chown d1/d5/da/f2a 26458490 1 2024-04-12T05:55:40.875 INFO:tasks.workunit.client.0.smithi008.stdout:3/779: mknod d9/d10/d19/d22/dec/c10a 0 2024-04-12T05:55:40.877 INFO:tasks.workunit.client.0.smithi008.stdout:4/717: dread - d1/d4f/fc7 zero size 2024-04-12T05:55:40.880 INFO:tasks.workunit.client.0.smithi008.stdout:8/783: dread fb [4194304,4194304] 0 2024-04-12T05:55:40.882 INFO:tasks.workunit.client.0.smithi008.stdout:2/930: creat d6/db/d60/f137 x:0 0 0 2024-04-12T05:55:40.885 INFO:tasks.workunit.client.0.smithi008.stdout:9/855: creat d3/df/d13/f122 x:0 0 0 2024-04-12T05:55:40.887 INFO:tasks.workunit.client.0.smithi008.stdout:0/768: creat d2/d11/d2a/ff4 x:0 0 0 2024-04-12T05:55:40.889 INFO:tasks.workunit.client.0.smithi008.stdout:3/780: creat d9/d10/d19/d3a/dad/dcf/f10b x:0 0 0 2024-04-12T05:55:40.891 INFO:tasks.workunit.client.0.smithi008.stdout:2/931: symlink d6/db/d6f/de2/d113/dec/l138 0 2024-04-12T05:55:40.891 INFO:tasks.workunit.client.0.smithi008.stdout:9/856: dread - d3/df/d13/d21/d5c/d8f/feb zero size 2024-04-12T05:55:40.892 INFO:tasks.workunit.client.0.smithi008.stdout:4/718: dread d1/d8/dc5/fcc [0,4194304] 0 2024-04-12T05:55:40.895 INFO:tasks.workunit.client.0.smithi008.stdout:0/769: chown d2/d11/d36/d42/da6/fbe 20938 1 2024-04-12T05:55:40.898 INFO:tasks.workunit.client.0.smithi008.stdout:4/719: mknod d1/d4f/dd5/cde 0 2024-04-12T05:55:40.901 INFO:tasks.workunit.client.0.smithi008.stdout:0/770: rename d2/d4b/da2/fc0 to d2/d11/d36/dd6/ff5 0 2024-04-12T05:55:40.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:40 smithi008 ceph-mon[39722]: pgmap v16: 65 pgs: 65 active+clean; 1.6 GiB data, 4.8 GiB used, 532 GiB / 536 GiB avail; 74 MiB/s rd, 191 MiB/s wr, 695 op/s 2024-04-12T05:55:40.907 INFO:tasks.workunit.client.0.smithi008.stdout:2/932: link d6/db/d60/d11e/d7d/f98 d6/db/d60/d84/f139 0 2024-04-12T05:55:40.909 INFO:tasks.workunit.client.0.smithi008.stdout:6/853: write df/d10/d20/d3e/f61 [169088,119448] 0 2024-04-12T05:55:40.915 INFO:tasks.workunit.client.0.smithi008.stdout:0/771: rename d2/d11/d36/d42/d86/d9b/cbf to d2/cf6 0 2024-04-12T05:55:40.916 INFO:tasks.workunit.client.0.smithi008.stdout:4/720: dwrite d1/d22/d29/f7c [0,4194304] 0 2024-04-12T05:55:40.922 INFO:tasks.workunit.client.0.smithi008.stdout:8/784: dread dd/d4f/d52/f69 [0,4194304] 0 2024-04-12T05:55:40.926 INFO:tasks.workunit.client.0.smithi008.stdout:0/772: mkdir d2/d24/d85/dde/df7 0 2024-04-12T05:55:40.932 INFO:tasks.workunit.client.0.smithi008.stdout:6/854: truncate df/d10/d20/f49 209725 0 2024-04-12T05:55:40.934 INFO:tasks.workunit.client.0.smithi008.stdout:4/721: creat d1/d22/d29/d30/d47/d28/d4a/fdf x:0 0 0 2024-04-12T05:55:40.934 INFO:tasks.workunit.client.0.smithi008.stdout:3/781: dread d9/d2e/d3d/f70 [0,4194304] 0 2024-04-12T05:55:40.934 INFO:tasks.workunit.client.0.smithi008.stdout:0/773: write d2/d24/fa0 [2812527,97672] 0 2024-04-12T05:55:40.937 INFO:tasks.workunit.client.0.smithi008.stdout:8/785: write dd/de/d16/db1/d97/fe5 [1296477,123102] 0 2024-04-12T05:55:40.938 INFO:tasks.workunit.client.0.smithi008.stdout:6/855: rmdir df/d10 39 2024-04-12T05:55:40.942 INFO:tasks.workunit.client.0.smithi008.stdout:9/857: dwrite d3/df/d10/f17 [0,4194304] 0 2024-04-12T05:55:40.944 INFO:tasks.workunit.client.0.smithi008.stdout:7/833: dwrite d1/d15/d43/d54/d58/d5e/d7f/fba [0,4194304] 0 2024-04-12T05:55:40.948 INFO:tasks.workunit.client.0.smithi008.stdout:3/782: fdatasync d9/d2e/f74 0 2024-04-12T05:55:40.950 INFO:tasks.workunit.client.0.smithi008.stdout:8/786: unlink dd/f1f 0 2024-04-12T05:55:40.957 INFO:tasks.workunit.client.0.smithi008.stdout:7/834: symlink d1/d15/d43/d5a/l120 0 2024-04-12T05:55:40.957 INFO:tasks.workunit.client.0.smithi008.stdout:0/774: getdents d2/d24/dbd/de9 0 2024-04-12T05:55:40.958 INFO:tasks.workunit.client.0.smithi008.stdout:9/858: rename d3/df/d13/d21/d2d/d30/de2/c10d to d3/d22/d2b/d58/d38/c123 0 2024-04-12T05:55:40.959 INFO:tasks.workunit.client.0.smithi008.stdout:3/783: creat d9/d10/d19/d3a/dad/dcf/de3/f10c x:0 0 0 2024-04-12T05:55:40.960 INFO:tasks.workunit.client.0.smithi008.stdout:8/787: truncate dd/de/d4c/fec 676581 0 2024-04-12T05:55:40.960 INFO:tasks.workunit.client.0.smithi008.stdout:2/933: dread d6/db/fd [0,4194304] 0 2024-04-12T05:55:40.965 INFO:tasks.workunit.client.0.smithi008.stdout:2/934: stat d6/db/d10/fda 0 2024-04-12T05:55:40.967 INFO:tasks.workunit.client.0.smithi008.stdout:6/856: dread df/d10/d15/d75/fba [0,4194304] 0 2024-04-12T05:55:40.967 INFO:tasks.workunit.client.0.smithi008.stdout:7/835: rename d1/l10d to d1/d5/da/d1d/l121 0 2024-04-12T05:55:40.968 INFO:tasks.workunit.client.0.smithi008.stdout:2/935: mknod d6/db/d60/d84/c13a 0 2024-04-12T05:55:40.969 INFO:tasks.workunit.client.0.smithi008.stdout:6/857: fsync df/d8c/f115 0 2024-04-12T05:55:40.969 INFO:tasks.workunit.client.0.smithi008.stdout:7/836: dread - d1/d5/da/d78/f11b zero size 2024-04-12T05:55:41.039 INFO:tasks.workunit.client.0.smithi008.stdout:8/788: rename dd/de/d16/d50/d56/d61/d77 to dd/de/d16/d10e 0 2024-04-12T05:55:41.039 INFO:tasks.workunit.client.0.smithi008.stdout:2/936: truncate d6/db/d6f/dea/f121 864511 0 2024-04-12T05:55:41.040 INFO:tasks.workunit.client.0.smithi008.stdout:4/722: truncate d1/d22/d29/d30/d47/d28/d4a/f95 1209904 0 2024-04-12T05:55:41.043 INFO:tasks.workunit.client.0.smithi008.stdout:4/723: mkdir d1/d8/de0 0 2024-04-12T05:55:41.045 INFO:tasks.workunit.client.0.smithi008.stdout:7/837: truncate d1/d5/da/d78/dd6/f116 820463 0 2024-04-12T05:55:41.045 INFO:tasks.workunit.client.0.smithi008.stdout:2/937: mkdir d6/db/d6f/de2/d113/d13b 0 2024-04-12T05:55:41.047 INFO:tasks.workunit.client.0.smithi008.stdout:3/784: dread d9/f82 [0,4194304] 0 2024-04-12T05:55:41.049 INFO:tasks.workunit.client.0.smithi008.stdout:8/789: getdents dd 0 2024-04-12T05:55:41.050 INFO:tasks.workunit.client.0.smithi008.stdout:9/859: dread d3/d22/d2b/d58/f1d [0,4194304] 0 2024-04-12T05:55:41.050 INFO:tasks.workunit.client.0.smithi008.stdout:6/858: dwrite df/d21/d4c/fe8 [0,4194304] 0 2024-04-12T05:55:41.050 INFO:tasks.workunit.client.0.smithi008.stdout:3/785: stat d9/d10/d19/d3a/d7c 0 2024-04-12T05:55:41.112 INFO:tasks.workunit.client.0.smithi008.stdout:2/938: fsync d6/f16 0 2024-04-12T05:55:41.112 INFO:tasks.workunit.client.0.smithi008.stdout:4/724: getdents d1/d22/d29/d30/d60/dd0/ddb 0 2024-04-12T05:55:41.114 INFO:tasks.workunit.client.0.smithi008.stdout:3/786: creat d9/d2e/d3d/f10d x:0 0 0 2024-04-12T05:55:41.115 INFO:tasks.workunit.client.0.smithi008.stdout:6/859: unlink df/d10/d15/d75/l8e 0 2024-04-12T05:55:41.200 INFO:tasks.workunit.client.0.smithi008.stdout:9/860: rename d3/d22/d2b/d4e to d3/d22/d2b/d58/d36/d124 0 2024-04-12T05:55:41.201 INFO:tasks.workunit.client.0.smithi008.stdout:7/838: creat d1/d15/d43/ddb/f122 x:0 0 0 2024-04-12T05:55:41.204 INFO:tasks.workunit.client.0.smithi008.stdout:3/787: creat d9/d11/d5b/f10e x:0 0 0 2024-04-12T05:55:41.204 INFO:tasks.workunit.client.0.smithi008.stdout:3/788: fdatasync d9/d10/f9c 0 2024-04-12T05:55:41.205 INFO:tasks.workunit.client.0.smithi008.stdout:9/861: mknod d3/d6a/c125 0 2024-04-12T05:55:41.206 INFO:tasks.workunit.client.0.smithi008.stdout:4/725: symlink d1/d22/d29/d30/d60/dd0/ddb/le1 0 2024-04-12T05:55:41.208 INFO:tasks.workunit.client.0.smithi008.stdout:9/862: truncate d3/d22/d2b/d58/d36/f3c 1383072 0 2024-04-12T05:55:41.209 INFO:tasks.workunit.client.0.smithi008.stdout:7/839: creat d1/d5/da/d11/d109/f123 x:0 0 0 2024-04-12T05:55:41.209 INFO:tasks.workunit.client.0.smithi008.stdout:7/840: stat d1/f6 0 2024-04-12T05:55:41.212 INFO:tasks.workunit.client.0.smithi008.stdout:6/860: rename df/d10/d15/d79/leb to df/l11c 0 2024-04-12T05:55:41.212 INFO:tasks.workunit.client.0.smithi008.stdout:2/939: getdents d6/db/d29 0 2024-04-12T05:55:41.215 INFO:tasks.workunit.client.0.smithi008.stdout:9/863: write d3/fa [2739711,25388] 0 2024-04-12T05:55:41.216 INFO:tasks.workunit.client.0.smithi008.stdout:9/864: fsync d3/df/d13/d21/d2d/d30/de2/f10a 0 2024-04-12T05:55:41.217 INFO:tasks.workunit.client.0.smithi008.stdout:7/841: creat d1/d5/da/d11/d56/f124 x:0 0 0 2024-04-12T05:55:41.218 INFO:tasks.workunit.client.0.smithi008.stdout:4/726: symlink d1/d4f/dcb/le2 0 2024-04-12T05:55:41.219 INFO:tasks.workunit.client.0.smithi008.stdout:2/940: readlink d6/db/d10/d23/ldf 0 2024-04-12T05:55:41.221 INFO:tasks.workunit.client.0.smithi008.stdout:3/789: unlink d9/dd/ce 0 2024-04-12T05:55:41.221 INFO:tasks.workunit.client.0.smithi008.stdout:2/941: chown d6/db/d60/d11e/d7d/la8 320 1 2024-04-12T05:55:41.222 INFO:tasks.workunit.client.0.smithi008.stdout:6/861: symlink df/d8c/db1/d10e/l11d 0 2024-04-12T05:55:41.224 INFO:tasks.workunit.client.0.smithi008.stdout:6/862: fsync df/d10/d15/dd0/dd1/fb7 0 2024-04-12T05:55:41.224 INFO:tasks.workunit.client.0.smithi008.stdout:7/842: mknod d1/d5/da/c125 0 2024-04-12T05:55:41.232 INFO:tasks.workunit.client.0.smithi008.stdout:6/863: dwrite f4 [0,4194304] 0 2024-04-12T05:55:41.234 INFO:tasks.workunit.client.0.smithi008.stdout:9/865: write d3/d22/d2b/d58/f27 [671078,26106] 0 2024-04-12T05:55:41.239 INFO:tasks.workunit.client.0.smithi008.stdout:4/727: link d1/db/db7/lc9 d1/d22/d29/d30/d60/dd6/le3 0 2024-04-12T05:55:41.240 INFO:tasks.workunit.client.0.smithi008.stdout:2/942: dread d6/db/d60/d57/dd9/fe9 [0,4194304] 0 2024-04-12T05:55:41.243 INFO:tasks.workunit.client.0.smithi008.stdout:4/728: fdatasync d1/d22/d29/d30/d60/fa3 0 2024-04-12T05:55:41.245 INFO:tasks.workunit.client.0.smithi008.stdout:6/864: dwrite df/d10/d15/dd0/ff8 [0,4194304] 0 2024-04-12T05:55:41.245 INFO:tasks.workunit.client.0.smithi008.stdout:3/790: dread d9/db7/fc1 [0,4194304] 0 2024-04-12T05:55:41.245 INFO:tasks.workunit.client.0.smithi008.stdout:2/943: chown d6/db/d60/f137 20658 1 2024-04-12T05:55:41.249 INFO:tasks.workunit.client.0.smithi008.stdout:7/843: readlink d1/d5/da/d1d/d82/d9f/lcd 0 2024-04-12T05:55:41.252 INFO:tasks.workunit.client.0.smithi008.stdout:4/729: creat d1/d4f/dd5/fe4 x:0 0 0 2024-04-12T05:55:41.252 INFO:tasks.workunit.client.0.smithi008.stdout:3/791: chown d9/d10/d4b/lf8 134855104 1 2024-04-12T05:55:41.255 INFO:tasks.workunit.client.0.smithi008.stdout:9/866: getdents d3/df/d13/d21/d5c/da4 0 2024-04-12T05:55:41.256 INFO:tasks.workunit.client.0.smithi008.stdout:6/865: dread df/d10/d15/dd0/dd1/f8d [0,4194304] 0 2024-04-12T05:55:41.257 INFO:tasks.workunit.client.0.smithi008.stdout:7/844: unlink d1/d5/l38 0 2024-04-12T05:55:41.261 INFO:tasks.workunit.client.0.smithi008.stdout:6/866: dread df/d10/d15/dd0/dd1/f82 [0,4194304] 0 2024-04-12T05:55:41.262 INFO:tasks.workunit.client.0.smithi008.stdout:4/730: rename d1/d22/d29/d30/d47/d28/f5c to d1/d22/d29/d46/d56/fe5 0 2024-04-12T05:55:41.263 INFO:tasks.workunit.client.0.smithi008.stdout:1/805: sync 2024-04-12T05:55:41.263 INFO:tasks.workunit.client.0.smithi008.stdout:7/845: symlink d1/d15/df7/l126 0 2024-04-12T05:55:41.269 INFO:tasks.workunit.client.0.smithi008.stdout:1/806: fdatasync d4/dd/d17/d26/fa9 0 2024-04-12T05:55:41.270 INFO:tasks.workunit.client.0.smithi008.stdout:7/846: creat d1/d5/da/d60/da1/f127 x:0 0 0 2024-04-12T05:55:41.271 INFO:tasks.workunit.client.0.smithi008.stdout:9/867: link d3/df/d13/d21/d2d/d75/lb7 d3/d22/d2b/d58/d90/ded/l126 0 2024-04-12T05:55:41.271 INFO:tasks.workunit.client.0.smithi008.stdout:2/944: dread d6/db/d38/d41/fad [0,4194304] 0 2024-04-12T05:55:41.273 INFO:tasks.workunit.client.0.smithi008.stdout:4/731: rename d1/d22/d29/d30/d47/l7d to d1/d8/le6 0 2024-04-12T05:55:41.277 INFO:tasks.workunit.client.0.smithi008.stdout:4/732: chown d1/d22/d29/d30/d47/d28/d4a 5 1 2024-04-12T05:55:41.279 INFO:tasks.workunit.client.0.smithi008.stdout:7/847: fsync d1/d5/da/d3c/fbe 0 2024-04-12T05:55:41.285 INFO:tasks.workunit.client.0.smithi008.stdout:1/807: write d4/dd/d17/f31 [3802467,12417] 0 2024-04-12T05:55:41.286 INFO:tasks.workunit.client.0.smithi008.stdout:3/792: dread d9/f1c [0,4194304] 0 2024-04-12T05:55:41.287 INFO:tasks.workunit.client.0.smithi008.stdout:9/868: getdents d3/d22/d2b/d58/d90/db1 0 2024-04-12T05:55:41.288 INFO:tasks.workunit.client.0.smithi008.stdout:1/808: stat d4/d33/d74 0 2024-04-12T05:55:41.290 INFO:tasks.workunit.client.0.smithi008.stdout:4/733: rename d1/d22/d29/d30/d47/d28/fae to d1/d22/d29/d46/da7/fe7 0 2024-04-12T05:55:41.296 INFO:tasks.workunit.client.0.smithi008.stdout:2/945: dwrite d6/db/f4b [0,4194304] 0 2024-04-12T05:55:41.296 INFO:tasks.workunit.client.0.smithi008.stdout:1/809: dread - d4/dd/d17/d91/f105 zero size 2024-04-12T05:55:41.296 INFO:tasks.workunit.client.0.smithi008.stdout:6/867: dwrite df/d21/f22 [4194304,4194304] 0 2024-04-12T05:55:41.297 INFO:tasks.workunit.client.0.smithi008.stdout:3/793: rmdir d9/d10/d19/d22 39 2024-04-12T05:55:41.300 INFO:tasks.workunit.client.0.smithi008.stdout:2/946: stat d6/db/d38/f75 0 2024-04-12T05:55:41.303 INFO:tasks.workunit.client.0.smithi008.stdout:6/868: mknod df/d8c/db1/d10e/c11e 0 2024-04-12T05:55:41.305 INFO:tasks.workunit.client.0.smithi008.stdout:7/848: dwrite d1/d5/da/d3c/f114 [0,4194304] 0 2024-04-12T05:55:41.305 INFO:tasks.workunit.client.0.smithi008.stdout:2/947: write d6/db/d60/d11e/fb4 [4266889,20089] 0 2024-04-12T05:55:41.306 INFO:tasks.workunit.client.0.smithi008.stdout:1/810: creat d4/d24/d62/dfd/f116 x:0 0 0 2024-04-12T05:55:41.306 INFO:tasks.workunit.client.0.smithi008.stdout:9/869: rename d3/df/d13/cde to d3/df/d13/d21/d2d/dbb/c127 0 2024-04-12T05:55:41.311 INFO:tasks.workunit.client.0.smithi008.stdout:9/870: chown d3/d22/d2b/c3a 199846491 1 2024-04-12T05:55:41.312 INFO:tasks.workunit.client.0.smithi008.stdout:2/948: symlink d6/l13c 0 2024-04-12T05:55:41.315 INFO:tasks.workunit.client.0.smithi008.stdout:7/849: mknod d1/d5/da/d60/da1/c128 0 2024-04-12T05:55:41.319 INFO:tasks.workunit.client.0.smithi008.stdout:7/850: stat d1/d5/da/d3c/d44/d8a/f3b 0 2024-04-12T05:55:41.322 INFO:tasks.workunit.client.0.smithi008.stdout:2/949: mkdir d6/db/d10/db2/d13d 0 2024-04-12T05:55:41.322 INFO:tasks.workunit.client.0.smithi008.stdout:4/734: dwrite d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:41.322 INFO:tasks.workunit.client.0.smithi008.stdout:9/871: readlink d3/d22/d2b/d58/d38/lc7 0 2024-04-12T05:55:41.325 INFO:tasks.workunit.client.0.smithi008.stdout:1/811: creat d4/dd/d17/f117 x:0 0 0 2024-04-12T05:55:41.334 INFO:tasks.workunit.client.0.smithi008.stdout:4/735: stat d1/d8/c1d 0 2024-04-12T05:55:41.336 INFO:tasks.workunit.client.0.smithi008.stdout:7/851: write d1/d15/d43/d54/d58/f67 [3816267,24051] 0 2024-04-12T05:55:41.341 INFO:tasks.workunit.client.0.smithi008.stdout:6/869: dwrite df/d8c/db1/ddc/d78/ff7 [0,4194304] 0 2024-04-12T05:55:41.341 INFO:tasks.workunit.client.0.smithi008.stdout:2/950: read d6/db/d6f/f88 [1674378,59586] 0 2024-04-12T05:55:41.342 INFO:tasks.workunit.client.0.smithi008.stdout:1/812: fdatasync d4/dd/d17/d66/fa8 0 2024-04-12T05:55:41.344 INFO:tasks.workunit.client.0.smithi008.stdout:4/736: chown d1/d22/d29/d46/d56/l81 265219297 1 2024-04-12T05:55:41.347 INFO:tasks.workunit.client.0.smithi008.stdout:3/794: dwrite d9/d10/f55 [0,4194304] 0 2024-04-12T05:55:41.350 INFO:tasks.workunit.client.0.smithi008.stdout:6/870: rmdir df/d10/d15/d75/dd6 39 2024-04-12T05:55:41.351 INFO:tasks.workunit.client.0.smithi008.stdout:2/951: chown d6/db/d6f/dea 80243 1 2024-04-12T05:55:41.353 INFO:tasks.workunit.client.0.smithi008.stdout:1/813: chown d4/d33/dc5/ce9 8698957 1 2024-04-12T05:55:41.358 INFO:tasks.workunit.client.0.smithi008.stdout:6/871: dread - df/d8c/db1/ddc/d95/fa7 zero size 2024-04-12T05:55:41.358 INFO:tasks.workunit.client.0.smithi008.stdout:3/795: creat d9/d10/d19/f10f x:0 0 0 2024-04-12T05:55:41.360 INFO:tasks.workunit.client.0.smithi008.stdout:4/737: rename d1/d22/d29/d30/d60/fa3 to d1/db/db7/fe8 0 2024-04-12T05:55:41.360 INFO:tasks.workunit.client.0.smithi008.stdout:7/852: dwrite d1/d5/da/d3c/fc1 [0,4194304] 0 2024-04-12T05:55:41.361 INFO:tasks.workunit.client.0.smithi008.stdout:8/790: sync 2024-04-12T05:55:41.361 INFO:tasks.workunit.client.0.smithi008.stdout:0/775: sync 2024-04-12T05:55:41.364 INFO:tasks.workunit.client.0.smithi008.stdout:2/952: read - d6/db/d10/d4c/f5a zero size 2024-04-12T05:55:41.368 INFO:tasks.workunit.client.0.smithi008.stdout:8/791: mknod dd/de/d4c/d66/d6c/c10f 0 2024-04-12T05:55:41.372 INFO:tasks.workunit.client.0.smithi008.stdout:9/872: dread d3/df/f28 [0,4194304] 0 2024-04-12T05:55:41.373 INFO:tasks.workunit.client.0.smithi008.stdout:8/792: write dd/de/d16/f28 [1886313,56791] 0 2024-04-12T05:55:41.374 INFO:tasks.workunit.client.0.smithi008.stdout:1/814: link d4/d6/d11/ddb/lf6 d4/d6/l118 0 2024-04-12T05:55:41.374 INFO:tasks.workunit.client.0.smithi008.stdout:4/738: symlink d1/d22/d29/d46/d56/daf/le9 0 2024-04-12T05:55:41.375 INFO:tasks.workunit.client.0.smithi008.stdout:0/776: fdatasync d2/d11/d2a/f5d 0 2024-04-12T05:55:41.376 INFO:tasks.workunit.client.0.smithi008.stdout:7/853: creat d1/d5/da/f129 x:0 0 0 2024-04-12T05:55:41.377 INFO:tasks.workunit.client.0.smithi008.stdout:6/872: write df/d21/d4c/f58 [1781633,120340] 0 2024-04-12T05:55:41.378 INFO:tasks.workunit.client.0.smithi008.stdout:2/953: symlink d6/db/d6f/de2/d113/db5/d122/l13e 0 2024-04-12T05:55:41.380 INFO:tasks.workunit.client.0.smithi008.stdout:4/739: fsync d1/d22/d29/d30/d47/d28/fa1 0 2024-04-12T05:55:41.384 INFO:tasks.workunit.client.0.smithi008.stdout:8/793: fsync dd/de/d16/db1/d97/fa3 0 2024-04-12T05:55:41.386 INFO:tasks.workunit.client.0.smithi008.stdout:4/740: fsync d1/d22/d29/d46/fb6 0 2024-04-12T05:55:41.386 INFO:tasks.workunit.client.0.smithi008.stdout:9/873: creat d3/d22/d2b/d58/d36/f128 x:0 0 0 2024-04-12T05:55:41.386 INFO:tasks.workunit.client.0.smithi008.stdout:7/854: dread d1/d5/da/d1d/ffe [0,4194304] 0 2024-04-12T05:55:41.386 INFO:tasks.workunit.client.0.smithi008.stdout:6/873: mknod df/d10/d20/d3e/db4/c11f 0 2024-04-12T05:55:41.388 INFO:tasks.workunit.client.0.smithi008.stdout:3/796: link d9/c40 d9/d10/d19/d3a/d67/c110 0 2024-04-12T05:55:41.390 INFO:tasks.workunit.client.0.smithi008.stdout:0/777: unlink d2/d11/d36/d42/da6/fbe 0 2024-04-12T05:55:41.394 INFO:tasks.workunit.client.0.smithi008.stdout:1/815: dwrite d4/dd/d17/f31 [0,4194304] 0 2024-04-12T05:55:41.395 INFO:tasks.workunit.client.0.smithi008.stdout:7/855: stat d1/d5/da/d3c/d44/c80 0 2024-04-12T05:55:41.399 INFO:tasks.workunit.client.0.smithi008.stdout:1/816: dread - d4/dd/d17/f103 zero size 2024-04-12T05:55:41.402 INFO:tasks.workunit.client.0.smithi008.stdout:9/874: creat d3/df/d13/d21/d2d/d30/d74/de9/df2/f129 x:0 0 0 2024-04-12T05:55:41.405 INFO:tasks.workunit.client.0.smithi008.stdout:0/778: mkdir d2/d11/d2a/db5/df8 0 2024-04-12T05:55:41.405 INFO:tasks.workunit.client.0.smithi008.stdout:2/954: symlink d6/db/d60/l13f 0 2024-04-12T05:55:41.406 INFO:tasks.workunit.client.0.smithi008.stdout:1/817: dread - d4/d1c/db9/dc1/fe5 zero size 2024-04-12T05:55:41.406 INFO:tasks.workunit.client.0.smithi008.stdout:6/874: read df/d10/d15/dd0/dd1/f5b [250348,28844] 0 2024-04-12T05:55:41.409 INFO:tasks.workunit.client.0.smithi008.stdout:6/875: readlink df/d8c/da4/l103 0 2024-04-12T05:55:41.409 INFO:tasks.workunit.client.0.smithi008.stdout:2/955: readlink d6/db/d10/d23/d35/l9d 0 2024-04-12T05:55:41.414 INFO:tasks.workunit.client.0.smithi008.stdout:7/856: mkdir d1/d5/da/d12a 0 2024-04-12T05:55:41.414 INFO:tasks.workunit.client.0.smithi008.stdout:0/779: mkdir d2/d11/d36/d82/df9 0 2024-04-12T05:55:41.414 INFO:tasks.workunit.client.0.smithi008.stdout:8/794: rename dd/d4f/df5/c106 to dd/de/d16/c110 0 2024-04-12T05:55:41.416 INFO:tasks.workunit.client.0.smithi008.stdout:1/818: mknod d4/d33/d74/d7a/c119 0 2024-04-12T05:55:41.417 INFO:tasks.workunit.client.0.smithi008.stdout:3/797: getdents d9/d37/d69 0 2024-04-12T05:55:41.421 INFO:tasks.workunit.client.0.smithi008.stdout:4/741: link d1/d22/d29/d46/l44 d1/d22/d29/d30/d60/dd6/lea 0 2024-04-12T05:55:41.424 INFO:tasks.workunit.client.0.smithi008.stdout:7/857: creat d1/d15/df7/f12b x:0 0 0 2024-04-12T05:55:41.424 INFO:tasks.workunit.client.0.smithi008.stdout:3/798: chown d9/d10/d4b/lf8 817678 1 2024-04-12T05:55:41.426 INFO:tasks.workunit.client.0.smithi008.stdout:2/956: dwrite d6/db/d60/d84/fcd [0,4194304] 0 2024-04-12T05:55:41.435 INFO:tasks.workunit.client.0.smithi008.stdout:4/742: mkdir d1/d8/dc5/deb 0 2024-04-12T05:55:41.435 INFO:tasks.workunit.client.0.smithi008.stdout:8/795: write dd/d4f/d52/f69 [2938982,67450] 0 2024-04-12T05:55:41.439 INFO:tasks.workunit.client.0.smithi008.stdout:2/957: chown d6/db/d60/c53 6 1 2024-04-12T05:55:41.441 INFO:tasks.workunit.client.0.smithi008.stdout:3/799: chown d9/d10/d19/d22/fce 0 1 2024-04-12T05:55:41.441 INFO:tasks.workunit.client.0.smithi008.stdout:4/743: fdatasync d1/d22/d29/d30/f57 0 2024-04-12T05:55:41.442 INFO:tasks.workunit.client.0.smithi008.stdout:9/875: truncate d3/d22/d2b/d58/d36/d124/f6b 3115596 0 2024-04-12T05:55:41.443 INFO:tasks.workunit.client.0.smithi008.stdout:6/876: dwrite df/dc1/fc4 [0,4194304] 0 2024-04-12T05:55:41.445 INFO:tasks.workunit.client.0.smithi008.stdout:8/796: rmdir dd/de/d16/d10d 39 2024-04-12T05:55:41.446 INFO:tasks.workunit.client.0.smithi008.stdout:7/858: rename d1/d5/fd1 to d1/d15/d43/d54/d77/f12c 0 2024-04-12T05:55:41.451 INFO:tasks.workunit.client.0.smithi008.stdout:3/800: creat d9/d10/d19/d3a/d7c/f111 x:0 0 0 2024-04-12T05:55:41.453 INFO:tasks.workunit.client.0.smithi008.stdout:4/744: truncate d1/db/f18 439331 0 2024-04-12T05:55:41.456 INFO:tasks.workunit.client.0.smithi008.stdout:3/801: fsync d9/d11/f49 0 2024-04-12T05:55:41.467 INFO:tasks.workunit.client.0.smithi008.stdout:0/780: dwrite d2/fc1 [0,4194304] 0 2024-04-12T05:55:41.467 INFO:tasks.workunit.client.0.smithi008.stdout:9/876: chown d3/df/d13/d21/d5c/da4/db5/f105 10759068 1 2024-04-12T05:55:41.468 INFO:tasks.workunit.client.0.smithi008.stdout:2/958: rename d6/db/d10/d23/l8e to d6/db/d29/dfd/l140 0 2024-04-12T05:55:41.472 INFO:tasks.workunit.client.0.smithi008.stdout:8/797: creat dd/de/d16/d50/d56/f111 x:0 0 0 2024-04-12T05:55:41.474 INFO:tasks.workunit.client.0.smithi008.stdout:9/877: chown d3/df/d13/d21/d5c/d8f/dab 16898825 1 2024-04-12T05:55:41.476 INFO:tasks.workunit.client.0.smithi008.stdout:1/819: dread d4/d33/d74/d30/d72/f96 [0,4194304] 0 2024-04-12T05:55:41.478 INFO:tasks.workunit.client.0.smithi008.stdout:1/820: fdatasync d4/d33/d37/fd2 0 2024-04-12T05:55:41.484 INFO:tasks.workunit.client.0.smithi008.stdout:1/821: chown d4/d33/d74/d7a/fdd 29799787 1 2024-04-12T05:55:41.485 INFO:tasks.workunit.client.0.smithi008.stdout:6/877: dwrite df/d8c/db1/ddc/d95/fb3 [0,4194304] 0 2024-04-12T05:55:41.487 INFO:tasks.workunit.client.0.smithi008.stdout:4/745: dwrite d1/db/db7/fe8 [0,4194304] 0 2024-04-12T05:55:41.501 INFO:tasks.workunit.client.0.smithi008.stdout:3/802: creat d9/d10/d105/f112 x:0 0 0 2024-04-12T05:55:41.502 INFO:tasks.workunit.client.0.smithi008.stdout:2/959: write d6/db/d60/d57/dd9/fe0 [4216845,91101] 0 2024-04-12T05:55:41.505 INFO:tasks.workunit.client.0.smithi008.stdout:6/878: creat df/d8c/db1/ddc/d105/f120 x:0 0 0 2024-04-12T05:55:41.511 INFO:tasks.workunit.client.0.smithi008.stdout:1/822: rename d4/d33/d74/d7a/f92 to d4/d6/d90/f11a 0 2024-04-12T05:55:41.516 INFO:tasks.workunit.client.0.smithi008.stdout:0/781: dread d2/d4b/da2/fb8 [0,4194304] 0 2024-04-12T05:55:41.517 INFO:tasks.workunit.client.0.smithi008.stdout:7/859: getdents d1/d15/d43/ddb 0 2024-04-12T05:55:41.522 INFO:tasks.workunit.client.0.smithi008.stdout:3/803: dread d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:41.524 INFO:tasks.workunit.client.0.smithi008.stdout:0/782: write d2/d11/d36/dd6/ff5 [574466,44492] 0 2024-04-12T05:55:41.525 INFO:tasks.workunit.client.0.smithi008.stdout:4/746: getdents d1/d22 0 2024-04-12T05:55:41.526 INFO:tasks.workunit.client.0.smithi008.stdout:4/747: chown d1/d8/l5e 1623983756 1 2024-04-12T05:55:41.527 INFO:tasks.workunit.client.0.smithi008.stdout:0/783: write d2/d11/fc5 [359702,34440] 0 2024-04-12T05:55:41.532 INFO:tasks.workunit.client.0.smithi008.stdout:6/879: rename df/d10/d15/d79/l118 to df/d10/l121 0 2024-04-12T05:55:41.533 INFO:tasks.workunit.client.0.smithi008.stdout:0/784: truncate d2/d11/d36/d42/da6/fd8 506044 0 2024-04-12T05:55:41.533 INFO:tasks.workunit.client.0.smithi008.stdout:4/748: mknod d1/d22/d29/d30/d60/dd6/cec 0 2024-04-12T05:55:41.534 INFO:tasks.workunit.client.0.smithi008.stdout:6/880: write df/d10/d15/d79/fe0 [3502312,53020] 0 2024-04-12T05:55:41.537 INFO:tasks.workunit.client.0.smithi008.stdout:4/749: dread - d1/d8/f9b zero size 2024-04-12T05:55:41.537 INFO:tasks.workunit.client.0.smithi008.stdout:7/860: rename d1/fd5 to d1/d15/d43/d54/d58/dc2/f12d 0 2024-04-12T05:55:41.537 INFO:tasks.workunit.client.0.smithi008.stdout:0/785: rename d2/d24/d85/dde to d2/d24/d85/dde/dfa 22 2024-04-12T05:55:41.540 INFO:tasks.workunit.client.0.smithi008.stdout:4/750: read d1/d22/d29/d46/d56/f80 [417497,42140] 0 2024-04-12T05:55:41.540 INFO:tasks.workunit.client.0.smithi008.stdout:6/881: mknod df/d10/d20/d3e/db4/df0/c122 0 2024-04-12T05:55:41.540 INFO:tasks.workunit.client.0.smithi008.stdout:7/861: read d1/d15/d43/d54/d77/f89 [1989308,28505] 0 2024-04-12T05:55:41.542 INFO:tasks.workunit.client.0.smithi008.stdout:0/786: unlink d2/l78 0 2024-04-12T05:55:41.542 INFO:tasks.workunit.client.0.smithi008.stdout:4/751: dread - d1/d22/d29/d30/d47/d28/fa6 zero size 2024-04-12T05:55:41.545 INFO:tasks.workunit.client.0.smithi008.stdout:9/878: write d3/df/d13/f51 [2972308,56733] 0 2024-04-12T05:55:41.546 INFO:tasks.workunit.client.0.smithi008.stdout:6/882: symlink df/d21/d4c/dfb/l123 0 2024-04-12T05:55:41.552 INFO:tasks.workunit.client.0.smithi008.stdout:0/787: unlink d2/d11/f23 0 2024-04-12T05:55:41.554 INFO:tasks.workunit.client.0.smithi008.stdout:4/752: mkdir d1/d22/d29/d46/d56/daf/dcf/ded 0 2024-04-12T05:55:41.556 INFO:tasks.workunit.client.0.smithi008.stdout:2/960: write d6/db/d60/f59 [3156481,69546] 0 2024-04-12T05:55:41.560 INFO:tasks.workunit.client.0.smithi008.stdout:1/823: write d4/d6/f40 [1187640,87917] 0 2024-04-12T05:55:41.562 INFO:tasks.workunit.client.0.smithi008.stdout:2/961: fsync d6/db/d29/dc2/f106 0 2024-04-12T05:55:41.564 INFO:tasks.workunit.client.0.smithi008.stdout:0/788: read d2/d24/d39/f3a [648885,122270] 0 2024-04-12T05:55:41.564 INFO:tasks.workunit.client.0.smithi008.stdout:9/879: fsync d3/d22/d2b/d58/d89/f8d 0 2024-04-12T05:55:41.566 INFO:tasks.workunit.client.0.smithi008.stdout:7/862: dread d1/d15/d43/d54/fc4 [0,4194304] 0 2024-04-12T05:55:41.572 INFO:tasks.workunit.client.0.smithi008.stdout:2/962: readlink d6/db/d60/d84/lc0 0 2024-04-12T05:55:41.574 INFO:tasks.workunit.client.0.smithi008.stdout:6/883: write df/d10/fb5 [4327145,15468] 0 2024-04-12T05:55:41.575 INFO:tasks.workunit.client.0.smithi008.stdout:4/753: write d1/d22/d29/d30/d47/d28/d4a/f8d [417129,1256] 0 2024-04-12T05:55:41.576 INFO:tasks.workunit.client.0.smithi008.stdout:9/880: mkdir d3/d22/d2b/d77/d82/d12a 0 2024-04-12T05:55:41.578 INFO:tasks.workunit.client.0.smithi008.stdout:0/789: write d2/d11/f64 [221668,48638] 0 2024-04-12T05:55:41.582 INFO:tasks.workunit.client.0.smithi008.stdout:8/798: dwrite dd/d30/d32/fb7 [0,4194304] 0 2024-04-12T05:55:41.585 INFO:tasks.workunit.client.0.smithi008.stdout:3/804: dwrite d9/d10/d19/d22/f4d [0,4194304] 0 2024-04-12T05:55:41.586 INFO:tasks.workunit.client.0.smithi008.stdout:4/754: truncate d1/d8/f55 4645189 0 2024-04-12T05:55:41.587 INFO:tasks.workunit.client.0.smithi008.stdout:0/790: stat d2/d24/d85/ddc 0 2024-04-12T05:55:41.587 INFO:tasks.workunit.client.0.smithi008.stdout:1/824: dread d4/f9 [0,4194304] 0 2024-04-12T05:55:41.589 INFO:tasks.workunit.client.0.smithi008.stdout:0/791: readlink d2/d24/l90 0 2024-04-12T05:55:41.592 INFO:tasks.workunit.client.0.smithi008.stdout:7/863: truncate d1/d5/da/d3c/d44/ff5 3711385 0 2024-04-12T05:55:41.592 INFO:tasks.workunit.client.0.smithi008.stdout:2/963: mknod d6/db/d60/d11e/d131/c141 0 2024-04-12T05:55:41.594 INFO:tasks.workunit.client.0.smithi008.stdout:4/755: rename d1/d22/c6a to d1/d4f/cee 0 2024-04-12T05:55:41.594 INFO:tasks.workunit.client.0.smithi008.stdout:1/825: write d4/dd/f9a [2117905,12655] 0 2024-04-12T05:55:41.596 INFO:tasks.workunit.client.0.smithi008.stdout:3/805: symlink d9/dd/l113 0 2024-04-12T05:55:41.597 INFO:tasks.workunit.client.0.smithi008.stdout:0/792: symlink d2/d11/lfb 0 2024-04-12T05:55:41.599 INFO:tasks.workunit.client.0.smithi008.stdout:0/793: fsync d2/d11/fb9 0 2024-04-12T05:55:41.601 INFO:tasks.workunit.client.0.smithi008.stdout:7/864: fdatasync d1/d15/fc8 0 2024-04-12T05:55:41.604 INFO:tasks.workunit.client.0.smithi008.stdout:6/884: write df/d10/f56 [544191,116535] 0 2024-04-12T05:55:41.604 INFO:tasks.workunit.client.0.smithi008.stdout:4/756: creat d1/d22/d29/d46/d56/daf/dcf/fef x:0 0 0 2024-04-12T05:55:41.607 INFO:tasks.workunit.client.0.smithi008.stdout:3/806: mkdir d9/d10/d19/d22/da8/d114 0 2024-04-12T05:55:41.609 INFO:tasks.workunit.client.0.smithi008.stdout:9/881: dwrite d3/fa [0,4194304] 0 2024-04-12T05:55:41.611 INFO:tasks.workunit.client.0.smithi008.stdout:6/885: stat df/d8c/da4/cb6 0 2024-04-12T05:55:41.616 INFO:tasks.workunit.client.0.smithi008.stdout:8/799: dread dd/d30/d3b/fb6 [0,4194304] 0 2024-04-12T05:55:41.617 INFO:tasks.workunit.client.0.smithi008.stdout:3/807: dwrite f1 [4194304,4194304] 0 2024-04-12T05:55:41.618 INFO:tasks.workunit.client.0.smithi008.stdout:3/808: readlink d9/d2e/lb5 0 2024-04-12T05:55:41.620 INFO:tasks.workunit.client.0.smithi008.stdout:0/794: creat d2/d24/dbd/de9/ffc x:0 0 0 2024-04-12T05:55:41.621 INFO:tasks.workunit.client.0.smithi008.stdout:6/886: creat df/d21/d4c/dfb/f124 x:0 0 0 2024-04-12T05:55:41.622 INFO:tasks.workunit.client.0.smithi008.stdout:1/826: link d4/d33/d74/d30/c4d d4/d1a/d57/c11b 0 2024-04-12T05:55:41.623 INFO:tasks.workunit.client.0.smithi008.stdout:3/809: truncate d9/d10/f6a 5820984 0 2024-04-12T05:55:41.625 INFO:tasks.workunit.client.0.smithi008.stdout:3/810: chown d9/d2e/f94 9 1 2024-04-12T05:55:41.627 INFO:tasks.workunit.client.0.smithi008.stdout:9/882: creat d3/df/d13/d21/d5c/d8f/dab/dda/f12b x:0 0 0 2024-04-12T05:55:41.628 INFO:tasks.workunit.client.0.smithi008.stdout:9/883: fdatasync d3/df/d13/d21/d2d/d30/de2/f10a 0 2024-04-12T05:55:41.632 INFO:tasks.workunit.client.0.smithi008.stdout:6/887: rename df/d8c/db1/d10e to df/d10/d15/dd0/dd1/d125 0 2024-04-12T05:55:41.638 INFO:tasks.workunit.client.0.smithi008.stdout:2/964: link d6/db/d38/dcf/c111 d6/db/d6f/dea/c142 0 2024-04-12T05:55:41.642 INFO:tasks.workunit.client.0.smithi008.stdout:0/795: truncate d2/d11/d36/d47/f91 435640 0 2024-04-12T05:55:41.643 INFO:tasks.workunit.client.0.smithi008.stdout:1/827: unlink d4/d1a/d57/dd6/l34 0 2024-04-12T05:55:41.643 INFO:tasks.workunit.client.0.smithi008.stdout:0/796: stat d2/f72 0 2024-04-12T05:55:41.646 INFO:tasks.workunit.client.0.smithi008.stdout:6/888: creat df/d21/d4c/f126 x:0 0 0 2024-04-12T05:55:41.646 INFO:tasks.workunit.client.0.smithi008.stdout:7/865: dwrite d1/d5/da/d3c/d44/d8a/f3b [0,4194304] 0 2024-04-12T05:55:41.646 INFO:tasks.workunit.client.0.smithi008.stdout:1/828: fdatasync d4/dd/d17/d91/da4/fea 0 2024-04-12T05:55:41.647 INFO:tasks.workunit.client.0.smithi008.stdout:2/965: mknod d6/db/d38/dcf/c143 0 2024-04-12T05:55:41.648 INFO:tasks.workunit.client.0.smithi008.stdout:8/800: link dd/de/c10 dd/de/d16/db1/c112 0 2024-04-12T05:55:41.648 INFO:tasks.workunit.client.0.smithi008.stdout:6/889: dread - df/d21/ff6 zero size 2024-04-12T05:55:41.651 INFO:tasks.workunit.client.0.smithi008.stdout:2/966: chown d6/db/d6f/da6/c110 906416914 1 2024-04-12T05:55:41.654 INFO:tasks.workunit.client.0.smithi008.stdout:8/801: chown dd/de/d16/d50/d56/d61/f63 23334587 1 2024-04-12T05:55:41.657 INFO:tasks.workunit.client.0.smithi008.stdout:3/811: creat d9/d37/f115 x:0 0 0 2024-04-12T05:55:41.659 INFO:tasks.workunit.client.0.smithi008.stdout:6/890: chown df/d10/l35 4 1 2024-04-12T05:55:41.659 INFO:tasks.workunit.client.0.smithi008.stdout:7/866: rename d1/d5/da/f2a to d1/d5/da/d78/f12e 0 2024-04-12T05:55:41.660 INFO:tasks.workunit.client.0.smithi008.stdout:0/797: creat d2/d4b/ffd x:0 0 0 2024-04-12T05:55:41.665 INFO:tasks.workunit.client.0.smithi008.stdout:9/884: dwrite d3/df/f86 [0,4194304] 0 2024-04-12T05:55:41.666 INFO:tasks.workunit.client.0.smithi008.stdout:6/891: readlink df/l11c 0 2024-04-12T05:55:41.667 INFO:tasks.workunit.client.0.smithi008.stdout:3/812: write d9/d10/d105/f112 [166424,111706] 0 2024-04-12T05:55:41.669 INFO:tasks.workunit.client.0.smithi008.stdout:6/892: truncate df/d10/ff2 168690 0 2024-04-12T05:55:41.676 INFO:tasks.workunit.client.0.smithi008.stdout:7/867: truncate d1/d5/f29 371919 0 2024-04-12T05:55:41.678 INFO:tasks.workunit.client.0.smithi008.stdout:2/967: link d6/db/d29/dc2/fd8 d6/db/d60/d4f/f144 0 2024-04-12T05:55:41.680 INFO:tasks.workunit.client.0.smithi008.stdout:3/813: mkdir d9/d10/d19/d3a/dad/dcf/d116 0 2024-04-12T05:55:41.683 INFO:tasks.workunit.client.0.smithi008.stdout:0/798: mknod d2/d11/d36/d42/cfe 0 2024-04-12T05:55:41.684 INFO:tasks.workunit.client.0.smithi008.stdout:1/829: link d4/d33/d37/c5b d4/d33/d74/d7a/db6/dc9/c11c 0 2024-04-12T05:55:41.687 INFO:tasks.workunit.client.0.smithi008.stdout:0/799: read d2/d11/fb9 [2074566,21915] 0 2024-04-12T05:55:41.688 INFO:tasks.workunit.client.0.smithi008.stdout:3/814: unlink d9/d11/d5b/f87 0 2024-04-12T05:55:41.689 INFO:tasks.workunit.client.0.smithi008.stdout:7/868: getdents d1/d15/d43/ddb 0 2024-04-12T05:55:41.691 INFO:tasks.workunit.client.0.smithi008.stdout:1/830: symlink d4/d6/d11/ddb/l11d 0 2024-04-12T05:55:41.692 INFO:tasks.workunit.client.0.smithi008.stdout:8/802: dwrite dd/f48 [4194304,4194304] 0 2024-04-12T05:55:41.693 INFO:tasks.workunit.client.0.smithi008.stdout:9/885: rename d3/df/d13/d21/d5c/da4/fb4 to d3/d22/d2b/d4a/f12c 0 2024-04-12T05:55:41.698 INFO:tasks.workunit.client.0.smithi008.stdout:0/800: mknod d2/d11/d2a/db5/cff 0 2024-04-12T05:55:41.701 INFO:tasks.workunit.client.0.smithi008.stdout:1/831: creat d4/d6/d90/f11e x:0 0 0 2024-04-12T05:55:41.706 INFO:tasks.workunit.client.0.smithi008.stdout:0/801: mkdir d2/d4b/da2/d100 0 2024-04-12T05:55:41.709 INFO:tasks.workunit.client.0.smithi008.stdout:9/886: mkdir d3/d22/d2b/d12d 0 2024-04-12T05:55:41.709 INFO:tasks.workunit.client.0.smithi008.stdout:6/893: dwrite df/d10/d15/d4f/f88 [0,4194304] 0 2024-04-12T05:55:41.711 INFO:tasks.workunit.client.0.smithi008.stdout:2/968: truncate d6/db/d10/d23/d35/fca 388272 0 2024-04-12T05:55:41.716 INFO:tasks.workunit.client.0.smithi008.stdout:1/832: read d4/dd/d17/f1f [266928,10416] 0 2024-04-12T05:55:41.716 INFO:tasks.workunit.client.0.smithi008.stdout:3/815: creat d9/d11/f117 x:0 0 0 2024-04-12T05:55:41.717 INFO:tasks.workunit.client.0.smithi008.stdout:0/802: rmdir d2/ddb 39 2024-04-12T05:55:41.717 INFO:tasks.workunit.client.0.smithi008.stdout:7/869: link d1/d5/da/d3c/d44/cad d1/d15/d43/d54/d58/d5e/c12f 0 2024-04-12T05:55:41.718 INFO:tasks.workunit.client.0.smithi008.stdout:0/803: dread - d2/d24/d85/fd2 zero size 2024-04-12T05:55:41.719 INFO:tasks.workunit.client.0.smithi008.stdout:1/833: write d4/d24/f32 [308288,91008] 0 2024-04-12T05:55:41.722 INFO:tasks.workunit.client.0.smithi008.stdout:6/894: creat df/d8c/dcc/d106/f127 x:0 0 0 2024-04-12T05:55:41.723 INFO:tasks.workunit.client.0.smithi008.stdout:2/969: unlink d6/db/d60/d11e/d7d/c83 0 2024-04-12T05:55:41.727 INFO:tasks.workunit.client.0.smithi008.stdout:9/887: getdents d3/df/d13/d21/d2d/d30/d74/d115 0 2024-04-12T05:55:41.729 INFO:tasks.workunit.client.0.smithi008.stdout:3/816: creat d9/d10/d19/d3a/d7c/f118 x:0 0 0 2024-04-12T05:55:41.733 INFO:tasks.workunit.client.0.smithi008.stdout:1/834: truncate d4/d33/d74/d7a/db6/fc7 275349 0 2024-04-12T05:55:41.737 INFO:tasks.workunit.client.0.smithi008.stdout:9/888: mknod d3/d22/d2b/d58/d36/c12e 0 2024-04-12T05:55:41.738 INFO:tasks.workunit.client.0.smithi008.stdout:3/817: symlink d9/d10/d105/l119 0 2024-04-12T05:55:41.739 INFO:tasks.workunit.client.0.smithi008.stdout:0/804: dread d2/d4b/f5f [0,4194304] 0 2024-04-12T05:55:41.742 INFO:tasks.workunit.client.0.smithi008.stdout:7/870: creat d1/f130 x:0 0 0 2024-04-12T05:55:41.742 INFO:tasks.workunit.client.0.smithi008.stdout:9/889: dread - d3/d22/d2b/d58/d89/ff7 zero size 2024-04-12T05:55:41.744 INFO:tasks.workunit.client.0.smithi008.stdout:6/895: getdents df/d10/d20/d3e/db4 0 2024-04-12T05:55:41.747 INFO:tasks.workunit.client.0.smithi008.stdout:3/818: truncate d9/dd/f7f 4553722 0 2024-04-12T05:55:41.749 INFO:tasks.workunit.client.0.smithi008.stdout:1/835: dwrite d4/d24/f32 [0,4194304] 0 2024-04-12T05:55:41.750 INFO:tasks.workunit.client.0.smithi008.stdout:7/871: mkdir d1/d5/da/d60/da1/d131 0 2024-04-12T05:55:41.751 INFO:tasks.workunit.client.0.smithi008.stdout:2/970: dread d6/db/d10/f55 [0,4194304] 0 2024-04-12T05:55:41.753 INFO:tasks.workunit.client.0.smithi008.stdout:0/805: write d2/d24/f79 [201609,81455] 0 2024-04-12T05:55:41.756 INFO:tasks.workunit.client.0.smithi008.stdout:7/872: readlink d1/d15/d43/d54/d58/dc2/lca 0 2024-04-12T05:55:41.756 INFO:tasks.workunit.client.0.smithi008.stdout:9/890: chown d3/df/d13/d21/d2d/d32/d52/db0/c118 236105 1 2024-04-12T05:55:41.758 INFO:tasks.workunit.client.0.smithi008.stdout:1/836: symlink d4/d33/d74/d46/d4c/l11f 0 2024-04-12T05:55:41.759 INFO:tasks.workunit.client.0.smithi008.stdout:3/819: symlink d9/d10/d19/d22/da8/d114/l11a 0 2024-04-12T05:55:41.759 INFO:tasks.workunit.client.0.smithi008.stdout:0/806: creat d2/d24/d39/f101 x:0 0 0 2024-04-12T05:55:41.760 INFO:tasks.workunit.client.0.smithi008.stdout:7/873: chown d1/d5/c10 3453882 1 2024-04-12T05:55:41.760 INFO:tasks.workunit.client.0.smithi008.stdout:6/896: rename df/d10/d15/dd0/dd1/f8d to df/d10/d15/dd0/dd1/def/f128 0 2024-04-12T05:55:41.766 INFO:tasks.workunit.client.0.smithi008.stdout:6/897: symlink df/d8c/da4/l129 0 2024-04-12T05:55:41.767 INFO:tasks.workunit.client.0.smithi008.stdout:0/807: symlink d2/d24/l102 0 2024-04-12T05:55:41.771 INFO:tasks.workunit.client.0.smithi008.stdout:2/971: dwrite d6/db/d6f/da6/f11a [0,4194304] 0 2024-04-12T05:55:41.771 INFO:tasks.workunit.client.0.smithi008.stdout:1/837: dread d4/d33/d74/d7a/db6/fc7 [0,4194304] 0 2024-04-12T05:55:41.776 INFO:tasks.workunit.client.0.smithi008.stdout:0/808: symlink d2/l103 0 2024-04-12T05:55:41.778 INFO:tasks.workunit.client.0.smithi008.stdout:7/874: write d1/f3 [939746,77120] 0 2024-04-12T05:55:41.780 INFO:tasks.workunit.client.0.smithi008.stdout:9/891: dwrite d3/df/d13/d21/d2d/d30/de2/f10a [0,4194304] 0 2024-04-12T05:55:41.780 INFO:tasks.workunit.client.0.smithi008.stdout:3/820: truncate d9/d2e/d3d/f6b 400187 0 2024-04-12T05:55:41.783 INFO:tasks.workunit.client.0.smithi008.stdout:1/838: creat d4/d33/d74/d30/d72/d7c/f120 x:0 0 0 2024-04-12T05:55:41.783 INFO:tasks.workunit.client.0.smithi008.stdout:0/809: creat d2/d11/d2a/db5/f104 x:0 0 0 2024-04-12T05:55:41.785 INFO:tasks.workunit.client.0.smithi008.stdout:6/898: creat df/d10/d15/f12a x:0 0 0 2024-04-12T05:55:41.787 INFO:tasks.workunit.client.0.smithi008.stdout:3/821: chown d9/d44/c4f 369376239 1 2024-04-12T05:55:41.788 INFO:tasks.workunit.client.0.smithi008.stdout:7/875: symlink d1/d5/da/d60/da1/l132 0 2024-04-12T05:55:41.792 INFO:tasks.workunit.client.0.smithi008.stdout:1/839: rename d4/dd/f39 to d4/d1c/d4f/d10b/f121 0 2024-04-12T05:55:41.794 INFO:tasks.workunit.client.0.smithi008.stdout:2/972: dwrite d6/db/d38/d41/ff7 [0,4194304] 0 2024-04-12T05:55:41.796 INFO:tasks.workunit.client.0.smithi008.stdout:3/822: truncate d9/d2e/d3d/fc6 4786377 0 2024-04-12T05:55:41.797 INFO:tasks.workunit.client.0.smithi008.stdout:9/892: rmdir d3/df/d13/d21/d2d/d32/d52/d120 0 2024-04-12T05:55:41.798 INFO:tasks.workunit.client.0.smithi008.stdout:1/840: fsync d4/d1c/f8f 0 2024-04-12T05:55:41.800 INFO:tasks.workunit.client.0.smithi008.stdout:6/899: rename df/d10/d15/f9d to df/d8c/db1/f12b 0 2024-04-12T05:55:41.802 INFO:tasks.workunit.client.0.smithi008.stdout:9/893: write d3/df/d13/d21/d5c/da4/db5/f105 [305595,36352] 0 2024-04-12T05:55:41.804 INFO:tasks.workunit.client.0.smithi008.stdout:2/973: unlink d6/db/d60/d84/fcd 0 2024-04-12T05:55:41.806 INFO:tasks.workunit.client.0.smithi008.stdout:1/841: unlink d4/d1a/f5f 0 2024-04-12T05:55:41.806 INFO:tasks.workunit.client.0.smithi008.stdout:3/823: dread d9/d37/d69/fe5 [0,4194304] 0 2024-04-12T05:55:41.807 INFO:tasks.workunit.client.0.smithi008.stdout:7/876: link d1/d5/da/d1d/ffe d1/d15/d43/ddb/f133 0 2024-04-12T05:55:41.812 INFO:tasks.workunit.client.0.smithi008.stdout:9/894: creat d3/df/d13/d21/d5c/d8f/dab/dda/f12f x:0 0 0 2024-04-12T05:55:41.817 INFO:tasks.workunit.client.0.smithi008.stdout:3/824: creat d9/d10/d19/d3a/dad/dcf/d116/f11b x:0 0 0 2024-04-12T05:55:41.819 INFO:tasks.workunit.client.0.smithi008.stdout:0/810: rename d2/d11/d36/l8c to d2/d11/d36/d82/l105 0 2024-04-12T05:55:41.819 INFO:tasks.workunit.client.0.smithi008.stdout:1/842: dwrite d4/d6/d90/f11e [0,4194304] 0 2024-04-12T05:55:41.819 INFO:tasks.workunit.client.0.smithi008.stdout:9/895: truncate d3/df/d13/d21/d2d/fa6 5007758 0 2024-04-12T05:55:41.822 INFO:tasks.workunit.client.0.smithi008.stdout:1/843: fsync d4/dd/f99 0 2024-04-12T05:55:41.824 INFO:tasks.workunit.client.0.smithi008.stdout:2/974: rename d6/db/dae to d6/db/d10/d23/d35/d11f/d145 0 2024-04-12T05:55:41.826 INFO:tasks.workunit.client.0.smithi008.stdout:3/825: truncate d9/db7/fc1 1741183 0 2024-04-12T05:55:41.829 INFO:tasks.workunit.client.0.smithi008.stdout:1/844: rename d4/d1a/d57/dd6/cc2 to d4/d24/d62/c122 0 2024-04-12T05:55:41.832 INFO:tasks.workunit.client.0.smithi008.stdout:3/826: mknod d9/d10/d19/d22/c11c 0 2024-04-12T05:55:41.832 INFO:tasks.workunit.client.0.smithi008.stdout:9/896: truncate d3/d22/d2b/d4a/f12c 723631 0 2024-04-12T05:55:41.834 INFO:tasks.workunit.client.0.smithi008.stdout:3/827: readlink d9/d10/d19/d3a/lfd 0 2024-04-12T05:55:41.834 INFO:tasks.workunit.client.0.smithi008.stdout:2/975: dwrite d6/db/d10/d23/fa4 [0,4194304] 0 2024-04-12T05:55:41.836 INFO:tasks.workunit.client.0.smithi008.stdout:1/845: rename d4/d6/c58 to d4/dd/d17/d26/c123 0 2024-04-12T05:55:41.839 INFO:tasks.workunit.client.0.smithi008.stdout:9/897: dread d3/df/f28 [0,4194304] 0 2024-04-12T05:55:41.843 INFO:tasks.workunit.client.0.smithi008.stdout:3/828: rename d9/d10/d19/d3a/d7c/cf0 to d9/d10/d19/d3a/d67/dda/c11d 0 2024-04-12T05:55:41.845 INFO:tasks.workunit.client.0.smithi008.stdout:1/846: getdents d4/d33/d74/d7a 0 2024-04-12T05:55:41.848 INFO:tasks.workunit.client.0.smithi008.stdout:3/829: rename d9/d2e/d3d/fc6 to d9/db7/f11e 0 2024-04-12T05:55:41.864 INFO:tasks.workunit.client.0.smithi008.stdout:1/847: dwrite d4/dd/f85 [0,4194304] 0 2024-04-12T05:55:41.866 INFO:tasks.workunit.client.0.smithi008.stdout:6/900: truncate df/d8c/db1/f12b 1014979 0 2024-04-12T05:55:41.868 INFO:tasks.workunit.client.0.smithi008.stdout:9/898: dread d3/d22/d2b/d58/d36/d124/faf [0,4194304] 0 2024-04-12T05:55:41.871 INFO:tasks.workunit.client.0.smithi008.stdout:1/848: mknod d4/d33/d74/d30/c124 0 2024-04-12T05:55:41.873 INFO:tasks.workunit.client.0.smithi008.stdout:6/901: dread - df/d8c/da4/dad/fc6 zero size 2024-04-12T05:55:41.874 INFO:tasks.workunit.client.0.smithi008.stdout:0/811: write d2/d11/d36/d47/f68 [594630,124709] 0 2024-04-12T05:55:41.877 INFO:tasks.workunit.client.0.smithi008.stdout:7/877: dwrite d1/d5/da/fd2 [0,4194304] 0 2024-04-12T05:55:41.878 INFO:tasks.workunit.client.0.smithi008.stdout:2/976: write d6/db/d60/d57/f76 [1760730,93777] 0 2024-04-12T05:55:41.879 INFO:tasks.workunit.client.0.smithi008.stdout:1/849: mkdir d4/d33/dc5/d125 0 2024-04-12T05:55:41.879 INFO:tasks.workunit.client.0.smithi008.stdout:0/812: fsync d2/d11/d2a/f7e 0 2024-04-12T05:55:41.883 INFO:tasks.workunit.client.0.smithi008.stdout:6/902: creat df/d8c/dcc/dee/f12c x:0 0 0 2024-04-12T05:55:41.884 INFO:tasks.workunit.client.0.smithi008.stdout:3/830: write d9/d2e/d3d/f6b [802452,86365] 0 2024-04-12T05:55:41.885 INFO:tasks.workunit.client.0.smithi008.stdout:7/878: stat d1/d5/da/d11/fbc 0 2024-04-12T05:55:41.885 INFO:tasks.workunit.client.0.smithi008.stdout:2/977: creat d6/db/d60/d84/f146 x:0 0 0 2024-04-12T05:55:41.886 INFO:tasks.workunit.client.0.smithi008.stdout:0/813: creat d2/d4b/f106 x:0 0 0 2024-04-12T05:55:41.890 INFO:tasks.workunit.client.0.smithi008.stdout:1/850: write d4/dd/d17/d26/f76 [7922275,13269] 0 2024-04-12T05:55:41.892 INFO:tasks.workunit.client.0.smithi008.stdout:1/851: fdatasync d4/d1c/db9/dc1/fe5 0 2024-04-12T05:55:41.894 INFO:tasks.workunit.client.0.smithi008.stdout:7/879: unlink d1/d5/da/d60/da1/lce 0 2024-04-12T05:55:41.897 INFO:tasks.workunit.client.0.smithi008.stdout:6/903: creat df/d8c/dcc/ddd/f12d x:0 0 0 2024-04-12T05:55:41.899 INFO:tasks.workunit.client.0.smithi008.stdout:0/814: symlink d2/ddb/l107 0 2024-04-12T05:55:41.899 INFO:tasks.workunit.client.0.smithi008.stdout:7/880: write d1/d15/d43/d54/d58/db7/ff8 [537571,109879] 0 2024-04-12T05:55:41.901 INFO:tasks.workunit.client.0.smithi008.stdout:3/831: creat d9/d44/f11f x:0 0 0 2024-04-12T05:55:41.902 INFO:tasks.workunit.client.0.smithi008.stdout:2/978: dwrite d6/db/fd [0,4194304] 0 2024-04-12T05:55:41.902 INFO:tasks.workunit.client.0.smithi008.stdout:0/815: readlink d2/d11/l71 0 2024-04-12T05:55:41.903 INFO:tasks.workunit.client.0.smithi008.stdout:6/904: write df/d10/ff2 [1059941,78965] 0 2024-04-12T05:55:41.906 INFO:tasks.workunit.client.0.smithi008.stdout:9/899: truncate d3/d22/d2b/d58/d38/f4c 2064195 0 2024-04-12T05:55:41.906 INFO:tasks.workunit.client.0.smithi008.stdout:7/881: read d1/d5/da/d3c/d44/d8a/f9b [1892809,123816] 0 2024-04-12T05:55:41.907 INFO:tasks.workunit.client.0.smithi008.stdout:2/979: readlink d6/db/d60/d4f/dbf/lc4 0 2024-04-12T05:55:41.907 INFO:tasks.workunit.client.0.smithi008.stdout:3/832: truncate f7 122384 0 2024-04-12T05:55:41.908 INFO:tasks.workunit.client.0.smithi008.stdout:0/816: write d2/d11/d2a/db5/feb [988185,127461] 0 2024-04-12T05:55:41.909 INFO:tasks.workunit.client.0.smithi008.stdout:1/852: rename d4/d33/dc5 to d4/dd/d21/d9f/d126 0 2024-04-12T05:55:41.911 INFO:tasks.workunit.client.0.smithi008.stdout:6/905: mkdir df/d10/d20/d3e/dfc/d10f/d12e 0 2024-04-12T05:55:41.912 INFO:tasks.workunit.client.0.smithi008.stdout:0/817: dread - d2/d24/d61/fed zero size 2024-04-12T05:55:41.912 INFO:tasks.workunit.client.0.smithi008.stdout:9/900: mknod d3/d22/d2b/d58/d36/d124/d60/c130 0 2024-04-12T05:55:41.913 INFO:tasks.workunit.client.0.smithi008.stdout:7/882: write d1/d15/d43/d54/d58/d5e/d7f/fba [4966911,22163] 0 2024-04-12T05:55:41.918 INFO:tasks.workunit.client.0.smithi008.stdout:9/901: rmdir d3/d22/d2b/d77 39 2024-04-12T05:55:41.921 INFO:tasks.workunit.client.0.smithi008.stdout:6/906: creat df/d10/d15/d4f/de9/f12f x:0 0 0 2024-04-12T05:55:41.922 INFO:tasks.workunit.client.0.smithi008.stdout:1/853: link d4/d33/c8e d4/d33/d74/c127 0 2024-04-12T05:55:41.923 INFO:tasks.workunit.client.0.smithi008.stdout:9/902: read - d3/df/d13/d21/d2d/d32/d52/f9f zero size 2024-04-12T05:55:41.929 INFO:tasks.workunit.client.0.smithi008.stdout:0/818: dwrite d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:41.929 INFO:tasks.workunit.client.0.smithi008.stdout:6/907: write df/d21/d4c/fe8 [6585023,14862] 0 2024-04-12T05:55:41.929 INFO:tasks.workunit.client.0.smithi008.stdout:2/980: dread d6/f15 [0,4194304] 0 2024-04-12T05:55:41.933 INFO:tasks.workunit.client.0.smithi008.stdout:9/903: creat d3/df/d13/d21/d5c/da4/db5/f131 x:0 0 0 2024-04-12T05:55:41.936 INFO:tasks.workunit.client.0.smithi008.stdout:3/833: write d9/d10/d19/d65/f93 [335558,3794] 0 2024-04-12T05:55:41.939 INFO:tasks.workunit.client.0.smithi008.stdout:1/854: rename d4/d33/f71 to d4/d33/f128 0 2024-04-12T05:55:41.940 INFO:tasks.workunit.client.0.smithi008.stdout:6/908: read df/d21/d4c/fbe [4551,108121] 0 2024-04-12T05:55:41.947 INFO:tasks.workunit.client.0.smithi008.stdout:6/909: truncate df/d10/d15/dd0/ff8 4442853 0 2024-04-12T05:55:41.947 INFO:tasks.workunit.client.0.smithi008.stdout:3/834: mkdir d9/d10/d4b/d120 0 2024-04-12T05:55:41.947 INFO:tasks.workunit.client.0.smithi008.stdout:0/819: dwrite d2/d11/d2a/f4a [0,4194304] 0 2024-04-12T05:55:41.947 INFO:tasks.workunit.client.0.smithi008.stdout:1/855: dread d4/d6/d90/f11a [0,4194304] 0 2024-04-12T05:55:41.950 INFO:tasks.workunit.client.0.smithi008.stdout:7/883: write d1/d5/da/d3c/d44/ff5 [3428074,121007] 0 2024-04-12T05:55:41.955 INFO:tasks.workunit.client.0.smithi008.stdout:6/910: symlink df/d8c/db1/l130 0 2024-04-12T05:55:41.958 INFO:tasks.workunit.client.0.smithi008.stdout:3/835: mkdir d9/d10/d105/d121 0 2024-04-12T05:55:41.958 INFO:tasks.workunit.client.0.smithi008.stdout:7/884: chown d1/d5/da/d3c/dd3 255 1 2024-04-12T05:55:41.959 INFO:tasks.workunit.client.0.smithi008.stdout:6/911: mknod df/d10/d15/d75/dd3/c131 0 2024-04-12T05:55:41.959 INFO:tasks.workunit.client.0.smithi008.stdout:0/820: creat d2/d24/d85/f108 x:0 0 0 2024-04-12T05:55:41.962 INFO:tasks.workunit.client.0.smithi008.stdout:9/904: dwrite d3/d22/d2b/d58/d38/f54 [0,4194304] 0 2024-04-12T05:55:41.962 INFO:tasks.workunit.client.0.smithi008.stdout:6/912: stat df/d10/d20/d3e 0 2024-04-12T05:55:41.963 INFO:tasks.workunit.client.0.smithi008.stdout:0/821: truncate d2/d24/d61/fed 12749 0 2024-04-12T05:55:41.966 INFO:tasks.workunit.client.0.smithi008.stdout:1/856: dread d4/d33/d74/d30/d72/d7c/f8c [0,4194304] 0 2024-04-12T05:55:41.966 INFO:tasks.workunit.client.0.smithi008.stdout:0/822: fsync d2/d24/d39/fae 0 2024-04-12T05:55:41.969 INFO:tasks.workunit.client.0.smithi008.stdout:3/836: dread d9/d10/d19/d65/f93 [0,4194304] 0 2024-04-12T05:55:41.970 INFO:tasks.workunit.client.0.smithi008.stdout:0/823: truncate d2/d24/d39/f101 739302 0 2024-04-12T05:55:41.972 INFO:tasks.workunit.client.0.smithi008.stdout:6/913: mkdir df/d10/d15/dd0/dd1/d125/d132 0 2024-04-12T05:55:41.973 INFO:tasks.workunit.client.0.smithi008.stdout:1/857: mkdir d4/d1c/db9/dc1/d129 0 2024-04-12T05:55:41.977 INFO:tasks.workunit.client.0.smithi008.stdout:1/858: rmdir d4/d33/d37 39 2024-04-12T05:55:41.978 INFO:tasks.workunit.client.0.smithi008.stdout:0/824: mkdir d2/d11/d2a/db5/df8/d109 0 2024-04-12T05:55:41.983 INFO:tasks.workunit.client.0.smithi008.stdout:0/825: symlink d2/d24/d85/dde/l10a 0 2024-04-12T05:55:41.984 INFO:tasks.workunit.client.0.smithi008.stdout:6/914: dwrite f8 [0,4194304] 0 2024-04-12T05:55:41.990 INFO:tasks.workunit.client.0.smithi008.stdout:1/859: rename d4/d33/d74/d30/d72/d7c/dd4/dd5/f106 to d4/d1c/f12a 0 2024-04-12T05:55:41.991 INFO:tasks.workunit.client.0.smithi008.stdout:1/860: dread d4/d6/d90/f11a [0,4194304] 0 2024-04-12T05:55:41.992 INFO:tasks.workunit.client.0.smithi008.stdout:0/826: symlink d2/d4b/l10b 0 2024-04-12T05:55:41.996 INFO:tasks.workunit.client.0.smithi008.stdout:9/905: write d3/df/d13/d21/f76 [153803,100594] 0 2024-04-12T05:55:41.999 INFO:tasks.workunit.client.0.smithi008.stdout:7/885: truncate d1/d5/da/d3c/f114 2127336 0 2024-04-12T05:55:42.000 INFO:tasks.workunit.client.0.smithi008.stdout:6/915: creat df/d10/d15/f133 x:0 0 0 2024-04-12T05:55:42.002 INFO:tasks.workunit.client.0.smithi008.stdout:9/906: mkdir d3/df/d13/d21/d5c/d8f/dab/dda/d132 0 2024-04-12T05:55:42.003 INFO:tasks.workunit.client.0.smithi008.stdout:7/886: fsync d1/d5/f42 0 2024-04-12T05:55:42.006 INFO:tasks.workunit.client.0.smithi008.stdout:9/907: chown d3/df/d13 11 1 2024-04-12T05:55:42.007 INFO:tasks.workunit.client.0.smithi008.stdout:6/916: creat df/d10/d15/d4f/f134 x:0 0 0 2024-04-12T05:55:42.007 INFO:tasks.workunit.client.0.smithi008.stdout:7/887: stat d1/d49/fec 0 2024-04-12T05:55:42.008 INFO:tasks.workunit.client.0.smithi008.stdout:1/861: dwrite d4/dd/d21/d9f/d100/fb0 [0,4194304] 0 2024-04-12T05:55:42.010 INFO:tasks.workunit.client.0.smithi008.stdout:6/917: fdatasync df/d10/d15/d4f/f88 0 2024-04-12T05:55:42.011 INFO:tasks.workunit.client.0.smithi008.stdout:1/862: readlink d4/d33/lf9 0 2024-04-12T05:55:42.015 INFO:tasks.workunit.client.0.smithi008.stdout:7/888: mkdir d1/d15/d43/d54/d58/dc2/d134 0 2024-04-12T05:55:42.019 INFO:tasks.workunit.client.0.smithi008.stdout:6/918: unlink df/d8c/db1/ddc/d105/f120 0 2024-04-12T05:55:42.026 INFO:tasks.workunit.client.0.smithi008.stdout:6/919: mkdir df/d10/d20/d3e/dfc/d135 0 2024-04-12T05:55:42.026 INFO:tasks.workunit.client.0.smithi008.stdout:0/827: dread d2/d11/d36/d82/d95/fb3 [0,4194304] 0 2024-04-12T05:55:42.026 INFO:tasks.workunit.client.0.smithi008.stdout:0/828: stat d2/d24/f32 0 2024-04-12T05:55:42.031 INFO:tasks.workunit.client.0.smithi008.stdout:6/920: chown df/c60 145537 1 2024-04-12T05:55:42.031 INFO:tasks.workunit.client.0.smithi008.stdout:0/829: symlink d2/d24/d85/ddc/l10c 0 2024-04-12T05:55:42.032 INFO:tasks.workunit.client.0.smithi008.stdout:1/863: getdents d4/d1a 0 2024-04-12T05:55:42.037 INFO:tasks.workunit.client.0.smithi008.stdout:6/921: write df/d10/d20/d3e/db4/fc8 [218279,127170] 0 2024-04-12T05:55:42.043 INFO:tasks.workunit.client.0.smithi008.stdout:0/830: truncate d2/d24/f32 4427840 0 2024-04-12T05:55:42.044 INFO:tasks.workunit.client.0.smithi008.stdout:9/908: dwrite d3/df/d13/d21/d2d/d32/d52/f11d [0,4194304] 0 2024-04-12T05:55:42.048 INFO:tasks.workunit.client.0.smithi008.stdout:0/831: dread - d2/d24/d85/ddc/fe3 zero size 2024-04-12T05:55:42.048 INFO:tasks.workunit.client.0.smithi008.stdout:7/889: dread d1/d5/da/d78/f12e [0,4194304] 0 2024-04-12T05:55:42.056 INFO:tasks.workunit.client.0.smithi008.stdout:0/832: read d2/d11/d2a/db5/fc2 [2421000,50564] 0 2024-04-12T05:55:42.056 INFO:tasks.workunit.client.0.smithi008.stdout:1/864: dwrite d4/d33/d74/d30/d72/d7c/f120 [0,4194304] 0 2024-04-12T05:55:42.063 INFO:tasks.workunit.client.0.smithi008.stdout:9/909: creat d3/d22/d2b/d58/f133 x:0 0 0 2024-04-12T05:55:42.063 INFO:tasks.workunit.client.0.smithi008.stdout:7/890: mknod d1/d5/da/d60/d7b/dff/c135 0 2024-04-12T05:55:42.063 INFO:tasks.workunit.client.0.smithi008.stdout:6/922: dwrite df/d10/d15/d75/dd6/f104 [0,4194304] 0 2024-04-12T05:55:42.065 INFO:tasks.workunit.client.0.smithi008.stdout:7/891: chown d1/d5/da/d3c/d44/d8a/l1b 15241 1 2024-04-12T05:55:42.067 INFO:tasks.workunit.client.0.smithi008.stdout:1/865: symlink d4/d33/d74/d7a/db6/l12b 0 2024-04-12T05:55:42.070 INFO:tasks.workunit.client.0.smithi008.stdout:7/892: chown d1/l8 1117 1 2024-04-12T05:55:42.071 INFO:tasks.workunit.client.0.smithi008.stdout:7/893: chown d1/d5/f4d 27904284 1 2024-04-12T05:55:42.072 INFO:tasks.workunit.client.0.smithi008.stdout:0/833: link d2/d11/d36/d82/d95/fb6 d2/d11/f10d 0 2024-04-12T05:55:42.072 INFO:tasks.workunit.client.0.smithi008.stdout:1/866: getdents d4/dd/d21/d9f/d126/d125 0 2024-04-12T05:55:42.073 INFO:tasks.workunit.client.0.smithi008.stdout:1/867: stat d4/d1c/db9/fe4 0 2024-04-12T05:55:42.074 INFO:tasks.workunit.client.0.smithi008.stdout:0/834: chown d2/d24/d61/c97 596 1 2024-04-12T05:55:42.077 INFO:tasks.workunit.client.0.smithi008.stdout:1/868: truncate d4/d1c/d4f/f6a 6501897 0 2024-04-12T05:55:42.078 INFO:tasks.workunit.client.0.smithi008.stdout:0/835: creat d2/d11/d2a/f10e x:0 0 0 2024-04-12T05:55:42.087 INFO:tasks.workunit.client.0.smithi008.stdout:1/869: dwrite d4/d1a/f9c [0,4194304] 0 2024-04-12T05:55:42.091 INFO:tasks.workunit.client.0.smithi008.stdout:1/870: write d4/d6/d90/f11e [4273517,48151] 0 2024-04-12T05:55:42.094 INFO:tasks.workunit.client.0.smithi008.stdout:0/836: write d2/d11/d36/d47/f74 [1014755,85909] 0 2024-04-12T05:55:42.098 INFO:tasks.workunit.client.0.smithi008.stdout:1/871: symlink d4/d24/d62/l12c 0 2024-04-12T05:55:42.101 INFO:tasks.workunit.client.0.smithi008.stdout:7/894: dwrite d1/d5/da/d3c/d44/f51 [4194304,4194304] 0 2024-04-12T05:55:42.105 INFO:tasks.workunit.client.0.smithi008.stdout:1/872: write d4/dd/d21/d9f/d100/fe2 [347802,118639] 0 2024-04-12T05:55:42.106 INFO:tasks.workunit.client.0.smithi008.stdout:0/837: dwrite d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:42.109 INFO:tasks.workunit.client.0.smithi008.stdout:1/873: read - d4/dd/d17/d91/f105 zero size 2024-04-12T05:55:42.111 INFO:tasks.workunit.client.0.smithi008.stdout:1/874: chown d4/d1a/d57/dd6/lff 0 1 2024-04-12T05:55:42.113 INFO:tasks.workunit.client.0.smithi008.stdout:0/838: creat d2/d24/d85/ddc/f10f x:0 0 0 2024-04-12T05:55:42.114 INFO:tasks.workunit.client.0.smithi008.stdout:0/839: dread - d2/d4b/da2/fd0 zero size 2024-04-12T05:55:42.116 INFO:tasks.workunit.client.0.smithi008.stdout:1/875: mkdir d4/d33/d74/d30/d72/d7c/dd4/d12d 0 2024-04-12T05:55:42.116 INFO:tasks.workunit.client.0.smithi008.stdout:7/895: rename d1/d5/da/d60/lcc to d1/d15/d43/l136 0 2024-04-12T05:55:42.117 INFO:tasks.workunit.client.0.smithi008.stdout:7/896: fsync d1/d5/da/d78/dab/ff2 0 2024-04-12T05:55:42.121 INFO:tasks.workunit.client.0.smithi008.stdout:7/897: mkdir d1/d15/d43/d54/d58/dc2/d137 0 2024-04-12T05:55:42.123 INFO:tasks.workunit.client.0.smithi008.stdout:7/898: fdatasync d1/d5/da/d11/f57 0 2024-04-12T05:55:42.128 INFO:tasks.workunit.client.0.smithi008.stdout:7/899: getdents d1/d5/da/d11/d56/d8f 0 2024-04-12T05:55:42.130 INFO:tasks.workunit.client.0.smithi008.stdout:7/900: write d1/d5/da/d11/d56/f124 [282770,110128] 0 2024-04-12T05:55:42.131 INFO:tasks.workunit.client.0.smithi008.stdout:7/901: fsync d1/d5/da/d1d/ff0 0 2024-04-12T05:55:42.132 INFO:tasks.workunit.client.0.smithi008.stdout:7/902: stat d1/d5/da/c85 0 2024-04-12T05:55:42.134 INFO:tasks.workunit.client.0.smithi008.stdout:1/876: truncate d4/dd/f89 760256 0 2024-04-12T05:55:42.135 INFO:tasks.workunit.client.0.smithi008.stdout:7/903: write d1/d5/da/d78/f88 [4618282,51538] 0 2024-04-12T05:55:42.141 INFO:tasks.workunit.client.0.smithi008.stdout:6/923: read df/d10/d15/f13 [195620,125563] 0 2024-04-12T05:55:42.143 INFO:tasks.workunit.client.0.smithi008.stdout:7/904: symlink d1/d5/da/d1d/l138 0 2024-04-12T05:55:42.145 INFO:tasks.workunit.client.0.smithi008.stdout:0/840: dwrite d2/d24/d39/f3a [0,4194304] 0 2024-04-12T05:55:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:7/905: write d1/d5/da/d11/d109/f123 [773079,10249] 0 2024-04-12T05:55:42.156 INFO:tasks.workunit.client.0.smithi008.stdout:6/924: dread df/d10/d15/d75/dd6/f104 [0,4194304] 0 2024-04-12T05:55:42.165 INFO:tasks.workunit.client.0.smithi008.stdout:7/906: unlink d1/d5/f29 0 2024-04-12T05:55:42.167 INFO:tasks.workunit.client.0.smithi008.stdout:0/841: dwrite d2/fe [0,4194304] 0 2024-04-12T05:55:42.168 INFO:tasks.workunit.client.0.smithi008.stdout:6/925: creat df/f136 x:0 0 0 2024-04-12T05:55:42.177 INFO:tasks.workunit.client.0.smithi008.stdout:7/907: fsync d1/d15/f59 0 2024-04-12T05:55:42.177 INFO:tasks.workunit.client.0.smithi008.stdout:0/842: creat d2/d11/d2a/db5/f110 x:0 0 0 2024-04-12T05:55:42.178 INFO:tasks.workunit.client.0.smithi008.stdout:9/910: dread d3/df/d13/d14/fa3 [0,4194304] 0 2024-04-12T05:55:42.180 INFO:tasks.workunit.client.0.smithi008.stdout:0/843: fdatasync d2/d24/d61/fed 0 2024-04-12T05:55:42.181 INFO:tasks.workunit.client.0.smithi008.stdout:7/908: truncate d1/d5/da/d60/da1/f127 456247 0 2024-04-12T05:55:42.183 INFO:tasks.workunit.client.0.smithi008.stdout:1/877: dread d4/d1c/db9/fe4 [0,4194304] 0 2024-04-12T05:55:42.187 INFO:tasks.workunit.client.0.smithi008.stdout:6/926: dwrite df/d21/d4c/f58 [0,4194304] 0 2024-04-12T05:55:42.188 INFO:tasks.workunit.client.0.smithi008.stdout:9/911: chown d3/d22/d2b/d77/d82/f106 1 1 2024-04-12T05:55:42.189 INFO:tasks.workunit.client.0.smithi008.stdout:1/878: creat d4/d33/d74/f12e x:0 0 0 2024-04-12T05:55:42.192 INFO:tasks.workunit.client.0.smithi008.stdout:0/844: rename d2/d11/d2a/f10e to d2/d11/d2a/db5/df8/f111 0 2024-04-12T05:55:42.194 INFO:tasks.workunit.client.0.smithi008.stdout:6/927: dread - df/d21/fb0 zero size 2024-04-12T05:55:42.196 INFO:tasks.workunit.client.0.smithi008.stdout:1/879: symlink d4/d1a/d57/l12f 0 2024-04-12T05:55:42.200 INFO:tasks.workunit.client.0.smithi008.stdout:1/880: creat d4/d1c/f130 x:0 0 0 2024-04-12T05:55:42.200 INFO:tasks.workunit.client.0.smithi008.stdout:9/912: dwrite d3/f25 [4194304,4194304] 0 2024-04-12T05:55:42.205 INFO:tasks.workunit.client.0.smithi008.stdout:9/913: read - d3/d22/f10e zero size 2024-04-12T05:55:42.207 INFO:tasks.workunit.client.0.smithi008.stdout:7/909: write d1/d5/da/d11/f57 [386960,26621] 0 2024-04-12T05:55:42.208 INFO:tasks.workunit.client.0.smithi008.stdout:0/845: dwrite d2/d11/fb9 [4194304,4194304] 0 2024-04-12T05:55:42.216 INFO:tasks.workunit.client.0.smithi008.stdout:9/914: unlink d3/df/d13/f51 0 2024-04-12T05:55:42.218 INFO:tasks.workunit.client.0.smithi008.stdout:7/910: creat d1/d5/da/d78/dd6/f139 x:0 0 0 2024-04-12T05:55:42.226 INFO:tasks.workunit.client.0.smithi008.stdout:9/915: fsync d3/df/d13/d21/d2d/d75/f94 0 2024-04-12T05:55:42.231 INFO:tasks.workunit.client.0.smithi008.stdout:9/916: mkdir d3/df/d13/d21/d5c/dd7/d134 0 2024-04-12T05:55:42.234 INFO:tasks.workunit.client.0.smithi008.stdout:1/881: truncate d4/dd/d17/d26/f76 2867781 0 2024-04-12T05:55:42.236 INFO:tasks.workunit.client.0.smithi008.stdout:9/917: truncate d3/d22/d2b/d58/f1e 1632696 0 2024-04-12T05:55:42.238 INFO:tasks.workunit.client.0.smithi008.stdout:7/911: creat d1/d5/da/d3c/f13a x:0 0 0 2024-04-12T05:55:42.240 INFO:tasks.workunit.client.0.smithi008.stdout:9/918: mkdir d3/d6a/d135 0 2024-04-12T05:55:42.243 INFO:tasks.workunit.client.0.smithi008.stdout:7/912: mknod d1/d15/d43/d54/d58/dc2/d134/c13b 0 2024-04-12T05:55:42.243 INFO:tasks.workunit.client.0.smithi008.stdout:9/919: write d3/d22/d2b/d58/d5d/fcd [355388,19540] 0 2024-04-12T05:55:42.246 INFO:tasks.workunit.client.0.smithi008.stdout:0/846: dwrite d2/d11/d36/f46 [0,4194304] 0 2024-04-12T05:55:42.251 INFO:tasks.workunit.client.0.smithi008.stdout:0/847: creat d2/d11/d2a/db5/df8/f112 x:0 0 0 2024-04-12T05:55:42.254 INFO:tasks.workunit.client.0.smithi008.stdout:7/913: getdents d1/d15/d43/d54/d58/dc2 0 2024-04-12T05:55:42.258 INFO:tasks.workunit.client.0.smithi008.stdout:7/914: symlink d1/d5/da/d1d/dc3/l13c 0 2024-04-12T05:55:42.258 INFO:tasks.workunit.client.0.smithi008.stdout:0/848: getdents d2/d4b 0 2024-04-12T05:55:42.260 INFO:tasks.workunit.client.0.smithi008.stdout:7/915: truncate d1/f6 4797506 0 2024-04-12T05:55:42.262 INFO:tasks.workunit.client.0.smithi008.stdout:7/916: chown d1/d5/da/d1d/d82/d9f 89163 1 2024-04-12T05:55:42.263 INFO:tasks.workunit.client.0.smithi008.stdout:1/882: dread d4/dd/f99 [0,4194304] 0 2024-04-12T05:55:42.266 INFO:tasks.workunit.client.0.smithi008.stdout:4/757: sync 2024-04-12T05:55:42.267 INFO:tasks.workunit.client.0.smithi008.stdout:0/849: unlink d2/d11/d2a/f4a 0 2024-04-12T05:55:42.267 INFO:tasks.workunit.client.0.smithi008.stdout:7/917: creat d1/d5/da/d11/d109/f13d x:0 0 0 2024-04-12T05:55:42.268 INFO:tasks.workunit.client.0.smithi008.stdout:1/883: mkdir d4/d33/d74/d7a/db6/dc9/d131 0 2024-04-12T05:55:42.271 INFO:tasks.workunit.client.0.smithi008.stdout:7/918: symlink d1/d5/da/d11/d109/l13e 0 2024-04-12T05:55:42.273 INFO:tasks.workunit.client.0.smithi008.stdout:0/850: creat d2/d24/f113 x:0 0 0 2024-04-12T05:55:42.274 INFO:tasks.workunit.client.0.smithi008.stdout:9/920: dread d3/d22/d2b/d58/da8/db6/fdc [0,4194304] 0 2024-04-12T05:55:42.275 INFO:tasks.workunit.client.0.smithi008.stdout:1/884: rename d4/d6/d11/f42 to d4/d1c/d4f/d10b/f132 0 2024-04-12T05:55:42.278 INFO:tasks.workunit.client.0.smithi008.stdout:7/919: rmdir d1/d15/d43/d54/d58/dc2/d134 39 2024-04-12T05:55:42.280 INFO:tasks.workunit.client.0.smithi008.stdout:1/885: creat d4/d1c/db9/f133 x:0 0 0 2024-04-12T05:55:42.282 INFO:tasks.workunit.client.0.smithi008.stdout:9/921: fdatasync d3/df/d13/d21/d2d/d30/de2/ff1 0 2024-04-12T05:55:42.282 INFO:tasks.workunit.client.0.smithi008.stdout:7/920: creat d1/d15/d43/d54/f13f x:0 0 0 2024-04-12T05:55:42.283 INFO:tasks.workunit.client.0.smithi008.stdout:7/921: readlink d1/d15/d43/d54/d58/dc2/lda 0 2024-04-12T05:55:42.284 INFO:tasks.workunit.client.0.smithi008.stdout:1/886: creat d4/d33/d74/d46/d4c/f134 x:0 0 0 2024-04-12T05:55:42.284 INFO:tasks.workunit.client.0.smithi008.stdout:7/922: stat d1/d15/d43/d54/d77/f96 0 2024-04-12T05:55:42.287 INFO:tasks.workunit.client.0.smithi008.stdout:1/887: creat d4/d1c/d4f/d10b/f135 x:0 0 0 2024-04-12T05:55:42.289 INFO:tasks.workunit.client.0.smithi008.stdout:7/923: mknod d1/d5/da/d3c/d44/d8a/c140 0 2024-04-12T05:55:42.289 INFO:tasks.workunit.client.0.smithi008.stdout:4/758: dread d1/d22/f4c [0,4194304] 0 2024-04-12T05:55:42.292 INFO:tasks.workunit.client.0.smithi008.stdout:7/924: readlink d1/d15/d43/d54/d58/l5f 0 2024-04-12T05:55:42.292 INFO:tasks.workunit.client.0.smithi008.stdout:7/925: dread - d1/d15/d43/f100 zero size 2024-04-12T05:55:42.299 INFO:tasks.workunit.client.0.smithi008.stdout:4/759: truncate d1/d8/dc5/fcc 413523 0 2024-04-12T05:55:42.303 INFO:tasks.workunit.client.0.smithi008.stdout:4/760: creat d1/d22/d29/d30/d47/d28/d74/ff0 x:0 0 0 2024-04-12T05:55:42.304 INFO:tasks.workunit.client.0.smithi008.stdout:7/926: dread d1/d15/d43/d54/d58/f67 [0,4194304] 0 2024-04-12T05:55:42.306 INFO:tasks.workunit.client.0.smithi008.stdout:4/761: mknod d1/d22/d29/d46/d56/daf/dcf/cf1 0 2024-04-12T05:55:42.306 INFO:tasks.workunit.client.0.smithi008.stdout:7/927: readlink d1/d15/d43/d54/d58/d5e/l81 0 2024-04-12T05:55:42.307 INFO:tasks.workunit.client.0.smithi008.stdout:4/762: chown d1/d22/d29/d30/d47/d28/d4a/d62/cb0 1954501 1 2024-04-12T05:55:42.309 INFO:tasks.workunit.client.0.smithi008.stdout:4/763: creat d1/d8/ff2 x:0 0 0 2024-04-12T05:55:42.313 INFO:tasks.workunit.client.0.smithi008.stdout:4/764: rename d1/d22/d29/d46/da7/ld3 to d1/d4f/dcb/lf3 0 2024-04-12T05:55:42.315 INFO:tasks.workunit.client.0.smithi008.stdout:4/765: chown d1/d8/l10 12872 1 2024-04-12T05:55:42.316 INFO:tasks.workunit.client.0.smithi008.stdout:7/928: write d1/d5/da/d78/fbd [541290,37498] 0 2024-04-12T05:55:42.317 INFO:tasks.workunit.client.0.smithi008.stdout:4/766: rmdir d1/d22/d29/d30/d47 39 2024-04-12T05:55:42.322 INFO:tasks.workunit.client.0.smithi008.stdout:4/767: write d1/d4f/f7f [603925,21107] 0 2024-04-12T05:55:42.324 INFO:tasks.workunit.client.0.smithi008.stdout:4/768: dread - d1/d22/d29/d30/d47/d28/d4a/fdf zero size 2024-04-12T05:55:42.336 INFO:tasks.workunit.client.0.smithi008.stdout:7/929: dread d1/d5/da/d11/f35 [8388608,4194304] 0 2024-04-12T05:55:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:4/769: dwrite d1/d8/dc5/fcc [0,4194304] 0 2024-04-12T05:55:42.339 INFO:tasks.workunit.client.0.smithi008.stdout:7/930: dread - d1/d5/da/d60/f11c zero size 2024-04-12T05:55:42.341 INFO:tasks.workunit.client.0.smithi008.stdout:4/770: rmdir d1/d4f/dcb 39 2024-04-12T05:55:42.345 INFO:tasks.workunit.client.0.smithi008.stdout:7/931: rename d1/d5/da/d3c/dd3/f101 to d1/d15/d43/d5a/f141 0 2024-04-12T05:55:42.346 INFO:tasks.workunit.client.0.smithi008.stdout:4/771: rmdir d1/d8/dc5 39 2024-04-12T05:55:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:4/772: chown d1/d22/d29/d46/d56/daf/dcf 252885 1 2024-04-12T05:55:42.347 INFO:tasks.workunit.client.0.smithi008.stdout:7/932: symlink d1/d5/d10b/l142 0 2024-04-12T05:55:42.351 INFO:tasks.workunit.client.0.smithi008.stdout:7/933: chown d1/d15/d43/d54/d58/db7/de3 2 1 2024-04-12T05:55:42.352 INFO:tasks.workunit.client.0.smithi008.stdout:4/773: mknod d1/db/cf4 0 2024-04-12T05:55:42.352 INFO:tasks.workunit.client.0.smithi008.stdout:4/774: chown d1/d22/la8 11 1 2024-04-12T05:55:42.357 INFO:tasks.workunit.client.0.smithi008.stdout:7/934: rename d1/d5/da/d11/d56/f124 to d1/d15/d43/d54/f143 0 2024-04-12T05:55:42.358 INFO:tasks.workunit.client.0.smithi008.stdout:7/935: fdatasync d1/d15/d43/d54/f143 0 2024-04-12T05:55:42.360 INFO:tasks.workunit.client.0.smithi008.stdout:7/936: write d1/d5/da/d60/d7b/fed [602211,25864] 0 2024-04-12T05:55:42.361 INFO:tasks.workunit.client.0.smithi008.stdout:4/775: write d1/d22/d29/d30/f91 [561876,60781] 0 2024-04-12T05:55:42.363 INFO:tasks.workunit.client.0.smithi008.stdout:7/937: getdents d1/d5/da/d1d/d82 0 2024-04-12T05:55:42.366 INFO:tasks.workunit.client.0.smithi008.stdout:4/776: mkdir d1/d22/d29/d46/d56/daf/dcf/ded/df5 0 2024-04-12T05:55:42.366 INFO:tasks.workunit.client.0.smithi008.stdout:7/938: dread d1/d15/fe6 [0,4194304] 0 2024-04-12T05:55:42.367 INFO:tasks.workunit.client.0.smithi008.stdout:4/777: write d1/db/db7/fbb [898894,114756] 0 2024-04-12T05:55:42.369 INFO:tasks.workunit.client.0.smithi008.stdout:7/939: mknod d1/d5/da/d11/d109/c144 0 2024-04-12T05:55:42.370 INFO:tasks.workunit.client.0.smithi008.stdout:7/940: readlink d1/d5/da/d1d/l6b 0 2024-04-12T05:55:42.372 INFO:tasks.workunit.client.0.smithi008.stdout:7/941: write d1/d15/d43/d54/d77/f12c [3504029,23276] 0 2024-04-12T05:55:42.372 INFO:tasks.workunit.client.0.smithi008.stdout:4/778: link d1/d22/d29/d30/d47/d28/d4a/fdf d1/d8/dc5/deb/ff6 0 2024-04-12T05:55:42.374 INFO:tasks.workunit.client.0.smithi008.stdout:7/942: truncate d1/d5/da/d1d/d82/f11a 428182 0 2024-04-12T05:55:42.379 INFO:tasks.workunit.client.0.smithi008.stdout:4/779: write d1/d22/d29/fd2 [687779,27210] 0 2024-04-12T05:55:42.383 INFO:tasks.workunit.client.0.smithi008.stdout:7/943: dwrite d1/d5/da/d78/f11b [0,4194304] 0 2024-04-12T05:55:42.384 INFO:tasks.workunit.client.0.smithi008.stdout:7/944: chown d1/d15/d43/ddb 1014263 1 2024-04-12T05:55:42.385 INFO:tasks.workunit.client.0.smithi008.stdout:4/780: mknod d1/d4f/dcb/cf7 0 2024-04-12T05:55:42.389 INFO:tasks.workunit.client.0.smithi008.stdout:7/945: rename d1/f130 to d1/d5/da/d1d/d82/f145 0 2024-04-12T05:55:42.390 INFO:tasks.workunit.client.0.smithi008.stdout:4/781: dread d1/d8/f55 [4194304,4194304] 0 2024-04-12T05:55:42.393 INFO:tasks.workunit.client.0.smithi008.stdout:4/782: unlink d1/d4f/dd5/cde 0 2024-04-12T05:55:42.395 INFO:tasks.workunit.client.0.smithi008.stdout:7/946: rename d1/d15/d43/d54/d58/dc2/d134 to d1/d5/da/d11/d103/d146 0 2024-04-12T05:55:42.396 INFO:tasks.workunit.client.0.smithi008.stdout:4/783: mkdir d1/d8/dc5/deb/df8 0 2024-04-12T05:55:42.399 INFO:tasks.workunit.client.0.smithi008.stdout:7/947: read d1/d5/da/f62 [878550,40805] 0 2024-04-12T05:55:42.401 INFO:tasks.workunit.client.0.smithi008.stdout:4/784: getdents d1/db/db7 0 2024-04-12T05:55:42.402 INFO:tasks.workunit.client.0.smithi008.stdout:7/948: fsync d1/d5/f32 0 2024-04-12T05:55:42.403 INFO:tasks.workunit.client.0.smithi008.stdout:4/785: write d1/d22/d29/d30/f52 [1440389,117729] 0 2024-04-12T05:55:42.405 INFO:tasks.workunit.client.0.smithi008.stdout:4/786: write d1/d22/d29/d46/d56/f63 [1199430,117293] 0 2024-04-12T05:55:42.406 INFO:tasks.workunit.client.0.smithi008.stdout:7/949: creat d1/d5/da/d60/da1/f147 x:0 0 0 2024-04-12T05:55:42.407 INFO:tasks.workunit.client.0.smithi008.stdout:4/787: write d1/d8/f37 [137268,77333] 0 2024-04-12T05:55:42.409 INFO:tasks.workunit.client.0.smithi008.stdout:4/788: fsync d1/d8/f9b 0 2024-04-12T05:55:42.411 INFO:tasks.workunit.client.0.smithi008.stdout:7/950: unlink d1/d5/da/d11/f35 0 2024-04-12T05:55:42.411 INFO:tasks.workunit.client.0.smithi008.stdout:7/951: dread - d1/d5/da/d60/fd7 zero size 2024-04-12T05:55:42.412 INFO:tasks.workunit.client.0.smithi008.stdout:7/952: chown d1/d5/da/d78/dab 2751 1 2024-04-12T05:55:42.413 INFO:tasks.workunit.client.0.smithi008.stdout:4/789: unlink d1/d8/l11 0 2024-04-12T05:55:42.414 INFO:tasks.workunit.client.0.smithi008.stdout:7/953: fsync d1/ffd 0 2024-04-12T05:55:42.415 INFO:tasks.workunit.client.0.smithi008.stdout:4/790: chown d1/d22/d29/d46/d56/ccd 1278954589 1 2024-04-12T05:55:42.422 INFO:tasks.workunit.client.0.smithi008.stdout:4/791: dwrite d1/d22/d29/d30/d47/d28/f3f [0,4194304] 0 2024-04-12T05:55:42.426 INFO:tasks.workunit.client.0.smithi008.stdout:7/954: write d1/d5/da/d78/dd6/fee [947298,53759] 0 2024-04-12T05:55:42.432 INFO:tasks.workunit.client.0.smithi008.stdout:4/792: creat d1/d8/ff9 x:0 0 0 2024-04-12T05:55:42.433 INFO:tasks.workunit.client.0.smithi008.stdout:4/793: chown d1/db/db7/fbb 410932336 1 2024-04-12T05:55:42.435 INFO:tasks.workunit.client.0.smithi008.stdout:4/794: mknod d1/d8/dc5/cfa 0 2024-04-12T05:55:42.437 INFO:tasks.workunit.client.0.smithi008.stdout:4/795: symlink d1/d8/dc5/lfb 0 2024-04-12T05:55:42.449 INFO:tasks.workunit.client.0.smithi008.stdout:4/796: dwrite d1/d8/f42 [0,4194304] 0 2024-04-12T05:55:42.453 INFO:tasks.workunit.client.0.smithi008.stdout:4/797: dread d1/d22/d29/fd2 [0,4194304] 0 2024-04-12T05:55:42.454 INFO:tasks.workunit.client.0.smithi008.stdout:7/955: dread d1/d5/da/d60/fe4 [0,4194304] 0 2024-04-12T05:55:42.458 INFO:tasks.workunit.client.0.smithi008.stdout:7/956: getdents d1/d5/da/d11/d56/d8f 0 2024-04-12T05:55:42.476 INFO:tasks.workunit.client.0.smithi008.stdout:7/957: dwrite d1/d5/da/d3c/dd3/f108 [0,4194304] 0 2024-04-12T05:55:42.478 INFO:tasks.workunit.client.0.smithi008.stdout:7/958: write d1/d5/da/d3c/dd3/f11f [818652,5098] 0 2024-04-12T05:55:42.517 INFO:tasks.workunit.client.0.smithi008.stdout:7/959: dwrite d1/d5/da/d3c/d44/f6f [0,4194304] 0 2024-04-12T05:55:42.523 INFO:tasks.workunit.client.0.smithi008.stdout:7/960: rename d1/d5/da/d3c/l64 to d1/d15/d43/d54/d58/db7/l148 0 2024-04-12T05:55:42.525 INFO:tasks.workunit.client.0.smithi008.stdout:7/961: getdents d1/d5/da/d3c/d117 0 2024-04-12T05:55:42.527 INFO:tasks.workunit.client.0.smithi008.stdout:7/962: truncate d1/d15/d43/d5a/f98 471380 0 2024-04-12T05:55:42.529 INFO:tasks.workunit.client.0.smithi008.stdout:8/803: sync 2024-04-12T05:55:42.531 INFO:tasks.workunit.client.0.smithi008.stdout:8/804: chown dd/de/d16/d50/d56/d61/l9d 1 1 2024-04-12T05:55:42.533 INFO:tasks.workunit.client.0.smithi008.stdout:8/805: write dd/de/f47 [10575004,28001] 0 2024-04-12T05:55:42.537 INFO:tasks.workunit.client.0.smithi008.stdout:7/963: write d1/d49/f4b [5537440,25735] 0 2024-04-12T05:55:42.538 INFO:tasks.workunit.client.0.smithi008.stdout:8/806: write dd/de/d16/d50/d5e/f8c [730226,58906] 0 2024-04-12T05:55:42.538 INFO:tasks.workunit.client.0.smithi008.stdout:7/964: stat d1/d15/d43/ddb 0 2024-04-12T05:55:42.540 INFO:tasks.workunit.client.0.smithi008.stdout:8/807: mkdir dd/d4f/df5/d113 0 2024-04-12T05:55:42.542 INFO:tasks.workunit.client.0.smithi008.stdout:7/965: mkdir d1/d149 0 2024-04-12T05:55:42.547 INFO:tasks.workunit.client.0.smithi008.stdout:7/966: rename d1/d15/d43/d54/d58/dc2/lca to d1/d5/da/d11/d103/d146/l14a 0 2024-04-12T05:55:42.548 INFO:tasks.workunit.client.0.smithi008.stdout:8/808: dwrite dd/de/f47 [4194304,4194304] 0 2024-04-12T05:55:42.550 INFO:tasks.workunit.client.0.smithi008.stdout:7/967: read - d1/d5/da/d11/d109/f13d zero size 2024-04-12T05:55:42.558 INFO:tasks.workunit.client.0.smithi008.stdout:7/968: rename d1/d15/d43/d54/d58/db7/l148 to d1/d15/d43/d10e/l14b 0 2024-04-12T05:55:42.561 INFO:tasks.workunit.client.0.smithi008.stdout:8/809: write dd/d65/ff2 [4219493,32508] 0 2024-04-12T05:55:42.565 INFO:tasks.workunit.client.0.smithi008.stdout:7/969: write d1/d49/f40 [3833212,103559] 0 2024-04-12T05:55:42.568 INFO:tasks.workunit.client.0.smithi008.stdout:7/970: fdatasync d1/d15/d43/d54/d58/db7/fb6 0 2024-04-12T05:55:42.571 INFO:tasks.workunit.client.0.smithi008.stdout:7/971: mknod d1/d5/da/d60/da1/d131/c14c 0 2024-04-12T05:55:42.576 INFO:tasks.workunit.client.0.smithi008.stdout:8/810: dwrite dd/de/d16/d50/d5e/fc5 [0,4194304] 0 2024-04-12T05:55:42.586 INFO:tasks.workunit.client.0.smithi008.stdout:7/972: dwrite d1/d5/da/d1d/d82/fb9 [4194304,4194304] 0 2024-04-12T05:55:42.590 INFO:tasks.workunit.client.0.smithi008.stdout:8/811: dwrite dd/de/d16/d50/d56/d61/d9c/ffa [0,4194304] 0 2024-04-12T05:55:42.594 INFO:tasks.workunit.client.0.smithi008.stdout:7/973: truncate d1/d5/da/d78/faa 716659 0 2024-04-12T05:55:42.595 INFO:tasks.workunit.client.0.smithi008.stdout:8/812: rename dd/de/d4c/d66/l100 to dd/de/d16/d50/d5e/l114 0 2024-04-12T05:55:42.596 INFO:tasks.workunit.client.0.smithi008.stdout:7/974: creat d1/d5/da/d60/da1/f14d x:0 0 0 2024-04-12T05:55:42.597 INFO:tasks.workunit.client.0.smithi008.stdout:8/813: chown dd/d65/c8b 1512750490 1 2024-04-12T05:55:42.601 INFO:tasks.workunit.client.0.smithi008.stdout:7/975: mkdir d1/d5/da/d11/d103/d146/d14e 0 2024-04-12T05:55:42.618 INFO:tasks.workunit.client.0.smithi008.stdout:8/814: dwrite dd/de/d16/d50/fd1 [0,4194304] 0 2024-04-12T05:55:42.621 INFO:tasks.workunit.client.0.smithi008.stdout:8/815: truncate dd/de/d16/d50/d56/f5d 4467733 0 2024-04-12T05:55:42.625 INFO:tasks.workunit.client.0.smithi008.stdout:8/816: rename dd/la8 to dd/de/d4c/d66/l115 0 2024-04-12T05:55:42.632 INFO:tasks.workunit.client.0.smithi008.stdout:8/817: mknod dd/de/d4c/d66/d6c/c116 0 2024-04-12T05:55:42.633 INFO:tasks.workunit.client.0.smithi008.stdout:8/818: stat dd/d4f/f96 0 2024-04-12T05:55:42.635 INFO:tasks.workunit.client.0.smithi008.stdout:8/819: truncate dd/d4f/f7b 1598412 0 2024-04-12T05:55:42.635 INFO:tasks.workunit.client.0.smithi008.stdout:8/820: stat dd/de/d16/dc8 0 2024-04-12T05:55:42.637 INFO:tasks.workunit.client.0.smithi008.stdout:8/821: symlink dd/d65/l117 0 2024-04-12T05:55:42.638 INFO:tasks.workunit.client.0.smithi008.stdout:8/822: chown dd/de/d4c/d66/d6d/dca 1 1 2024-04-12T05:55:42.640 INFO:tasks.workunit.client.0.smithi008.stdout:8/823: symlink dd/de/d16/db1/l118 0 2024-04-12T05:55:42.650 INFO:tasks.workunit.client.0.smithi008.stdout:8/824: dread dd/d30/d3b/db5/df0/fc3 [0,4194304] 0 2024-04-12T05:55:42.654 INFO:tasks.workunit.client.0.smithi008.stdout:8/825: creat dd/de/d4c/d66/d6d/f119 x:0 0 0 2024-04-12T05:55:42.659 INFO:tasks.workunit.client.0.smithi008.stdout:8/826: creat dd/de/d4c/d66/d6d/f11a x:0 0 0 2024-04-12T05:55:42.664 INFO:tasks.workunit.client.0.smithi008.stdout:8/827: read dd/de/d16/d10d/f6f [1611862,116052] 0 2024-04-12T05:55:42.667 INFO:tasks.workunit.client.0.smithi008.stdout:8/828: rename dd/de/d16/f72 to dd/de/d16/d50/d5e/dbe/f11b 0 2024-04-12T05:55:42.675 INFO:tasks.workunit.client.0.smithi008.stdout:8/829: dread dd/d30/d3b/fb6 [0,4194304] 0 2024-04-12T05:55:42.688 INFO:tasks.workunit.client.0.smithi008.stdout:8/830: dread dd/de/d16/d10e/fb8 [0,4194304] 0 2024-04-12T05:55:42.691 INFO:tasks.workunit.client.0.smithi008.stdout:8/831: rmdir dd/de/d16/d50/d5e/dbe/dd6/df6 0 2024-04-12T05:55:42.693 INFO:tasks.workunit.client.0.smithi008.stdout:8/832: creat dd/de/d16/db1/d97/dcf/f11c x:0 0 0 2024-04-12T05:55:42.794 INFO:tasks.workunit.client.0.smithi008.stdout:2/981: sync 2024-04-12T05:55:42.795 INFO:tasks.workunit.client.0.smithi008.stdout:2/982: chown d6/db/d60/d4f/lcb 263444 1 2024-04-12T05:55:42.800 INFO:tasks.workunit.client.0.smithi008.stdout:2/983: getdents d6/db/d10/d23 0 2024-04-12T05:55:42.801 INFO:tasks.workunit.client.0.smithi008.stdout:2/984: dread - d6/db/d60/d4f/f119 zero size 2024-04-12T05:55:42.802 INFO:tasks.workunit.client.0.smithi008.stdout:2/985: chown d6/db/d38/dcf/c143 63 1 2024-04-12T05:55:42.805 INFO:tasks.workunit.client.0.smithi008.stdout:2/986: rmdir d6/db/d10/dbc 0 2024-04-12T05:55:42.809 INFO:tasks.workunit.client.0.smithi008.stdout:2/987: dread d6/db/fd [0,4194304] 0 2024-04-12T05:55:42.811 INFO:tasks.workunit.client.0.smithi008.stdout:2/988: mkdir d6/db/d29/d147 0 2024-04-12T05:55:42.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:42 smithi194 ceph-mon[49771]: pgmap v17: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 72 MiB/s rd, 179 MiB/s wr, 658 op/s 2024-04-12T05:55:42.829 INFO:tasks.workunit.client.0.smithi008.stdout:2/989: dread d6/db/f9e [0,4194304] 0 2024-04-12T05:55:42.831 INFO:tasks.workunit.client.0.smithi008.stdout:2/990: write d6/db/d10/d23/f4d [5819061,854] 0 2024-04-12T05:55:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:42 smithi008 ceph-mon[39722]: pgmap v17: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 72 MiB/s rd, 179 MiB/s wr, 658 op/s 2024-04-12T05:55:42.968 INFO:tasks.workunit.client.0.smithi008.stdout:3/837: sync 2024-04-12T05:55:42.969 INFO:tasks.workunit.client.0.smithi008.stdout:6/928: sync 2024-04-12T05:55:42.970 INFO:tasks.workunit.client.0.smithi008.stdout:3/838: stat d9/d2e/d3d/f80 0 2024-04-12T05:55:42.971 INFO:tasks.workunit.client.0.smithi008.stdout:3/839: chown d9/db7 453 1 2024-04-12T05:55:42.973 INFO:tasks.workunit.client.0.smithi008.stdout:6/929: unlink df/d8c/db1/ddc/d95/fa7 0 2024-04-12T05:55:42.975 INFO:tasks.workunit.client.0.smithi008.stdout:6/930: write df/d21/d4c/dfb/f124 [308822,115874] 0 2024-04-12T05:55:42.978 INFO:tasks.workunit.client.0.smithi008.stdout:6/931: creat df/d10/d15/dd0/f137 x:0 0 0 2024-04-12T05:55:42.980 INFO:tasks.workunit.client.0.smithi008.stdout:6/932: read - df/d10/d15/dd0/f137 zero size 2024-04-12T05:55:42.984 INFO:tasks.workunit.client.0.smithi008.stdout:3/840: dread d9/d11/f98 [0,4194304] 0 2024-04-12T05:55:42.987 INFO:tasks.workunit.client.0.smithi008.stdout:3/841: creat d9/d37/d69/f122 x:0 0 0 2024-04-12T05:55:42.989 INFO:tasks.workunit.client.0.smithi008.stdout:3/842: stat d9/d1e/c21 0 2024-04-12T05:55:42.992 INFO:tasks.workunit.client.0.smithi008.stdout:3/843: symlink d9/d10/d19/d22/da8/d114/l123 0 2024-04-12T05:55:42.992 INFO:tasks.workunit.client.0.smithi008.stdout:6/933: dread df/d21/f43 [0,4194304] 0 2024-04-12T05:55:42.992 INFO:tasks.workunit.client.0.smithi008.stdout:6/934: chown df/c60 1145 1 2024-04-12T05:55:42.995 INFO:tasks.workunit.client.0.smithi008.stdout:3/844: creat d9/d10/d19/d22/da8/d114/f124 x:0 0 0 2024-04-12T05:55:42.995 INFO:tasks.workunit.client.0.smithi008.stdout:6/935: dread - df/d21/f96 zero size 2024-04-12T05:55:42.995 INFO:tasks.workunit.client.0.smithi008.stdout:3/845: chown d9/d11/d5b 56456 1 2024-04-12T05:55:42.996 INFO:tasks.workunit.client.0.smithi008.stdout:3/846: readlink d9/dd/l113 0 2024-04-12T05:55:42.998 INFO:tasks.workunit.client.0.smithi008.stdout:6/936: truncate df/d21/f81 542039 0 2024-04-12T05:55:43.000 INFO:tasks.workunit.client.0.smithi008.stdout:6/937: chown df/d10/d20/d3e/ld7 218630 1 2024-04-12T05:55:43.004 INFO:tasks.workunit.client.0.smithi008.stdout:3/847: dwrite d9/d10/f33 [0,4194304] 0 2024-04-12T05:55:43.007 INFO:tasks.workunit.client.0.smithi008.stdout:6/938: rename df/d8c/db1/ddc/d78/d8b/fdb to df/d8c/f138 0 2024-04-12T05:55:43.009 INFO:tasks.workunit.client.0.smithi008.stdout:6/939: read df/d10/ff2 [728893,55239] 0 2024-04-12T05:55:43.012 INFO:tasks.workunit.client.0.smithi008.stdout:3/848: link d9/f1c d9/f125 0 2024-04-12T05:55:43.013 INFO:tasks.workunit.client.0.smithi008.stdout:3/849: truncate d9/dd/fa6 1833806 0 2024-04-12T05:55:43.016 INFO:tasks.workunit.client.0.smithi008.stdout:3/850: mknod d9/d37/d69/c126 0 2024-04-12T05:55:43.017 INFO:tasks.workunit.client.0.smithi008.stdout:3/851: dread - d9/d10/d19/d22/fdc zero size 2024-04-12T05:55:43.020 INFO:tasks.workunit.client.0.smithi008.stdout:3/852: unlink d9/d44/f11f 0 2024-04-12T05:55:43.023 INFO:tasks.workunit.client.0.smithi008.stdout:3/853: creat d9/d11/f127 x:0 0 0 2024-04-12T05:55:43.024 INFO:tasks.workunit.client.0.smithi008.stdout:3/854: dread - d9/dd/f53 zero size 2024-04-12T05:55:43.031 INFO:tasks.workunit.client.0.smithi008.stdout:3/855: dwrite d9/dd/fa6 [0,4194304] 0 2024-04-12T05:55:43.033 INFO:tasks.workunit.client.0.smithi008.stdout:3/856: chown d9/d10/d19/d65/f8e 2 1 2024-04-12T05:55:43.045 INFO:tasks.workunit.client.0.smithi008.stdout:3/857: dwrite d9/d2e/d3d/f6b [0,4194304] 0 2024-04-12T05:55:43.130 INFO:tasks.workunit.client.0.smithi008.stdout:0/851: sync 2024-04-12T05:55:43.131 INFO:tasks.workunit.client.0.smithi008.stdout:0/852: dread - d2/d11/d36/d47/fd9 zero size 2024-04-12T05:55:43.135 INFO:tasks.workunit.client.0.smithi008.stdout:0/853: rename d2/d24/d39/f3a to d2/d11/d36/d82/df9/f114 0 2024-04-12T05:55:43.136 INFO:tasks.workunit.client.0.smithi008.stdout:0/854: write d2/d11/d36/f46 [5359547,40812] 0 2024-04-12T05:55:43.139 INFO:tasks.workunit.client.0.smithi008.stdout:0/855: rename d2/d11/d36/d47/f2c to d2/d24/d85/dde/df7/f115 0 2024-04-12T05:55:43.141 INFO:tasks.workunit.client.0.smithi008.stdout:0/856: fdatasync d2/d11/d36/d47/f8b 0 2024-04-12T05:55:43.143 INFO:tasks.workunit.client.0.smithi008.stdout:0/857: symlink d2/d11/d2a/db5/df8/d109/l116 0 2024-04-12T05:55:43.145 INFO:tasks.workunit.client.0.smithi008.stdout:0/858: symlink d2/d11/d36/l117 0 2024-04-12T05:55:43.151 INFO:tasks.workunit.client.0.smithi008.stdout:0/859: truncate d2/d11/fb9 7144014 0 2024-04-12T05:55:43.153 INFO:tasks.workunit.client.0.smithi008.stdout:0/860: mknod d2/d24/d39/c118 0 2024-04-12T05:55:43.156 INFO:tasks.workunit.client.0.smithi008.stdout:0/861: rename d2/d11/l1e to d2/d11/d2a/db5/df8/l119 0 2024-04-12T05:55:43.163 INFO:tasks.workunit.client.0.smithi008.stdout:0/862: truncate d2/d11/f2e 221360 0 2024-04-12T05:55:43.165 INFO:tasks.workunit.client.0.smithi008.stdout:0/863: write d2/d11/d36/d82/fb4 [1165094,20620] 0 2024-04-12T05:55:43.168 INFO:tasks.workunit.client.0.smithi008.stdout:0/864: dread d2/d11/d36/d82/d95/f98 [0,4194304] 0 2024-04-12T05:55:43.179 INFO:tasks.workunit.client.0.smithi008.stdout:0/865: dwrite d2/d24/d61/f92 [0,4194304] 0 2024-04-12T05:55:43.181 INFO:tasks.workunit.client.0.smithi008.stdout:0/866: chown d2/d11/d36/d82/c93 112 1 2024-04-12T05:55:43.184 INFO:tasks.workunit.client.0.smithi008.stdout:0/867: creat d2/d4b/f11a x:0 0 0 2024-04-12T05:55:43.186 INFO:tasks.workunit.client.0.smithi008.stdout:0/868: truncate d2/d11/f40 1090687 0 2024-04-12T05:55:43.192 INFO:tasks.workunit.client.0.smithi008.stdout:0/869: dwrite d2/d11/d36/d82/d95/fb3 [0,4194304] 0 2024-04-12T05:55:43.194 INFO:tasks.workunit.client.0.smithi008.stdout:0/870: chown d2/d11/d36/d82/d95/lce 4376 1 2024-04-12T05:55:43.196 INFO:tasks.workunit.client.0.smithi008.stdout:0/871: mkdir d2/ddb/d11b 0 2024-04-12T05:55:43.200 INFO:tasks.workunit.client.0.smithi008.stdout:0/872: truncate d2/d11/d36/d42/da6/fb7 887781 0 2024-04-12T05:55:43.204 INFO:tasks.workunit.client.0.smithi008.stdout:0/873: dwrite d2/fa [0,4194304] 0 2024-04-12T05:55:43.207 INFO:tasks.workunit.client.0.smithi008.stdout:0/874: chown d2/d11/lc3 6 1 2024-04-12T05:55:43.210 INFO:tasks.workunit.client.0.smithi008.stdout:0/875: unlink d2/d4b/cd3 0 2024-04-12T05:55:43.211 INFO:tasks.workunit.client.0.smithi008.stdout:0/876: chown d2/l103 161 1 2024-04-12T05:55:43.212 INFO:tasks.workunit.client.0.smithi008.stdout:0/877: write d2/d11/d2a/db5/feb [38308,79784] 0 2024-04-12T05:55:43.214 INFO:tasks.workunit.client.0.smithi008.stdout:0/878: mkdir d2/ddb/d11c 0 2024-04-12T05:55:43.216 INFO:tasks.workunit.client.0.smithi008.stdout:0/879: creat d2/d24/d61/f11d x:0 0 0 2024-04-12T05:55:43.222 INFO:tasks.workunit.client.0.smithi008.stdout:0/880: dwrite d2/d24/d39/fae [0,4194304] 0 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:7/976: sync 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:4/798: sync 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:1/888: sync 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:9/922: sync 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:6/940: sync 2024-04-12T05:55:43.237 INFO:tasks.workunit.client.0.smithi008.stdout:8/833: sync 2024-04-12T05:55:43.238 INFO:tasks.workunit.client.0.smithi008.stdout:2/991: sync 2024-04-12T05:55:43.240 INFO:tasks.workunit.client.0.smithi008.stdout:1/889: readlink d4/d33/d74/da2/lf5 0 2024-04-12T05:55:43.242 INFO:tasks.workunit.client.0.smithi008.stdout:4/799: creat d1/d22/d29/d30/d60/ffc x:0 0 0 2024-04-12T05:55:43.243 INFO:tasks.workunit.client.0.smithi008.stdout:6/941: fsync df/d8c/db1/ddc/f7f 0 2024-04-12T05:55:43.245 INFO:tasks.workunit.client.0.smithi008.stdout:7/977: creat d1/d15/f14f x:0 0 0 2024-04-12T05:55:43.245 INFO:tasks.workunit.client.0.smithi008.stdout:8/834: truncate dd/de/d16/f45 134374 0 2024-04-12T05:55:43.248 INFO:tasks.workunit.client.0.smithi008.stdout:9/923: rename d3/d22/d2b/d58/fdf to d3/df/d13/d21/d2d/d30/d74/d115/f136 0 2024-04-12T05:55:43.249 INFO:tasks.workunit.client.0.smithi008.stdout:0/881: dread d2/d11/d2a/f37 [0,4194304] 0 2024-04-12T05:55:43.250 INFO:tasks.workunit.client.0.smithi008.stdout:4/800: symlink d1/d22/d29/d30/d47/d28/d4a/lfd 0 2024-04-12T05:55:43.251 INFO:tasks.workunit.client.0.smithi008.stdout:4/801: dread - d1/d4f/dd5/fe4 zero size 2024-04-12T05:55:43.255 INFO:tasks.workunit.client.0.smithi008.stdout:2/992: write d6/db/d10/d4c/f5a [291002,103897] 0 2024-04-12T05:55:43.256 INFO:tasks.workunit.client.0.smithi008.stdout:9/924: dwrite d3/df/f86 [0,4194304] 0 2024-04-12T05:55:43.258 INFO:tasks.workunit.client.0.smithi008.stdout:7/978: symlink d1/d15/d43/d54/l150 0 2024-04-12T05:55:43.258 INFO:tasks.workunit.client.0.smithi008.stdout:8/835: symlink dd/de/d16/d10d/l11d 0 2024-04-12T05:55:43.259 INFO:tasks.workunit.client.0.smithi008.stdout:8/836: chown dd/de/d16/d50/d5e 50214317 1 2024-04-12T05:55:43.262 INFO:tasks.workunit.client.0.smithi008.stdout:6/942: mknod df/d21/d4c/dfb/d10a/c139 0 2024-04-12T05:55:43.267 INFO:tasks.workunit.client.0.smithi008.stdout:0/882: mknod d2/d4b/da2/d100/c11e 0 2024-04-12T05:55:43.268 INFO:tasks.workunit.client.0.smithi008.stdout:0/883: fsync d2/d11/d2a/db5/feb 0 2024-04-12T05:55:43.268 INFO:tasks.workunit.client.0.smithi008.stdout:6/943: mknod df/d8c/db1/c13a 0 2024-04-12T05:55:43.273 INFO:tasks.workunit.client.0.smithi008.stdout:8/837: symlink dd/de/d4c/d66/d6c/df4/l11e 0 2024-04-12T05:55:43.276 INFO:tasks.workunit.client.0.smithi008.stdout:2/993: write d6/db/d10/d23/d35/ff3 [61500,52848] 0 2024-04-12T05:55:43.279 INFO:tasks.workunit.client.0.smithi008.stdout:9/925: link d3/df/d13/d21/d2d/d30/d74/de9/df2/f103 d3/d22/d2b/f137 0 2024-04-12T05:55:43.280 INFO:tasks.workunit.client.0.smithi008.stdout:8/838: write dd/d4f/d52/f69 [3837918,24541] 0 2024-04-12T05:55:43.283 INFO:tasks.workunit.client.0.smithi008.stdout:0/884: dread d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:43.284 INFO:tasks.workunit.client.0.smithi008.stdout:2/994: chown d6/db/d10/d23/f108 1111846 1 2024-04-12T05:55:43.286 INFO:tasks.workunit.client.0.smithi008.stdout:6/944: dwrite df/d8c/dcc/dee/f107 [0,4194304] 0 2024-04-12T05:55:43.289 INFO:tasks.workunit.client.0.smithi008.stdout:4/802: dwrite d1/d22/d29/d46/d56/f92 [0,4194304] 0 2024-04-12T05:55:43.290 INFO:tasks.workunit.client.0.smithi008.stdout:7/979: dread d1/f2d [4194304,4194304] 0 2024-04-12T05:55:43.290 INFO:tasks.workunit.client.0.smithi008.stdout:8/839: fsync dd/de/d16/d10e/fd2 0 2024-04-12T05:55:43.292 INFO:tasks.workunit.client.0.smithi008.stdout:9/926: chown d3/d22/d2b/d58/d36/d124/d60/c98 2595302 1 2024-04-12T05:55:43.296 INFO:tasks.workunit.client.0.smithi008.stdout:6/945: creat df/dc1/f13b x:0 0 0 2024-04-12T05:55:43.296 INFO:tasks.workunit.client.0.smithi008.stdout:9/927: stat d3/d22/d2b/d58/d36/d124/f61 0 2024-04-12T05:55:43.298 INFO:tasks.workunit.client.0.smithi008.stdout:7/980: write d1/d15/d43/d54/d58/d5e/d7f/fba [3204086,11533] 0 2024-04-12T05:55:43.300 INFO:tasks.workunit.client.0.smithi008.stdout:6/946: readlink df/d10/d20/d3e/ld7 0 2024-04-12T05:55:43.301 INFO:tasks.workunit.client.0.smithi008.stdout:8/840: mkdir dd/de/d16/d50/df1/d11f 0 2024-04-12T05:55:43.305 INFO:tasks.workunit.client.0.smithi008.stdout:8/841: rmdir dd/d30/d32 39 2024-04-12T05:55:43.305 INFO:tasks.workunit.client.0.smithi008.stdout:0/885: getdents d2/d11/d36/d47 0 2024-04-12T05:55:43.307 INFO:tasks.workunit.client.0.smithi008.stdout:2/995: write d6/db/d10/fda [219526,72994] 0 2024-04-12T05:55:43.309 INFO:tasks.workunit.client.0.smithi008.stdout:0/886: fsync d2/d11/d36/d47/ff2 0 2024-04-12T05:55:43.309 INFO:tasks.workunit.client.0.smithi008.stdout:4/803: write d1/d22/d29/d46/da7/fe7 [6387452,74834] 0 2024-04-12T05:55:43.310 INFO:tasks.workunit.client.0.smithi008.stdout:8/842: mknod dd/d30/d3b/c120 0 2024-04-12T05:55:43.313 INFO:tasks.workunit.client.0.smithi008.stdout:6/947: truncate df/d10/d15/d79/fe0 1941583 0 2024-04-12T05:55:43.315 INFO:tasks.workunit.client.0.smithi008.stdout:0/887: symlink d2/d24/d61/l11f 0 2024-04-12T05:55:43.315 INFO:tasks.workunit.client.0.smithi008.stdout:8/843: mknod dd/de/c121 0 2024-04-12T05:55:43.320 INFO:tasks.workunit.client.0.smithi008.stdout:4/804: truncate d1/d22/d29/d30/d47/d28/fa1 279367 0 2024-04-12T05:55:43.320 INFO:tasks.workunit.client.0.smithi008.stdout:7/981: dread d1/d15/d43/d54/d77/f12c [0,4194304] 0 2024-04-12T05:55:43.325 INFO:tasks.workunit.client.0.smithi008.stdout:0/888: creat d2/d11/d2a/db5/df8/d109/f120 x:0 0 0 2024-04-12T05:55:43.326 INFO:tasks.workunit.client.0.smithi008.stdout:9/928: dwrite d3/df/d13/d21/d5c/dd7/f8c [0,4194304] 0 2024-04-12T05:55:43.329 INFO:tasks.workunit.client.0.smithi008.stdout:6/948: link df/dc1/fc4 df/d8c/db1/ddc/f13c 0 2024-04-12T05:55:43.331 INFO:tasks.workunit.client.0.smithi008.stdout:2/996: dread d6/db/d6f/da6/f11a [0,4194304] 0 2024-04-12T05:55:43.332 INFO:tasks.workunit.client.0.smithi008.stdout:4/805: fsync d1/d22/d29/d30/f34 0 2024-04-12T05:55:43.333 INFO:tasks.workunit.client.0.smithi008.stdout:9/929: dread - d3/df/d13/d21/d5c/d8f/feb zero size 2024-04-12T05:55:43.337 INFO:tasks.workunit.client.0.smithi008.stdout:9/930: fsync d3/df/d13/fff 0 2024-04-12T05:55:43.338 INFO:tasks.workunit.client.0.smithi008.stdout:4/806: write d1/d4f/dd5/fe4 [566802,94536] 0 2024-04-12T05:55:43.341 INFO:tasks.workunit.client.0.smithi008.stdout:8/844: truncate dd/de/d16/f2e 4811004 0 2024-04-12T05:55:43.342 INFO:tasks.workunit.client.0.smithi008.stdout:7/982: dwrite d1/d5/da/d78/dd6/f116 [0,4194304] 0 2024-04-12T05:55:43.344 INFO:tasks.workunit.client.0.smithi008.stdout:0/889: creat d2/d11/d2a/f121 x:0 0 0 2024-04-12T05:55:43.345 INFO:tasks.workunit.client.0.smithi008.stdout:3/858: sync 2024-04-12T05:55:43.346 INFO:tasks.workunit.client.0.smithi008.stdout:2/997: dread d6/db/d10/d23/d35/fca [0,4194304] 0 2024-04-12T05:55:43.347 INFO:tasks.workunit.client.0.smithi008.stdout:8/845: chown dd/de/d16/f45 100979 1 2024-04-12T05:55:43.348 INFO:tasks.workunit.client.0.smithi008.stdout:0/890: write d2/fe [2543575,114336] 0 2024-04-12T05:55:43.350 INFO:tasks.workunit.client.0.smithi008.stdout:4/807: mknod d1/d22/d29/d30/d47/d28/d4a/cfe 0 2024-04-12T05:55:43.351 INFO:tasks.workunit.client.0.smithi008.stdout:6/949: link df/d10/l17 df/d8c/db1/ddc/l13d 0 2024-04-12T05:55:43.354 INFO:tasks.workunit.client.0.smithi008.stdout:7/983: getdents d1/d149 0 2024-04-12T05:55:43.354 INFO:tasks.workunit.client.0.smithi008.stdout:2/998: unlink d6/db/d38/dcf/fee 0 2024-04-12T05:55:43.356 INFO:tasks.workunit.client.0.smithi008.stdout:9/931: creat d3/df/d13/d14/f138 x:0 0 0 2024-04-12T05:55:43.356 INFO:tasks.workunit.client.0.smithi008.stdout:7/984: write d1/d15/d43/f11e [994031,101431] 0 2024-04-12T05:55:43.358 INFO:tasks.workunit.client.0.smithi008.stdout:2/999: fsync d6/db/d60/d4f/d126/f112 0 2024-04-12T05:55:43.359 INFO:tasks.workunit.client.0.smithi008.stdout:4/808: symlink d1/d22/d29/d30/d60/dd0/ddb/lff 0 2024-04-12T05:55:43.360 INFO:tasks.workunit.client.0.smithi008.stdout:6/950: chown df/d10/d15/fda 97444 1 2024-04-12T05:55:43.363 INFO:tasks.workunit.client.0.smithi008.stdout:0/891: dread d2/d11/d2a/f37 [0,4194304] 0 2024-04-12T05:55:43.363 INFO:tasks.workunit.client.0.smithi008.stdout:7/985: read d1/d15/d43/f11e [372836,31050] 0 2024-04-12T05:55:43.365 INFO:tasks.workunit.client.0.smithi008.stdout:8/846: rename dd/de/d4c/l9e to dd/dab/l122 0 2024-04-12T05:55:43.366 INFO:tasks.workunit.client.0.smithi008.stdout:7/986: stat d1/d5/d10b/l142 0 2024-04-12T05:55:43.367 INFO:tasks.workunit.client.0.smithi008.stdout:3/859: dwrite d9/d11/f49 [0,4194304] 0 2024-04-12T05:55:43.368 INFO:tasks.workunit.client.0.smithi008.stdout:9/932: truncate d3/df/d13/d21/d5c/da4/fd4 1757235 0 2024-04-12T05:55:43.371 INFO:tasks.workunit.client.0.smithi008.stdout:0/892: rename d2/l29 to d2/d24/d85/ddc/l122 0 2024-04-12T05:55:43.373 INFO:tasks.workunit.client.0.smithi008.stdout:8/847: mknod dd/d30/c123 0 2024-04-12T05:55:43.373 INFO:tasks.workunit.client.0.smithi008.stdout:7/987: write d1/f6 [5244530,125125] 0 2024-04-12T05:55:43.375 INFO:tasks.workunit.client.0.smithi008.stdout:7/988: dread - d1/d5/da/d78/dd6/f139 zero size 2024-04-12T05:55:43.375 INFO:tasks.workunit.client.0.smithi008.stdout:3/860: rename d9/d10/d19/d3a/d7c/f111 to d9/d10/d19/d22/da8/f128 0 2024-04-12T05:55:43.377 INFO:tasks.workunit.client.0.smithi008.stdout:3/861: chown d9/d2e/f59 892845203 1 2024-04-12T05:55:43.379 INFO:tasks.workunit.client.0.smithi008.stdout:0/893: symlink d2/d11/d36/d82/l123 0 2024-04-12T05:55:43.381 INFO:tasks.workunit.client.0.smithi008.stdout:9/933: rename d3/df/d13/d21/d5c/d8f/dab/dda/d132 to d3/d22/d139 0 2024-04-12T05:55:43.388 INFO:tasks.workunit.client.0.smithi008.stdout:3/862: dread d9/d1e/f35 [0,4194304] 0 2024-04-12T05:55:43.389 INFO:tasks.workunit.client.0.smithi008.stdout:0/894: fdatasync d2/d24/d61/fad 0 2024-04-12T05:55:43.393 INFO:tasks.workunit.client.0.smithi008.stdout:4/809: dread d1/d22/d29/d30/f90 [0,4194304] 0 2024-04-12T05:55:43.401 INFO:tasks.workunit.client.0.smithi008.stdout:1/890: sync 2024-04-12T05:55:43.405 INFO:tasks.workunit.client.0.smithi008.stdout:3/863: dwrite d9/d2e/d3d/ffe [0,4194304] 0 2024-04-12T05:55:43.408 INFO:tasks.workunit.client.0.smithi008.stdout:1/891: creat d4/dd/d17/d66/f136 x:0 0 0 2024-04-12T05:55:43.410 INFO:tasks.workunit.client.0.smithi008.stdout:0/895: dwrite d2/d11/d2a/db5/f110 [0,4194304] 0 2024-04-12T05:55:43.411 INFO:tasks.workunit.client.0.smithi008.stdout:7/989: dwrite d1/f55 [0,4194304] 0 2024-04-12T05:55:43.414 INFO:tasks.workunit.client.0.smithi008.stdout:0/896: readlink d2/d11/d36/l58 0 2024-04-12T05:55:43.416 INFO:tasks.workunit.client.0.smithi008.stdout:9/934: dread d3/df/d13/d21/d5c/dd7/fbe [0,4194304] 0 2024-04-12T05:55:43.418 INFO:tasks.workunit.client.0.smithi008.stdout:4/810: dwrite d1/d22/d29/d46/fdd [0,4194304] 0 2024-04-12T05:55:43.418 INFO:tasks.workunit.client.0.smithi008.stdout:1/892: creat d4/d33/d74/d46/d4c/f137 x:0 0 0 2024-04-12T05:55:43.421 INFO:tasks.workunit.client.0.smithi008.stdout:4/811: fsync d1/d22/d29/d30/f52 0 2024-04-12T05:55:43.424 INFO:tasks.workunit.client.0.smithi008.stdout:9/935: creat d3/df/d13/d21/d2d/d30/d74/d115/f13a x:0 0 0 2024-04-12T05:55:43.425 INFO:tasks.workunit.client.0.smithi008.stdout:4/812: chown d1/db/c17 14834 1 2024-04-12T05:55:43.426 INFO:tasks.workunit.client.0.smithi008.stdout:7/990: creat d1/d15/f151 x:0 0 0 2024-04-12T05:55:43.428 INFO:tasks.workunit.client.0.smithi008.stdout:7/991: chown d1/d49 15981168 1 2024-04-12T05:55:43.429 INFO:tasks.workunit.client.0.smithi008.stdout:0/897: dread d2/d11/f25 [0,4194304] 0 2024-04-12T05:55:43.431 INFO:tasks.workunit.client.0.smithi008.stdout:9/936: readlink d3/df/d13/d21/d2d/d32/d52/l57 0 2024-04-12T05:55:43.433 INFO:tasks.workunit.client.0.smithi008.stdout:1/893: unlink d4/d1c/d4f/ceb 0 2024-04-12T05:55:43.438 INFO:tasks.workunit.client.0.smithi008.stdout:4/813: rename d1/d22/d29/d30/fd8 to d1/db/f100 0 2024-04-12T05:55:43.438 INFO:tasks.workunit.client.0.smithi008.stdout:7/992: rmdir d1/d15/d43/d54 39 2024-04-12T05:55:43.439 INFO:tasks.workunit.client.0.smithi008.stdout:1/894: write d4/d33/d74/d7a/ff1 [561374,28400] 0 2024-04-12T05:55:43.440 INFO:tasks.workunit.client.0.smithi008.stdout:3/864: write d9/d2e/f94 [288933,10060] 0 2024-04-12T05:55:43.446 INFO:tasks.workunit.client.0.smithi008.stdout:9/937: dwrite d3/d6a/f9e [4194304,4194304] 0 2024-04-12T05:55:43.450 INFO:tasks.workunit.client.0.smithi008.stdout:0/898: dwrite d2/d24/f79 [4194304,4194304] 0 2024-04-12T05:55:43.451 INFO:tasks.workunit.client.0.smithi008.stdout:1/895: getdents d4 0 2024-04-12T05:55:43.453 INFO:tasks.workunit.client.0.smithi008.stdout:0/899: write d2/fc1 [1160621,90404] 0 2024-04-12T05:55:43.456 INFO:tasks.workunit.client.0.smithi008.stdout:9/938: unlink d3/df/d13/d21/d2d/d75/f94 0 2024-04-12T05:55:43.456 INFO:tasks.workunit.client.0.smithi008.stdout:7/993: write d1/d5/da/d1d/fc5 [562028,103707] 0 2024-04-12T05:55:43.458 INFO:tasks.workunit.client.0.smithi008.stdout:1/896: symlink d4/dd/d17/d66/l138 0 2024-04-12T05:55:43.460 INFO:tasks.workunit.client.0.smithi008.stdout:7/994: stat d1/d5/fc9 0 2024-04-12T05:55:43.460 INFO:tasks.workunit.client.0.smithi008.stdout:9/939: creat d3/d22/d2b/d77/d82/f13b x:0 0 0 2024-04-12T05:55:43.462 INFO:tasks.workunit.client.0.smithi008.stdout:1/897: fsync d4/dd/d17/fb2 0 2024-04-12T05:55:43.463 INFO:tasks.workunit.client.0.smithi008.stdout:9/940: stat d3/df/d13/d21/d5c/dd7/fbe 0 2024-04-12T05:55:43.463 INFO:tasks.workunit.client.0.smithi008.stdout:7/995: mknod d1/d5/da/d11/d56/c152 0 2024-04-12T05:55:43.467 INFO:tasks.workunit.client.0.smithi008.stdout:7/996: mkdir d1/d5/da/d11/d56/d153 0 2024-04-12T05:55:43.468 INFO:tasks.workunit.client.0.smithi008.stdout:3/865: dread d9/d2e/d3d/f70 [0,4194304] 0 2024-04-12T05:55:43.468 INFO:tasks.workunit.client.0.smithi008.stdout:9/941: mknod d3/df/d13/d21/d2d/d30/d74/d115/c13c 0 2024-04-12T05:55:43.471 INFO:tasks.workunit.client.0.smithi008.stdout:0/900: truncate d2/d11/f69 3958920 0 2024-04-12T05:55:43.471 INFO:tasks.workunit.client.0.smithi008.stdout:0/901: chown d2/d24/d39 387 1 2024-04-12T05:55:43.472 INFO:tasks.workunit.client.0.smithi008.stdout:7/997: fdatasync d1/d5/da/d11/f69 0 2024-04-12T05:55:43.475 INFO:tasks.workunit.client.0.smithi008.stdout:7/998: dread - d1/d5/da/d60/d7b/f104 zero size 2024-04-12T05:55:43.475 INFO:tasks.workunit.client.0.smithi008.stdout:1/898: dread d4/d7/fe [0,4194304] 0 2024-04-12T05:55:43.480 INFO:tasks.workunit.client.0.smithi008.stdout:9/942: rename d3/df/f28 to d3/d22/d2b/d58/d5d/f13d 0 2024-04-12T05:55:43.481 INFO:tasks.workunit.client.0.smithi008.stdout:0/902: creat d2/d11/d36/d42/da6/f124 x:0 0 0 2024-04-12T05:55:43.484 INFO:tasks.workunit.client.0.smithi008.stdout:9/943: readlink d3/df/d13/d21/d2d/d32/ld0 0 2024-04-12T05:55:43.488 INFO:tasks.workunit.client.0.smithi008.stdout:7/999: dwrite d1/d15/d43/f100 [0,4194304] 0 2024-04-12T05:55:43.509 INFO:tasks.workunit.client.0.smithi008.stdout:1/899: symlink d4/d24/d62/dfd/l139 0 2024-04-12T05:55:43.510 INFO:tasks.workunit.client.0.smithi008.stdout:0/903: rename d2/d24/f113 to d2/ddb/d11c/f125 0 2024-04-12T05:55:43.514 INFO:tasks.workunit.client.0.smithi008.stdout:3/866: getdents d9/d10/d19/d3a/d67 0 2024-04-12T05:55:43.515 INFO:tasks.workunit.client.0.smithi008.stdout:3/867: dread - d9/dd/f53 zero size 2024-04-12T05:55:43.516 INFO:tasks.workunit.client.0.smithi008.stdout:1/900: mkdir d4/dd/d21/d13a 0 2024-04-12T05:55:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:1/901: read - d4/dd/d17/fb2 zero size 2024-04-12T05:55:43.517 INFO:tasks.workunit.client.0.smithi008.stdout:0/904: mknod d2/d11/d2a/db5/df8/d109/c126 0 2024-04-12T05:55:43.520 INFO:tasks.workunit.client.0.smithi008.stdout:0/905: mknod d2/d11/d2a/db5/df8/c127 0 2024-04-12T05:55:43.522 INFO:tasks.workunit.client.0.smithi008.stdout:3/868: truncate d9/dd/f7f 798223 0 2024-04-12T05:55:43.523 INFO:tasks.workunit.client.0.smithi008.stdout:0/906: unlink d2/d11/c2b 0 2024-04-12T05:55:43.524 INFO:tasks.workunit.client.0.smithi008.stdout:0/907: readlink d2/d24/l102 0 2024-04-12T05:55:43.525 INFO:tasks.workunit.client.0.smithi008.stdout:0/908: chown d2/d11/d36/d47/f8b 184663580 1 2024-04-12T05:55:43.525 INFO:tasks.workunit.client.0.smithi008.stdout:0/909: chown d2/d4b/f99 11950781 1 2024-04-12T05:55:43.526 INFO:tasks.workunit.client.0.smithi008.stdout:3/869: unlink d9/d11/d5b/lf7 0 2024-04-12T05:55:43.527 INFO:tasks.workunit.client.0.smithi008.stdout:0/910: write d2/d11/d2a/fda [205419,59659] 0 2024-04-12T05:55:43.528 INFO:tasks.workunit.client.0.smithi008.stdout:0/911: chown d2/d4b/fc6 105 1 2024-04-12T05:55:43.528 INFO:tasks.workunit.client.0.smithi008.stdout:3/870: fsync d9/d10/d19/d65/f81 0 2024-04-12T05:55:43.530 INFO:tasks.workunit.client.0.smithi008.stdout:3/871: chown d9/d10/d19/f50 96669 1 2024-04-12T05:55:43.532 INFO:tasks.workunit.client.0.smithi008.stdout:3/872: mknod d9/d10/d19/d3a/dad/dcf/c129 0 2024-04-12T05:55:43.533 INFO:tasks.workunit.client.0.smithi008.stdout:3/873: stat d9/d10/d19/d22/dec/c100 0 2024-04-12T05:55:43.535 INFO:tasks.workunit.client.0.smithi008.stdout:3/874: mknod d9/d2e/dee/c12a 0 2024-04-12T05:55:43.537 INFO:tasks.workunit.client.0.smithi008.stdout:3/875: mknod d9/d10/d19/c12b 0 2024-04-12T05:55:43.541 INFO:tasks.workunit.client.0.smithi008.stdout:3/876: truncate d9/d10/d19/d22/d2c/f83 471478 0 2024-04-12T05:55:43.545 INFO:tasks.workunit.client.0.smithi008.stdout:9/944: write d3/df/d13/d21/d2d/d30/f4d [149186,11224] 0 2024-04-12T05:55:43.548 INFO:tasks.workunit.client.0.smithi008.stdout:9/945: mknod d3/dae/dba/c13e 0 2024-04-12T05:55:43.549 INFO:tasks.workunit.client.0.smithi008.stdout:9/946: readlink d3/l8 0 2024-04-12T05:55:43.550 INFO:tasks.workunit.client.0.smithi008.stdout:1/902: truncate d4/d1c/d4f/d10b/f121 1947316 0 2024-04-12T05:55:43.553 INFO:tasks.workunit.client.0.smithi008.stdout:1/903: symlink d4/dd/d21/d9f/d100/l13b 0 2024-04-12T05:55:43.556 INFO:tasks.workunit.client.0.smithi008.stdout:3/877: write d9/dd/f2f [8588439,36002] 0 2024-04-12T05:55:43.558 INFO:tasks.workunit.client.0.smithi008.stdout:1/904: rmdir d4/dd/d21/d9f/d126/d125 0 2024-04-12T05:55:43.558 INFO:tasks.workunit.client.0.smithi008.stdout:3/878: dread - d9/d10/d19/d3a/d67/f97 zero size 2024-04-12T05:55:43.566 INFO:tasks.workunit.client.0.smithi008.stdout:3/879: creat d9/d10/f12c x:0 0 0 2024-04-12T05:55:43.568 INFO:tasks.workunit.client.0.smithi008.stdout:9/947: dread d3/f25 [0,4194304] 0 2024-04-12T05:55:43.572 INFO:tasks.workunit.client.0.smithi008.stdout:9/948: mkdir d3/df/d13/d21/d5c/dd7/d134/d13f 0 2024-04-12T05:55:43.577 INFO:tasks.workunit.client.0.smithi008.stdout:9/949: mknod d3/d22/d2b/d12d/c140 0 2024-04-12T05:55:43.578 INFO:tasks.workunit.client.0.smithi008.stdout:3/880: dwrite d9/d11/d5b/fcd [0,4194304] 0 2024-04-12T05:55:43.580 INFO:tasks.workunit.client.0.smithi008.stdout:1/905: dwrite d4/dd/d17/d66/fbd [0,4194304] 0 2024-04-12T05:55:43.583 INFO:tasks.workunit.client.0.smithi008.stdout:1/906: write d4/dd/d17/d66/fe3 [4623096,20817] 0 2024-04-12T05:55:43.587 INFO:tasks.workunit.client.0.smithi008.stdout:3/881: write d9/d37/d69/f122 [445766,39255] 0 2024-04-12T05:55:43.590 INFO:tasks.workunit.client.0.smithi008.stdout:9/950: creat d3/d22/d2b/d77/d82/d12a/f141 x:0 0 0 2024-04-12T05:55:43.592 INFO:tasks.workunit.client.0.smithi008.stdout:9/951: readlink d3/df/d13/d21/d2d/d32/ld0 0 2024-04-12T05:55:43.594 INFO:tasks.workunit.client.0.smithi008.stdout:3/882: truncate d9/f125 3257543 0 2024-04-12T05:55:43.595 INFO:tasks.workunit.client.0.smithi008.stdout:3/883: write d9/d10/f33 [4764168,16413] 0 2024-04-12T05:55:43.595 INFO:tasks.workunit.client.0.smithi008.stdout:9/952: creat d3/df/d13/d21/d5c/d8f/f142 x:0 0 0 2024-04-12T05:55:43.599 INFO:tasks.workunit.client.0.smithi008.stdout:9/953: truncate d3/d22/d2b/d77/ff6 114338 0 2024-04-12T05:55:43.601 INFO:tasks.workunit.client.0.smithi008.stdout:9/954: creat d3/d22/f143 x:0 0 0 2024-04-12T05:55:43.605 INFO:tasks.workunit.client.0.smithi008.stdout:3/884: dwrite d9/d10/d105/f112 [0,4194304] 0 2024-04-12T05:55:43.605 INFO:tasks.workunit.client.0.smithi008.stdout:9/955: fdatasync d3/df/d13/d21/d2d/d30/de2/ff1 0 2024-04-12T05:55:43.607 INFO:tasks.workunit.client.0.smithi008.stdout:1/907: dwrite d4/dd/d17/f1f [0,4194304] 0 2024-04-12T05:55:43.608 INFO:tasks.workunit.client.0.smithi008.stdout:3/885: write d9/dd/f2f [397967,30230] 0 2024-04-12T05:55:43.611 INFO:tasks.workunit.client.0.smithi008.stdout:1/908: write d4/d24/faa [3048075,41520] 0 2024-04-12T05:55:43.612 INFO:tasks.workunit.client.0.smithi008.stdout:9/956: truncate d3/df/d13/d21/d2d/d30/f4d 1059942 0 2024-04-12T05:55:43.650 INFO:tasks.workunit.client.0.smithi008.stdout:1/909: symlink d4/d6/d11/ddb/l13c 0 2024-04-12T05:55:43.655 INFO:tasks.workunit.client.0.smithi008.stdout:1/910: mknod d4/dd/d21/d9f/d100/c13d 0 2024-04-12T05:55:43.657 INFO:tasks.workunit.client.0.smithi008.stdout:9/957: creat d3/d22/d2b/d58/d90/df9/dfa/f144 x:0 0 0 2024-04-12T05:55:43.662 INFO:tasks.workunit.client.0.smithi008.stdout:1/911: rename d4/dd/f89 to d4/dd/d17/d26/f13e 0 2024-04-12T05:55:43.663 INFO:tasks.workunit.client.0.smithi008.stdout:9/958: creat d3/d22/d2b/d77/d82/f145 x:0 0 0 2024-04-12T05:55:43.666 INFO:tasks.workunit.client.0.smithi008.stdout:1/912: chown d4/d1a/d57/c11b 201532 1 2024-04-12T05:55:43.667 INFO:tasks.workunit.client.0.smithi008.stdout:1/913: chown d4/dd/d17/f64 7 1 2024-04-12T05:55:43.671 INFO:tasks.workunit.client.0.smithi008.stdout:9/959: link d3/d22/f10e d3/df/d13/d21/d5c/dd7/d134/f146 0 2024-04-12T05:55:43.672 INFO:tasks.workunit.client.0.smithi008.stdout:1/914: dread - d4/d33/d74/d30/d72/d7c/fa1 zero size 2024-04-12T05:55:43.676 INFO:tasks.workunit.client.0.smithi008.stdout:1/915: rename d4/d24/d62/dfd to d4/dd/d13f 0 2024-04-12T05:55:43.677 INFO:tasks.workunit.client.0.smithi008.stdout:9/960: truncate d3/d22/d2b/d58/da8/db6/fcb 2940215 0 2024-04-12T05:55:43.679 INFO:tasks.workunit.client.0.smithi008.stdout:9/961: write d3/df/d13/d14/f138 [455438,72983] 0 2024-04-12T05:55:43.681 INFO:tasks.workunit.client.0.smithi008.stdout:1/916: rename d4/dd/d17/d26/fdf to d4/d6/d11/ddb/f140 0 2024-04-12T05:55:43.686 INFO:tasks.workunit.client.0.smithi008.stdout:9/962: rename d3/f35 to d3/df/d13/d21/f147 0 2024-04-12T05:55:43.691 INFO:tasks.workunit.client.0.smithi008.stdout:1/917: write d4/d1c/d4f/d10b/f121 [1428182,8029] 0 2024-04-12T05:55:43.693 INFO:tasks.workunit.client.0.smithi008.stdout:1/918: chown d4/dd/d17/d91/da4/fea 16194237 1 2024-04-12T05:55:43.695 INFO:tasks.workunit.client.0.smithi008.stdout:1/919: fdatasync d4/d33/d74/d30/d72/f96 0 2024-04-12T05:55:43.697 INFO:tasks.workunit.client.0.smithi008.stdout:1/920: dread d4/dd/d17/d26/f35 [0,4194304] 0 2024-04-12T05:55:43.704 INFO:tasks.workunit.client.0.smithi008.stdout:1/921: dwrite d4/d33/d74/d30/d72/f113 [0,4194304] 0 2024-04-12T05:55:43.708 INFO:tasks.workunit.client.0.smithi008.stdout:9/963: dread d3/d22/d2b/d58/d89/f8d [0,4194304] 0 2024-04-12T05:55:43.708 INFO:tasks.workunit.client.0.smithi008.stdout:1/922: mknod d4/d33/d74/d7a/db6/dc9/d131/c141 0 2024-04-12T05:55:43.711 INFO:tasks.workunit.client.0.smithi008.stdout:9/964: unlink d3/d22/d2b/d77/d82/f106 0 2024-04-12T05:55:43.712 INFO:tasks.workunit.client.0.smithi008.stdout:1/923: mknod d4/d33/d74/c142 0 2024-04-12T05:55:43.714 INFO:tasks.workunit.client.0.smithi008.stdout:9/965: creat d3/df/d13/d21/d2d/d30/d74/d115/f148 x:0 0 0 2024-04-12T05:55:43.719 INFO:tasks.workunit.client.0.smithi008.stdout:9/966: getdents d3/df/d13/d21/d5c/d8f 0 2024-04-12T05:55:43.720 INFO:tasks.workunit.client.0.smithi008.stdout:1/924: write d4/d33/d74/f43 [1694538,30541] 0 2024-04-12T05:55:43.721 INFO:tasks.workunit.client.0.smithi008.stdout:1/925: chown d4/d1c/db9 9634469 1 2024-04-12T05:55:43.721 INFO:tasks.workunit.client.0.smithi008.stdout:9/967: dread - d3/d22/d2b/d58/d90/df9/dfa/f144 zero size 2024-04-12T05:55:43.724 INFO:tasks.workunit.client.0.smithi008.stdout:9/968: unlink d3/d22/d2b/d58/d36/d124/f11b 0 2024-04-12T05:55:43.731 INFO:tasks.workunit.client.0.smithi008.stdout:1/926: dwrite d4/d6/f59 [4194304,4194304] 0 2024-04-12T05:55:43.732 INFO:tasks.workunit.client.0.smithi008.stdout:9/969: dread d3/df/d13/d21/d5c/dd7/f8c [0,4194304] 0 2024-04-12T05:55:43.734 INFO:tasks.workunit.client.0.smithi008.stdout:1/927: read d4/d7/f4e [195940,118430] 0 2024-04-12T05:55:43.735 INFO:tasks.workunit.client.0.smithi008.stdout:9/970: creat d3/d22/d2b/d58/da8/f149 x:0 0 0 2024-04-12T05:55:43.736 INFO:tasks.workunit.client.0.smithi008.stdout:9/971: write d3/d22/d2b/fea [603197,4721] 0 2024-04-12T05:55:43.738 INFO:tasks.workunit.client.0.smithi008.stdout:9/972: symlink d3/df/d13/d21/d2d/d75/l14a 0 2024-04-12T05:55:43.740 INFO:tasks.workunit.client.0.smithi008.stdout:9/973: write d3/df/d13/d21/d5c/d8f/feb [23764,102124] 0 2024-04-12T05:55:43.751 INFO:tasks.workunit.client.0.smithi008.stdout:9/974: dread d3/d22/d2b/d58/d36/d124/d60/f6d [0,4194304] 0 2024-04-12T05:55:43.755 INFO:tasks.workunit.client.0.smithi008.stdout:9/975: link d3/df/d13/d21/d2d/d30/d74/cd2 d3/df/d13/d21/d2d/d30/c14b 0 2024-04-12T05:55:43.756 INFO:tasks.workunit.client.0.smithi008.stdout:9/976: rmdir d3/dae/dba 39 2024-04-12T05:55:43.756 INFO:tasks.workunit.client.0.smithi008.stdout:9/977: chown d3/df/d13/d14/d95 1313594516 1 2024-04-12T05:55:43.758 INFO:tasks.workunit.client.0.smithi008.stdout:9/978: write d3/df/d13/d21/d2d/d30/d74/d115/f136 [841052,124487] 0 2024-04-12T05:55:43.759 INFO:tasks.workunit.client.0.smithi008.stdout:9/979: rmdir d3/d22/d2b/d58/d38 39 2024-04-12T05:55:43.761 INFO:tasks.workunit.client.0.smithi008.stdout:9/980: write d3/d22/d2b/d58/f133 [597136,118778] 0 2024-04-12T05:55:43.770 INFO:tasks.workunit.client.0.smithi008.stdout:8/848: sync 2024-04-12T05:55:43.770 INFO:tasks.workunit.client.0.smithi008.stdout:6/951: sync 2024-04-12T05:55:43.772 INFO:tasks.workunit.client.0.smithi008.stdout:6/952: write df/d8c/db1/ddc/d9b/fcb [3490429,130967] 0 2024-04-12T05:55:43.772 INFO:tasks.workunit.client.0.smithi008.stdout:8/849: chown dd/de/d16/d50/d5e/dbe/dd6/cfe 0 1 2024-04-12T05:55:43.774 INFO:tasks.workunit.client.0.smithi008.stdout:8/850: fsync dd/de/d4c/d66/ff9 0 2024-04-12T05:55:43.776 INFO:tasks.workunit.client.0.smithi008.stdout:6/953: truncate df/d8c/db1/ddc/f13c 2409688 0 2024-04-12T05:55:43.781 INFO:tasks.workunit.client.0.smithi008.stdout:6/954: unlink df/d21/d4c/lca 0 2024-04-12T05:55:43.782 INFO:tasks.workunit.client.0.smithi008.stdout:6/955: mkdir df/d10/d15/d4f/de9/d13e 0 2024-04-12T05:55:43.786 INFO:tasks.workunit.client.0.smithi008.stdout:6/956: rename df/d8c/db1/ddc/f98 to df/d8c/dcc/dee/f13f 0 2024-04-12T05:55:43.787 INFO:tasks.workunit.client.0.smithi008.stdout:8/851: write dd/de/d16/d50/d56/d61/f105 [2724376,55518] 0 2024-04-12T05:55:43.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:43 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:43.888 INFO:tasks.workunit.client.0.smithi008.stdout:6/957: link df/d10/d15/d4f/f134 df/d21/f140 0 2024-04-12T05:55:43.893 INFO:tasks.workunit.client.0.smithi008.stdout:6/958: rename df/d21/d4c/f126 to df/d10/d20/d3e/db4/df0/f141 0 2024-04-12T05:55:43.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:43 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:43.914 INFO:tasks.workunit.client.0.smithi008.stdout:6/959: creat df/d10/d15/dd0/f142 x:0 0 0 2024-04-12T05:55:43.915 INFO:tasks.workunit.client.0.smithi008.stdout:6/960: stat df/d10/d20/d3e/f72 0 2024-04-12T05:55:43.918 INFO:tasks.workunit.client.0.smithi008.stdout:6/961: creat df/d10/d20/d3e/db4/f143 x:0 0 0 2024-04-12T05:55:43.918 INFO:tasks.workunit.client.0.smithi008.stdout:6/962: stat df/d8c/da4/dad/fd5 0 2024-04-12T05:55:43.919 INFO:tasks.workunit.client.0.smithi008.stdout:6/963: write df/d10/d20/d3e/f72 [3850358,16854] 0 2024-04-12T05:55:43.922 INFO:tasks.workunit.client.0.smithi008.stdout:6/964: creat df/d21/d4c/dfb/d10a/f144 x:0 0 0 2024-04-12T05:55:43.922 INFO:tasks.workunit.client.0.smithi008.stdout:6/965: readlink df/d8c/da4/ld2 0 2024-04-12T05:55:43.924 INFO:tasks.workunit.client.0.smithi008.stdout:4/814: sync 2024-04-12T05:55:43.924 INFO:tasks.workunit.client.0.smithi008.stdout:0/912: sync 2024-04-12T05:55:43.926 INFO:tasks.workunit.client.0.smithi008.stdout:0/913: read d2/fc1 [3268343,126678] 0 2024-04-12T05:55:43.929 INFO:tasks.workunit.client.0.smithi008.stdout:4/815: creat d1/d22/d29/d46/d56/daf/dcf/f101 x:0 0 0 2024-04-12T05:55:43.929 INFO:tasks.workunit.client.0.smithi008.stdout:0/914: symlink d2/ddb/d11c/l128 0 2024-04-12T05:55:43.930 INFO:tasks.workunit.client.0.smithi008.stdout:6/966: dwrite df/d10/d15/dd0/dd1/f6c [0,4194304] 0 2024-04-12T05:55:43.932 INFO:tasks.workunit.client.0.smithi008.stdout:6/967: read df/d21/d4c/fc7 [345991,111859] 0 2024-04-12T05:55:43.933 INFO:tasks.workunit.client.0.smithi008.stdout:6/968: chown df/d10/d15/dd0/dd1/d112/lea 0 1 2024-04-12T05:55:43.934 INFO:tasks.workunit.client.0.smithi008.stdout:4/816: mknod d1/d8/dc5/deb/c102 0 2024-04-12T05:55:43.935 INFO:tasks.workunit.client.0.smithi008.stdout:0/915: readlink d2/d24/d85/ddc/l122 0 2024-04-12T05:55:43.936 INFO:tasks.workunit.client.0.smithi008.stdout:6/969: mkdir df/d8c/db1/d145 0 2024-04-12T05:55:43.949 INFO:tasks.workunit.client.0.smithi008.stdout:4/817: dwrite d1/d4f/f7f [0,4194304] 0 2024-04-12T05:55:43.951 INFO:tasks.workunit.client.0.smithi008.stdout:0/916: dwrite d2/d11/d36/d42/da6/fd8 [0,4194304] 0 2024-04-12T05:55:43.951 INFO:tasks.workunit.client.0.smithi008.stdout:6/970: dwrite df/d10/f56 [0,4194304] 0 2024-04-12T05:55:43.952 INFO:tasks.workunit.client.0.smithi008.stdout:4/818: chown d1/d22/d29/d30/d47/d28/d4a/lac 32978164 1 2024-04-12T05:55:43.957 INFO:tasks.workunit.client.0.smithi008.stdout:0/917: creat d2/d11/d36/d42/da6/dd7/f129 x:0 0 0 2024-04-12T05:55:43.959 INFO:tasks.workunit.client.0.smithi008.stdout:4/819: mknod d1/d22/d29/d46/d56/daf/dcf/ded/c103 0 2024-04-12T05:55:43.960 INFO:tasks.workunit.client.0.smithi008.stdout:6/971: rmdir df/d8c/dcc 39 2024-04-12T05:55:43.962 INFO:tasks.workunit.client.0.smithi008.stdout:4/820: chown d1/d22/d29/d30/d47/d28/f79 12 1 2024-04-12T05:55:43.962 INFO:tasks.workunit.client.0.smithi008.stdout:0/918: rmdir d2/d11/d36/d42 39 2024-04-12T05:55:43.963 INFO:tasks.workunit.client.0.smithi008.stdout:0/919: chown d2/lca 0 1 2024-04-12T05:55:43.964 INFO:tasks.workunit.client.0.smithi008.stdout:6/972: mkdir df/d10/d15/dd0/dd1/d146 0 2024-04-12T05:55:43.965 INFO:tasks.workunit.client.0.smithi008.stdout:6/973: stat df/d10/d15/f2a 0 2024-04-12T05:55:43.968 INFO:tasks.workunit.client.0.smithi008.stdout:0/920: dread - d2/d11/d36/d42/da6/f124 zero size 2024-04-12T05:55:43.970 INFO:tasks.workunit.client.0.smithi008.stdout:6/974: rename df/lc2 to df/d21/l147 0 2024-04-12T05:55:43.973 INFO:tasks.workunit.client.0.smithi008.stdout:6/975: symlink df/d10/d20/l148 0 2024-04-12T05:55:43.974 INFO:tasks.workunit.client.0.smithi008.stdout:0/921: getdents d2/d24/d61 0 2024-04-12T05:55:43.977 INFO:tasks.workunit.client.0.smithi008.stdout:0/922: mknod d2/d11/d36/d42/da6/c12a 0 2024-04-12T05:55:43.979 INFO:tasks.workunit.client.0.smithi008.stdout:6/976: rename df/d10/d20/c34 to df/d10/d15/d79/c149 0 2024-04-12T05:55:43.981 INFO:tasks.workunit.client.0.smithi008.stdout:6/977: write df/d10/d15/d75/f9f [1769802,118521] 0 2024-04-12T05:55:43.983 INFO:tasks.workunit.client.0.smithi008.stdout:0/923: getdents d2/d11/d36 0 2024-04-12T05:55:43.984 INFO:tasks.workunit.client.0.smithi008.stdout:0/924: write d2/d11/d36/d82/fb4 [1800876,83142] 0 2024-04-12T05:55:43.984 INFO:tasks.workunit.client.0.smithi008.stdout:6/978: rmdir df/dc1 39 2024-04-12T05:55:43.989 INFO:tasks.workunit.client.0.smithi008.stdout:0/925: write d2/d24/d39/fae [4341641,64051] 0 2024-04-12T05:55:43.989 INFO:tasks.workunit.client.0.smithi008.stdout:6/979: read df/d10/d15/dd0/dd1/fb7 [1197714,63294] 0 2024-04-12T05:55:43.991 INFO:tasks.workunit.client.0.smithi008.stdout:0/926: creat d2/d11/d36/d42/d86/d9b/f12b x:0 0 0 2024-04-12T05:55:43.992 INFO:tasks.workunit.client.0.smithi008.stdout:0/927: write d2/d11/d36/d47/f94 [1241752,2420] 0 2024-04-12T05:55:43.994 INFO:tasks.workunit.client.0.smithi008.stdout:6/980: mkdir df/d21/d14a 0 2024-04-12T05:55:44.003 INFO:tasks.workunit.client.0.smithi008.stdout:4/821: dwrite d1/d4f/f5a [0,4194304] 0 2024-04-12T05:55:44.007 INFO:tasks.workunit.client.0.smithi008.stdout:3/886: sync 2024-04-12T05:55:44.007 INFO:tasks.workunit.client.0.smithi008.stdout:9/981: sync 2024-04-12T05:55:44.008 INFO:tasks.workunit.client.0.smithi008.stdout:1/928: sync 2024-04-12T05:55:44.008 INFO:tasks.workunit.client.0.smithi008.stdout:8/852: sync 2024-04-12T05:55:44.009 INFO:tasks.workunit.client.0.smithi008.stdout:0/928: dread d2/f72 [0,4194304] 0 2024-04-12T05:55:44.010 INFO:tasks.workunit.client.0.smithi008.stdout:4/822: dread d1/d22/d29/fd2 [0,4194304] 0 2024-04-12T05:55:44.010 INFO:tasks.workunit.client.0.smithi008.stdout:9/982: chown d3/df/d13/f122 0 1 2024-04-12T05:55:44.014 INFO:tasks.workunit.client.0.smithi008.stdout:3/887: creat d9/d10/d19/d3a/d67/f12d x:0 0 0 2024-04-12T05:55:44.015 INFO:tasks.workunit.client.0.smithi008.stdout:1/929: mkdir d4/d24/d143 0 2024-04-12T05:55:44.016 INFO:tasks.workunit.client.0.smithi008.stdout:4/823: write d1/d8/ff9 [251376,81365] 0 2024-04-12T05:55:44.018 INFO:tasks.workunit.client.0.smithi008.stdout:6/981: dwrite df/d8c/db1/ddc/d78/f91 [0,4194304] 0 2024-04-12T05:55:44.019 INFO:tasks.workunit.client.0.smithi008.stdout:0/929: rename d2/d4b/da2/fb8 to d2/ddb/d11b/f12c 0 2024-04-12T05:55:44.022 INFO:tasks.workunit.client.0.smithi008.stdout:8/853: write dd/ff [2730057,3958] 0 2024-04-12T05:55:44.023 INFO:tasks.workunit.client.0.smithi008.stdout:9/983: fdatasync d3/df/d13/d21/d2d/d30/faa 0 2024-04-12T05:55:44.023 INFO:tasks.workunit.client.0.smithi008.stdout:1/930: write d4/d1c/f130 [93710,71612] 0 2024-04-12T05:55:44.026 INFO:tasks.workunit.client.0.smithi008.stdout:0/930: creat d2/ddb/d11c/f12d x:0 0 0 2024-04-12T05:55:44.028 INFO:tasks.workunit.client.0.smithi008.stdout:1/931: fdatasync d4/d33/d74/d46/d4c/f134 0 2024-04-12T05:55:44.029 INFO:tasks.workunit.client.0.smithi008.stdout:6/982: dread df/d21/d4c/f52 [0,4194304] 0 2024-04-12T05:55:44.036 INFO:tasks.workunit.client.0.smithi008.stdout:8/854: rename dd/de/d4c/d66/d6c/dc0 to dd/de/d16/d50/d5e/dbe/dd6/d124 0 2024-04-12T05:55:44.037 INFO:tasks.workunit.client.0.smithi008.stdout:6/983: creat df/d10/d15/dd0/dd1/d112/f14b x:0 0 0 2024-04-12T05:55:44.038 INFO:tasks.workunit.client.0.smithi008.stdout:1/932: creat d4/dd/d17/d66/f144 x:0 0 0 2024-04-12T05:55:44.038 INFO:tasks.workunit.client.0.smithi008.stdout:4/824: link d1/d22/d29/d30/l32 d1/d22/d29/d30/d47/d28/d4a/d62/l104 0 2024-04-12T05:55:44.042 INFO:tasks.workunit.client.0.smithi008.stdout:8/855: write dd/d4f/d52/f79 [5119182,55602] 0 2024-04-12T05:55:44.045 INFO:tasks.workunit.client.0.smithi008.stdout:1/933: mknod d4/dd/c145 0 2024-04-12T05:55:44.050 INFO:tasks.workunit.client.0.smithi008.stdout:3/888: dwrite d9/d10/d19/d22/da8/fd7 [0,4194304] 0 2024-04-12T05:55:44.050 INFO:tasks.workunit.client.0.smithi008.stdout:6/984: write df/d21/d4c/fc7 [145325,62786] 0 2024-04-12T05:55:44.050 INFO:tasks.workunit.client.0.smithi008.stdout:0/931: dwrite d2/d4b/da2/fd0 [0,4194304] 0 2024-04-12T05:55:44.056 INFO:tasks.workunit.client.0.smithi008.stdout:8/856: creat dd/de/d16/db1/d97/dcf/f125 x:0 0 0 2024-04-12T05:55:44.057 INFO:tasks.workunit.client.0.smithi008.stdout:3/889: read - d9/d10/d19/d65/f81 zero size 2024-04-12T05:55:44.061 INFO:tasks.workunit.client.0.smithi008.stdout:1/934: rename d4/d1c to d4/d33/d74/d7a/db6/dc9/d146 0 2024-04-12T05:55:44.061 INFO:tasks.workunit.client.0.smithi008.stdout:9/984: dwrite d3/fb [0,4194304] 0 2024-04-12T05:55:44.063 INFO:tasks.workunit.client.0.smithi008.stdout:6/985: symlink df/d8c/db1/ddc/d105/l14c 0 2024-04-12T05:55:44.064 INFO:tasks.workunit.client.0.smithi008.stdout:0/932: mkdir d2/d11/d36/dd6/d12e 0 2024-04-12T05:55:44.067 INFO:tasks.workunit.client.0.smithi008.stdout:8/857: unlink dd/de/d16/d50/d5e/f92 0 2024-04-12T05:55:44.070 INFO:tasks.workunit.client.0.smithi008.stdout:8/858: chown dd/d30/d3b/f74 23976 1 2024-04-12T05:55:44.071 INFO:tasks.workunit.client.0.smithi008.stdout:0/933: mknod d2/d24/d85/ddc/c12f 0 2024-04-12T05:55:44.078 INFO:tasks.workunit.client.0.smithi008.stdout:8/859: mkdir dd/dab/d126 0 2024-04-12T05:55:44.082 INFO:tasks.workunit.client.0.smithi008.stdout:6/986: mkdir df/d10/d15/d4f/de9/d13e/d14d 0 2024-04-12T05:55:44.082 INFO:tasks.workunit.client.0.smithi008.stdout:9/985: dwrite d3/df/d13/d21/d2d/d30/d74/ff8 [4194304,4194304] 0 2024-04-12T05:55:44.082 INFO:tasks.workunit.client.0.smithi008.stdout:1/935: readlink d4/d33/l9d 0 2024-04-12T05:55:44.088 INFO:tasks.workunit.client.0.smithi008.stdout:0/934: read d2/d4b/da2/fa3 [2176963,99326] 0 2024-04-12T05:55:44.089 INFO:tasks.workunit.client.0.smithi008.stdout:3/890: truncate d9/d10/d19/d22/da8/fb2 667309 0 2024-04-12T05:55:44.091 INFO:tasks.workunit.client.0.smithi008.stdout:0/935: write d2/d11/d2a/f73 [3002884,32028] 0 2024-04-12T05:55:44.092 INFO:tasks.workunit.client.0.smithi008.stdout:3/891: readlink d9/d11/d5b/lb1 0 2024-04-12T05:55:44.094 INFO:tasks.workunit.client.0.smithi008.stdout:6/987: getdents df/d10/d20/d3e/dfc 0 2024-04-12T05:55:44.095 INFO:tasks.workunit.client.0.smithi008.stdout:1/936: write d4/dd/d17/d66/fbd [3754540,21173] 0 2024-04-12T05:55:44.100 INFO:tasks.workunit.client.0.smithi008.stdout:9/986: dwrite d3/df/d13/d21/d5c/d8f/f142 [0,4194304] 0 2024-04-12T05:55:44.107 INFO:tasks.workunit.client.0.smithi008.stdout:3/892: rename d9/d10/d19/d22/l77 to d9/d10/d19/d22/da8/l12e 0 2024-04-12T05:55:44.111 INFO:tasks.workunit.client.0.smithi008.stdout:1/937: unlink d4/d33/d74/d30/ccf 0 2024-04-12T05:55:44.111 INFO:tasks.workunit.client.0.smithi008.stdout:3/893: write d9/dd/fef [322644,116297] 0 2024-04-12T05:55:44.116 INFO:tasks.workunit.client.0.smithi008.stdout:9/987: dwrite d3/df/d13/d21/d2d/d32/d52/f11d [4194304,4194304] 0 2024-04-12T05:55:44.121 INFO:tasks.workunit.client.0.smithi008.stdout:6/988: dwrite df/d10/d15/f13 [4194304,4194304] 0 2024-04-12T05:55:44.123 INFO:tasks.workunit.client.0.smithi008.stdout:0/936: creat d2/d11/d2a/f130 x:0 0 0 2024-04-12T05:55:44.123 INFO:tasks.workunit.client.0.smithi008.stdout:6/989: dread - df/d10/d15/d4f/f134 zero size 2024-04-12T05:55:44.124 INFO:tasks.workunit.client.0.smithi008.stdout:9/988: write d3/df/d13/d21/d2d/d30/d74/f83 [3103755,48644] 0 2024-04-12T05:55:44.129 INFO:tasks.workunit.client.0.smithi008.stdout:1/938: creat d4/d33/d74/d7a/db6/dc9/f147 x:0 0 0 2024-04-12T05:55:44.130 INFO:tasks.workunit.client.0.smithi008.stdout:3/894: fdatasync d9/d10/d19/d22/d2c/f9a 0 2024-04-12T05:55:44.136 INFO:tasks.workunit.client.0.smithi008.stdout:9/989: stat d3/df/l79 0 2024-04-12T05:55:44.136 INFO:tasks.workunit.client.0.smithi008.stdout:1/939: mkdir d4/dd/d17/d26/d148 0 2024-04-12T05:55:44.139 INFO:tasks.workunit.client.0.smithi008.stdout:0/937: rmdir d2 39 2024-04-12T05:55:44.140 INFO:tasks.workunit.client.0.smithi008.stdout:9/990: truncate d3/df/d13/d21/d5c/dd7/d134/f146 658446 0 2024-04-12T05:55:44.143 INFO:tasks.workunit.client.0.smithi008.stdout:9/991: chown d3/df/f63 25139871 1 2024-04-12T05:55:44.145 INFO:tasks.workunit.client.0.smithi008.stdout:1/940: rename d4/dd/d17/c2d to d4/d33/d74/d7a/db6/dc9/d146/db9/dc1/c149 0 2024-04-12T05:55:44.148 INFO:tasks.workunit.client.0.smithi008.stdout:6/990: dwrite df/f1f [0,4194304] 0 2024-04-12T05:55:44.149 INFO:tasks.workunit.client.0.smithi008.stdout:6/991: read - df/d10/d15/dd0/f137 zero size 2024-04-12T05:55:44.149 INFO:tasks.workunit.client.0.smithi008.stdout:9/992: symlink d3/df/d13/d21/d5c/da4/l14c 0 2024-04-12T05:55:44.151 INFO:tasks.workunit.client.0.smithi008.stdout:6/992: write df/d10/d15/f133 [363881,101474] 0 2024-04-12T05:55:44.154 INFO:tasks.workunit.client.0.smithi008.stdout:6/993: write df/d10/d20/d3e/db4/f143 [138797,71543] 0 2024-04-12T05:55:44.155 INFO:tasks.workunit.client.0.smithi008.stdout:1/941: getdents d4/d33/d74/da2 0 2024-04-12T05:55:44.159 INFO:tasks.workunit.client.0.smithi008.stdout:9/993: dread d3/d22/d2b/d58/d36/f3c [0,4194304] 0 2024-04-12T05:55:44.159 INFO:tasks.workunit.client.0.smithi008.stdout:6/994: symlink df/d8c/db1/d145/l14e 0 2024-04-12T05:55:44.163 INFO:tasks.workunit.client.0.smithi008.stdout:0/938: dwrite d2/d11/d36/d47/f6d [0,4194304] 0 2024-04-12T05:55:44.165 INFO:tasks.workunit.client.0.smithi008.stdout:1/942: getdents d4/d1a 0 2024-04-12T05:55:44.168 INFO:tasks.workunit.client.0.smithi008.stdout:9/994: chown d3/d22/d2b/d58/d38/c123 1981694 1 2024-04-12T05:55:44.169 INFO:tasks.workunit.client.0.smithi008.stdout:1/943: creat d4/d33/d74/d7a/db6/f14a x:0 0 0 2024-04-12T05:55:44.169 INFO:tasks.workunit.client.0.smithi008.stdout:0/939: chown d2/d11/d36/d42/da6/f124 116922 1 2024-04-12T05:55:44.171 INFO:tasks.workunit.client.0.smithi008.stdout:1/944: readlink d4/d33/d74/d30/d72/d7c/dd4/lec 0 2024-04-12T05:55:44.175 INFO:tasks.workunit.client.0.smithi008.stdout:9/995: getdents d3/d22/d2b/d58/d38 0 2024-04-12T05:55:44.175 INFO:tasks.workunit.client.0.smithi008.stdout:1/945: write d4/d6/f59 [158965,89495] 0 2024-04-12T05:55:44.178 INFO:tasks.workunit.client.0.smithi008.stdout:9/996: mkdir d3/d22/d2b/d77/d82/d14d 0 2024-04-12T05:55:44.179 INFO:tasks.workunit.client.0.smithi008.stdout:0/940: write d2/d11/d36/d82/d95/f98 [210806,115493] 0 2024-04-12T05:55:44.180 INFO:tasks.workunit.client.0.smithi008.stdout:0/941: dread - d2/d11/d36/d42/da6/f124 zero size 2024-04-12T05:55:44.182 INFO:tasks.workunit.client.0.smithi008.stdout:1/946: rename d4/dd/d21/d9f/d100/c114 to d4/d33/d74/da2/c14b 0 2024-04-12T05:55:44.183 INFO:tasks.workunit.client.0.smithi008.stdout:9/997: dread d3/d22/d2b/d58/d5d/fcd [0,4194304] 0 2024-04-12T05:55:44.183 INFO:tasks.workunit.client.0.smithi008.stdout:0/942: chown d2/d11/d36/d47/cc4 164039931 1 2024-04-12T05:55:44.184 INFO:tasks.workunit.client.0.smithi008.stdout:0/943: fsync d2/d11/fc5 0 2024-04-12T05:55:44.185 INFO:tasks.workunit.client.0.smithi008.stdout:9/998: write d3/df/d10/f17 [600410,43459] 0 2024-04-12T05:55:44.186 INFO:tasks.workunit.client.0.smithi008.stdout:1/947: truncate d4/d33/d74/d46/f7e 883769 0 2024-04-12T05:55:44.188 INFO:tasks.workunit.client.0.smithi008.stdout:1/948: chown d4/d1a 1 1 2024-04-12T05:55:44.190 INFO:tasks.workunit.client.0.smithi008.stdout:6/995: dread fd [4194304,4194304] 0 2024-04-12T05:55:44.192 INFO:tasks.workunit.client.0.smithi008.stdout:9/999: symlink d3/df/d13/d21/d5c/l14e 0 2024-04-12T05:55:44.192 INFO:tasks.workunit.client.0.smithi008.stdout:0/944: getdents d2/d24/d85/dde/df7 0 2024-04-12T05:55:44.195 INFO:tasks.workunit.client.0.smithi008.stdout:6/996: symlink df/d8c/dcc/l14f 0 2024-04-12T05:55:44.196 INFO:tasks.workunit.client.0.smithi008.stdout:0/945: mknod d2/d11/d36/d42/d86/c131 0 2024-04-12T05:55:44.197 INFO:tasks.workunit.client.0.smithi008.stdout:1/949: creat d4/d1a/d57/f14c x:0 0 0 2024-04-12T05:55:44.199 INFO:tasks.workunit.client.0.smithi008.stdout:1/950: unlink d4/dd/c145 0 2024-04-12T05:55:44.201 INFO:tasks.workunit.client.0.smithi008.stdout:1/951: chown d4/dd/d17/cc3 703 1 2024-04-12T05:55:44.202 INFO:tasks.workunit.client.0.smithi008.stdout:0/946: dwrite d2/f72 [0,4194304] 0 2024-04-12T05:55:44.205 INFO:tasks.workunit.client.0.smithi008.stdout:1/952: mkdir d4/d33/d74/d7a/db6/dc9/d131/d14d 0 2024-04-12T05:55:44.209 INFO:tasks.workunit.client.0.smithi008.stdout:1/953: symlink d4/l14e 0 2024-04-12T05:55:44.213 INFO:tasks.workunit.client.0.smithi008.stdout:1/954: creat d4/d24/f14f x:0 0 0 2024-04-12T05:55:44.220 INFO:tasks.workunit.client.0.smithi008.stdout:0/947: dread d2/d11/d36/d82/d95/fba [0,4194304] 0 2024-04-12T05:55:44.228 INFO:tasks.workunit.client.0.smithi008.stdout:0/948: dwrite d2/d4b/f106 [0,4194304] 0 2024-04-12T05:55:44.228 INFO:tasks.workunit.client.0.smithi008.stdout:1/955: write d4/d33/d74/d46/f7e [764573,118464] 0 2024-04-12T05:55:44.239 INFO:tasks.workunit.client.0.smithi008.stdout:1/956: dwrite d4/d33/d74/da2/f108 [0,4194304] 0 2024-04-12T05:55:44.239 INFO:tasks.workunit.client.0.smithi008.stdout:0/949: dwrite d2/d24/dbd/de9/ffc [0,4194304] 0 2024-04-12T05:55:44.240 INFO:tasks.workunit.client.0.smithi008.stdout:1/957: chown d4/d24/l3a 3 1 2024-04-12T05:55:44.251 INFO:tasks.workunit.client.0.smithi008.stdout:0/950: dwrite d2/d11/d36/d42/da6/f124 [0,4194304] 0 2024-04-12T05:55:44.259 INFO:tasks.workunit.client.0.smithi008.stdout:0/951: write d2/d24/d61/f92 [4011283,94031] 0 2024-04-12T05:55:44.270 INFO:tasks.workunit.client.0.smithi008.stdout:1/958: dwrite d4/dd/d17/f63 [0,4194304] 0 2024-04-12T05:55:44.274 INFO:tasks.workunit.client.0.smithi008.stdout:0/952: truncate d2/d11/d36/d82/d95/fba 2061745 0 2024-04-12T05:55:44.277 INFO:tasks.workunit.client.0.smithi008.stdout:0/953: creat d2/d24/d61/f132 x:0 0 0 2024-04-12T05:55:44.287 INFO:tasks.workunit.client.0.smithi008.stdout:0/954: write d2/d11/f2e [887877,103081] 0 2024-04-12T05:55:44.290 INFO:tasks.workunit.client.0.smithi008.stdout:0/955: creat d2/d24/dbd/de9/f133 x:0 0 0 2024-04-12T05:55:44.292 INFO:tasks.workunit.client.0.smithi008.stdout:0/956: creat d2/d11/d36/dd6/dea/f134 x:0 0 0 2024-04-12T05:55:44.294 INFO:tasks.workunit.client.0.smithi008.stdout:0/957: mkdir d2/d4b/da2/d100/d135 0 2024-04-12T05:55:44.297 INFO:tasks.workunit.client.0.smithi008.stdout:1/959: dread d4/d7/ff [0,4194304] 0 2024-04-12T05:55:44.300 INFO:tasks.workunit.client.0.smithi008.stdout:1/960: mkdir d4/dd/dba/d150 0 2024-04-12T05:55:44.300 INFO:tasks.workunit.client.0.smithi008.stdout:0/958: dwrite d2/f72 [0,4194304] 0 2024-04-12T05:55:44.301 INFO:tasks.workunit.client.0.smithi008.stdout:1/961: dread - d4/dd/d17/d91/f105 zero size 2024-04-12T05:55:44.301 INFO:tasks.workunit.client.0.smithi008.stdout:0/959: chown d2/d4b/da2/fe6 161 1 2024-04-12T05:55:44.305 INFO:tasks.workunit.client.0.smithi008.stdout:0/960: mkdir d2/d24/d85/dde/df7/d136 0 2024-04-12T05:55:44.309 INFO:tasks.workunit.client.0.smithi008.stdout:0/961: unlink d2/d11/d36/d42/d86/f96 0 2024-04-12T05:55:44.311 INFO:tasks.workunit.client.0.smithi008.stdout:1/962: dwrite d4/dd/d17/d66/f144 [0,4194304] 0 2024-04-12T05:55:44.313 INFO:tasks.workunit.client.0.smithi008.stdout:0/962: read d2/d4b/da2/fe6 [531673,5555] 0 2024-04-12T05:55:44.317 INFO:tasks.workunit.client.0.smithi008.stdout:0/963: mknod d2/d4b/da2/d100/c137 0 2024-04-12T05:55:44.322 INFO:tasks.workunit.client.0.smithi008.stdout:0/964: dread d2/d11/d36/d82/d95/fba [0,4194304] 0 2024-04-12T05:55:44.324 INFO:tasks.workunit.client.0.smithi008.stdout:0/965: rmdir d2 39 2024-04-12T05:55:44.324 INFO:tasks.workunit.client.0.smithi008.stdout:1/963: dread d4/d6/f53 [0,4194304] 0 2024-04-12T05:55:44.327 INFO:tasks.workunit.client.0.smithi008.stdout:1/964: chown d4/dd/d17/l54 0 1 2024-04-12T05:55:44.329 INFO:tasks.workunit.client.0.smithi008.stdout:1/965: mknod d4/c151 0 2024-04-12T05:55:44.333 INFO:tasks.workunit.client.0.smithi008.stdout:1/966: getdents d4/dd/d17/d91 0 2024-04-12T05:55:44.334 INFO:tasks.workunit.client.0.smithi008.stdout:1/967: chown d4/d33/d74/d30/f5d 3663 1 2024-04-12T05:55:44.335 INFO:tasks.workunit.client.0.smithi008.stdout:1/968: write d4/dd/d17/f1f [3426133,69393] 0 2024-04-12T05:55:44.339 INFO:tasks.workunit.client.0.smithi008.stdout:1/969: chown d4/d24/d62/c122 11982759 1 2024-04-12T05:55:44.341 INFO:tasks.workunit.client.0.smithi008.stdout:1/970: creat d4/dd/d21/d98/f152 x:0 0 0 2024-04-12T05:55:44.342 INFO:tasks.workunit.client.0.smithi008.stdout:0/966: dread d2/d11/d2a/db5/fd1 [0,4194304] 0 2024-04-12T05:55:44.342 INFO:tasks.workunit.client.0.smithi008.stdout:0/967: stat d2/d24/d61/l11f 0 2024-04-12T05:55:44.343 INFO:tasks.workunit.client.0.smithi008.stdout:1/971: truncate d4/dd/d17/d26/fa9 4698617 0 2024-04-12T05:55:44.344 INFO:tasks.workunit.client.0.smithi008.stdout:0/968: fdatasync d2/d11/d36/d47/fb2 0 2024-04-12T05:55:44.345 INFO:tasks.workunit.client.0.smithi008.stdout:0/969: fdatasync d2/d24/d61/f132 0 2024-04-12T05:55:44.347 INFO:tasks.workunit.client.0.smithi008.stdout:1/972: rename d4/d33/d74/d7a/db6/dc9/d131/d14d to d4/dd/d21/d153 0 2024-04-12T05:55:44.348 INFO:tasks.workunit.client.0.smithi008.stdout:0/970: dread d2/d11/d36/d82/d95/fba [0,4194304] 0 2024-04-12T05:55:44.349 INFO:tasks.workunit.client.0.smithi008.stdout:0/971: fsync d2/d11/d36/d47/f87 0 2024-04-12T05:55:44.349 INFO:tasks.workunit.client.0.smithi008.stdout:1/973: truncate d4/f7f 915242 0 2024-04-12T05:55:44.351 INFO:tasks.workunit.client.0.smithi008.stdout:0/972: mknod d2/d11/d36/d42/d86/c138 0 2024-04-12T05:55:44.352 INFO:tasks.workunit.client.0.smithi008.stdout:1/974: creat d4/d24/d143/f154 x:0 0 0 2024-04-12T05:55:44.353 INFO:tasks.workunit.client.0.smithi008.stdout:1/975: write d4/dd/d17/d66/fe3 [1552752,73522] 0 2024-04-12T05:55:44.354 INFO:tasks.workunit.client.0.smithi008.stdout:0/973: unlink d2/d4b/f106 0 2024-04-12T05:55:44.354 INFO:tasks.workunit.client.0.smithi008.stdout:0/974: chown d2/d24/c8e 4474115 1 2024-04-12T05:55:44.356 INFO:tasks.workunit.client.0.smithi008.stdout:0/975: read d2/d11/d36/d47/fb2 [256455,80470] 0 2024-04-12T05:55:44.357 INFO:tasks.workunit.client.0.smithi008.stdout:1/976: read d4/d1a/f9c [696138,72141] 0 2024-04-12T05:55:44.359 INFO:tasks.workunit.client.0.smithi008.stdout:0/976: write d2/d11/fb0 [4892082,124587] 0 2024-04-12T05:55:44.361 INFO:tasks.workunit.client.0.smithi008.stdout:1/977: rename d4/d33/d74/c142 to d4/dd/c155 0 2024-04-12T05:55:44.363 INFO:tasks.workunit.client.0.smithi008.stdout:0/977: chown d2/d11/d36/d82/fd4 7 1 2024-04-12T05:55:44.367 INFO:tasks.workunit.client.0.smithi008.stdout:1/978: getdents d4/d33/d74/d30/d72 0 2024-04-12T05:55:44.368 INFO:tasks.workunit.client.0.smithi008.stdout:1/979: chown d4/l14e 129470673 1 2024-04-12T05:55:44.374 INFO:tasks.workunit.client.0.smithi008.stdout:1/980: dwrite d4/d33/d74/d7a/db6/dc9/d146/d4f/d10b/f121 [0,4194304] 0 2024-04-12T05:55:44.376 INFO:tasks.workunit.client.0.smithi008.stdout:1/981: mknod d4/d33/d74/d46/c156 0 2024-04-12T05:55:44.380 INFO:tasks.workunit.client.0.smithi008.stdout:1/982: creat d4/dd/dba/d150/f157 x:0 0 0 2024-04-12T05:55:44.382 INFO:tasks.workunit.client.0.smithi008.stdout:1/983: creat d4/d33/d74/d7a/f158 x:0 0 0 2024-04-12T05:55:44.394 INFO:tasks.workunit.client.0.smithi008.stdout:1/984: dread d4/d6/d90/fd3 [0,4194304] 0 2024-04-12T05:55:44.398 INFO:tasks.workunit.client.0.smithi008.stdout:1/985: write d4/d33/d74/d30/f49 [1820048,102858] 0 2024-04-12T05:55:44.398 INFO:tasks.workunit.client.0.smithi008.stdout:1/986: dread - d4/dd/d21/d98/f152 zero size 2024-04-12T05:55:44.401 INFO:tasks.workunit.client.0.smithi008.stdout:1/987: rename d4/dd/d17/f64 to d4/d6/d90/f159 0 2024-04-12T05:55:44.403 INFO:tasks.workunit.client.0.smithi008.stdout:1/988: chown d4/d33/d74/d7a/db6/dc9/d146/db9/dc1/c149 51775 1 2024-04-12T05:55:44.405 INFO:tasks.workunit.client.0.smithi008.stdout:1/989: rename d4/d33/d74/d7a/fb4 to d4/d33/d74/d46/f15a 0 2024-04-12T05:55:44.411 INFO:tasks.workunit.client.0.smithi008.stdout:1/990: dwrite d4/d24/faa [0,4194304] 0 2024-04-12T05:55:44.418 INFO:tasks.workunit.client.0.smithi008.stdout:1/991: fsync d4/d1a/d57/dd6/f81 0 2024-04-12T05:55:44.419 INFO:tasks.workunit.client.0.smithi008.stdout:1/992: truncate d4/d33/d74/d7a/f158 607291 0 2024-04-12T05:55:44.421 INFO:tasks.workunit.client.0.smithi008.stdout:1/993: creat d4/d33/d74/d30/d72/d7c/dd4/f15b x:0 0 0 2024-04-12T05:55:44.426 INFO:tasks.workunit.client.0.smithi008.stdout:1/994: rename d4/d33/d74/d30/d72/d7c/dd4/dd5/f104 to d4/dd/d21/d9f/f15c 0 2024-04-12T05:55:44.436 INFO:tasks.workunit.client.0.smithi008.stdout:1/995: dread d4/dd/d21/d9f/d100/fb0 [0,4194304] 0 2024-04-12T05:55:44.440 INFO:tasks.workunit.client.0.smithi008.stdout:1/996: getdents d4/d33/d74 0 2024-04-12T05:55:44.441 INFO:tasks.workunit.client.0.smithi008.stdout:1/997: truncate d4/d33/d74/d7a/db6/dc9/d146/d4f/d10b/f135 21236 0 2024-04-12T05:55:44.442 INFO:tasks.workunit.client.0.smithi008.stdout:1/998: stat d4/d33/d74/d46/f7e 0 2024-04-12T05:55:44.444 INFO:tasks.workunit.client.0.smithi008.stdout:1/999: mknod d4/dd/d21/d13a/c15d 0 2024-04-12T05:55:44.605 INFO:tasks.workunit.client.0.smithi008.stdout:4/825: sync 2024-04-12T05:55:44.607 INFO:tasks.workunit.client.0.smithi008.stdout:4/826: rename d1/d4f/c9a to d1/d22/d29/d46/d56/daf/dcf/ded/c105 0 2024-04-12T05:55:44.610 INFO:tasks.workunit.client.0.smithi008.stdout:4/827: creat d1/d8/dc5/f106 x:0 0 0 2024-04-12T05:55:44.611 INFO:tasks.workunit.client.0.smithi008.stdout:4/828: write d1/d22/d29/f7c [977673,40416] 0 2024-04-12T05:55:44.614 INFO:tasks.workunit.client.0.smithi008.stdout:4/829: unlink d1/d22/f4c 0 2024-04-12T05:55:44.615 INFO:tasks.workunit.client.0.smithi008.stdout:4/830: dread - d1/d22/d29/d30/d60/ffc zero size 2024-04-12T05:55:44.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/831: link d1/d22/d29/d30/d47/d28/d4a/d62/l104 d1/l107 0 2024-04-12T05:55:44.621 INFO:tasks.workunit.client.0.smithi008.stdout:4/832: creat d1/db/f108 x:0 0 0 2024-04-12T05:55:44.622 INFO:tasks.workunit.client.0.smithi008.stdout:4/833: write d1/d22/d29/d30/d47/d28/fa6 [683433,27416] 0 2024-04-12T05:55:44.627 INFO:tasks.workunit.client.0.smithi008.stdout:4/834: getdents d1/d22/d29/d46/d56 0 2024-04-12T05:55:44.629 INFO:tasks.workunit.client.0.smithi008.stdout:4/835: mkdir d1/d4f/dd5/d109 0 2024-04-12T05:55:44.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/836: rename d1/d22/d29/d30/d47/c75 to d1/d22/d29/d46/d56/daf/dcf/ded/c10a 0 2024-04-12T05:55:44.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/837: chown d1/db/c3d 0 1 2024-04-12T05:55:44.636 INFO:tasks.workunit.client.0.smithi008.stdout:4/838: write d1/d8/f25 [1057622,87154] 0 2024-04-12T05:55:44.638 INFO:tasks.workunit.client.0.smithi008.stdout:4/839: getdents d1/db/db7 0 2024-04-12T05:55:44.640 INFO:tasks.workunit.client.0.smithi008.stdout:4/840: truncate d1/fa0 985838 0 2024-04-12T05:55:44.650 INFO:tasks.workunit.client.0.smithi008.stdout:8/860: sync 2024-04-12T05:55:44.650 INFO:tasks.workunit.client.0.smithi008.stdout:3/895: sync 2024-04-12T05:55:44.650 INFO:tasks.workunit.client.0.smithi008.stdout:6/997: sync 2024-04-12T05:55:44.651 INFO:tasks.workunit.client.0.smithi008.stdout:0/978: sync 2024-04-12T05:55:44.653 INFO:tasks.workunit.client.0.smithi008.stdout:3/896: mkdir d9/d10/d4b/d12f 0 2024-04-12T05:55:44.656 INFO:tasks.workunit.client.0.smithi008.stdout:0/979: chown d2/d24/d39/c5b 2847808 1 2024-04-12T05:55:44.658 INFO:tasks.workunit.client.0.smithi008.stdout:6/998: mkdir df/d10/d15/dd0/d150 0 2024-04-12T05:55:44.660 INFO:tasks.workunit.client.0.smithi008.stdout:8/861: creat dd/de/d16/d50/d56/d61/f127 x:0 0 0 2024-04-12T05:55:44.661 INFO:tasks.workunit.client.0.smithi008.stdout:3/897: truncate d9/d10/d19/d22/fce 4541478 0 2024-04-12T05:55:44.663 INFO:tasks.workunit.client.0.smithi008.stdout:0/980: creat d2/d11/d36/d42/d86/d9b/f139 x:0 0 0 2024-04-12T05:55:44.667 INFO:tasks.workunit.client.0.smithi008.stdout:4/841: dwrite d1/d22/d29/d30/d47/d28/d74/f9c [0,4194304] 0 2024-04-12T05:55:44.669 INFO:tasks.workunit.client.0.smithi008.stdout:0/981: chown d2/d11/d2a/f9d 2384 1 2024-04-12T05:55:44.669 INFO:tasks.workunit.client.0.smithi008.stdout:8/862: stat dd/de/d16/db1/d97/fa3 0 2024-04-12T05:55:44.676 INFO:tasks.workunit.client.0.smithi008.stdout:3/898: rename d9/d10/d19/d22/da8/d114 to d9/d37/d130 0 2024-04-12T05:55:44.677 INFO:tasks.workunit.client.0.smithi008.stdout:3/899: readlink d9/d11/l2a 0 2024-04-12T05:55:44.678 INFO:tasks.workunit.client.0.smithi008.stdout:4/842: creat d1/d22/d29/d30/d60/f10b x:0 0 0 2024-04-12T05:55:44.679 INFO:tasks.workunit.client.0.smithi008.stdout:3/900: dread - d9/d10/d19/d3a/d67/f97 zero size 2024-04-12T05:55:44.681 INFO:tasks.workunit.client.0.smithi008.stdout:0/982: creat d2/d24/dbd/de9/f13a x:0 0 0 2024-04-12T05:55:44.681 INFO:tasks.workunit.client.0.smithi008.stdout:8/863: write dd/d30/d3b/f9b [666259,61265] 0 2024-04-12T05:55:44.682 INFO:tasks.workunit.client.0.smithi008.stdout:6/999: dread df/d8c/db1/f12b [0,4194304] 0 2024-04-12T05:55:44.683 INFO:tasks.workunit.client.0.smithi008.stdout:4/843: mknod d1/d22/d29/d46/d56/c10c 0 2024-04-12T05:55:44.685 INFO:tasks.workunit.client.0.smithi008.stdout:3/901: symlink d9/d10/d19/d22/da8/dc4/l131 0 2024-04-12T05:55:44.687 INFO:tasks.workunit.client.0.smithi008.stdout:3/902: chown d9/d10/d19/d22/dec/c10a 8 1 2024-04-12T05:55:44.687 INFO:tasks.workunit.client.0.smithi008.stdout:4/844: unlink d1/d4f/fc7 0 2024-04-12T05:55:44.689 INFO:tasks.workunit.client.0.smithi008.stdout:8/864: mknod dd/de/d16/d50/d5e/dbe/dd6/d124/c128 0 2024-04-12T05:55:44.689 INFO:tasks.workunit.client.0.smithi008.stdout:4/845: read d1/d8/f55 [2308940,535] 0 2024-04-12T05:55:44.690 INFO:tasks.workunit.client.0.smithi008.stdout:3/903: fdatasync d9/d1e/f35 0 2024-04-12T05:55:44.692 INFO:tasks.workunit.client.0.smithi008.stdout:0/983: rename d2/d11/d2a/c4c to d2/d24/d85/dde/c13b 0 2024-04-12T05:55:44.696 INFO:tasks.workunit.client.0.smithi008.stdout:0/984: write d2/d11/d36/d42/fbb [870679,107557] 0 2024-04-12T05:55:44.697 INFO:tasks.workunit.client.0.smithi008.stdout:4/846: dread d1/d22/d29/d30/f91 [0,4194304] 0 2024-04-12T05:55:44.701 INFO:tasks.workunit.client.0.smithi008.stdout:0/985: write d2/d24/d85/ddc/f10f [20236,129292] 0 2024-04-12T05:55:44.702 INFO:tasks.workunit.client.0.smithi008.stdout:8/865: dwrite dd/d4f/d52/f79 [0,4194304] 0 2024-04-12T05:55:44.708 INFO:tasks.workunit.client.0.smithi008.stdout:8/866: getdents dd/de/d16/d50/df1/d11f 0 2024-04-12T05:55:44.708 INFO:tasks.workunit.client.0.smithi008.stdout:4/847: write d1/d8/f15 [2609018,121739] 0 2024-04-12T05:55:44.717 INFO:tasks.workunit.client.0.smithi008.stdout:3/904: dwrite d9/d10/d19/d22/f1f [0,4194304] 0 2024-04-12T05:55:44.722 INFO:tasks.workunit.client.0.smithi008.stdout:0/986: dread d2/d11/d36/d47/f8b [0,4194304] 0 2024-04-12T05:55:44.722 INFO:tasks.workunit.client.0.smithi008.stdout:3/905: creat d9/f132 x:0 0 0 2024-04-12T05:55:44.723 INFO:tasks.workunit.client.0.smithi008.stdout:4/848: write d1/db/f18 [463022,97736] 0 2024-04-12T05:55:44.724 INFO:tasks.workunit.client.0.smithi008.stdout:3/906: chown d9/d2e/d3d/f70 2447 1 2024-04-12T05:55:44.727 INFO:tasks.workunit.client.0.smithi008.stdout:4/849: chown d1/d22/d29/d46/d56/c7b 813878 1 2024-04-12T05:55:44.730 INFO:tasks.workunit.client.0.smithi008.stdout:3/907: symlink d9/d44/d107/l133 0 2024-04-12T05:55:44.730 INFO:tasks.workunit.client.0.smithi008.stdout:4/850: dread - d1/d22/d29/d46/fb6 zero size 2024-04-12T05:55:44.731 INFO:tasks.workunit.client.0.smithi008.stdout:8/867: dwrite dd/d4f/f96 [0,4194304] 0 2024-04-12T05:55:44.732 INFO:tasks.workunit.client.0.smithi008.stdout:0/987: rename d2/f5 to d2/d11/d36/d42/d86/d9b/f13c 0 2024-04-12T05:55:44.735 INFO:tasks.workunit.client.0.smithi008.stdout:0/988: creat d2/d24/f13d x:0 0 0 2024-04-12T05:55:44.738 INFO:tasks.workunit.client.0.smithi008.stdout:0/989: stat d2/d11/d36/d42/f62 0 2024-04-12T05:55:44.740 INFO:tasks.workunit.client.0.smithi008.stdout:4/851: dwrite d1/d8/f9b [0,4194304] 0 2024-04-12T05:55:44.809 INFO:tasks.workunit.client.0.smithi008.stdout:4/852: rename d1/db/f8f to d1/d22/d29/d30/f10d 0 2024-04-12T05:55:44.811 INFO:tasks.workunit.client.0.smithi008.stdout:4/853: chown d1/d22/d29/d30/d47/d28/f79 655767 1 2024-04-12T05:55:44.812 INFO:tasks.workunit.client.0.smithi008.stdout:4/854: dread - d1/d8/f5f zero size 2024-04-12T05:55:44.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:44 smithi194 ceph-mon[49771]: pgmap v18: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 138 MiB/s wr, 508 op/s 2024-04-12T05:55:44.820 INFO:tasks.workunit.client.0.smithi008.stdout:0/990: dwrite d2/d11/d36/d42/d86/d9b/f13c [4194304,4194304] 0 2024-04-12T05:55:44.825 INFO:tasks.workunit.client.0.smithi008.stdout:4/855: dwrite d1/d4f/f89 [0,4194304] 0 2024-04-12T05:55:44.825 INFO:tasks.workunit.client.0.smithi008.stdout:0/991: fdatasync d2/d11/d36/d42/da6/fa7 0 2024-04-12T05:55:44.827 INFO:tasks.workunit.client.0.smithi008.stdout:4/856: write d1/d22/d29/d30/d60/f10b [966897,117020] 0 2024-04-12T05:55:44.830 INFO:tasks.workunit.client.0.smithi008.stdout:4/857: fdatasync d1/d22/fda 0 2024-04-12T05:55:44.831 INFO:tasks.workunit.client.0.smithi008.stdout:0/992: chown d2/cf6 16168 1 2024-04-12T05:55:44.832 INFO:tasks.workunit.client.0.smithi008.stdout:4/858: mknod d1/d22/d29/d30/c10e 0 2024-04-12T05:55:44.835 INFO:tasks.workunit.client.0.smithi008.stdout:0/993: link d2/d11/d36/d42/d86/d9b/f13c d2/d4b/da2/f13e 0 2024-04-12T05:55:44.839 INFO:tasks.workunit.client.0.smithi008.stdout:4/859: getdents d1 0 2024-04-12T05:55:44.849 INFO:tasks.workunit.client.0.smithi008.stdout:4/860: truncate d1/d8/f9b 1858170 0 2024-04-12T05:55:44.850 INFO:tasks.workunit.client.0.smithi008.stdout:4/861: read - d1/d22/d29/d46/fb6 zero size 2024-04-12T05:55:44.853 INFO:tasks.workunit.client.0.smithi008.stdout:0/994: dwrite d2/d11/fb9 [4194304,4194304] 0 2024-04-12T05:55:44.858 INFO:tasks.workunit.client.0.smithi008.stdout:0/995: read d2/d24/f79 [5320100,39545] 0 2024-04-12T05:55:44.862 INFO:tasks.workunit.client.0.smithi008.stdout:0/996: creat d2/d11/d36/d42/da6/f13f x:0 0 0 2024-04-12T05:55:44.862 INFO:tasks.workunit.client.0.smithi008.stdout:4/862: dwrite d1/d22/d29/d30/d47/d28/d74/fa9 [0,4194304] 0 2024-04-12T05:55:44.864 INFO:tasks.workunit.client.0.smithi008.stdout:0/997: chown d2/d4b/da2/fd0 243 1 2024-04-12T05:55:44.867 INFO:tasks.workunit.client.0.smithi008.stdout:4/863: unlink d1/d22/d29/d30/d47/lbc 0 2024-04-12T05:55:44.868 INFO:tasks.workunit.client.0.smithi008.stdout:4/864: dread - d1/d22/d29/d46/d56/fc4 zero size 2024-04-12T05:55:44.870 INFO:tasks.workunit.client.0.smithi008.stdout:0/998: mkdir d2/ddb/d11b/d140 0 2024-04-12T05:55:44.876 INFO:tasks.workunit.client.0.smithi008.stdout:0/999: mkdir d2/d11/d2a/db5/df8/d109/d141 0 2024-04-12T05:55:44.878 INFO:tasks.workunit.client.0.smithi008.stdout:4/865: dwrite d1/db/f18 [0,4194304] 0 2024-04-12T05:55:44.886 INFO:tasks.workunit.client.0.smithi008.stdout:4/866: dwrite d1/db/f108 [0,4194304] 0 2024-04-12T05:55:44.887 INFO:tasks.workunit.client.0.smithi008.stdout:4/867: chown d1/d4f/f89 116112780 1 2024-04-12T05:55:44.898 INFO:tasks.workunit.client.0.smithi008.stdout:4/868: write d1/f20 [4705601,4145] 0 2024-04-12T05:55:44.901 INFO:tasks.workunit.client.0.smithi008.stdout:4/869: symlink d1/d22/d29/d30/d60/l10f 0 2024-04-12T05:55:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:44 smithi008 ceph-mon[39722]: pgmap v18: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 59 MiB/s rd, 138 MiB/s wr, 508 op/s 2024-04-12T05:55:44.903 INFO:tasks.workunit.client.0.smithi008.stdout:4/870: creat d1/d22/f110 x:0 0 0 2024-04-12T05:55:44.904 INFO:tasks.workunit.client.0.smithi008.stdout:4/871: write d1/db/db7/fe8 [678431,18448] 0 2024-04-12T05:55:44.907 INFO:tasks.workunit.client.0.smithi008.stdout:4/872: symlink d1/d8/dc5/deb/df8/l111 0 2024-04-12T05:55:44.908 INFO:tasks.workunit.client.0.smithi008.stdout:4/873: write d1/d22/d29/d46/f76 [686351,108025] 0 2024-04-12T05:55:44.910 INFO:tasks.workunit.client.0.smithi008.stdout:4/874: dread - d1/d22/d29/d30/d47/d28/fbf zero size 2024-04-12T05:55:44.911 INFO:tasks.workunit.client.0.smithi008.stdout:4/875: stat d1/d22/d29/d30/d60/l10f 0 2024-04-12T05:55:44.913 INFO:tasks.workunit.client.0.smithi008.stdout:4/876: creat d1/d22/d29/d30/d47/d28/d4a/d62/f112 x:0 0 0 2024-04-12T05:55:44.922 INFO:tasks.workunit.client.0.smithi008.stdout:4/877: dwrite d1/d4f/fa5 [0,4194304] 0 2024-04-12T05:55:44.925 INFO:tasks.workunit.client.0.smithi008.stdout:4/878: creat d1/d22/d29/d30/d60/dd6/f113 x:0 0 0 2024-04-12T05:55:44.929 INFO:tasks.workunit.client.0.smithi008.stdout:4/879: creat d1/d8/dc5/deb/df8/f114 x:0 0 0 2024-04-12T05:55:44.930 INFO:tasks.workunit.client.0.smithi008.stdout:4/880: chown d1/d22/d29/d30/d47/f48 4 1 2024-04-12T05:55:44.931 INFO:tasks.workunit.client.0.smithi008.stdout:4/881: chown d1/d8/dc5/deb/df8 34200546 1 2024-04-12T05:55:44.935 INFO:tasks.workunit.client.0.smithi008.stdout:4/882: dread d1/d8/f37 [0,4194304] 0 2024-04-12T05:55:44.945 INFO:tasks.workunit.client.0.smithi008.stdout:4/883: dwrite d1/db/f18 [4194304,4194304] 0 2024-04-12T05:55:44.951 INFO:tasks.workunit.client.0.smithi008.stdout:4/884: getdents d1/d22/d29/d30/d60/dd6 0 2024-04-12T05:55:44.953 INFO:tasks.workunit.client.0.smithi008.stdout:4/885: mknod d1/d4f/dd5/d109/c115 0 2024-04-12T05:55:44.956 INFO:tasks.workunit.client.0.smithi008.stdout:4/886: symlink d1/d22/d29/d46/d56/daf/dcf/ded/df5/l116 0 2024-04-12T05:55:44.957 INFO:tasks.workunit.client.0.smithi008.stdout:4/887: chown d1/d4f/dd5/d109 960 1 2024-04-12T05:55:44.958 INFO:tasks.workunit.client.0.smithi008.stdout:4/888: write d1/d8/f25 [4468986,780] 0 2024-04-12T05:55:44.959 INFO:tasks.workunit.client.0.smithi008.stdout:4/889: truncate d1/d22/d29/d46/fb6 242222 0 2024-04-12T05:55:44.960 INFO:tasks.workunit.client.0.smithi008.stdout:4/890: read d1/d4f/f5a [1793570,25222] 0 2024-04-12T05:55:44.964 INFO:tasks.workunit.client.0.smithi008.stdout:4/891: creat d1/d8/f117 x:0 0 0 2024-04-12T05:55:44.968 INFO:tasks.workunit.client.0.smithi008.stdout:4/892: write d1/d4f/f9f [923840,94082] 0 2024-04-12T05:55:44.976 INFO:tasks.workunit.client.0.smithi008.stdout:4/893: dwrite d1/d22/d29/d30/f90 [4194304,4194304] 0 2024-04-12T05:55:44.985 INFO:tasks.workunit.client.0.smithi008.stdout:4/894: dread d1/d22/d29/d30/f88 [0,4194304] 0 2024-04-12T05:55:44.987 INFO:tasks.workunit.client.0.smithi008.stdout:4/895: creat d1/db/db7/f118 x:0 0 0 2024-04-12T05:55:44.992 INFO:tasks.workunit.client.0.smithi008.stdout:4/896: dread d1/d4f/f7f [0,4194304] 0 2024-04-12T05:55:44.992 INFO:tasks.workunit.client.0.smithi008.stdout:3/908: sync 2024-04-12T05:55:44.992 INFO:tasks.workunit.client.0.smithi008.stdout:8/868: sync 2024-04-12T05:55:44.993 INFO:tasks.workunit.client.0.smithi008.stdout:4/897: chown d1/d22/d29/d30/d47/d28/f79 8 1 2024-04-12T05:55:44.994 INFO:tasks.workunit.client.0.smithi008.stdout:8/869: mkdir dd/d4f/df5/d129 0 2024-04-12T05:55:44.995 INFO:tasks.workunit.client.0.smithi008.stdout:4/898: chown d1/d22/d29/d30/d60/l10f 10 1 2024-04-12T05:55:44.996 INFO:tasks.workunit.client.0.smithi008.stdout:8/870: chown dd/d30/d3b/db5/df0/fe6 825997820 1 2024-04-12T05:55:44.997 INFO:tasks.workunit.client.0.smithi008.stdout:3/909: getdents d9/d11 0 2024-04-12T05:55:44.998 INFO:tasks.workunit.client.0.smithi008.stdout:4/899: rmdir d1/d4f/dd5 39 2024-04-12T05:55:44.999 INFO:tasks.workunit.client.0.smithi008.stdout:4/900: stat d1/db/db7/cce 0 2024-04-12T05:55:45.000 INFO:tasks.workunit.client.0.smithi008.stdout:8/871: mkdir dd/de/d16/db1/d12a 0 2024-04-12T05:55:45.001 INFO:tasks.workunit.client.0.smithi008.stdout:3/910: read d9/d1e/f5a [489324,20432] 0 2024-04-12T05:55:45.006 INFO:tasks.workunit.client.0.smithi008.stdout:8/872: write dd/d4f/f7b [1903672,59668] 0 2024-04-12T05:55:45.007 INFO:tasks.workunit.client.0.smithi008.stdout:4/901: dwrite d1/d4f/f9f [0,4194304] 0 2024-04-12T05:55:45.009 INFO:tasks.workunit.client.0.smithi008.stdout:8/873: chown dd/de/d16/d50/d5e/lbf 38 1 2024-04-12T05:55:45.009 INFO:tasks.workunit.client.0.smithi008.stdout:4/902: fdatasync d1/d22/d29/d30/d60/ffc 0 2024-04-12T05:55:45.013 INFO:tasks.workunit.client.0.smithi008.stdout:4/903: creat d1/d22/d29/d30/d60/dd6/f119 x:0 0 0 2024-04-12T05:55:45.015 INFO:tasks.workunit.client.0.smithi008.stdout:4/904: dread - d1/d22/d29/d30/d60/ffc zero size 2024-04-12T05:55:45.015 INFO:tasks.workunit.client.0.smithi008.stdout:8/874: creat dd/de/d16/f12b x:0 0 0 2024-04-12T05:55:45.017 INFO:tasks.workunit.client.0.smithi008.stdout:4/905: creat d1/d22/d29/d30/d60/dd6/f11a x:0 0 0 2024-04-12T05:55:45.018 INFO:tasks.workunit.client.0.smithi008.stdout:8/875: creat dd/de/f12c x:0 0 0 2024-04-12T05:55:45.020 INFO:tasks.workunit.client.0.smithi008.stdout:8/876: chown dd/de/d4c/d66/d6d/fde 284427 1 2024-04-12T05:55:45.023 INFO:tasks.workunit.client.0.smithi008.stdout:8/877: read - dd/de/d16/d50/d56/d61/d9c/ffc zero size 2024-04-12T05:55:45.034 INFO:tasks.workunit.client.0.smithi008.stdout:8/878: dwrite dd/d30/d32/fbd [0,4194304] 0 2024-04-12T05:55:45.042 INFO:tasks.workunit.client.0.smithi008.stdout:8/879: dwrite dd/de/d16/d10e/fb8 [4194304,4194304] 0 2024-04-12T05:55:45.047 INFO:tasks.workunit.client.0.smithi008.stdout:8/880: rmdir dd/de/d16/db1/d12a 0 2024-04-12T05:55:45.049 INFO:tasks.workunit.client.0.smithi008.stdout:8/881: read dd/f4e [2698675,86240] 0 2024-04-12T05:55:45.052 INFO:tasks.workunit.client.0.smithi008.stdout:8/882: mknod dd/dab/d126/c12d 0 2024-04-12T05:55:45.053 INFO:tasks.workunit.client.0.smithi008.stdout:8/883: chown dd/d30/d3b/db5/df0/fc3 3 1 2024-04-12T05:55:45.056 INFO:tasks.workunit.client.0.smithi008.stdout:8/884: link dd/de/d16/le7 dd/d65/l12e 0 2024-04-12T05:55:45.059 INFO:tasks.workunit.client.0.smithi008.stdout:8/885: chown dd/de/d16/db1/d97/dcf/f125 43366 1 2024-04-12T05:55:45.059 INFO:tasks.workunit.client.0.smithi008.stdout:8/886: fdatasync dd/d4f/f96 0 2024-04-12T05:55:45.065 INFO:tasks.workunit.client.0.smithi008.stdout:8/887: write dd/de/fda [176522,102827] 0 2024-04-12T05:55:45.071 INFO:tasks.workunit.client.0.smithi008.stdout:8/888: truncate dd/d30/d32/fb7 1819872 0 2024-04-12T05:55:45.075 INFO:tasks.workunit.client.0.smithi008.stdout:8/889: write dd/d4f/d52/f60 [436442,109517] 0 2024-04-12T05:55:45.077 INFO:tasks.workunit.client.0.smithi008.stdout:8/890: creat dd/de/d16/dc8/f12f x:0 0 0 2024-04-12T05:55:45.080 INFO:tasks.workunit.client.0.smithi008.stdout:8/891: creat dd/de/d16/d50/d5e/dbe/dd6/f130 x:0 0 0 2024-04-12T05:55:45.082 INFO:tasks.workunit.client.0.smithi008.stdout:8/892: creat dd/de/d16/d50/df1/f131 x:0 0 0 2024-04-12T05:55:45.083 INFO:tasks.workunit.client.0.smithi008.stdout:8/893: fsync dd/de/d4c/d66/d6d/fde 0 2024-04-12T05:55:45.084 INFO:tasks.workunit.client.0.smithi008.stdout:8/894: readlink dd/de/d16/d50/d5e/dbe/dd6/l101 0 2024-04-12T05:55:45.302 INFO:tasks.workunit.client.0.smithi008.stdout:4/906: sync 2024-04-12T05:55:45.302 INFO:tasks.workunit.client.0.smithi008.stdout:8/895: sync 2024-04-12T05:55:45.302 INFO:tasks.workunit.client.0.smithi008.stdout:3/911: sync 2024-04-12T05:55:45.304 INFO:tasks.workunit.client.0.smithi008.stdout:3/912: creat d9/d37/dd8/f134 x:0 0 0 2024-04-12T05:55:45.306 INFO:tasks.workunit.client.0.smithi008.stdout:8/896: mkdir dd/de/d16/d50/d5e/dbe/d132 0 2024-04-12T05:55:45.306 INFO:tasks.workunit.client.0.smithi008.stdout:3/913: dread - d9/d10/d19/d3a/d67/ffb zero size 2024-04-12T05:55:45.310 INFO:tasks.workunit.client.0.smithi008.stdout:8/897: symlink dd/d4f/df5/d129/l133 0 2024-04-12T05:55:45.310 INFO:tasks.workunit.client.0.smithi008.stdout:3/914: write d9/d10/d19/d3a/dad/feb [5028047,63091] 0 2024-04-12T05:55:45.312 INFO:tasks.workunit.client.0.smithi008.stdout:4/907: dwrite d1/d22/d29/d30/d47/d28/d4a/d62/f112 [0,4194304] 0 2024-04-12T05:55:45.313 INFO:tasks.workunit.client.0.smithi008.stdout:8/898: stat dd/d30/d32/fbd 0 2024-04-12T05:55:45.315 INFO:tasks.workunit.client.0.smithi008.stdout:3/915: symlink d9/db7/l135 0 2024-04-12T05:55:45.316 INFO:tasks.workunit.client.0.smithi008.stdout:4/908: write d1/db/f100 [680136,30639] 0 2024-04-12T05:55:45.316 INFO:tasks.workunit.client.0.smithi008.stdout:3/916: chown d9/d10/d19/d3a/fe6 1 1 2024-04-12T05:55:45.317 INFO:tasks.workunit.client.0.smithi008.stdout:4/909: chown d1/d8/dc5 38 1 2024-04-12T05:55:45.317 INFO:tasks.workunit.client.0.smithi008.stdout:3/917: chown d9/d11/lc5 27673 1 2024-04-12T05:55:45.317 INFO:tasks.workunit.client.0.smithi008.stdout:8/899: rmdir dd/de/d16/d50/df1 39 2024-04-12T05:55:45.318 INFO:tasks.workunit.client.0.smithi008.stdout:3/918: readlink d9/d37/lb9 0 2024-04-12T05:55:45.320 INFO:tasks.workunit.client.0.smithi008.stdout:4/910: chown d1/d22/d29/d30/cc2 77261 1 2024-04-12T05:55:45.322 INFO:tasks.workunit.client.0.smithi008.stdout:8/900: truncate dd/de/d16/db1/d97/dcf/f125 38532 0 2024-04-12T05:55:45.323 INFO:tasks.workunit.client.0.smithi008.stdout:4/911: dread d1/d22/d29/fd2 [0,4194304] 0 2024-04-12T05:55:45.323 INFO:tasks.workunit.client.0.smithi008.stdout:8/901: chown dd/de/d16/db1/d97/dcf/ldc 217 1 2024-04-12T05:55:45.326 INFO:tasks.workunit.client.0.smithi008.stdout:3/919: rename d9/d10/d19/d3a/dad/dcf to d9/d2e/d136 0 2024-04-12T05:55:45.327 INFO:tasks.workunit.client.0.smithi008.stdout:3/920: chown d9/d10/d19/d65/f8e 30462 1 2024-04-12T05:55:45.327 INFO:tasks.workunit.client.0.smithi008.stdout:3/921: chown d9/dd/ff6 782923 1 2024-04-12T05:55:45.328 INFO:tasks.workunit.client.0.smithi008.stdout:8/902: truncate dd/de/f29 1672843 0 2024-04-12T05:55:45.330 INFO:tasks.workunit.client.0.smithi008.stdout:8/903: mknod dd/de/d16/d50/d56/c134 0 2024-04-12T05:55:45.330 INFO:tasks.workunit.client.0.smithi008.stdout:8/904: chown dd/d4f/df5/d113 740 1 2024-04-12T05:55:45.332 INFO:tasks.workunit.client.0.smithi008.stdout:3/922: getdents d9/d10/d4b 0 2024-04-12T05:55:45.339 INFO:tasks.workunit.client.0.smithi008.stdout:3/923: link d9/d1e/c6c d9/c137 0 2024-04-12T05:55:45.340 INFO:tasks.workunit.client.0.smithi008.stdout:8/905: dwrite dd/de/f103 [0,4194304] 0 2024-04-12T05:55:45.342 INFO:tasks.workunit.client.0.smithi008.stdout:8/906: dread - dd/de/d16/d50/d56/f90 zero size 2024-04-12T05:55:45.348 INFO:tasks.workunit.client.0.smithi008.stdout:8/907: dread fb [4194304,4194304] 0 2024-04-12T05:55:45.350 INFO:tasks.workunit.client.0.smithi008.stdout:8/908: creat dd/de/d16/d10d/f135 x:0 0 0 2024-04-12T05:55:45.351 INFO:tasks.workunit.client.0.smithi008.stdout:3/924: dread d9/d2e/d3d/f70 [0,4194304] 0 2024-04-12T05:55:45.352 INFO:tasks.workunit.client.0.smithi008.stdout:8/909: chown dd/de/d4c/d66/d6d/c10a 2 1 2024-04-12T05:55:45.355 INFO:tasks.workunit.client.0.smithi008.stdout:8/910: creat dd/dab/f136 x:0 0 0 2024-04-12T05:55:45.356 INFO:tasks.workunit.client.0.smithi008.stdout:8/911: read dd/de/d16/db1/f104 [105993,4539] 0 2024-04-12T05:55:45.358 INFO:tasks.workunit.client.0.smithi008.stdout:8/912: creat dd/d4f/d52/da5/f137 x:0 0 0 2024-04-12T05:55:45.360 INFO:tasks.workunit.client.0.smithi008.stdout:8/913: mknod dd/d4f/df5/d129/c138 0 2024-04-12T05:55:45.361 INFO:tasks.workunit.client.0.smithi008.stdout:3/925: write d9/d10/d19/d65/f8e [1222766,73460] 0 2024-04-12T05:55:45.362 INFO:tasks.workunit.client.0.smithi008.stdout:3/926: fsync d9/d2e/d136/d116/f11b 0 2024-04-12T05:55:45.362 INFO:tasks.workunit.client.0.smithi008.stdout:8/914: truncate dd/d30/f78 173654 0 2024-04-12T05:55:45.364 INFO:tasks.workunit.client.0.smithi008.stdout:8/915: write dd/de/d16/d50/d5e/dbe/dd6/f130 [218253,109939] 0 2024-04-12T05:55:45.365 INFO:tasks.workunit.client.0.smithi008.stdout:3/927: unlink d9/d2e/d136/de3/l109 0 2024-04-12T05:55:45.366 INFO:tasks.workunit.client.0.smithi008.stdout:8/916: fdatasync dd/de/f2f 0 2024-04-12T05:55:45.368 INFO:tasks.workunit.client.0.smithi008.stdout:3/928: creat d9/d37/dd8/f138 x:0 0 0 2024-04-12T05:55:45.369 INFO:tasks.workunit.client.0.smithi008.stdout:8/917: creat dd/de/d16/dc8/f139 x:0 0 0 2024-04-12T05:55:45.371 INFO:tasks.workunit.client.0.smithi008.stdout:3/929: dread d9/d2e/f94 [0,4194304] 0 2024-04-12T05:55:45.375 INFO:tasks.workunit.client.0.smithi008.stdout:3/930: creat d9/dd/f139 x:0 0 0 2024-04-12T05:55:45.380 INFO:tasks.workunit.client.0.smithi008.stdout:8/918: dread dd/d4f/f5f [0,4194304] 0 2024-04-12T05:55:45.385 INFO:tasks.workunit.client.0.smithi008.stdout:8/919: link dd/de/d16/d50/d5e/dbe/f11b dd/de/d16/d50/f13a 0 2024-04-12T05:55:45.388 INFO:tasks.workunit.client.0.smithi008.stdout:3/931: dwrite d9/d11/d5b/f8d [0,4194304] 0 2024-04-12T05:55:45.391 INFO:tasks.workunit.client.0.smithi008.stdout:3/932: creat d9/d10/d19/d65/f13a x:0 0 0 2024-04-12T05:55:45.393 INFO:tasks.workunit.client.0.smithi008.stdout:8/920: rename dd/de/d16/d50/d56/d61/fbb to dd/de/d16/d50/d5e/dbe/d132/f13b 0 2024-04-12T05:55:45.395 INFO:tasks.workunit.client.0.smithi008.stdout:8/921: chown dd/d65/l12e 1883393 1 2024-04-12T05:55:45.395 INFO:tasks.workunit.client.0.smithi008.stdout:8/922: readlink dd/d4f/l59 0 2024-04-12T05:55:45.396 INFO:tasks.workunit.client.0.smithi008.stdout:8/923: chown dd/de/d16/d10e 212 1 2024-04-12T05:55:45.402 INFO:tasks.workunit.client.0.smithi008.stdout:8/924: dwrite dd/d4f/d52/da5/f137 [0,4194304] 0 2024-04-12T05:55:45.403 INFO:tasks.workunit.client.0.smithi008.stdout:8/925: stat dd/d4f/d52/f69 0 2024-04-12T05:55:45.413 INFO:tasks.workunit.client.0.smithi008.stdout:8/926: write fb [2992870,125995] 0 2024-04-12T05:55:45.417 INFO:tasks.workunit.client.0.smithi008.stdout:8/927: unlink dd/de/f2a 0 2024-04-12T05:55:45.426 INFO:tasks.workunit.client.0.smithi008.stdout:8/928: dwrite dd/f4e [4194304,4194304] 0 2024-04-12T05:55:45.431 INFO:tasks.workunit.client.0.smithi008.stdout:8/929: creat dd/de/d16/d50/d5e/dbe/dd6/d108/f13c x:0 0 0 2024-04-12T05:55:45.431 INFO:tasks.workunit.client.0.smithi008.stdout:8/930: stat dd/d30/l37 0 2024-04-12T05:55:45.436 INFO:tasks.workunit.client.0.smithi008.stdout:8/931: rename dd/de/d16/d50/d56/d61/d9c to dd/d4f/df5/d113/d13d 0 2024-04-12T05:55:45.443 INFO:tasks.workunit.client.0.smithi008.stdout:8/932: dwrite dd/f49 [0,4194304] 0 2024-04-12T05:55:45.444 INFO:tasks.workunit.client.0.smithi008.stdout:8/933: readlink dd/d4f/d52/lcd 0 2024-04-12T05:55:45.447 INFO:tasks.workunit.client.0.smithi008.stdout:8/934: mkdir dd/de/d16/d50/d13e 0 2024-04-12T05:55:45.454 INFO:tasks.workunit.client.0.smithi008.stdout:8/935: dwrite dd/de/f27 [0,4194304] 0 2024-04-12T05:55:45.480 INFO:tasks.workunit.client.0.smithi008.stdout:8/936: dread dd/d30/d32/f3a [0,4194304] 0 2024-04-12T05:55:45.486 INFO:tasks.workunit.client.0.smithi008.stdout:8/937: write dd/de/d16/d50/d5e/f85 [987604,51648] 0 2024-04-12T05:55:45.524 INFO:tasks.workunit.client.0.smithi008.stdout:8/938: creat dd/de/f13f x:0 0 0 2024-04-12T05:55:45.525 INFO:tasks.workunit.client.0.smithi008.stdout:8/939: chown dd/de/d16/d10e/fb8 0 1 2024-04-12T05:55:45.551 INFO:tasks.workunit.client.0.smithi008.stdout:8/940: dwrite dd/de/d16/fba [0,4194304] 0 2024-04-12T05:55:45.552 INFO:tasks.workunit.client.0.smithi008.stdout:8/941: dread - dd/de/d4c/d66/d6d/dca/f10b zero size 2024-04-12T05:55:45.554 INFO:tasks.workunit.client.0.smithi008.stdout:8/942: creat dd/de/d16/d50/d5e/dbe/dd6/d108/f140 x:0 0 0 2024-04-12T05:55:45.557 INFO:tasks.workunit.client.0.smithi008.stdout:8/943: creat dd/de/d16/d50/df1/d11f/f141 x:0 0 0 2024-04-12T05:55:45.560 INFO:tasks.workunit.client.0.smithi008.stdout:8/944: creat dd/de/d16/d50/d13e/f142 x:0 0 0 2024-04-12T05:55:45.566 INFO:tasks.workunit.client.0.smithi008.stdout:8/945: dwrite dd/de/d16/d50/d5e/dbe/dd6/f130 [0,4194304] 0 2024-04-12T05:55:45.569 INFO:tasks.workunit.client.0.smithi008.stdout:8/946: chown dd/f99 12295793 1 2024-04-12T05:55:45.571 INFO:tasks.workunit.client.0.smithi008.stdout:8/947: mknod dd/de/d16/db1/c143 0 2024-04-12T05:55:45.574 INFO:tasks.workunit.client.0.smithi008.stdout:8/948: mkdir dd/d4f/df5/d129/d144 0 2024-04-12T05:55:45.575 INFO:tasks.workunit.client.0.smithi008.stdout:8/949: chown dd/de/d16/d50/d5e/lc9 66856048 1 2024-04-12T05:55:45.577 INFO:tasks.workunit.client.0.smithi008.stdout:8/950: unlink dd/de/d16/f2e 0 2024-04-12T05:55:45.580 INFO:tasks.workunit.client.0.smithi008.stdout:8/951: write dd/d30/d3b/fb6 [75450,70394] 0 2024-04-12T05:55:45.583 INFO:tasks.workunit.client.0.smithi008.stdout:8/952: mknod dd/de/d16/d50/d5e/dbe/dd6/d124/c145 0 2024-04-12T05:55:45.602 INFO:tasks.workunit.client.0.smithi008.stdout:4/912: sync 2024-04-12T05:55:45.603 INFO:tasks.workunit.client.0.smithi008.stdout:3/933: sync 2024-04-12T05:55:45.604 INFO:tasks.workunit.client.0.smithi008.stdout:8/953: dwrite dd/f1a [0,4194304] 0 2024-04-12T05:55:45.606 INFO:tasks.workunit.client.0.smithi008.stdout:3/934: readlink d9/d10/d19/l3b 0 2024-04-12T05:55:45.606 INFO:tasks.workunit.client.0.smithi008.stdout:4/913: truncate d1/d22/d29/d46/d56/f63 4478879 0 2024-04-12T05:55:45.608 INFO:tasks.workunit.client.0.smithi008.stdout:3/935: write d9/d2e/d3d/f6b [1633117,64929] 0 2024-04-12T05:55:45.611 INFO:tasks.workunit.client.0.smithi008.stdout:3/936: fsync d9/d11/d5b/f10e 0 2024-04-12T05:55:45.613 INFO:tasks.workunit.client.0.smithi008.stdout:3/937: readlink d9/d10/d105/l119 0 2024-04-12T05:55:45.614 INFO:tasks.workunit.client.0.smithi008.stdout:8/954: write dd/de/d16/dc8/fe3 [447007,32245] 0 2024-04-12T05:55:45.615 INFO:tasks.workunit.client.0.smithi008.stdout:3/938: chown d9/d37/lea 231 1 2024-04-12T05:55:45.619 INFO:tasks.workunit.client.0.smithi008.stdout:4/914: dwrite d1/d22/d29/d30/d60/ffc [0,4194304] 0 2024-04-12T05:55:45.626 INFO:tasks.workunit.client.0.smithi008.stdout:4/915: mkdir d1/d22/d29/d30/d11b 0 2024-04-12T05:55:45.627 INFO:tasks.workunit.client.0.smithi008.stdout:8/955: truncate dd/de/f27 2609186 0 2024-04-12T05:55:45.630 INFO:tasks.workunit.client.0.smithi008.stdout:4/916: creat d1/d22/d29/d46/d56/daf/dcf/f11c x:0 0 0 2024-04-12T05:55:45.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/917: chown d1/d22/d29/d46/da7/fe7 97635 1 2024-04-12T05:55:45.636 INFO:tasks.workunit.client.0.smithi008.stdout:3/939: dwrite d9/f92 [0,4194304] 0 2024-04-12T05:55:45.644 INFO:tasks.workunit.client.0.smithi008.stdout:4/918: creat d1/d22/d29/d30/d47/f11d x:0 0 0 2024-04-12T05:55:45.648 INFO:tasks.workunit.client.0.smithi008.stdout:8/956: dread dd/d30/d3b/db5/df0/fc2 [0,4194304] 0 2024-04-12T05:55:45.701 INFO:tasks.workunit.client.0.smithi008.stdout:4/919: rmdir d1/d8 39 2024-04-12T05:55:45.826 INFO:tasks.workunit.client.0.smithi008.stdout:3/940: sync 2024-04-12T05:55:45.857 INFO:tasks.workunit.client.0.smithi008.stdout:3/941: creat d9/d10/d19/d22/da8/f13b x:0 0 0 2024-04-12T05:55:45.858 INFO:tasks.workunit.client.0.smithi008.stdout:8/957: getdents dd/dab/d126 0 2024-04-12T05:55:45.872 INFO:tasks.workunit.client.0.smithi008.stdout:3/942: dwrite d9/d10/d105/f112 [4194304,4194304] 0 2024-04-12T05:55:45.874 INFO:tasks.workunit.client.0.smithi008.stdout:8/958: dread dd/de/d16/d10d/f7d [0,4194304] 0 2024-04-12T05:55:45.877 INFO:tasks.workunit.client.0.smithi008.stdout:8/959: readlink dd/de/d16/d50/d56/d61/l9d 0 2024-04-12T05:55:45.909 INFO:tasks.workunit.client.0.smithi008.stdout:4/920: sync 2024-04-12T05:55:46.010 INFO:tasks.workunit.client.0.smithi008.stdout:8/960: symlink dd/de/d16/d50/d5e/dbe/dd6/d108/l146 0 2024-04-12T05:55:46.011 INFO:tasks.workunit.client.0.smithi008.stdout:3/943: mknod d9/dd/c13c 0 2024-04-12T05:55:46.431 INFO:tasks.workunit.client.0.smithi008.stdout:8/961: creat dd/d4f/d52/f147 x:0 0 0 2024-04-12T05:55:46.432 INFO:tasks.workunit.client.0.smithi008.stdout:4/921: mknod d1/d8/c11e 0 2024-04-12T05:55:46.433 INFO:tasks.workunit.client.0.smithi008.stdout:8/962: mknod dd/d30/d3b/db5/c148 0 2024-04-12T05:55:46.437 INFO:tasks.workunit.client.0.smithi008.stdout:3/944: write d9/d2e/d3d/f51 [936663,74388] 0 2024-04-12T05:55:46.438 INFO:tasks.workunit.client.0.smithi008.stdout:8/963: dread dd/de/d16/fba [0,4194304] 0 2024-04-12T05:55:46.445 INFO:tasks.workunit.client.0.smithi008.stdout:3/945: getdents d9/d10/d19/d22/da8/dc4 0 2024-04-12T05:55:46.448 INFO:tasks.workunit.client.0.smithi008.stdout:4/922: dwrite d1/d22/d29/d30/d47/f26 [0,4194304] 0 2024-04-12T05:55:46.448 INFO:tasks.workunit.client.0.smithi008.stdout:3/946: symlink d9/d37/d69/l13d 0 2024-04-12T05:55:46.452 INFO:tasks.workunit.client.0.smithi008.stdout:3/947: creat d9/d2e/d136/d116/f13e x:0 0 0 2024-04-12T05:55:46.453 INFO:tasks.workunit.client.0.smithi008.stdout:3/948: stat d9/d10/d19/d3a/dad/fdd 0 2024-04-12T05:55:46.454 INFO:tasks.workunit.client.0.smithi008.stdout:8/964: truncate dd/ff 784729 0 2024-04-12T05:55:46.457 INFO:tasks.workunit.client.0.smithi008.stdout:8/965: read dd/de/d16/d50/d5e/dbe/f11b [3669798,17469] 0 2024-04-12T05:55:46.457 INFO:tasks.workunit.client.0.smithi008.stdout:3/949: symlink d9/d10/d19/d22/da8/l13f 0 2024-04-12T05:55:46.459 INFO:tasks.workunit.client.0.smithi008.stdout:4/923: truncate d1/d22/d29/d30/d47/f48 412662 0 2024-04-12T05:55:46.461 INFO:tasks.workunit.client.0.smithi008.stdout:8/966: symlink dd/d65/l149 0 2024-04-12T05:55:46.463 INFO:tasks.workunit.client.0.smithi008.stdout:4/924: creat d1/d8/dc5/deb/f11f x:0 0 0 2024-04-12T05:55:46.464 INFO:tasks.workunit.client.0.smithi008.stdout:8/967: write dd/de/d16/d10d/f6f [4823891,54457] 0 2024-04-12T05:55:46.465 INFO:tasks.workunit.client.0.smithi008.stdout:3/950: creat d9/d2e/f140 x:0 0 0 2024-04-12T05:55:46.467 INFO:tasks.workunit.client.0.smithi008.stdout:4/925: creat d1/d22/d29/d30/d60/dd0/ddb/f120 x:0 0 0 2024-04-12T05:55:46.468 INFO:tasks.workunit.client.0.smithi008.stdout:3/951: fdatasync d9/d10/d4b/fa7 0 2024-04-12T05:55:46.472 INFO:tasks.workunit.client.0.smithi008.stdout:8/968: mknod dd/d30/d32/c14a 0 2024-04-12T05:55:46.474 INFO:tasks.workunit.client.0.smithi008.stdout:4/926: mkdir d1/d22/d29/d30/d47/d28/d4a/d121 0 2024-04-12T05:55:46.476 INFO:tasks.workunit.client.0.smithi008.stdout:4/927: read d1/d4f/fa5 [2635070,76498] 0 2024-04-12T05:55:46.481 INFO:tasks.workunit.client.0.smithi008.stdout:3/952: dread d9/d2e/d3d/f6b [0,4194304] 0 2024-04-12T05:55:46.482 INFO:tasks.workunit.client.0.smithi008.stdout:3/953: chown d9/d11/f49 257031 1 2024-04-12T05:55:46.485 INFO:tasks.workunit.client.0.smithi008.stdout:4/928: rename d1/c1e to d1/d4f/dd5/c122 0 2024-04-12T05:55:46.486 INFO:tasks.workunit.client.0.smithi008.stdout:3/954: stat d9/d10/d19/d22/d2c/la4 0 2024-04-12T05:55:46.489 INFO:tasks.workunit.client.0.smithi008.stdout:3/955: mknod d9/d2e/d3d/c141 0 2024-04-12T05:55:46.490 INFO:tasks.workunit.client.0.smithi008.stdout:3/956: dread - d9/dd/f53 zero size 2024-04-12T05:55:46.491 INFO:tasks.workunit.client.0.smithi008.stdout:4/929: getdents d1/d4f/dcb 0 2024-04-12T05:55:46.493 INFO:tasks.workunit.client.0.smithi008.stdout:3/957: mkdir d9/d10/d4b/d120/d142 0 2024-04-12T05:55:46.498 INFO:tasks.workunit.client.0.smithi008.stdout:4/930: mkdir d1/d22/d29/d30/d60/dd0/d123 0 2024-04-12T05:55:46.502 INFO:tasks.workunit.client.0.smithi008.stdout:8/969: dwrite dd/d30/f78 [0,4194304] 0 2024-04-12T05:55:46.531 INFO:tasks.workunit.client.0.smithi008.stdout:3/958: fdatasync d9/d2e/d3d/f70 0 2024-04-12T05:55:46.533 INFO:tasks.workunit.client.0.smithi008.stdout:4/931: creat d1/d22/d29/d46/d56/daf/f124 x:0 0 0 2024-04-12T05:55:46.534 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:46 smithi008 ceph-mon[39722]: pgmap v19: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 87 MiB/s rd, 198 MiB/s wr, 733 op/s 2024-04-12T05:55:46.535 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:46 smithi008 ceph-mon[39722]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:55:46.537 INFO:tasks.workunit.client.0.smithi008.stdout:3/959: mknod d9/d10/d19/d3a/dad/c143 0 2024-04-12T05:55:46.542 INFO:tasks.workunit.client.0.smithi008.stdout:8/970: link dd/dab/cef dd/de/d16/db1/d97/c14b 0 2024-04-12T05:55:46.543 INFO:tasks.workunit.client.0.smithi008.stdout:8/971: stat dd/de/d16/d50/d5e/dbe/dd6/cfe 0 2024-04-12T05:55:46.546 INFO:tasks.workunit.client.0.smithi008.stdout:8/972: creat dd/de/d16/d50/d5e/dbe/dd6/d108/f14c x:0 0 0 2024-04-12T05:55:46.548 INFO:tasks.workunit.client.0.smithi008.stdout:8/973: mknod dd/de/c14d 0 2024-04-12T05:55:46.553 INFO:tasks.workunit.client.0.smithi008.stdout:8/974: getdents dd/dab 0 2024-04-12T05:55:46.557 INFO:tasks.workunit.client.0.smithi008.stdout:8/975: rename dd/dab/cef to dd/de/d16/d50/d5e/dbe/c14e 0 2024-04-12T05:55:46.560 INFO:tasks.workunit.client.0.smithi008.stdout:8/976: creat dd/d4f/df5/d113/d13d/f14f x:0 0 0 2024-04-12T05:55:46.560 INFO:tasks.workunit.client.0.smithi008.stdout:8/977: dread - dd/dab/f136 zero size 2024-04-12T05:55:46.563 INFO:tasks.workunit.client.0.smithi008.stdout:8/978: write dd/de/d4c/d66/d6d/f11a [18666,32165] 0 2024-04-12T05:55:46.565 INFO:tasks.workunit.client.0.smithi008.stdout:8/979: creat dd/d4f/df5/d113/d13d/f150 x:0 0 0 2024-04-12T05:55:46.576 INFO:tasks.workunit.client.0.smithi008.stdout:4/932: truncate d1/d8/f15 1310919 0 2024-04-12T05:55:46.581 INFO:tasks.workunit.client.0.smithi008.stdout:8/980: write dd/f99 [5335058,78864] 0 2024-04-12T05:55:46.582 INFO:tasks.workunit.client.0.smithi008.stdout:3/960: dwrite d9/f25 [0,4194304] 0 2024-04-12T05:55:46.585 INFO:tasks.workunit.client.0.smithi008.stdout:8/981: write dd/d4f/df5/d113/d13d/f150 [627833,20895] 0 2024-04-12T05:55:46.586 INFO:tasks.workunit.client.0.smithi008.stdout:4/933: link d1/d22/d29/d30/d47/d28/c49 d1/d4f/c125 0 2024-04-12T05:55:46.588 INFO:tasks.workunit.client.0.smithi008.stdout:4/934: chown d1/d4f/dd5/fdc 206 1 2024-04-12T05:55:46.588 INFO:tasks.workunit.client.0.smithi008.stdout:8/982: truncate dd/de/d16/fba 4242325 0 2024-04-12T05:55:46.589 INFO:tasks.workunit.client.0.smithi008.stdout:3/961: dread d9/d10/f9c [0,4194304] 0 2024-04-12T05:55:46.597 INFO:tasks.workunit.client.0.smithi008.stdout:4/935: dread d1/fa0 [0,4194304] 0 2024-04-12T05:55:46.600 INFO:tasks.workunit.client.0.smithi008.stdout:4/936: creat d1/d8/dc5/deb/f126 x:0 0 0 2024-04-12T05:55:46.600 INFO:tasks.workunit.client.0.smithi008.stdout:8/983: write dd/de/f29 [2554828,25809] 0 2024-04-12T05:55:46.602 INFO:tasks.workunit.client.0.smithi008.stdout:8/984: fsync dd/d4f/d52/da5/f137 0 2024-04-12T05:55:46.603 INFO:tasks.workunit.client.0.smithi008.stdout:8/985: dread - dd/de/d4c/d66/d6d/f119 zero size 2024-04-12T05:55:46.603 INFO:tasks.workunit.client.0.smithi008.stdout:4/937: symlink d1/d8/dc5/l127 0 2024-04-12T05:55:46.607 INFO:tasks.workunit.client.0.smithi008.stdout:4/938: truncate d1/d22/d29/d30/d47/d28/d74/fb3 288289 0 2024-04-12T05:55:46.608 INFO:tasks.workunit.client.0.smithi008.stdout:4/939: chown d1/d22/d29/d46/d56/daf/dcf/cf1 307780 1 2024-04-12T05:55:46.609 INFO:tasks.workunit.client.0.smithi008.stdout:8/986: write dd/f22 [4409569,65946] 0 2024-04-12T05:55:46.610 INFO:tasks.workunit.client.0.smithi008.stdout:4/940: symlink d1/d8/l128 0 2024-04-12T05:55:46.614 INFO:tasks.workunit.client.0.smithi008.stdout:8/987: dread dd/d30/d32/fbd [0,4194304] 0 2024-04-12T05:55:46.620 INFO:tasks.workunit.client.0.smithi008.stdout:4/941: write d1/d22/d29/d30/d47/d28/d74/fb3 [1200652,6441] 0 2024-04-12T05:55:46.620 INFO:tasks.workunit.client.0.smithi008.stdout:8/988: write dd/de/d4c/fcb [3694439,75594] 0 2024-04-12T05:55:46.622 INFO:tasks.workunit.client.0.smithi008.stdout:4/942: write d1/d22/d29/d46/fb6 [658528,28079] 0 2024-04-12T05:55:46.623 INFO:tasks.workunit.client.0.smithi008.stdout:3/962: dread d9/d1e/f42 [4194304,4194304] 0 2024-04-12T05:55:46.625 INFO:tasks.workunit.client.0.smithi008.stdout:3/963: write d9/d10/d19/d65/f8e [625850,73192] 0 2024-04-12T05:55:46.628 INFO:tasks.workunit.client.0.smithi008.stdout:8/989: getdents dd/d30/d3b 0 2024-04-12T05:55:46.628 INFO:tasks.workunit.client.0.smithi008.stdout:4/943: write d1/d4f/f5a [3379999,4462] 0 2024-04-12T05:55:46.629 INFO:tasks.workunit.client.0.smithi008.stdout:3/964: creat d9/d10/d19/d3a/d7c/f144 x:0 0 0 2024-04-12T05:55:46.631 INFO:tasks.workunit.client.0.smithi008.stdout:4/944: creat d1/db/f129 x:0 0 0 2024-04-12T05:55:46.632 INFO:tasks.workunit.client.0.smithi008.stdout:8/990: creat dd/de/d4c/d66/d6c/df4/f151 x:0 0 0 2024-04-12T05:55:46.633 INFO:tasks.workunit.client.0.smithi008.stdout:3/965: symlink d9/d44/l145 0 2024-04-12T05:55:46.634 INFO:tasks.workunit.client.0.smithi008.stdout:4/945: symlink d1/d22/d29/d30/d47/d28/d4a/l12a 0 2024-04-12T05:55:46.642 INFO:tasks.workunit.client.0.smithi008.stdout:4/946: dwrite d1/d22/d29/d30/d47/d28/d74/ff0 [0,4194304] 0 2024-04-12T05:55:46.716 INFO:tasks.workunit.client.0.smithi008.stdout:4/947: rmdir d1/d22/d29/d46/d56/daf 39 2024-04-12T05:55:46.750 INFO:tasks.workunit.client.0.smithi008.stdout:4/948: readlink d1/d22/d29/d30/d60/dd6/lea 0 2024-04-12T05:55:46.751 INFO:tasks.workunit.client.0.smithi008.stdout:8/991: creat dd/de/f152 x:0 0 0 2024-04-12T05:55:46.752 INFO:tasks.workunit.client.0.smithi008.stdout:3/966: dwrite d9/d11/f85 [0,4194304] 0 2024-04-12T05:55:46.753 INFO:tasks.workunit.client.0.smithi008.stdout:4/949: rmdir d1/d8 39 2024-04-12T05:55:46.753 INFO:tasks.workunit.client.0.smithi008.stdout:8/992: truncate dd/d4f/d52/f147 341778 0 2024-04-12T05:55:46.757 INFO:tasks.workunit.client.0.smithi008.stdout:3/967: symlink d9/d37/d69/l146 0 2024-04-12T05:55:46.759 INFO:tasks.workunit.client.0.smithi008.stdout:3/968: chown d9/d37/c86 6 1 2024-04-12T05:55:46.761 INFO:tasks.workunit.client.0.smithi008.stdout:4/950: creat d1/d8/dc5/deb/f12b x:0 0 0 2024-04-12T05:55:46.767 INFO:tasks.workunit.client.0.smithi008.stdout:8/993: dwrite dd/de/d16/d10d/f135 [0,4194304] 0 2024-04-12T05:55:46.767 INFO:tasks.workunit.client.0.smithi008.stdout:8/994: read - dd/de/d16/dc8/f139 zero size 2024-04-12T05:55:46.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:46 smithi194 ceph-mon[49771]: pgmap v19: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 87 MiB/s rd, 198 MiB/s wr, 733 op/s 2024-04-12T05:55:46.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:46 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:55:46.925 INFO:tasks.workunit.client.0.smithi008.stdout:3/969: creat d9/d1e/f147 x:0 0 0 2024-04-12T05:55:46.987 INFO:tasks.workunit.client.0.smithi008.stdout:4/951: mknod d1/d22/d29/d30/d60/c12c 0 2024-04-12T05:55:46.987 INFO:tasks.workunit.client.0.smithi008.stdout:4/952: dread - d1/d22/d29/d30/d60/dd6/f11a zero size 2024-04-12T05:55:46.988 INFO:tasks.workunit.client.0.smithi008.stdout:4/953: chown d1/d8/l14 1312286 1 2024-04-12T05:55:46.989 INFO:tasks.workunit.client.0.smithi008.stdout:4/954: chown d1/d8/fc3 122 1 2024-04-12T05:55:46.999 INFO:tasks.workunit.client.0.smithi008.stdout:3/970: rename d9/d10/d19/f50 to d9/f148 0 2024-04-12T05:55:47.001 INFO:tasks.workunit.client.0.smithi008.stdout:4/955: creat d1/d22/d29/d30/f12d x:0 0 0 2024-04-12T05:55:47.001 INFO:tasks.workunit.client.0.smithi008.stdout:3/971: dread - d9/d10/d19/d3a/d7c/f144 zero size 2024-04-12T05:55:47.001 INFO:tasks.workunit.client.0.smithi008.stdout:8/995: write dd/fe9 [912431,105628] 0 2024-04-12T05:55:47.003 INFO:tasks.workunit.client.0.smithi008.stdout:4/956: write d1/d22/d29/d30/d60/dd6/f113 [265393,63032] 0 2024-04-12T05:55:47.005 INFO:tasks.workunit.client.0.smithi008.stdout:4/957: chown d1/d22/d29/d30/c58 2026 1 2024-04-12T05:55:47.005 INFO:tasks.workunit.client.0.smithi008.stdout:3/972: mkdir d9/d10/d19/d3a/dad/d149 0 2024-04-12T05:55:47.009 INFO:tasks.workunit.client.0.smithi008.stdout:8/996: creat dd/de/d16/d50/d5e/dbe/dd6/d108/f153 x:0 0 0 2024-04-12T05:55:47.010 INFO:tasks.workunit.client.0.smithi008.stdout:3/973: creat d9/d10/d19/f14a x:0 0 0 2024-04-12T05:55:47.011 INFO:tasks.workunit.client.0.smithi008.stdout:8/997: write dd/f4e [4011115,106516] 0 2024-04-12T05:55:47.016 INFO:tasks.workunit.client.0.smithi008.stdout:4/958: dwrite d1/db/f108 [0,4194304] 0 2024-04-12T05:55:47.018 INFO:tasks.workunit.client.0.smithi008.stdout:3/974: write d9/d1e/f5a [327485,32698] 0 2024-04-12T05:55:47.020 INFO:tasks.workunit.client.0.smithi008.stdout:4/959: read d1/d22/d29/d30/f57 [67733,87459] 0 2024-04-12T05:55:47.028 INFO:tasks.workunit.client.0.smithi008.stdout:3/975: dwrite d9/d10/d19/d22/da8/f13b [0,4194304] 0 2024-04-12T05:55:47.032 INFO:tasks.workunit.client.0.smithi008.stdout:3/976: mkdir d9/d10/d105/d121/d14b 0 2024-04-12T05:55:47.036 INFO:tasks.workunit.client.0.smithi008.stdout:4/960: truncate d1/d8/f55 4358137 0 2024-04-12T05:55:47.038 INFO:tasks.workunit.client.0.smithi008.stdout:4/961: dread d1/fa0 [0,4194304] 0 2024-04-12T05:55:47.038 INFO:tasks.workunit.client.0.smithi008.stdout:3/977: link d9/d10/d19/d3a/d67/f97 d9/d10/d19/d3a/d67/f14c 0 2024-04-12T05:55:47.047 INFO:tasks.workunit.client.0.smithi008.stdout:3/978: dwrite d9/d10/d105/f112 [0,4194304] 0 2024-04-12T05:55:47.048 INFO:tasks.workunit.client.0.smithi008.stdout:4/962: dwrite d1/d22/d29/d30/d47/d28/fa6 [0,4194304] 0 2024-04-12T05:55:47.052 INFO:tasks.workunit.client.0.smithi008.stdout:4/963: mknod d1/d22/d29/c12e 0 2024-04-12T05:55:47.055 INFO:tasks.workunit.client.0.smithi008.stdout:4/964: read - d1/d22/d29/d30/d47/d28/d4a/fdf zero size 2024-04-12T05:55:47.062 INFO:tasks.workunit.client.0.smithi008.stdout:4/965: creat d1/d8/dc5/deb/f12f x:0 0 0 2024-04-12T05:55:47.063 INFO:tasks.workunit.client.0.smithi008.stdout:4/966: creat d1/f130 x:0 0 0 2024-04-12T05:55:47.068 INFO:tasks.workunit.client.0.smithi008.stdout:4/967: link d1/d22/d29/d30/d60/l69 d1/d22/d29/d46/l131 0 2024-04-12T05:55:47.072 INFO:tasks.workunit.client.0.smithi008.stdout:3/979: write d9/dd/f7f [428933,44603] 0 2024-04-12T05:55:47.077 INFO:tasks.workunit.client.0.smithi008.stdout:4/968: link d1/d22/d29/d46/d56/daf/dcf/cf1 d1/d22/d29/d30/d60/dd0/c132 0 2024-04-12T05:55:47.080 INFO:tasks.workunit.client.0.smithi008.stdout:4/969: chown d1/d22/d29/d30/d60/dd6/cec 13596 1 2024-04-12T05:55:47.083 INFO:tasks.workunit.client.0.smithi008.stdout:3/980: dwrite d9/d10/d19/d22/d2c/f83 [0,4194304] 0 2024-04-12T05:55:47.093 INFO:tasks.workunit.client.0.smithi008.stdout:4/970: dwrite d1/db/db7/fba [0,4194304] 0 2024-04-12T05:55:47.095 INFO:tasks.workunit.client.0.smithi008.stdout:4/971: chown d1/d22/d29/d30/d60/c99 3264 1 2024-04-12T05:55:47.098 INFO:tasks.workunit.client.0.smithi008.stdout:4/972: stat d1/d8/f55 0 2024-04-12T05:55:47.099 INFO:tasks.workunit.client.0.smithi008.stdout:4/973: write d1/f130 [183631,23060] 0 2024-04-12T05:55:47.102 INFO:tasks.workunit.client.0.smithi008.stdout:4/974: creat d1/d4f/dcb/f133 x:0 0 0 2024-04-12T05:55:47.105 INFO:tasks.workunit.client.0.smithi008.stdout:4/975: creat d1/d8/dc5/f134 x:0 0 0 2024-04-12T05:55:47.108 INFO:tasks.workunit.client.0.smithi008.stdout:4/976: unlink d1/d22/d29/d46/d56/fc4 0 2024-04-12T05:55:47.111 INFO:tasks.workunit.client.0.smithi008.stdout:4/977: symlink d1/d22/d29/d30/d11b/l135 0 2024-04-12T05:55:47.113 INFO:tasks.workunit.client.0.smithi008.stdout:4/978: read d1/d22/d29/d46/fdd [2751633,48073] 0 2024-04-12T05:55:47.114 INFO:tasks.workunit.client.0.smithi008.stdout:3/981: dread d9/d11/f49 [0,4194304] 0 2024-04-12T05:55:47.122 INFO:tasks.workunit.client.0.smithi008.stdout:3/982: getdents d9/d10/d105 0 2024-04-12T05:55:47.127 INFO:tasks.workunit.client.0.smithi008.stdout:3/983: rename d9/d1e/lbb to d9/db7/l14d 0 2024-04-12T05:55:47.128 INFO:tasks.workunit.client.0.smithi008.stdout:4/979: dwrite d1/d8/f55 [4194304,4194304] 0 2024-04-12T05:55:47.129 INFO:tasks.workunit.client.0.smithi008.stdout:3/984: chown d9/d44/f9f 51 1 2024-04-12T05:55:47.137 INFO:tasks.workunit.client.0.smithi008.stdout:3/985: rename d9/d10/d19/d22/da8 to d9/d37/d14e 0 2024-04-12T05:55:47.139 INFO:tasks.workunit.client.0.smithi008.stdout:4/980: dwrite d1/d4f/dd5/fdc [0,4194304] 0 2024-04-12T05:55:47.145 INFO:tasks.workunit.client.0.smithi008.stdout:3/986: rename d9/d10/d19/d3a/d7c/f144 to d9/d44/d107/f14f 0 2024-04-12T05:55:47.151 INFO:tasks.workunit.client.0.smithi008.stdout:4/981: dwrite d1/d22/d29/f7c [0,4194304] 0 2024-04-12T05:55:47.151 INFO:tasks.workunit.client.0.smithi008.stdout:3/987: getdents d9/d11/d5b 0 2024-04-12T05:55:47.152 INFO:tasks.workunit.client.0.smithi008.stdout:3/988: chown d9/d44/fe4 328859 1 2024-04-12T05:55:47.157 INFO:tasks.workunit.client.0.smithi008.stdout:4/982: readlink d1/d22/d29/d30/d47/d28/d4a/lfd 0 2024-04-12T05:55:47.158 INFO:tasks.workunit.client.0.smithi008.stdout:4/983: write d1/d8/f117 [257505,117221] 0 2024-04-12T05:55:47.162 INFO:tasks.workunit.client.0.smithi008.stdout:4/984: readlink d1/d8/le6 0 2024-04-12T05:55:47.162 INFO:tasks.workunit.client.0.smithi008.stdout:3/989: truncate d9/d44/f9f 777969 0 2024-04-12T05:55:47.164 INFO:tasks.workunit.client.0.smithi008.stdout:3/990: readlink d9/d37/l54 0 2024-04-12T05:55:47.167 INFO:tasks.workunit.client.0.smithi008.stdout:3/991: creat d9/d44/d107/f150 x:0 0 0 2024-04-12T05:55:47.168 INFO:tasks.workunit.client.0.smithi008.stdout:3/992: dread - d9/d37/dd8/f134 zero size 2024-04-12T05:55:47.169 INFO:tasks.workunit.client.0.smithi008.stdout:4/985: dread d1/d22/d29/d30/d47/d28/d74/ff0 [0,4194304] 0 2024-04-12T05:55:47.169 INFO:tasks.workunit.client.0.smithi008.stdout:3/993: stat d9/d10/d19/f14a 0 2024-04-12T05:55:47.172 INFO:tasks.workunit.client.0.smithi008.stdout:3/994: creat d9/d10/f151 x:0 0 0 2024-04-12T05:55:47.174 INFO:tasks.workunit.client.0.smithi008.stdout:3/995: readlink d9/d37/l71 0 2024-04-12T05:55:47.176 INFO:tasks.workunit.client.0.smithi008.stdout:3/996: mknod d9/d2e/d136/c152 0 2024-04-12T05:55:47.178 INFO:tasks.workunit.client.0.smithi008.stdout:3/997: chown d9/d37/d69/fe5 36345 1 2024-04-12T05:55:47.178 INFO:tasks.workunit.client.0.smithi008.stdout:4/986: dwrite d1/d22/d29/d30/d47/d28/d74/ff0 [4194304,4194304] 0 2024-04-12T05:55:47.181 INFO:tasks.workunit.client.0.smithi008.stdout:3/998: readlink d9/d10/d4b/le1 0 2024-04-12T05:55:47.182 INFO:tasks.workunit.client.0.smithi008.stdout:4/987: creat d1/d4f/dcb/f136 x:0 0 0 2024-04-12T05:55:47.184 INFO:tasks.workunit.client.0.smithi008.stdout:3/999: mkdir d9/d2e/d136/de3/d153 0 2024-04-12T05:55:47.186 INFO:tasks.workunit.client.0.smithi008.stdout:4/988: unlink d1/d22/d29/d30/d60/l10f 0 2024-04-12T05:55:47.192 INFO:tasks.workunit.client.0.smithi008.stdout:4/989: getdents d1/d22/d29/d30/d60/dd0/ddb 0 2024-04-12T05:55:47.194 INFO:tasks.workunit.client.0.smithi008.stdout:4/990: truncate d1/d22/d29/d30/d47/d28/fa1 401214 0 2024-04-12T05:55:47.196 INFO:tasks.workunit.client.0.smithi008.stdout:4/991: write d1/d22/d29/d30/d47/d28/d74/f9c [3716305,14908] 0 2024-04-12T05:55:47.199 INFO:tasks.workunit.client.0.smithi008.stdout:4/992: mknod d1/d22/d29/c137 0 2024-04-12T05:55:47.201 INFO:tasks.workunit.client.0.smithi008.stdout:4/993: write d1/d22/d29/d30/d60/f10b [1866303,58224] 0 2024-04-12T05:55:47.208 INFO:tasks.workunit.client.0.smithi008.stdout:4/994: dwrite d1/d22/d29/d30/d60/ffc [0,4194304] 0 2024-04-12T05:55:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:48.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:47 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:55:48.136 INFO:tasks.workunit.client.0.smithi008.stdout:8/998: sync 2024-04-12T05:55:48.138 INFO:tasks.workunit.client.0.smithi008.stdout:8/999: creat dd/d4f/f154 x:0 0 0 2024-04-12T05:55:48.277 INFO:tasks.workunit.client.0.smithi008.stdout:4/995: sync 2024-04-12T05:55:48.279 INFO:tasks.workunit.client.0.smithi008.stdout:4/996: mknod d1/d8/c138 0 2024-04-12T05:55:48.285 INFO:tasks.workunit.client.0.smithi008.stdout:4/997: link d1/d22/d29/d46/l54 d1/d4f/dd5/l139 0 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:47 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:55:48.434 INFO:tasks.workunit.client.0.smithi008.stdout:4/998: sync 2024-04-12T05:55:48.435 INFO:tasks.workunit.client.0.smithi008.stdout:4/999: truncate d1/d22/d29/d46/d56/daf/dcf/f101 1004300 0 2024-04-12T05:55:48.438 INFO:tasks.workunit.client.0.smithi008.stderr:+ rm -rf -- ./tmp.TkQo1dS5dN 2024-04-12T05:55:49.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:48 smithi194 ceph-mon[49771]: pgmap v20: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 56 MiB/s rd, 122 MiB/s wr, 480 op/s 2024-04-12T05:55:49.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:48 smithi194 ceph-mon[49771]: Adjusting osd_memory_target on smithi008 to 1228M 2024-04-12T05:55:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:48 smithi194 ceph-mon[49771]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:55:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:48 smithi194 ceph-mon[49771]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:55:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:48 smithi008 ceph-mon[39722]: pgmap v20: 65 pgs: 65 active+clean; 2.0 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 56 MiB/s rd, 122 MiB/s wr, 480 op/s 2024-04-12T05:55:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:48 smithi008 ceph-mon[39722]: Adjusting osd_memory_target on smithi008 to 1228M 2024-04-12T05:55:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:48 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:55:49.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:48 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:55:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:50 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:50.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:50 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:51.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:51 smithi008 ceph-mon[39722]: pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 61 MiB/s rd, 140 MiB/s wr, 624 op/s 2024-04-12T05:55:51.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:51 smithi008 ceph-mon[39722]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:55:51.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:51 smithi008 ceph-mon[39722]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:55:51.517 INFO:tasks.workunit.client.1.smithi194.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-12T05:55:51.526 INFO:tasks.workunit.client.1.smithi194.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T05:55:51.526 INFO:tasks.workunit.client.1.smithi194.stderr:+ make 2024-04-12T05:55:51.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:51 smithi194 ceph-mon[49771]: pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 61 MiB/s rd, 140 MiB/s wr, 624 op/s 2024-04-12T05:55:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:51 smithi194 ceph-mon[49771]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:55:51.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:51 smithi194 ceph-mon[49771]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:55:51.662 INFO:tasks.workunit.client.1.smithi194.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-12T05:55:53.024 INFO:tasks.workunit.client.1.smithi194.stderr:++ readlink -f fsstress 2024-04-12T05:55:53.025 INFO:tasks.workunit.client.1.smithi194.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-12T05:55:53.025 INFO:tasks.workunit.client.1.smithi194.stderr:+ popd 2024-04-12T05:55:53.026 INFO:tasks.workunit.client.1.smithi194.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-12T05:55:53.026 INFO:tasks.workunit.client.1.smithi194.stderr:+ popd 2024-04-12T05:55:53.027 INFO:tasks.workunit.client.1.smithi194.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-12T05:55:53.027 INFO:tasks.workunit.client.1.smithi194.stderr:++ mktemp -d -p . 2024-04-12T05:55:53.030 INFO:tasks.workunit.client.1.smithi194.stderr:+ T=./tmp.5iwGLtTk4c 2024-04-12T05:55:53.030 INFO:tasks.workunit.client.1.smithi194.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.5iwGLtTk4c -l 1 -n 1000 -p 10 -v 2024-04-12T05:55:53.032 INFO:tasks.workunit.client.1.smithi194.stdout:seed = 1712870495 2024-04-12T05:55:53.035 INFO:tasks.workunit.client.1.smithi194.stdout:0/0: dread - no filename 2024-04-12T05:55:53.036 INFO:tasks.workunit.client.1.smithi194.stdout:0/1: rename - no filename 2024-04-12T05:55:53.036 INFO:tasks.workunit.client.1.smithi194.stdout:0/2: rmdir - no directory 2024-04-12T05:55:53.036 INFO:tasks.workunit.client.1.smithi194.stdout:0/3: write - no filename 2024-04-12T05:55:53.036 INFO:tasks.workunit.client.1.smithi194.stdout:0/4: fdatasync - no filename 2024-04-12T05:55:53.036 INFO:tasks.workunit.client.1.smithi194.stdout:0/5: dread - no filename 2024-04-12T05:55:53.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/0: unlink - no file 2024-04-12T05:55:53.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/1: rmdir - no directory 2024-04-12T05:55:53.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/2: rmdir - no directory 2024-04-12T05:55:53.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/3: fdatasync - no filename 2024-04-12T05:55:53.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/4: unlink - no file 2024-04-12T05:55:53.038 INFO:tasks.workunit.client.1.smithi194.stdout:2/0: readlink - no filename 2024-04-12T05:55:53.038 INFO:tasks.workunit.client.1.smithi194.stdout:2/1: rename - no filename 2024-04-12T05:55:53.038 INFO:tasks.workunit.client.1.smithi194.stdout:2/2: rmdir - no directory 2024-04-12T05:55:53.038 INFO:tasks.workunit.client.1.smithi194.stdout:1/5: mkdir d0 0 2024-04-12T05:55:53.039 INFO:tasks.workunit.client.1.smithi194.stdout:1/6: dread - no filename 2024-04-12T05:55:53.040 INFO:tasks.workunit.client.1.smithi194.stdout:3/0: chown . 178653916 1 2024-04-12T05:55:53.040 INFO:tasks.workunit.client.1.smithi194.stdout:3/1: chown . 75 1 2024-04-12T05:55:53.040 INFO:tasks.workunit.client.1.smithi194.stdout:3/2: dread - no filename 2024-04-12T05:55:53.041 INFO:tasks.workunit.client.1.smithi194.stdout:2/3: mknod c0 0 2024-04-12T05:55:53.041 INFO:tasks.workunit.client.1.smithi194.stdout:1/7: mkdir d0/d1 0 2024-04-12T05:55:53.041 INFO:tasks.workunit.client.1.smithi194.stdout:1/8: truncate - no filename 2024-04-12T05:55:53.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/9: dread - no filename 2024-04-12T05:55:53.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/10: fdatasync - no filename 2024-04-12T05:55:53.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/11: write - no filename 2024-04-12T05:55:53.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/12: fsync - no filename 2024-04-12T05:55:53.043 INFO:tasks.workunit.client.1.smithi194.stdout:4/0: truncate - no filename 2024-04-12T05:55:53.043 INFO:tasks.workunit.client.1.smithi194.stdout:4/1: write - no filename 2024-04-12T05:55:53.043 INFO:tasks.workunit.client.1.smithi194.stdout:4/2: dread - no filename 2024-04-12T05:55:53.044 INFO:tasks.workunit.client.1.smithi194.stdout:2/4: rename c0 to c1 0 2024-04-12T05:55:53.045 INFO:tasks.workunit.client.1.smithi194.stdout:1/13: creat d0/f2 x:0 0 0 2024-04-12T05:55:53.046 INFO:tasks.workunit.client.1.smithi194.stdout:3/3: mkdir d0 0 2024-04-12T05:55:53.046 INFO:tasks.workunit.client.1.smithi194.stdout:3/4: write - no filename 2024-04-12T05:55:53.049 INFO:tasks.workunit.client.1.smithi194.stdout:5/0: dread - no filename 2024-04-12T05:55:53.049 INFO:tasks.workunit.client.1.smithi194.stdout:5/1: write - no filename 2024-04-12T05:55:53.049 INFO:tasks.workunit.client.1.smithi194.stdout:5/2: rename - no filename 2024-04-12T05:55:53.049 INFO:tasks.workunit.client.1.smithi194.stdout:4/3: creat f0 x:0 0 0 2024-04-12T05:55:53.050 INFO:tasks.workunit.client.1.smithi194.stdout:3/5: getdents d0 0 2024-04-12T05:55:53.052 INFO:tasks.workunit.client.1.smithi194.stdout:4/4: creat f1 x:0 0 0 2024-04-12T05:55:53.053 INFO:tasks.workunit.client.1.smithi194.stdout:4/5: chown f1 458 1 2024-04-12T05:55:53.054 INFO:tasks.workunit.client.1.smithi194.stdout:5/3: creat f0 x:0 0 0 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:3/6: mkdir d0/d1 0 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:6/0: mknod c0 0 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:6/1: dread - no filename 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:6/2: dread - no filename 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:6/3: write - no filename 2024-04-12T05:55:53.056 INFO:tasks.workunit.client.1.smithi194.stdout:3/7: chown d0/d1 167 1 2024-04-12T05:55:53.058 INFO:tasks.workunit.client.1.smithi194.stdout:5/4: mkdir d1 0 2024-04-12T05:55:53.059 INFO:tasks.workunit.client.1.smithi194.stdout:4/6: unlink f1 0 2024-04-12T05:55:53.059 INFO:tasks.workunit.client.1.smithi194.stdout:1/14: dwrite d0/f2 [0,4194304] 0 2024-04-12T05:55:53.061 INFO:tasks.workunit.client.1.smithi194.stdout:9/0: write - no filename 2024-04-12T05:55:53.061 INFO:tasks.workunit.client.1.smithi194.stdout:1/15: write d0/f2 [2144141,16275] 0 2024-04-12T05:55:53.061 INFO:tasks.workunit.client.1.smithi194.stdout:9/1: chown . 33151 1 2024-04-12T05:55:53.063 INFO:tasks.workunit.client.1.smithi194.stdout:7/0: mknod c0 0 2024-04-12T05:55:53.066 INFO:tasks.workunit.client.1.smithi194.stdout:4/7: mkdir d2 0 2024-04-12T05:55:53.070 INFO:tasks.workunit.client.1.smithi194.stdout:1/16: creat d0/f3 x:0 0 0 2024-04-12T05:55:53.071 INFO:tasks.workunit.client.1.smithi194.stdout:3/8: symlink d0/d1/l2 0 2024-04-12T05:55:53.072 INFO:tasks.workunit.client.1.smithi194.stdout:5/5: symlink d1/l2 0 2024-04-12T05:55:53.072 INFO:tasks.workunit.client.1.smithi194.stdout:8/0: symlink l0 0 2024-04-12T05:55:53.073 INFO:tasks.workunit.client.1.smithi194.stdout:8/1: stat l0 0 2024-04-12T05:55:53.074 INFO:tasks.workunit.client.1.smithi194.stdout:6/4: link c0 c1 0 2024-04-12T05:55:53.075 INFO:tasks.workunit.client.1.smithi194.stdout:9/2: creat f0 x:0 0 0 2024-04-12T05:55:53.077 INFO:tasks.workunit.client.1.smithi194.stdout:1/17: rename d0/f3 to d0/f4 0 2024-04-12T05:55:53.078 INFO:tasks.workunit.client.1.smithi194.stdout:1/18: write d0/f2 [3716494,45827] 0 2024-04-12T05:55:53.078 INFO:tasks.workunit.client.1.smithi194.stdout:5/6: creat d1/f3 x:0 0 0 2024-04-12T05:55:53.079 INFO:tasks.workunit.client.1.smithi194.stdout:1/19: dread - d0/f4 zero size 2024-04-12T05:55:53.080 INFO:tasks.workunit.client.1.smithi194.stdout:3/9: symlink d0/l3 0 2024-04-12T05:55:53.080 INFO:tasks.workunit.client.1.smithi194.stdout:1/20: dread - d0/f4 zero size 2024-04-12T05:55:53.081 INFO:tasks.workunit.client.1.smithi194.stdout:4/8: mkdir d2/d3 0 2024-04-12T05:55:53.081 INFO:tasks.workunit.client.1.smithi194.stdout:1/21: chown d0/d1 5 1 2024-04-12T05:55:53.082 INFO:tasks.workunit.client.1.smithi194.stdout:4/9: write f0 [778914,70451] 0 2024-04-12T05:55:53.082 INFO:tasks.workunit.client.1.smithi194.stdout:8/2: creat f1 x:0 0 0 2024-04-12T05:55:53.083 INFO:tasks.workunit.client.1.smithi194.stdout:4/10: truncate f0 1420162 0 2024-04-12T05:55:53.084 INFO:tasks.workunit.client.1.smithi194.stdout:9/3: mkdir d1 0 2024-04-12T05:55:53.087 INFO:tasks.workunit.client.1.smithi194.stdout:5/7: mkdir d1/d4 0 2024-04-12T05:55:53.088 INFO:tasks.workunit.client.1.smithi194.stdout:5/8: dread - f0 zero size 2024-04-12T05:55:53.090 INFO:tasks.workunit.client.1.smithi194.stdout:5/9: readlink d1/l2 0 2024-04-12T05:55:53.090 INFO:tasks.workunit.client.1.smithi194.stdout:1/22: write d0/f4 [912548,63404] 0 2024-04-12T05:55:53.090 INFO:tasks.workunit.client.1.smithi194.stdout:6/5: chown c0 14420069 1 2024-04-12T05:55:53.094 INFO:tasks.workunit.client.1.smithi194.stdout:4/11: dwrite f0 [0,4194304] 0 2024-04-12T05:55:53.095 INFO:tasks.workunit.client.1.smithi194.stdout:9/4: getdents d1 0 2024-04-12T05:55:53.095 INFO:tasks.workunit.client.1.smithi194.stdout:9/5: stat f0 0 2024-04-12T05:55:53.096 INFO:tasks.workunit.client.1.smithi194.stdout:9/6: truncate f0 519234 0 2024-04-12T05:55:53.099 INFO:tasks.workunit.client.1.smithi194.stdout:9/7: dread f0 [0,4194304] 0 2024-04-12T05:55:53.100 INFO:tasks.workunit.client.1.smithi194.stdout:6/6: chown c0 423632475 1 2024-04-12T05:55:53.100 INFO:tasks.workunit.client.1.smithi194.stdout:6/7: write - no filename 2024-04-12T05:55:53.100 INFO:tasks.workunit.client.1.smithi194.stdout:9/8: write f0 [625578,69233] 0 2024-04-12T05:55:53.100 INFO:tasks.workunit.client.1.smithi194.stdout:4/12: mknod d2/c4 0 2024-04-12T05:55:53.104 INFO:tasks.workunit.client.1.smithi194.stdout:5/10: link d1/f3 d1/d4/f5 0 2024-04-12T05:55:53.105 INFO:tasks.workunit.client.1.smithi194.stdout:1/23: rmdir d0/d1 0 2024-04-12T05:55:53.110 INFO:tasks.workunit.client.1.smithi194.stdout:4/13: dwrite f0 [0,4194304] 0 2024-04-12T05:55:53.112 INFO:tasks.workunit.client.1.smithi194.stdout:6/8: rename c0 to c2 0 2024-04-12T05:55:53.112 INFO:tasks.workunit.client.1.smithi194.stdout:6/9: read - no filename 2024-04-12T05:55:53.113 INFO:tasks.workunit.client.1.smithi194.stdout:9/9: symlink d1/l2 0 2024-04-12T05:55:53.115 INFO:tasks.workunit.client.1.smithi194.stdout:4/14: symlink d2/l5 0 2024-04-12T05:55:53.117 INFO:tasks.workunit.client.1.smithi194.stdout:6/10: mkdir d3 0 2024-04-12T05:55:53.117 INFO:tasks.workunit.client.1.smithi194.stdout:6/11: dwrite - no filename 2024-04-12T05:55:53.117 INFO:tasks.workunit.client.1.smithi194.stdout:6/12: dwrite - no filename 2024-04-12T05:55:53.118 INFO:tasks.workunit.client.1.smithi194.stdout:9/10: fdatasync f0 0 2024-04-12T05:55:53.118 INFO:tasks.workunit.client.1.smithi194.stdout:9/11: readlink d1/l2 0 2024-04-12T05:55:53.120 INFO:tasks.workunit.client.1.smithi194.stdout:6/13: creat d3/f4 x:0 0 0 2024-04-12T05:55:53.122 INFO:tasks.workunit.client.1.smithi194.stdout:9/12: rename d1/l2 to d1/l3 0 2024-04-12T05:55:53.125 INFO:tasks.workunit.client.1.smithi194.stdout:9/13: symlink d1/l4 0 2024-04-12T05:55:53.129 INFO:tasks.workunit.client.1.smithi194.stdout:9/14: dread f0 [0,4194304] 0 2024-04-12T05:55:53.130 INFO:tasks.workunit.client.1.smithi194.stdout:6/14: dwrite d3/f4 [0,4194304] 0 2024-04-12T05:55:53.133 INFO:tasks.workunit.client.1.smithi194.stdout:9/15: creat d1/f5 x:0 0 0 2024-04-12T05:55:53.135 INFO:tasks.workunit.client.1.smithi194.stdout:6/15: mkdir d3/d5 0 2024-04-12T05:55:53.138 INFO:tasks.workunit.client.1.smithi194.stdout:6/16: mknod d3/d5/c6 0 2024-04-12T05:55:53.140 INFO:tasks.workunit.client.1.smithi194.stdout:6/17: rmdir d3/d5 39 2024-04-12T05:55:53.142 INFO:tasks.workunit.client.1.smithi194.stdout:9/16: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:55:53.143 INFO:tasks.workunit.client.1.smithi194.stdout:9/17: write d1/f5 [2085580,19750] 0 2024-04-12T05:55:53.145 INFO:tasks.workunit.client.1.smithi194.stdout:6/18: unlink d3/f4 0 2024-04-12T05:55:53.147 INFO:tasks.workunit.client.1.smithi194.stdout:9/18: mknod d1/c6 0 2024-04-12T05:55:53.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/19: symlink d3/l7 0 2024-04-12T05:55:53.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/20: fsync - no filename 2024-04-12T05:55:53.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/21: dread - no filename 2024-04-12T05:55:53.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/22: dread - no filename 2024-04-12T05:55:53.191 INFO:tasks.workunit.client.1.smithi194.stdout:1/24: write d0/f2 [4365705,107142] 0 2024-04-12T05:55:53.194 INFO:tasks.workunit.client.1.smithi194.stdout:1/25: fsync d0/f2 0 2024-04-12T05:55:53.199 INFO:tasks.workunit.client.1.smithi194.stdout:3/10: getdents d0 0 2024-04-12T05:55:53.200 INFO:tasks.workunit.client.1.smithi194.stdout:3/11: write - no filename 2024-04-12T05:55:53.200 INFO:tasks.workunit.client.1.smithi194.stdout:3/12: write - no filename 2024-04-12T05:55:53.200 INFO:tasks.workunit.client.1.smithi194.stdout:3/13: rename d0/d1 to d0/d1/d4 22 2024-04-12T05:55:53.201 INFO:tasks.workunit.client.1.smithi194.stdout:8/3: getdents . 0 2024-04-12T05:55:53.202 INFO:tasks.workunit.client.1.smithi194.stdout:8/4: truncate f1 83069 0 2024-04-12T05:55:53.203 INFO:tasks.workunit.client.1.smithi194.stdout:5/11: getdents d1 0 2024-04-12T05:55:53.204 INFO:tasks.workunit.client.1.smithi194.stdout:5/12: truncate d1/d4/f5 29475 0 2024-04-12T05:55:53.205 INFO:tasks.workunit.client.1.smithi194.stdout:8/5: mkdir d2 0 2024-04-12T05:55:53.205 INFO:tasks.workunit.client.1.smithi194.stdout:3/14: link d0/d1/l2 d0/d1/l5 0 2024-04-12T05:55:53.206 INFO:tasks.workunit.client.1.smithi194.stdout:3/15: write - no filename 2024-04-12T05:55:53.206 INFO:tasks.workunit.client.1.smithi194.stdout:3/16: write - no filename 2024-04-12T05:55:53.206 INFO:tasks.workunit.client.1.smithi194.stdout:5/13: dread d1/f3 [0,4194304] 0 2024-04-12T05:55:53.206 INFO:tasks.workunit.client.1.smithi194.stdout:5/14: fdatasync f0 0 2024-04-12T05:55:53.209 INFO:tasks.workunit.client.1.smithi194.stdout:8/6: dread f1 [0,4194304] 0 2024-04-12T05:55:53.210 INFO:tasks.workunit.client.1.smithi194.stdout:8/7: readlink l0 0 2024-04-12T05:55:53.210 INFO:tasks.workunit.client.1.smithi194.stdout:8/8: dread f1 [0,4194304] 0 2024-04-12T05:55:53.210 INFO:tasks.workunit.client.1.smithi194.stdout:1/26: rmdir d0 39 2024-04-12T05:55:53.211 INFO:tasks.workunit.client.1.smithi194.stdout:3/17: mknod d0/d1/c6 0 2024-04-12T05:55:53.212 INFO:tasks.workunit.client.1.smithi194.stdout:9/19: truncate f0 580939 0 2024-04-12T05:55:53.214 INFO:tasks.workunit.client.1.smithi194.stdout:8/9: unlink f1 0 2024-04-12T05:55:53.214 INFO:tasks.workunit.client.1.smithi194.stdout:8/10: write - no filename 2024-04-12T05:55:53.214 INFO:tasks.workunit.client.1.smithi194.stdout:8/11: fsync - no filename 2024-04-12T05:55:53.215 INFO:tasks.workunit.client.1.smithi194.stdout:8/12: chown l0 12 1 2024-04-12T05:55:53.215 INFO:tasks.workunit.client.1.smithi194.stdout:8/13: chown l0 13515 1 2024-04-12T05:55:53.216 INFO:tasks.workunit.client.1.smithi194.stdout:1/27: write d0/f2 [3334360,33135] 0 2024-04-12T05:55:53.216 INFO:tasks.workunit.client.1.smithi194.stdout:3/18: symlink d0/l7 0 2024-04-12T05:55:53.218 INFO:tasks.workunit.client.1.smithi194.stdout:1/28: write d0/f2 [5130759,62964] 0 2024-04-12T05:55:53.220 INFO:tasks.workunit.client.1.smithi194.stdout:3/19: symlink d0/d1/l8 0 2024-04-12T05:55:53.221 INFO:tasks.workunit.client.1.smithi194.stdout:8/14: symlink d2/l3 0 2024-04-12T05:55:53.221 INFO:tasks.workunit.client.1.smithi194.stdout:8/15: write - no filename 2024-04-12T05:55:53.221 INFO:tasks.workunit.client.1.smithi194.stdout:8/16: fsync - no filename 2024-04-12T05:55:53.222 INFO:tasks.workunit.client.1.smithi194.stdout:1/29: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.224 INFO:tasks.workunit.client.1.smithi194.stdout:3/20: creat d0/d1/f9 x:0 0 0 2024-04-12T05:55:53.226 INFO:tasks.workunit.client.1.smithi194.stdout:8/17: mknod d2/c4 0 2024-04-12T05:55:53.229 INFO:tasks.workunit.client.1.smithi194.stdout:6/23: creat d3/d5/f8 x:0 0 0 2024-04-12T05:55:53.230 INFO:tasks.workunit.client.1.smithi194.stdout:8/18: mknod d2/c5 0 2024-04-12T05:55:53.230 INFO:tasks.workunit.client.1.smithi194.stdout:8/19: dwrite - no filename 2024-04-12T05:55:53.233 INFO:tasks.workunit.client.1.smithi194.stdout:1/30: dwrite d0/f2 [0,4194304] 0 2024-04-12T05:55:53.237 INFO:tasks.workunit.client.1.smithi194.stdout:8/20: rename l0 to d2/l6 0 2024-04-12T05:55:53.239 INFO:tasks.workunit.client.1.smithi194.stdout:1/31: creat d0/f5 x:0 0 0 2024-04-12T05:55:53.241 INFO:tasks.workunit.client.1.smithi194.stdout:3/21: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.244 INFO:tasks.workunit.client.1.smithi194.stdout:8/21: mknod d2/c7 0 2024-04-12T05:55:53.244 INFO:tasks.workunit.client.1.smithi194.stdout:1/32: chown d0/f2 4802 1 2024-04-12T05:55:53.245 INFO:tasks.workunit.client.1.smithi194.stdout:1/33: dread - d0/f5 zero size 2024-04-12T05:55:53.249 INFO:tasks.workunit.client.1.smithi194.stdout:6/24: getdents d3 0 2024-04-12T05:55:53.250 INFO:tasks.workunit.client.1.smithi194.stdout:3/22: symlink d0/d1/la 0 2024-04-12T05:55:53.251 INFO:tasks.workunit.client.1.smithi194.stdout:8/22: mkdir d2/d8 0 2024-04-12T05:55:53.252 INFO:tasks.workunit.client.1.smithi194.stdout:5/15: dwrite d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:53.253 INFO:tasks.workunit.client.1.smithi194.stdout:8/23: chown d2/c4 2 1 2024-04-12T05:55:53.253 INFO:tasks.workunit.client.1.smithi194.stdout:8/24: dread - no filename 2024-04-12T05:55:53.253 INFO:tasks.workunit.client.1.smithi194.stdout:8/25: write - no filename 2024-04-12T05:55:53.254 INFO:tasks.workunit.client.1.smithi194.stdout:8/26: chown d2/d8 108 1 2024-04-12T05:55:53.255 INFO:tasks.workunit.client.1.smithi194.stdout:6/25: creat d3/d5/f9 x:0 0 0 2024-04-12T05:55:53.260 INFO:tasks.workunit.client.1.smithi194.stdout:6/26: chown c1 2193 1 2024-04-12T05:55:53.261 INFO:tasks.workunit.client.1.smithi194.stdout:5/16: mkdir d1/d4/d6 0 2024-04-12T05:55:53.263 INFO:tasks.workunit.client.1.smithi194.stdout:3/23: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.263 INFO:tasks.workunit.client.1.smithi194.stdout:8/27: mkdir d2/d8/d9 0 2024-04-12T05:55:53.265 INFO:tasks.workunit.client.1.smithi194.stdout:5/17: symlink d1/l7 0 2024-04-12T05:55:53.272 INFO:tasks.workunit.client.1.smithi194.stdout:8/28: mkdir d2/d8/da 0 2024-04-12T05:55:53.275 INFO:tasks.workunit.client.1.smithi194.stdout:5/18: getdents d1/d4/d6 0 2024-04-12T05:55:53.275 INFO:tasks.workunit.client.1.smithi194.stdout:8/29: stat d2/c4 0 2024-04-12T05:55:53.275 INFO:tasks.workunit.client.1.smithi194.stdout:8/30: dread - no filename 2024-04-12T05:55:53.275 INFO:tasks.workunit.client.1.smithi194.stdout:1/34: write d0/f4 [905197,35294] 0 2024-04-12T05:55:53.277 INFO:tasks.workunit.client.1.smithi194.stdout:6/27: dwrite d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.278 INFO:tasks.workunit.client.1.smithi194.stdout:5/19: mkdir d1/d8 0 2024-04-12T05:55:53.279 INFO:tasks.workunit.client.1.smithi194.stdout:3/24: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.280 INFO:tasks.workunit.client.1.smithi194.stdout:8/31: creat d2/fb x:0 0 0 2024-04-12T05:55:53.282 INFO:tasks.workunit.client.1.smithi194.stdout:1/35: rename d0/f5 to d0/f6 0 2024-04-12T05:55:53.284 INFO:tasks.workunit.client.1.smithi194.stdout:8/32: mknod d2/cc 0 2024-04-12T05:55:53.287 INFO:tasks.workunit.client.1.smithi194.stdout:5/20: rename d1 to d1/d4/d6/d9 22 2024-04-12T05:55:53.288 INFO:tasks.workunit.client.1.smithi194.stdout:3/25: chown d0/d1/l5 532474 1 2024-04-12T05:55:53.292 INFO:tasks.workunit.client.1.smithi194.stdout:8/33: mkdir d2/dd 0 2024-04-12T05:55:53.294 INFO:tasks.workunit.client.1.smithi194.stdout:3/26: creat d0/fb x:0 0 0 2024-04-12T05:55:53.298 INFO:tasks.workunit.client.1.smithi194.stdout:8/34: mkdir d2/d8/d9/de 0 2024-04-12T05:55:53.300 INFO:tasks.workunit.client.1.smithi194.stdout:6/28: dwrite d3/d5/f8 [0,4194304] 0 2024-04-12T05:55:53.303 INFO:tasks.workunit.client.1.smithi194.stdout:6/29: write d3/d5/f8 [2721297,118672] 0 2024-04-12T05:55:53.308 INFO:tasks.workunit.client.1.smithi194.stdout:1/36: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.309 INFO:tasks.workunit.client.1.smithi194.stdout:8/35: truncate d2/fb 699836 0 2024-04-12T05:55:53.309 INFO:tasks.workunit.client.1.smithi194.stdout:8/36: chown d2/c7 350 1 2024-04-12T05:55:53.312 INFO:tasks.workunit.client.1.smithi194.stdout:1/37: stat d0/f2 0 2024-04-12T05:55:53.314 INFO:tasks.workunit.client.1.smithi194.stdout:5/21: dwrite d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:53.315 INFO:tasks.workunit.client.1.smithi194.stdout:5/22: readlink d1/l2 0 2024-04-12T05:55:53.317 INFO:tasks.workunit.client.1.smithi194.stdout:1/38: rmdir d0 39 2024-04-12T05:55:53.320 INFO:tasks.workunit.client.1.smithi194.stdout:3/27: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.321 INFO:tasks.workunit.client.1.smithi194.stdout:8/37: link d2/cc d2/d8/cf 0 2024-04-12T05:55:53.322 INFO:tasks.workunit.client.1.smithi194.stdout:1/39: symlink d0/l7 0 2024-04-12T05:55:53.328 INFO:tasks.workunit.client.1.smithi194.stdout:3/28: mkdir d0/dc 0 2024-04-12T05:55:53.328 INFO:tasks.workunit.client.1.smithi194.stdout:6/30: truncate d3/d5/f9 1436156 0 2024-04-12T05:55:53.329 INFO:tasks.workunit.client.1.smithi194.stdout:1/40: fsync d0/f6 0 2024-04-12T05:55:53.330 INFO:tasks.workunit.client.1.smithi194.stdout:8/38: mkdir d2/d8/d9/de/d10 0 2024-04-12T05:55:53.331 INFO:tasks.workunit.client.1.smithi194.stdout:1/41: truncate d0/f6 715602 0 2024-04-12T05:55:53.332 INFO:tasks.workunit.client.1.smithi194.stdout:3/29: creat d0/d1/fd x:0 0 0 2024-04-12T05:55:53.333 INFO:tasks.workunit.client.1.smithi194.stdout:5/23: dread d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:53.336 INFO:tasks.workunit.client.1.smithi194.stdout:5/24: creat d1/fa x:0 0 0 2024-04-12T05:55:53.337 INFO:tasks.workunit.client.1.smithi194.stdout:8/39: dread d2/fb [0,4194304] 0 2024-04-12T05:55:53.342 INFO:tasks.workunit.client.1.smithi194.stdout:1/42: dwrite d0/f2 [0,4194304] 0 2024-04-12T05:55:53.342 INFO:tasks.workunit.client.1.smithi194.stdout:8/40: dread d2/fb [0,4194304] 0 2024-04-12T05:55:53.348 INFO:tasks.workunit.client.1.smithi194.stdout:6/31: dread d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.348 INFO:tasks.workunit.client.1.smithi194.stdout:8/41: write d2/fb [171436,8171] 0 2024-04-12T05:55:53.349 INFO:tasks.workunit.client.1.smithi194.stdout:3/30: dwrite d0/fb [0,4194304] 0 2024-04-12T05:55:53.351 INFO:tasks.workunit.client.1.smithi194.stdout:1/43: rename d0/f2 to d0/f8 0 2024-04-12T05:55:53.352 INFO:tasks.workunit.client.1.smithi194.stdout:5/25: dread d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:53.353 INFO:tasks.workunit.client.1.smithi194.stdout:3/31: chown d0/l3 966753 1 2024-04-12T05:55:53.354 INFO:tasks.workunit.client.1.smithi194.stdout:6/32: write d3/d5/f8 [300360,95047] 0 2024-04-12T05:55:53.356 INFO:tasks.workunit.client.1.smithi194.stdout:8/42: rename d2/l3 to d2/d8/l11 0 2024-04-12T05:55:53.358 INFO:tasks.workunit.client.1.smithi194.stdout:5/26: stat d1/f3 0 2024-04-12T05:55:53.360 INFO:tasks.workunit.client.1.smithi194.stdout:3/32: mkdir d0/dc/de 0 2024-04-12T05:55:53.361 INFO:tasks.workunit.client.1.smithi194.stdout:8/43: dread d2/fb [0,4194304] 0 2024-04-12T05:55:53.361 INFO:tasks.workunit.client.1.smithi194.stdout:5/27: write f0 [342903,73140] 0 2024-04-12T05:55:53.364 INFO:tasks.workunit.client.1.smithi194.stdout:3/33: truncate d0/d1/f9 4869141 0 2024-04-12T05:55:53.364 INFO:tasks.workunit.client.1.smithi194.stdout:1/44: dread d0/f8 [0,4194304] 0 2024-04-12T05:55:53.365 INFO:tasks.workunit.client.1.smithi194.stdout:8/44: creat d2/f12 x:0 0 0 2024-04-12T05:55:53.366 INFO:tasks.workunit.client.1.smithi194.stdout:5/28: truncate d1/fa 541247 0 2024-04-12T05:55:53.366 INFO:tasks.workunit.client.1.smithi194.stdout:3/34: mknod d0/cf 0 2024-04-12T05:55:53.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/45: creat d0/f9 x:0 0 0 2024-04-12T05:55:53.369 INFO:tasks.workunit.client.1.smithi194.stdout:3/35: mkdir d0/d1/d10 0 2024-04-12T05:55:53.372 INFO:tasks.workunit.client.1.smithi194.stdout:6/33: dwrite d3/d5/f8 [4194304,4194304] 0 2024-04-12T05:55:53.374 INFO:tasks.workunit.client.1.smithi194.stdout:5/29: getdents d1 0 2024-04-12T05:55:53.375 INFO:tasks.workunit.client.1.smithi194.stdout:1/46: rename d0/f6 to d0/fa 0 2024-04-12T05:55:53.377 INFO:tasks.workunit.client.1.smithi194.stdout:8/45: symlink d2/d8/d9/de/d10/l13 0 2024-04-12T05:55:53.379 INFO:tasks.workunit.client.1.smithi194.stdout:8/46: write d2/f12 [497556,86972] 0 2024-04-12T05:55:53.381 INFO:tasks.workunit.client.1.smithi194.stdout:6/34: symlink d3/d5/la 0 2024-04-12T05:55:53.381 INFO:tasks.workunit.client.1.smithi194.stdout:5/30: fdatasync d1/f3 0 2024-04-12T05:55:53.383 INFO:tasks.workunit.client.1.smithi194.stdout:5/31: chown d1/d4 134647 1 2024-04-12T05:55:53.383 INFO:tasks.workunit.client.1.smithi194.stdout:3/36: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.392 INFO:tasks.workunit.client.1.smithi194.stdout:8/47: dread d2/f12 [0,4194304] 0 2024-04-12T05:55:53.392 INFO:tasks.workunit.client.1.smithi194.stdout:5/32: rename d1/d8 to d1/d4/d6/db 0 2024-04-12T05:55:53.396 INFO:tasks.workunit.client.1.smithi194.stdout:8/48: rmdir d2/d8/d9/de 39 2024-04-12T05:55:53.399 INFO:tasks.workunit.client.1.smithi194.stdout:1/47: dwrite d0/f8 [0,4194304] 0 2024-04-12T05:55:53.401 INFO:tasks.workunit.client.1.smithi194.stdout:5/33: link f0 d1/d4/d6/db/fc 0 2024-04-12T05:55:53.402 INFO:tasks.workunit.client.1.smithi194.stdout:3/37: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:53.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:53 smithi008 ceph-mon[39722]: pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 85 MiB/s wr, 388 op/s 2024-04-12T05:55:53.410 INFO:tasks.workunit.client.1.smithi194.stdout:8/49: getdents d2/d8/d9/de 0 2024-04-12T05:55:53.410 INFO:tasks.workunit.client.1.smithi194.stdout:6/35: dwrite d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.418 INFO:tasks.workunit.client.1.smithi194.stdout:1/48: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.419 INFO:tasks.workunit.client.1.smithi194.stdout:3/38: dwrite d0/fb [4194304,4194304] 0 2024-04-12T05:55:53.419 INFO:tasks.workunit.client.1.smithi194.stdout:5/34: dread f0 [0,4194304] 0 2024-04-12T05:55:53.420 INFO:tasks.workunit.client.1.smithi194.stdout:1/49: chown d0/l7 113950277 1 2024-04-12T05:55:53.421 INFO:tasks.workunit.client.1.smithi194.stdout:3/39: stat d0/d1/la 0 2024-04-12T05:55:53.421 INFO:tasks.workunit.client.1.smithi194.stdout:1/50: dread - d0/f9 zero size 2024-04-12T05:55:53.422 INFO:tasks.workunit.client.1.smithi194.stdout:1/51: chown d0/f8 15075 1 2024-04-12T05:55:53.422 INFO:tasks.workunit.client.1.smithi194.stdout:1/52: chown d0 1116 1 2024-04-12T05:55:53.426 INFO:tasks.workunit.client.1.smithi194.stdout:8/50: link d2/c5 d2/d8/da/c14 0 2024-04-12T05:55:53.427 INFO:tasks.workunit.client.1.smithi194.stdout:1/53: unlink d0/f8 0 2024-04-12T05:55:53.429 INFO:tasks.workunit.client.1.smithi194.stdout:5/35: link f0 d1/fd 0 2024-04-12T05:55:53.429 INFO:tasks.workunit.client.1.smithi194.stdout:5/36: fsync d1/fa 0 2024-04-12T05:55:53.431 INFO:tasks.workunit.client.1.smithi194.stdout:1/54: unlink d0/fa 0 2024-04-12T05:55:53.433 INFO:tasks.workunit.client.1.smithi194.stdout:8/51: unlink d2/d8/cf 0 2024-04-12T05:55:53.434 INFO:tasks.workunit.client.1.smithi194.stdout:6/36: truncate d3/d5/f9 344805 0 2024-04-12T05:55:53.434 INFO:tasks.workunit.client.1.smithi194.stdout:8/52: truncate d2/fb 1509317 0 2024-04-12T05:55:53.435 INFO:tasks.workunit.client.1.smithi194.stdout:5/37: symlink d1/le 0 2024-04-12T05:55:53.435 INFO:tasks.workunit.client.1.smithi194.stdout:6/37: rename d3/d5 to d3/d5/db 22 2024-04-12T05:55:53.436 INFO:tasks.workunit.client.1.smithi194.stdout:8/53: readlink d2/d8/d9/de/d10/l13 0 2024-04-12T05:55:53.439 INFO:tasks.workunit.client.1.smithi194.stdout:8/54: rename d2/dd to d2/d8/d15 0 2024-04-12T05:55:53.442 INFO:tasks.workunit.client.1.smithi194.stdout:8/55: creat d2/d8/d15/f16 x:0 0 0 2024-04-12T05:55:53.462 INFO:tasks.workunit.client.1.smithi194.stdout:1/55: rmdir d0 39 2024-04-12T05:55:53.463 INFO:tasks.workunit.client.1.smithi194.stdout:8/56: chown d2/c4 30 1 2024-04-12T05:55:53.463 INFO:tasks.workunit.client.1.smithi194.stdout:5/38: dwrite d1/fd [0,4194304] 0 2024-04-12T05:55:53.466 INFO:tasks.workunit.client.1.smithi194.stdout:8/57: write d2/d8/d15/f16 [192992,55540] 0 2024-04-12T05:55:53.469 INFO:tasks.workunit.client.1.smithi194.stdout:6/38: dwrite d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.477 INFO:tasks.workunit.client.1.smithi194.stdout:8/58: dwrite d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:55:53.484 INFO:tasks.workunit.client.1.smithi194.stdout:6/39: dwrite d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.486 INFO:tasks.workunit.client.1.smithi194.stdout:6/40: read d3/d5/f8 [7641989,63076] 0 2024-04-12T05:55:53.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:53 smithi194 ceph-mon[49771]: pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 85 MiB/s wr, 388 op/s 2024-04-12T05:55:53.604 INFO:tasks.workunit.client.1.smithi194.stdout:1/56: chown d0/l7 653929244 1 2024-04-12T05:55:53.606 INFO:tasks.workunit.client.1.smithi194.stdout:5/39: mkdir d1/d4/df 0 2024-04-12T05:55:53.606 INFO:tasks.workunit.client.1.smithi194.stdout:5/40: stat d1/l2 0 2024-04-12T05:55:53.609 INFO:tasks.workunit.client.1.smithi194.stdout:8/59: rename d2/d8/da to d2/d8/d15/d17 0 2024-04-12T05:55:53.611 INFO:tasks.workunit.client.1.smithi194.stdout:6/41: rmdir d3 39 2024-04-12T05:55:53.612 INFO:tasks.workunit.client.1.smithi194.stdout:1/57: unlink d0/f9 0 2024-04-12T05:55:53.615 INFO:tasks.workunit.client.1.smithi194.stdout:5/41: creat d1/d4/d6/db/f10 x:0 0 0 2024-04-12T05:55:53.615 INFO:tasks.workunit.client.1.smithi194.stdout:8/60: rmdir d2/d8/d15 39 2024-04-12T05:55:53.618 INFO:tasks.workunit.client.1.smithi194.stdout:1/58: fdatasync d0/f4 0 2024-04-12T05:55:53.622 INFO:tasks.workunit.client.1.smithi194.stdout:6/42: unlink d3/d5/la 0 2024-04-12T05:55:53.624 INFO:tasks.workunit.client.1.smithi194.stdout:8/61: unlink d2/d8/l11 0 2024-04-12T05:55:53.625 INFO:tasks.workunit.client.1.smithi194.stdout:5/42: dread f0 [0,4194304] 0 2024-04-12T05:55:53.625 INFO:tasks.workunit.client.1.smithi194.stdout:1/59: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.628 INFO:tasks.workunit.client.1.smithi194.stdout:5/43: write d1/d4/d6/db/f10 [239335,127894] 0 2024-04-12T05:55:53.629 INFO:tasks.workunit.client.1.smithi194.stdout:1/60: symlink d0/lb 0 2024-04-12T05:55:53.633 INFO:tasks.workunit.client.1.smithi194.stdout:8/62: creat d2/d8/d15/d17/f18 x:0 0 0 2024-04-12T05:55:53.633 INFO:tasks.workunit.client.1.smithi194.stdout:6/43: dread d3/d5/f8 [4194304,4194304] 0 2024-04-12T05:55:53.634 INFO:tasks.workunit.client.1.smithi194.stdout:1/61: symlink d0/lc 0 2024-04-12T05:55:53.638 INFO:tasks.workunit.client.1.smithi194.stdout:6/44: fsync d3/d5/f8 0 2024-04-12T05:55:53.643 INFO:tasks.workunit.client.1.smithi194.stdout:8/63: dwrite d2/fb [0,4194304] 0 2024-04-12T05:55:53.643 INFO:tasks.workunit.client.1.smithi194.stdout:1/62: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.645 INFO:tasks.workunit.client.1.smithi194.stdout:6/45: rename d3/l7 to d3/lc 0 2024-04-12T05:55:53.646 INFO:tasks.workunit.client.1.smithi194.stdout:8/64: mknod d2/d8/d9/de/d10/c19 0 2024-04-12T05:55:53.647 INFO:tasks.workunit.client.1.smithi194.stdout:6/46: chown d3/d5 1 1 2024-04-12T05:55:53.648 INFO:tasks.workunit.client.1.smithi194.stdout:1/63: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.649 INFO:tasks.workunit.client.1.smithi194.stdout:8/65: mkdir d2/d8/d9/d1a 0 2024-04-12T05:55:53.652 INFO:tasks.workunit.client.1.smithi194.stdout:8/66: readlink d2/l6 0 2024-04-12T05:55:53.654 INFO:tasks.workunit.client.1.smithi194.stdout:6/47: dread d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.654 INFO:tasks.workunit.client.1.smithi194.stdout:1/64: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.658 INFO:tasks.workunit.client.1.smithi194.stdout:1/65: unlink d0/l7 0 2024-04-12T05:55:53.661 INFO:tasks.workunit.client.1.smithi194.stdout:1/66: mkdir d0/dd 0 2024-04-12T05:55:53.662 INFO:tasks.workunit.client.1.smithi194.stdout:1/67: chown d0 0 1 2024-04-12T05:55:53.662 INFO:tasks.workunit.client.1.smithi194.stdout:8/67: dwrite d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:55:53.665 INFO:tasks.workunit.client.1.smithi194.stdout:6/48: dwrite d3/d5/f8 [4194304,4194304] 0 2024-04-12T05:55:53.667 INFO:tasks.workunit.client.1.smithi194.stdout:1/68: rename d0/lb to d0/le 0 2024-04-12T05:55:53.668 INFO:tasks.workunit.client.1.smithi194.stdout:1/69: truncate d0/f4 4759833 0 2024-04-12T05:55:53.669 INFO:tasks.workunit.client.1.smithi194.stdout:8/68: creat d2/d8/d15/d17/f1b x:0 0 0 2024-04-12T05:55:53.673 INFO:tasks.workunit.client.1.smithi194.stdout:6/49: mknod d3/cd 0 2024-04-12T05:55:53.673 INFO:tasks.workunit.client.1.smithi194.stdout:8/69: truncate d2/d8/d15/d17/f18 129019 0 2024-04-12T05:55:53.673 INFO:tasks.workunit.client.1.smithi194.stdout:8/70: chown d2 488678 1 2024-04-12T05:55:53.676 INFO:tasks.workunit.client.1.smithi194.stdout:8/71: creat d2/d8/d9/de/d10/f1c x:0 0 0 2024-04-12T05:55:53.677 INFO:tasks.workunit.client.1.smithi194.stdout:8/72: chown d2/d8/d9/de 9711012 1 2024-04-12T05:55:53.719 INFO:tasks.workunit.client.1.smithi194.stdout:5/44: rmdir d1/d4/d6/db 39 2024-04-12T05:55:53.719 INFO:tasks.workunit.client.1.smithi194.stdout:5/45: stat d1 0 2024-04-12T05:55:53.723 INFO:tasks.workunit.client.1.smithi194.stdout:5/46: truncate d1/d4/d6/db/fc 4086231 0 2024-04-12T05:55:53.729 INFO:tasks.workunit.client.1.smithi194.stdout:6/50: unlink d3/lc 0 2024-04-12T05:55:53.731 INFO:tasks.workunit.client.1.smithi194.stdout:6/51: write d3/d5/f9 [436880,68822] 0 2024-04-12T05:55:53.733 INFO:tasks.workunit.client.1.smithi194.stdout:1/70: fsync d0/f4 0 2024-04-12T05:55:53.734 INFO:tasks.workunit.client.1.smithi194.stdout:5/47: fsync f0 0 2024-04-12T05:55:53.738 INFO:tasks.workunit.client.1.smithi194.stdout:5/48: rmdir d1/d4 39 2024-04-12T05:55:53.740 INFO:tasks.workunit.client.1.smithi194.stdout:5/49: stat d1/d4/d6/db/fc 0 2024-04-12T05:55:53.741 INFO:tasks.workunit.client.1.smithi194.stdout:6/52: truncate d3/d5/f8 2054345 0 2024-04-12T05:55:53.741 INFO:tasks.workunit.client.1.smithi194.stdout:1/71: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.741 INFO:tasks.workunit.client.1.smithi194.stdout:1/72: stat d0/f4 0 2024-04-12T05:55:53.745 INFO:tasks.workunit.client.1.smithi194.stdout:6/53: creat d3/fe x:0 0 0 2024-04-12T05:55:53.747 INFO:tasks.workunit.client.1.smithi194.stdout:1/73: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.747 INFO:tasks.workunit.client.1.smithi194.stdout:6/54: write d3/d5/f9 [3783702,60631] 0 2024-04-12T05:55:53.750 INFO:tasks.workunit.client.1.smithi194.stdout:6/55: rename d3/cd to d3/d5/cf 0 2024-04-12T05:55:53.752 INFO:tasks.workunit.client.1.smithi194.stdout:1/74: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.756 INFO:tasks.workunit.client.1.smithi194.stdout:6/56: dread d3/d5/f9 [0,4194304] 0 2024-04-12T05:55:53.756 INFO:tasks.workunit.client.1.smithi194.stdout:6/57: dread - d3/fe zero size 2024-04-12T05:55:53.760 INFO:tasks.workunit.client.1.smithi194.stdout:1/75: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.761 INFO:tasks.workunit.client.1.smithi194.stdout:6/58: mknod d3/d5/c10 0 2024-04-12T05:55:53.767 INFO:tasks.workunit.client.1.smithi194.stdout:6/59: rename d3/d5/c6 to d3/d5/c11 0 2024-04-12T05:55:53.770 INFO:tasks.workunit.client.1.smithi194.stdout:1/76: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.778 INFO:tasks.workunit.client.1.smithi194.stdout:1/77: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.779 INFO:tasks.workunit.client.1.smithi194.stdout:6/60: dwrite d3/fe [0,4194304] 0 2024-04-12T05:55:53.781 INFO:tasks.workunit.client.1.smithi194.stdout:1/78: mkdir d0/df 0 2024-04-12T05:55:53.782 INFO:tasks.workunit.client.1.smithi194.stdout:1/79: read d0/f4 [4361201,65400] 0 2024-04-12T05:55:53.790 INFO:tasks.workunit.client.1.smithi194.stdout:1/80: creat d0/dd/f10 x:0 0 0 2024-04-12T05:55:53.793 INFO:tasks.workunit.client.1.smithi194.stdout:1/81: getdents d0/df 0 2024-04-12T05:55:53.796 INFO:tasks.workunit.client.1.smithi194.stdout:1/82: write d0/f4 [287461,62078] 0 2024-04-12T05:55:53.799 INFO:tasks.workunit.client.1.smithi194.stdout:6/61: write d3/d5/f8 [986517,33343] 0 2024-04-12T05:55:53.800 INFO:tasks.workunit.client.1.smithi194.stdout:1/83: mkdir d0/d11 0 2024-04-12T05:55:53.802 INFO:tasks.workunit.client.1.smithi194.stdout:1/84: write d0/f4 [3340374,61300] 0 2024-04-12T05:55:53.812 INFO:tasks.workunit.client.1.smithi194.stdout:6/62: dwrite d3/d5/f9 [4194304,4194304] 0 2024-04-12T05:55:53.815 INFO:tasks.workunit.client.1.smithi194.stdout:1/85: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.816 INFO:tasks.workunit.client.1.smithi194.stdout:1/86: dread - d0/dd/f10 zero size 2024-04-12T05:55:53.817 INFO:tasks.workunit.client.1.smithi194.stdout:1/87: chown d0/df 1 1 2024-04-12T05:55:53.822 INFO:tasks.workunit.client.1.smithi194.stdout:1/88: write d0/dd/f10 [849683,84003] 0 2024-04-12T05:55:53.822 INFO:tasks.workunit.client.1.smithi194.stdout:1/89: write d0/f4 [2852895,129432] 0 2024-04-12T05:55:53.830 INFO:tasks.workunit.client.1.smithi194.stdout:1/90: rename d0/dd/f10 to d0/f12 0 2024-04-12T05:55:53.836 INFO:tasks.workunit.client.1.smithi194.stdout:1/91: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.902 INFO:tasks.workunit.client.1.smithi194.stdout:1/92: symlink d0/dd/l13 0 2024-04-12T05:55:53.910 INFO:tasks.workunit.client.1.smithi194.stdout:1/93: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:53.974 INFO:tasks.workunit.client.1.smithi194.stdout:1/94: mknod d0/d11/c14 0 2024-04-12T05:55:53.980 INFO:tasks.workunit.client.1.smithi194.stdout:1/95: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:53.982 INFO:tasks.workunit.client.1.smithi194.stdout:1/96: rename d0/lc to d0/d11/l15 0 2024-04-12T05:55:53.990 INFO:tasks.workunit.client.1.smithi194.stdout:1/97: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:53.991 INFO:tasks.workunit.client.1.smithi194.stdout:1/98: write d0/f4 [1361335,11353] 0 2024-04-12T05:55:53.993 INFO:tasks.workunit.client.1.smithi194.stdout:1/99: mknod d0/d11/c16 0 2024-04-12T05:55:54.001 INFO:tasks.workunit.client.1.smithi194.stdout:1/100: dwrite d0/f12 [4194304,4194304] 0 2024-04-12T05:55:54.010 INFO:tasks.workunit.client.1.smithi194.stdout:1/101: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:55:54.012 INFO:tasks.workunit.client.1.smithi194.stdout:1/102: fdatasync d0/f4 0 2024-04-12T05:55:54.014 INFO:tasks.workunit.client.1.smithi194.stdout:1/103: write d0/f12 [7355909,14425] 0 2024-04-12T05:55:54.020 INFO:tasks.workunit.client.1.smithi194.stdout:1/104: link d0/le d0/dd/l17 0 2024-04-12T05:55:54.024 INFO:tasks.workunit.client.1.smithi194.stdout:1/105: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:54.025 INFO:tasks.workunit.client.1.smithi194.stdout:1/106: write d0/f12 [7270292,108275] 0 2024-04-12T05:55:54.027 INFO:tasks.workunit.client.1.smithi194.stdout:1/107: write d0/f12 [2022088,24541] 0 2024-04-12T05:55:54.028 INFO:tasks.workunit.client.1.smithi194.stdout:1/108: readlink d0/d11/l15 0 2024-04-12T05:55:54.029 INFO:tasks.workunit.client.1.smithi194.stdout:1/109: readlink d0/d11/l15 0 2024-04-12T05:55:54.034 INFO:tasks.workunit.client.1.smithi194.stdout:1/110: dwrite d0/f12 [4194304,4194304] 0 2024-04-12T05:55:54.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/111: mkdir d0/d11/d18 0 2024-04-12T05:55:54.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/112: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:54.044 INFO:tasks.workunit.client.1.smithi194.stdout:1/113: mkdir d0/d11/d18/d19 0 2024-04-12T05:55:54.051 INFO:tasks.workunit.client.1.smithi194.stdout:1/114: rename d0/dd to d0/d11/d18/d19/d1a 0 2024-04-12T05:55:54.056 INFO:tasks.workunit.client.1.smithi194.stdout:1/115: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:54.057 INFO:tasks.workunit.client.1.smithi194.stdout:1/116: chown d0/d11 23778048 1 2024-04-12T05:55:54.057 INFO:tasks.workunit.client.1.smithi194.stdout:1/117: chown d0/d11/l15 7070140 1 2024-04-12T05:55:54.062 INFO:tasks.workunit.client.1.smithi194.stdout:1/118: rename d0/d11/l15 to d0/df/l1b 0 2024-04-12T05:55:54.067 INFO:tasks.workunit.client.1.smithi194.stdout:1/119: dread d0/f12 [4194304,4194304] 0 2024-04-12T05:55:54.070 INFO:tasks.workunit.client.1.smithi194.stdout:1/120: dread d0/f12 [0,4194304] 0 2024-04-12T05:55:54.077 INFO:tasks.workunit.client.1.smithi194.stdout:1/121: dwrite d0/f12 [4194304,4194304] 0 2024-04-12T05:55:54.078 INFO:tasks.workunit.client.1.smithi194.stdout:1/122: fsync d0/f4 0 2024-04-12T05:55:54.079 INFO:tasks.workunit.client.1.smithi194.stdout:1/123: fdatasync d0/f4 0 2024-04-12T05:55:54.083 INFO:tasks.workunit.client.1.smithi194.stdout:1/124: readlink d0/d11/d18/d19/d1a/l17 0 2024-04-12T05:55:54.084 INFO:tasks.workunit.client.1.smithi194.stdout:1/125: mkdir d0/d11/d18/d19/d1c 0 2024-04-12T05:55:54.085 INFO:tasks.workunit.client.1.smithi194.stdout:1/126: chown d0/d11/d18/d19/d1a/l13 1 1 2024-04-12T05:55:54.087 INFO:tasks.workunit.client.1.smithi194.stdout:1/127: mknod d0/d11/d18/d19/d1c/c1d 0 2024-04-12T05:55:54.092 INFO:tasks.workunit.client.1.smithi194.stdout:1/128: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:54.097 INFO:tasks.workunit.client.1.smithi194.stdout:1/129: dread d0/f12 [4194304,4194304] 0 2024-04-12T05:55:54.099 INFO:tasks.workunit.client.1.smithi194.stdout:1/130: creat d0/df/f1e x:0 0 0 2024-04-12T05:55:54.101 INFO:tasks.workunit.client.1.smithi194.stdout:1/131: creat d0/d11/d18/d19/f1f x:0 0 0 2024-04-12T05:55:55.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:55 smithi008 ceph-mon[39722]: pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 79 MiB/s wr, 369 op/s 2024-04-12T05:55:55.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:55 smithi194 ceph-mon[49771]: pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 32 MiB/s rd, 79 MiB/s wr, 369 op/s 2024-04-12T05:55:57.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: pgmap v24: 65 pgs: 65 active+clean; 1.5 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 95 MiB/s wr, 493 op/s 2024-04-12T05:55:57.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: Upgrade: Need to upgrade myself (mgr.smithi194.xbiyaj) 2024-04-12T05:55:57.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:56 smithi008 ceph-mon[39722]: Upgrade: Need to upgrade myself (mgr.smithi194.xbiyaj) 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: pgmap v24: 65 pgs: 65 active+clean; 1.5 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 95 MiB/s wr, 493 op/s 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: Upgrade: Need to upgrade myself (mgr.smithi194.xbiyaj) 2024-04-12T05:55:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:56 smithi194 ceph-mon[49771]: Upgrade: Need to upgrade myself (mgr.smithi194.xbiyaj) 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/6: sync 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:5/50: sync 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/7: read - no filename 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:4/15: sync 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/8: dwrite - no filename 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/9: rename - no filename 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/10: rmdir - no directory 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/11: dread - no filename 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/12: dwrite - no filename 2024-04-12T05:55:57.673 INFO:tasks.workunit.client.1.smithi194.stdout:0/13: dwrite - no filename 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:0/14: unlink - no file 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:0/15: rename - no filename 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:7/1: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:9/20: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:3/40: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:2/5: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:2/6: readlink - no filename 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:1/132: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:6/63: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:6/64: readlink - no filename 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:8/73: sync 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:0/16: chown . 12180119 1 2024-04-12T05:55:57.674 INFO:tasks.workunit.client.1.smithi194.stdout:0/17: dwrite - no filename 2024-04-12T05:55:57.675 INFO:tasks.workunit.client.1.smithi194.stdout:5/51: chown d1/f3 227 1 2024-04-12T05:55:57.675 INFO:tasks.workunit.client.1.smithi194.stdout:6/65: write d3/fe [1678744,70566] 0 2024-04-12T05:55:57.676 INFO:tasks.workunit.client.1.smithi194.stdout:7/2: unlink c0 0 2024-04-12T05:55:57.676 INFO:tasks.workunit.client.1.smithi194.stdout:7/3: truncate - no filename 2024-04-12T05:55:57.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/18: mkdir d0 0 2024-04-12T05:55:57.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/19: fdatasync - no filename 2024-04-12T05:55:57.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/20: dwrite - no filename 2024-04-12T05:55:57.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/21: truncate - no filename 2024-04-12T05:55:57.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/22: dwrite - no filename 2024-04-12T05:55:57.680 INFO:tasks.workunit.client.1.smithi194.stdout:3/41: stat d0/d1/l8 0 2024-04-12T05:55:57.680 INFO:tasks.workunit.client.1.smithi194.stdout:9/21: dread d1/f5 [0,4194304] 0 2024-04-12T05:55:57.681 INFO:tasks.workunit.client.1.smithi194.stdout:4/16: creat d2/f6 x:0 0 0 2024-04-12T05:55:57.683 INFO:tasks.workunit.client.1.smithi194.stdout:8/74: creat d2/d8/d9/de/f1d x:0 0 0 2024-04-12T05:55:57.684 INFO:tasks.workunit.client.1.smithi194.stdout:6/66: dwrite d3/fe [0,4194304] 0 2024-04-12T05:55:57.685 INFO:tasks.workunit.client.1.smithi194.stdout:6/67: fdatasync d3/d5/f8 0 2024-04-12T05:55:57.686 INFO:tasks.workunit.client.1.smithi194.stdout:1/133: creat d0/d11/d18/d19/d1c/f20 x:0 0 0 2024-04-12T05:55:57.692 INFO:tasks.workunit.client.1.smithi194.stdout:4/17: symlink d2/l7 0 2024-04-12T05:55:57.693 INFO:tasks.workunit.client.1.smithi194.stdout:9/22: mknod d1/c7 0 2024-04-12T05:55:57.697 INFO:tasks.workunit.client.1.smithi194.stdout:6/68: mkdir d3/d5/d12 0 2024-04-12T05:55:57.698 INFO:tasks.workunit.client.1.smithi194.stdout:1/134: creat d0/d11/f21 x:0 0 0 2024-04-12T05:55:57.699 INFO:tasks.workunit.client.1.smithi194.stdout:0/23: creat d0/f1 x:0 0 0 2024-04-12T05:55:57.700 INFO:tasks.workunit.client.1.smithi194.stdout:0/24: dread - d0/f1 zero size 2024-04-12T05:55:57.701 INFO:tasks.workunit.client.1.smithi194.stdout:5/52: creat d1/f11 x:0 0 0 2024-04-12T05:55:57.702 INFO:tasks.workunit.client.1.smithi194.stdout:5/53: write d1/d4/f5 [786129,38699] 0 2024-04-12T05:55:57.702 INFO:tasks.workunit.client.1.smithi194.stdout:2/7: link c1 c2 0 2024-04-12T05:55:57.703 INFO:tasks.workunit.client.1.smithi194.stdout:5/54: fdatasync d1/f11 0 2024-04-12T05:55:57.705 INFO:tasks.workunit.client.1.smithi194.stdout:8/75: rename d2/cc to d2/d8/d9/de/c1e 0 2024-04-12T05:55:57.706 INFO:tasks.workunit.client.1.smithi194.stdout:8/76: write d2/fb [425840,85176] 0 2024-04-12T05:55:57.707 INFO:tasks.workunit.client.1.smithi194.stdout:1/135: creat d0/d11/d18/d19/d1c/f22 x:0 0 0 2024-04-12T05:55:57.709 INFO:tasks.workunit.client.1.smithi194.stdout:0/25: unlink d0/f1 0 2024-04-12T05:55:57.710 INFO:tasks.workunit.client.1.smithi194.stdout:0/26: readlink - no filename 2024-04-12T05:55:57.710 INFO:tasks.workunit.client.1.smithi194.stdout:0/27: dwrite - no filename 2024-04-12T05:55:57.710 INFO:tasks.workunit.client.1.smithi194.stdout:4/18: symlink d2/d3/l8 0 2024-04-12T05:55:57.712 INFO:tasks.workunit.client.1.smithi194.stdout:2/8: creat f3 x:0 0 0 2024-04-12T05:55:57.713 INFO:tasks.workunit.client.1.smithi194.stdout:2/9: dread - f3 zero size 2024-04-12T05:55:57.713 INFO:tasks.workunit.client.1.smithi194.stdout:9/23: rename d1/l3 to d1/l8 0 2024-04-12T05:55:57.717 INFO:tasks.workunit.client.1.smithi194.stdout:1/136: dwrite d0/d11/f21 [0,4194304] 0 2024-04-12T05:55:57.718 INFO:tasks.workunit.client.1.smithi194.stdout:1/137: chown d0/d11/d18/d19/d1c/f22 998820155 1 2024-04-12T05:55:57.719 INFO:tasks.workunit.client.1.smithi194.stdout:0/28: creat d0/f2 x:0 0 0 2024-04-12T05:55:57.720 INFO:tasks.workunit.client.1.smithi194.stdout:4/19: creat d2/f9 x:0 0 0 2024-04-12T05:55:57.723 INFO:tasks.workunit.client.1.smithi194.stdout:9/24: fsync d1/f5 0 2024-04-12T05:55:57.723 INFO:tasks.workunit.client.1.smithi194.stdout:9/25: fsync d1/f5 0 2024-04-12T05:55:57.725 INFO:tasks.workunit.client.1.smithi194.stdout:2/10: rename f3 to f4 0 2024-04-12T05:55:57.728 INFO:tasks.workunit.client.1.smithi194.stdout:1/138: fsync d0/f12 0 2024-04-12T05:55:57.729 INFO:tasks.workunit.client.1.smithi194.stdout:0/29: mknod d0/c3 0 2024-04-12T05:55:57.730 INFO:tasks.workunit.client.1.smithi194.stdout:1/139: write d0/f12 [5667156,99667] 0 2024-04-12T05:55:57.731 INFO:tasks.workunit.client.1.smithi194.stdout:4/20: symlink d2/d3/la 0 2024-04-12T05:55:57.733 INFO:tasks.workunit.client.1.smithi194.stdout:9/26: rename d1/l4 to d1/l9 0 2024-04-12T05:55:57.735 INFO:tasks.workunit.client.1.smithi194.stdout:6/69: getdents d3 0 2024-04-12T05:55:57.737 INFO:tasks.workunit.client.1.smithi194.stdout:6/70: write d3/d5/f8 [442372,118269] 0 2024-04-12T05:55:57.740 INFO:tasks.workunit.client.1.smithi194.stdout:1/140: mkdir d0/d11/d18/d19/d1a/d23 0 2024-04-12T05:55:57.741 INFO:tasks.workunit.client.1.smithi194.stdout:9/27: symlink d1/la 0 2024-04-12T05:55:57.742 INFO:tasks.workunit.client.1.smithi194.stdout:2/11: chown c2 25 1 2024-04-12T05:55:57.742 INFO:tasks.workunit.client.1.smithi194.stdout:6/71: dread d3/d5/f9 [4194304,4194304] 0 2024-04-12T05:55:57.743 INFO:tasks.workunit.client.1.smithi194.stdout:2/12: dread - f4 zero size 2024-04-12T05:55:57.743 INFO:tasks.workunit.client.1.smithi194.stdout:6/72: stat d3/fe 0 2024-04-12T05:55:57.745 INFO:tasks.workunit.client.1.smithi194.stdout:1/141: chown d0/df/l1b 125173284 1 2024-04-12T05:55:57.746 INFO:tasks.workunit.client.1.smithi194.stdout:9/28: creat d1/fb x:0 0 0 2024-04-12T05:55:57.746 INFO:tasks.workunit.client.1.smithi194.stdout:9/29: readlink d1/la 0 2024-04-12T05:55:57.750 INFO:tasks.workunit.client.1.smithi194.stdout:6/73: rename d3/d5/f9 to d3/d5/f13 0 2024-04-12T05:55:57.750 INFO:tasks.workunit.client.1.smithi194.stdout:1/142: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:57.755 INFO:tasks.workunit.client.1.smithi194.stdout:1/143: dread - d0/d11/d18/d19/f1f zero size 2024-04-12T05:55:57.757 INFO:tasks.workunit.client.1.smithi194.stdout:2/13: dwrite f4 [0,4194304] 0 2024-04-12T05:55:57.762 INFO:tasks.workunit.client.1.smithi194.stdout:1/144: rename d0/df/f1e to d0/d11/d18/d19/d1c/f24 0 2024-04-12T05:55:57.763 INFO:tasks.workunit.client.1.smithi194.stdout:6/74: dwrite d3/d5/f13 [4194304,4194304] 0 2024-04-12T05:55:57.768 INFO:tasks.workunit.client.1.smithi194.stdout:6/75: unlink c1 0 2024-04-12T05:55:57.771 INFO:tasks.workunit.client.1.smithi194.stdout:6/76: creat d3/d5/f14 x:0 0 0 2024-04-12T05:55:57.774 INFO:tasks.workunit.client.1.smithi194.stdout:6/77: dread d3/d5/f8 [0,4194304] 0 2024-04-12T05:55:57.783 INFO:tasks.workunit.client.1.smithi194.stdout:6/78: dwrite d3/d5/f8 [0,4194304] 0 2024-04-12T05:55:57.784 INFO:tasks.workunit.client.1.smithi194.stdout:6/79: fdatasync d3/d5/f13 0 2024-04-12T05:55:57.880 INFO:tasks.workunit.client.1.smithi194.stdout:7/4: sync 2024-04-12T05:55:57.882 INFO:tasks.workunit.client.1.smithi194.stdout:7/5: symlink l1 0 2024-04-12T05:55:57.883 INFO:tasks.workunit.client.1.smithi194.stdout:7/6: write - no filename 2024-04-12T05:55:57.883 INFO:tasks.workunit.client.1.smithi194.stdout:7/7: write - no filename 2024-04-12T05:55:57.884 INFO:tasks.workunit.client.1.smithi194.stdout:7/8: stat l1 0 2024-04-12T05:55:57.887 INFO:tasks.workunit.client.1.smithi194.stdout:7/9: creat f2 x:0 0 0 2024-04-12T05:55:57.887 INFO:tasks.workunit.client.1.smithi194.stdout:7/10: readlink l1 0 2024-04-12T05:55:57.898 INFO:tasks.workunit.client.1.smithi194.stdout:5/55: getdents d1 0 2024-04-12T05:55:57.898 INFO:tasks.workunit.client.1.smithi194.stdout:8/77: getdents d2/d8/d9/de 0 2024-04-12T05:55:57.901 INFO:tasks.workunit.client.1.smithi194.stdout:8/78: truncate d2/d8/d9/de/d10/f1c 166273 0 2024-04-12T05:55:57.904 INFO:tasks.workunit.client.1.smithi194.stdout:5/56: unlink d1/d4/d6/db/f10 0 2024-04-12T05:55:57.904 INFO:tasks.workunit.client.1.smithi194.stdout:8/79: write d2/d8/d15/d17/f1b [266779,85155] 0 2024-04-12T05:55:57.913 INFO:tasks.workunit.client.1.smithi194.stdout:9/30: rename d1/l8 to d1/lc 0 2024-04-12T05:55:57.915 INFO:tasks.workunit.client.1.smithi194.stdout:0/30: rmdir d0 39 2024-04-12T05:55:57.916 INFO:tasks.workunit.client.1.smithi194.stdout:4/21: fsync d2/f9 0 2024-04-12T05:55:57.923 INFO:tasks.workunit.client.1.smithi194.stdout:5/57: dwrite d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:57.924 INFO:tasks.workunit.client.1.smithi194.stdout:9/31: mknod d1/cd 0 2024-04-12T05:55:57.925 INFO:tasks.workunit.client.1.smithi194.stdout:0/31: truncate d0/f2 222554 0 2024-04-12T05:55:57.925 INFO:tasks.workunit.client.1.smithi194.stdout:1/145: rmdir d0/d11 39 2024-04-12T05:55:57.926 INFO:tasks.workunit.client.1.smithi194.stdout:4/22: symlink d2/lb 0 2024-04-12T05:55:57.926 INFO:tasks.workunit.client.1.smithi194.stdout:6/80: rename d3/d5 to d3/d15 0 2024-04-12T05:55:57.930 INFO:tasks.workunit.client.1.smithi194.stdout:0/32: creat d0/f4 x:0 0 0 2024-04-12T05:55:57.930 INFO:tasks.workunit.client.1.smithi194.stdout:4/23: stat d2/d3 0 2024-04-12T05:55:57.931 INFO:tasks.workunit.client.1.smithi194.stdout:6/81: chown d3/d15 1218866 1 2024-04-12T05:55:57.931 INFO:tasks.workunit.client.1.smithi194.stdout:7/11: fsync f2 0 2024-04-12T05:55:57.931 INFO:tasks.workunit.client.1.smithi194.stdout:4/24: fsync d2/f9 0 2024-04-12T05:55:57.931 INFO:tasks.workunit.client.1.smithi194.stdout:9/32: creat d1/fe x:0 0 0 2024-04-12T05:55:57.933 INFO:tasks.workunit.client.1.smithi194.stdout:9/33: chown d1/f5 35757 1 2024-04-12T05:55:57.934 INFO:tasks.workunit.client.1.smithi194.stdout:7/12: dread - f2 zero size 2024-04-12T05:55:57.935 INFO:tasks.workunit.client.1.smithi194.stdout:9/34: write d1/fb [181256,60018] 0 2024-04-12T05:55:57.936 INFO:tasks.workunit.client.1.smithi194.stdout:0/33: mkdir d0/d5 0 2024-04-12T05:55:57.939 INFO:tasks.workunit.client.1.smithi194.stdout:1/146: write d0/d11/d18/d19/d1c/f22 [36275,118177] 0 2024-04-12T05:55:57.940 INFO:tasks.workunit.client.1.smithi194.stdout:2/14: dwrite f4 [4194304,4194304] 0 2024-04-12T05:55:57.942 INFO:tasks.workunit.client.1.smithi194.stdout:7/13: chown l1 4324 1 2024-04-12T05:55:57.943 INFO:tasks.workunit.client.1.smithi194.stdout:5/58: link d1/f3 d1/d4/d6/db/f12 0 2024-04-12T05:55:57.943 INFO:tasks.workunit.client.1.smithi194.stdout:8/80: dwrite d2/f12 [0,4194304] 0 2024-04-12T05:55:57.949 INFO:tasks.workunit.client.1.smithi194.stdout:6/82: rmdir d3/d15/d12 0 2024-04-12T05:55:57.949 INFO:tasks.workunit.client.1.smithi194.stdout:9/35: write d1/f5 [426904,30799] 0 2024-04-12T05:55:57.949 INFO:tasks.workunit.client.1.smithi194.stdout:7/14: chown l1 1 1 2024-04-12T05:55:57.951 INFO:tasks.workunit.client.1.smithi194.stdout:8/81: write d2/d8/d15/d17/f18 [465733,94492] 0 2024-04-12T05:55:57.955 INFO:tasks.workunit.client.1.smithi194.stdout:4/25: creat d2/fc x:0 0 0 2024-04-12T05:55:57.955 INFO:tasks.workunit.client.1.smithi194.stdout:2/15: rename f4 to f5 0 2024-04-12T05:55:57.956 INFO:tasks.workunit.client.1.smithi194.stdout:4/26: dread - d2/fc zero size 2024-04-12T05:55:57.956 INFO:tasks.workunit.client.1.smithi194.stdout:1/147: fdatasync d0/d11/d18/d19/d1c/f24 0 2024-04-12T05:55:57.957 INFO:tasks.workunit.client.1.smithi194.stdout:4/27: fdatasync f0 0 2024-04-12T05:55:57.958 INFO:tasks.workunit.client.1.smithi194.stdout:4/28: write d2/f9 [210690,21983] 0 2024-04-12T05:55:57.960 INFO:tasks.workunit.client.1.smithi194.stdout:1/148: write d0/d11/d18/d19/f1f [789083,105071] 0 2024-04-12T05:55:57.960 INFO:tasks.workunit.client.1.smithi194.stdout:9/36: unlink d1/cd 0 2024-04-12T05:55:57.964 INFO:tasks.workunit.client.1.smithi194.stdout:1/149: write d0/d11/f21 [4281414,118404] 0 2024-04-12T05:55:57.967 INFO:tasks.workunit.client.1.smithi194.stdout:0/34: rmdir d0 39 2024-04-12T05:55:57.967 INFO:tasks.workunit.client.1.smithi194.stdout:2/16: write f5 [4521896,473] 0 2024-04-12T05:55:57.968 INFO:tasks.workunit.client.1.smithi194.stdout:1/150: stat d0/d11/d18/d19/d1a 0 2024-04-12T05:55:57.969 INFO:tasks.workunit.client.1.smithi194.stdout:8/82: dread d2/d8/d15/d17/f18 [0,4194304] 0 2024-04-12T05:55:57.973 INFO:tasks.workunit.client.1.smithi194.stdout:1/151: write d0/d11/d18/d19/f1f [76853,88179] 0 2024-04-12T05:55:57.975 INFO:tasks.workunit.client.1.smithi194.stdout:5/59: creat d1/d4/d6/f13 x:0 0 0 2024-04-12T05:55:57.976 INFO:tasks.workunit.client.1.smithi194.stdout:6/83: creat d3/f16 x:0 0 0 2024-04-12T05:55:57.978 INFO:tasks.workunit.client.1.smithi194.stdout:4/29: dwrite d2/f9 [0,4194304] 0 2024-04-12T05:55:57.980 INFO:tasks.workunit.client.1.smithi194.stdout:5/60: write d1/f11 [960975,61074] 0 2024-04-12T05:55:57.983 INFO:tasks.workunit.client.1.smithi194.stdout:1/152: creat d0/df/f25 x:0 0 0 2024-04-12T05:55:57.985 INFO:tasks.workunit.client.1.smithi194.stdout:9/37: link d1/c6 d1/cf 0 2024-04-12T05:55:57.986 INFO:tasks.workunit.client.1.smithi194.stdout:2/17: dwrite f5 [0,4194304] 0 2024-04-12T05:55:57.989 INFO:tasks.workunit.client.1.smithi194.stdout:4/30: symlink d2/d3/ld 0 2024-04-12T05:55:57.989 INFO:tasks.workunit.client.1.smithi194.stdout:0/35: creat d0/d5/f6 x:0 0 0 2024-04-12T05:55:57.993 INFO:tasks.workunit.client.1.smithi194.stdout:9/38: stat d1/l9 0 2024-04-12T05:55:57.995 INFO:tasks.workunit.client.1.smithi194.stdout:4/31: mknod d2/ce 0 2024-04-12T05:55:57.998 INFO:tasks.workunit.client.1.smithi194.stdout:0/36: mknod d0/d5/c7 0 2024-04-12T05:55:58.000 INFO:tasks.workunit.client.1.smithi194.stdout:4/32: chown d2/f6 436 1 2024-04-12T05:55:58.004 INFO:tasks.workunit.client.1.smithi194.stdout:6/84: dwrite d3/f16 [0,4194304] 0 2024-04-12T05:55:58.007 INFO:tasks.workunit.client.1.smithi194.stdout:6/85: chown d3/d15 0 1 2024-04-12T05:55:58.007 INFO:tasks.workunit.client.1.smithi194.stdout:6/86: readlink - no filename 2024-04-12T05:55:58.008 INFO:tasks.workunit.client.1.smithi194.stdout:8/83: truncate d2/d8/d15/f16 2119684 0 2024-04-12T05:55:58.008 INFO:tasks.workunit.client.1.smithi194.stdout:4/33: chown d2/f9 1438967 1 2024-04-12T05:55:58.008 INFO:tasks.workunit.client.1.smithi194.stdout:1/153: dwrite d0/df/f25 [0,4194304] 0 2024-04-12T05:55:58.009 INFO:tasks.workunit.client.1.smithi194.stdout:5/61: write d1/d4/d6/db/f12 [5180528,104711] 0 2024-04-12T05:55:58.010 INFO:tasks.workunit.client.1.smithi194.stdout:2/18: dwrite f5 [4194304,4194304] 0 2024-04-12T05:55:58.013 INFO:tasks.workunit.client.1.smithi194.stdout:9/39: dread d1/f5 [0,4194304] 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:0/37: link d0/f2 d0/d5/f8 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:8/84: symlink d2/d8/d9/de/l1f 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:9/40: fsync d1/fe 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:2/19: creat f6 x:0 0 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:5/62: creat d1/d4/f14 x:0 0 0 2024-04-12T05:55:58.021 INFO:tasks.workunit.client.1.smithi194.stdout:4/34: creat d2/d3/ff x:0 0 0 2024-04-12T05:55:58.027 INFO:tasks.workunit.client.1.smithi194.stdout:0/38: rename d0/d5 to d0/d5/d9 22 2024-04-12T05:55:58.027 INFO:tasks.workunit.client.1.smithi194.stdout:1/154: creat d0/d11/f26 x:0 0 0 2024-04-12T05:55:58.027 INFO:tasks.workunit.client.1.smithi194.stdout:8/85: read d2/d8/d15/d17/f1b [111836,119505] 0 2024-04-12T05:55:58.027 INFO:tasks.workunit.client.1.smithi194.stdout:9/41: rmdir d1 39 2024-04-12T05:55:58.027 INFO:tasks.workunit.client.1.smithi194.stdout:4/35: symlink d2/d3/l10 0 2024-04-12T05:55:58.029 INFO:tasks.workunit.client.1.smithi194.stdout:0/39: mkdir d0/da 0 2024-04-12T05:55:58.029 INFO:tasks.workunit.client.1.smithi194.stdout:5/63: mkdir d1/d4/d6/db/d15 0 2024-04-12T05:55:58.034 INFO:tasks.workunit.client.1.smithi194.stdout:2/20: dwrite f6 [0,4194304] 0 2024-04-12T05:55:58.034 INFO:tasks.workunit.client.1.smithi194.stdout:2/21: readlink - no filename 2024-04-12T05:55:58.034 INFO:tasks.workunit.client.1.smithi194.stdout:0/40: truncate d0/f4 497819 0 2024-04-12T05:55:58.034 INFO:tasks.workunit.client.1.smithi194.stdout:4/36: mkdir d2/d11 0 2024-04-12T05:55:58.034 INFO:tasks.workunit.client.1.smithi194.stdout:9/42: write d1/fe [814043,53320] 0 2024-04-12T05:55:58.035 INFO:tasks.workunit.client.1.smithi194.stdout:4/37: fdatasync d2/f9 0 2024-04-12T05:55:58.036 INFO:tasks.workunit.client.1.smithi194.stdout:1/155: mkdir d0/d11/d18/d19/d1a/d23/d27 0 2024-04-12T05:55:58.038 INFO:tasks.workunit.client.1.smithi194.stdout:9/43: creat d1/f10 x:0 0 0 2024-04-12T05:55:58.043 INFO:tasks.workunit.client.1.smithi194.stdout:5/64: dread d1/f11 [0,4194304] 0 2024-04-12T05:55:58.044 INFO:tasks.workunit.client.1.smithi194.stdout:4/38: chown d2/d3/l10 19085967 1 2024-04-12T05:55:58.044 INFO:tasks.workunit.client.1.smithi194.stdout:9/44: dread - d1/f10 zero size 2024-04-12T05:55:58.047 INFO:tasks.workunit.client.1.smithi194.stdout:4/39: chown d2/d3 1133297 1 2024-04-12T05:55:58.047 INFO:tasks.workunit.client.1.smithi194.stdout:0/41: fsync d0/f2 0 2024-04-12T05:55:58.053 INFO:tasks.workunit.client.1.smithi194.stdout:6/87: truncate d3/fe 2383621 0 2024-04-12T05:55:58.055 INFO:tasks.workunit.client.1.smithi194.stdout:8/86: dwrite d2/d8/d15/d17/f1b [0,4194304] 0 2024-04-12T05:55:58.056 INFO:tasks.workunit.client.1.smithi194.stdout:4/40: creat d2/d3/f12 x:0 0 0 2024-04-12T05:55:58.057 INFO:tasks.workunit.client.1.smithi194.stdout:2/22: dwrite f6 [0,4194304] 0 2024-04-12T05:55:58.058 INFO:tasks.workunit.client.1.smithi194.stdout:2/23: readlink - no filename 2024-04-12T05:55:58.058 INFO:tasks.workunit.client.1.smithi194.stdout:6/88: read - d3/d15/f14 zero size 2024-04-12T05:55:58.062 INFO:tasks.workunit.client.1.smithi194.stdout:0/42: dread d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.063 INFO:tasks.workunit.client.1.smithi194.stdout:1/156: dread d0/d11/f21 [0,4194304] 0 2024-04-12T05:55:58.064 INFO:tasks.workunit.client.1.smithi194.stdout:4/41: write d2/d3/ff [547944,85083] 0 2024-04-12T05:55:58.066 INFO:tasks.workunit.client.1.smithi194.stdout:5/65: write d1/fd [2269424,19842] 0 2024-04-12T05:55:58.067 INFO:tasks.workunit.client.1.smithi194.stdout:9/45: dwrite d1/fe [0,4194304] 0 2024-04-12T05:55:58.067 INFO:tasks.workunit.client.1.smithi194.stdout:0/43: dread d0/f2 [0,4194304] 0 2024-04-12T05:55:58.067 INFO:tasks.workunit.client.1.smithi194.stdout:2/24: mkdir d7 0 2024-04-12T05:55:58.067 INFO:tasks.workunit.client.1.smithi194.stdout:8/87: mkdir d2/d8/d15/d17/d20 0 2024-04-12T05:55:58.072 INFO:tasks.workunit.client.1.smithi194.stdout:1/157: truncate d0/d11/f21 4560665 0 2024-04-12T05:55:58.072 INFO:tasks.workunit.client.1.smithi194.stdout:0/44: dread d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.074 INFO:tasks.workunit.client.1.smithi194.stdout:9/46: creat d1/f11 x:0 0 0 2024-04-12T05:55:58.077 INFO:tasks.workunit.client.1.smithi194.stdout:8/88: write d2/d8/d9/de/d10/f1c [124322,44113] 0 2024-04-12T05:55:58.081 INFO:tasks.workunit.client.1.smithi194.stdout:1/158: dread - d0/d11/d18/d19/d1c/f24 zero size 2024-04-12T05:55:58.083 INFO:tasks.workunit.client.1.smithi194.stdout:2/25: rename c2 to d7/c8 0 2024-04-12T05:55:58.085 INFO:tasks.workunit.client.1.smithi194.stdout:9/47: symlink d1/l12 0 2024-04-12T05:55:58.086 INFO:tasks.workunit.client.1.smithi194.stdout:6/89: dwrite d3/f16 [0,4194304] 0 2024-04-12T05:55:58.089 INFO:tasks.workunit.client.1.smithi194.stdout:8/89: write d2/d8/d15/d17/f1b [1239864,54568] 0 2024-04-12T05:55:58.091 INFO:tasks.workunit.client.1.smithi194.stdout:2/26: rmdir d7 39 2024-04-12T05:55:58.092 INFO:tasks.workunit.client.1.smithi194.stdout:4/42: dwrite f0 [0,4194304] 0 2024-04-12T05:55:58.093 INFO:tasks.workunit.client.1.smithi194.stdout:5/66: dwrite d1/f3 [4194304,4194304] 0 2024-04-12T05:55:58.095 INFO:tasks.workunit.client.1.smithi194.stdout:9/48: creat d1/f13 x:0 0 0 2024-04-12T05:55:58.097 INFO:tasks.workunit.client.1.smithi194.stdout:4/43: write f0 [580742,116917] 0 2024-04-12T05:55:58.097 INFO:tasks.workunit.client.1.smithi194.stdout:2/27: chown f5 486 1 2024-04-12T05:55:58.099 INFO:tasks.workunit.client.1.smithi194.stdout:4/44: readlink d2/d3/l10 0 2024-04-12T05:55:58.099 INFO:tasks.workunit.client.1.smithi194.stdout:8/90: fsync d2/d8/d15/d17/f18 0 2024-04-12T05:55:58.104 INFO:tasks.workunit.client.1.smithi194.stdout:9/49: dread d1/fb [0,4194304] 0 2024-04-12T05:55:58.105 INFO:tasks.workunit.client.1.smithi194.stdout:4/45: chown d2/lb 133 1 2024-04-12T05:55:58.106 INFO:tasks.workunit.client.1.smithi194.stdout:6/90: creat d3/f17 x:0 0 0 2024-04-12T05:55:58.109 INFO:tasks.workunit.client.1.smithi194.stdout:8/91: read d2/d8/d15/d17/f1b [3081664,66983] 0 2024-04-12T05:55:58.109 INFO:tasks.workunit.client.1.smithi194.stdout:1/159: getdents d0/df 0 2024-04-12T05:55:58.110 INFO:tasks.workunit.client.1.smithi194.stdout:0/45: dwrite d0/f2 [0,4194304] 0 2024-04-12T05:55:58.110 INFO:tasks.workunit.client.1.smithi194.stdout:0/46: readlink - no filename 2024-04-12T05:55:58.112 INFO:tasks.workunit.client.1.smithi194.stdout:5/67: link d1/d4/f14 d1/d4/f16 0 2024-04-12T05:55:58.115 INFO:tasks.workunit.client.1.smithi194.stdout:2/28: rename c1 to d7/c9 0 2024-04-12T05:55:58.117 INFO:tasks.workunit.client.1.smithi194.stdout:0/47: symlink d0/lb 0 2024-04-12T05:55:58.118 INFO:tasks.workunit.client.1.smithi194.stdout:8/92: mknod d2/d8/d9/de/d10/c21 0 2024-04-12T05:55:58.120 INFO:tasks.workunit.client.1.smithi194.stdout:2/29: chown f5 252072322 1 2024-04-12T05:55:58.121 INFO:tasks.workunit.client.1.smithi194.stdout:5/68: rename d1/d4/d6 to d1/d4/d6/d17 22 2024-04-12T05:55:58.123 INFO:tasks.workunit.client.1.smithi194.stdout:4/46: dread d2/f9 [0,4194304] 0 2024-04-12T05:55:58.123 INFO:tasks.workunit.client.1.smithi194.stdout:6/91: creat d3/f18 x:0 0 0 2024-04-12T05:55:58.126 INFO:tasks.workunit.client.1.smithi194.stdout:2/30: creat d7/fa x:0 0 0 2024-04-12T05:55:58.127 INFO:tasks.workunit.client.1.smithi194.stdout:5/69: dread - d1/d4/f14 zero size 2024-04-12T05:55:58.128 INFO:tasks.workunit.client.1.smithi194.stdout:4/47: dread - d2/fc zero size 2024-04-12T05:55:58.131 INFO:tasks.workunit.client.1.smithi194.stdout:5/70: chown d1/f3 177508 1 2024-04-12T05:55:58.131 INFO:tasks.workunit.client.1.smithi194.stdout:1/160: dwrite d0/df/f25 [0,4194304] 0 2024-04-12T05:55:58.136 INFO:tasks.workunit.client.1.smithi194.stdout:6/92: rmdir d3/d15 39 2024-04-12T05:55:58.136 INFO:tasks.workunit.client.1.smithi194.stdout:9/50: dwrite d1/fb [0,4194304] 0 2024-04-12T05:55:58.139 INFO:tasks.workunit.client.1.smithi194.stdout:2/31: mknod d7/cb 0 2024-04-12T05:55:58.140 INFO:tasks.workunit.client.1.smithi194.stdout:3/42: sync 2024-04-12T05:55:58.142 INFO:tasks.workunit.client.1.smithi194.stdout:1/161: write d0/d11/d18/d19/d1c/f22 [1138201,2613] 0 2024-04-12T05:55:58.146 INFO:tasks.workunit.client.1.smithi194.stdout:8/93: write d2/d8/d15/d17/f18 [1157477,45338] 0 2024-04-12T05:55:58.147 INFO:tasks.workunit.client.1.smithi194.stdout:9/51: creat d1/f14 x:0 0 0 2024-04-12T05:55:58.147 INFO:tasks.workunit.client.1.smithi194.stdout:5/71: mknod d1/d4/df/c18 0 2024-04-12T05:55:58.148 INFO:tasks.workunit.client.1.smithi194.stdout:3/43: creat d0/f11 x:0 0 0 2024-04-12T05:55:58.150 INFO:tasks.workunit.client.1.smithi194.stdout:1/162: read d0/f4 [2807848,24930] 0 2024-04-12T05:55:58.152 INFO:tasks.workunit.client.1.smithi194.stdout:0/48: dwrite d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.152 INFO:tasks.workunit.client.1.smithi194.stdout:9/52: write d1/f10 [355712,78750] 0 2024-04-12T05:55:58.154 INFO:tasks.workunit.client.1.smithi194.stdout:3/44: write d0/fb [2249448,115077] 0 2024-04-12T05:55:58.156 INFO:tasks.workunit.client.1.smithi194.stdout:4/48: dwrite d2/f9 [0,4194304] 0 2024-04-12T05:55:58.161 INFO:tasks.workunit.client.1.smithi194.stdout:8/94: read d2/d8/d15/d17/f1b [1587838,114250] 0 2024-04-12T05:55:58.162 INFO:tasks.workunit.client.1.smithi194.stdout:9/53: chown d1/lc 755272 1 2024-04-12T05:55:58.163 INFO:tasks.workunit.client.1.smithi194.stdout:4/49: creat d2/d3/f13 x:0 0 0 2024-04-12T05:55:58.172 INFO:tasks.workunit.client.1.smithi194.stdout:4/50: write f0 [2512714,49957] 0 2024-04-12T05:55:58.179 INFO:tasks.workunit.client.1.smithi194.stdout:4/51: unlink d2/lb 0 2024-04-12T05:55:58.181 INFO:tasks.workunit.client.1.smithi194.stdout:8/95: rmdir d2/d8/d15/d17/d20 0 2024-04-12T05:55:58.182 INFO:tasks.workunit.client.1.smithi194.stdout:1/163: dwrite d0/df/f25 [0,4194304] 0 2024-04-12T05:55:58.183 INFO:tasks.workunit.client.1.smithi194.stdout:2/32: truncate f5 6709144 0 2024-04-12T05:55:58.183 INFO:tasks.workunit.client.1.smithi194.stdout:4/52: dread - d2/fc zero size 2024-04-12T05:55:58.186 INFO:tasks.workunit.client.1.smithi194.stdout:0/49: dread d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.192 INFO:tasks.workunit.client.1.smithi194.stdout:5/72: dwrite d1/d4/d6/db/f12 [0,4194304] 0 2024-04-12T05:55:58.194 INFO:tasks.workunit.client.1.smithi194.stdout:3/45: dwrite d0/d1/fd [0,4194304] 0 2024-04-12T05:55:58.194 INFO:tasks.workunit.client.1.smithi194.stdout:4/53: dread - d2/d3/f13 zero size 2024-04-12T05:55:58.196 INFO:tasks.workunit.client.1.smithi194.stdout:0/50: fdatasync d0/f2 0 2024-04-12T05:55:58.198 INFO:tasks.workunit.client.1.smithi194.stdout:8/96: unlink d2/d8/d15/d17/c14 0 2024-04-12T05:55:58.199 INFO:tasks.workunit.client.1.smithi194.stdout:5/73: chown d1/d4/f5 1904 1 2024-04-12T05:55:58.202 INFO:tasks.workunit.client.1.smithi194.stdout:3/46: write d0/d1/f9 [503506,64133] 0 2024-04-12T05:55:58.202 INFO:tasks.workunit.client.1.smithi194.stdout:6/93: dwrite d3/fe [0,4194304] 0 2024-04-12T05:55:58.204 INFO:tasks.workunit.client.1.smithi194.stdout:2/33: link d7/c9 d7/cc 0 2024-04-12T05:55:58.204 INFO:tasks.workunit.client.1.smithi194.stdout:0/51: write d0/d5/f8 [4585915,121934] 0 2024-04-12T05:55:58.210 INFO:tasks.workunit.client.1.smithi194.stdout:5/74: write d1/d4/d6/db/f12 [1862834,88609] 0 2024-04-12T05:55:58.214 INFO:tasks.workunit.client.1.smithi194.stdout:5/75: write f0 [979291,98157] 0 2024-04-12T05:55:58.218 INFO:tasks.workunit.client.1.smithi194.stdout:9/54: write f0 [1117685,7051] 0 2024-04-12T05:55:58.223 INFO:tasks.workunit.client.1.smithi194.stdout:5/76: chown d1/d4/d6/db 130 1 2024-04-12T05:55:58.224 INFO:tasks.workunit.client.1.smithi194.stdout:6/94: dwrite d3/f18 [0,4194304] 0 2024-04-12T05:55:58.224 INFO:tasks.workunit.client.1.smithi194.stdout:3/47: creat d0/dc/f12 x:0 0 0 2024-04-12T05:55:58.224 INFO:tasks.workunit.client.1.smithi194.stdout:2/34: creat d7/fd x:0 0 0 2024-04-12T05:55:58.225 INFO:tasks.workunit.client.1.smithi194.stdout:4/54: creat d2/d11/f14 x:0 0 0 2024-04-12T05:55:58.230 INFO:tasks.workunit.client.1.smithi194.stdout:9/55: write d1/f5 [449273,34877] 0 2024-04-12T05:55:58.230 INFO:tasks.workunit.client.1.smithi194.stdout:4/55: symlink d2/l15 0 2024-04-12T05:55:58.232 INFO:tasks.workunit.client.1.smithi194.stdout:1/164: dwrite d0/d11/d18/d19/d1c/f24 [0,4194304] 0 2024-04-12T05:55:58.233 INFO:tasks.workunit.client.1.smithi194.stdout:5/77: chown d1/d4/df/c18 149 1 2024-04-12T05:55:58.234 INFO:tasks.workunit.client.1.smithi194.stdout:4/56: write d2/d11/f14 [643281,4863] 0 2024-04-12T05:55:58.235 INFO:tasks.workunit.client.1.smithi194.stdout:0/52: dread d0/f2 [0,4194304] 0 2024-04-12T05:55:58.235 INFO:tasks.workunit.client.1.smithi194.stdout:3/48: symlink d0/l13 0 2024-04-12T05:55:58.242 INFO:tasks.workunit.client.1.smithi194.stdout:6/95: chown d3/d15/f8 1288450 1 2024-04-12T05:55:58.244 INFO:tasks.workunit.client.1.smithi194.stdout:6/96: write d3/f16 [2571879,66251] 0 2024-04-12T05:55:58.245 INFO:tasks.workunit.client.1.smithi194.stdout:4/57: chown d2/l5 55 1 2024-04-12T05:55:58.246 INFO:tasks.workunit.client.1.smithi194.stdout:9/56: mknod d1/c15 0 2024-04-12T05:55:58.247 INFO:tasks.workunit.client.1.smithi194.stdout:8/97: truncate d2/fb 229428 0 2024-04-12T05:55:58.248 INFO:tasks.workunit.client.1.smithi194.stdout:5/78: dread d1/f11 [0,4194304] 0 2024-04-12T05:55:58.250 INFO:tasks.workunit.client.1.smithi194.stdout:6/97: rmdir d3/d15 39 2024-04-12T05:55:58.254 INFO:tasks.workunit.client.1.smithi194.stdout:4/58: creat d2/f16 x:0 0 0 2024-04-12T05:55:58.256 INFO:tasks.workunit.client.1.smithi194.stdout:4/59: chown d2/d3 54510 1 2024-04-12T05:55:58.257 INFO:tasks.workunit.client.1.smithi194.stdout:4/60: dread - d2/f6 zero size 2024-04-12T05:55:58.257 INFO:tasks.workunit.client.1.smithi194.stdout:2/35: chown f5 0 1 2024-04-12T05:55:58.259 INFO:tasks.workunit.client.1.smithi194.stdout:1/165: link d0/d11/c14 d0/d11/c28 0 2024-04-12T05:55:58.263 INFO:tasks.workunit.client.1.smithi194.stdout:3/49: truncate d0/fb 5904178 0 2024-04-12T05:55:58.263 INFO:tasks.workunit.client.1.smithi194.stdout:9/57: dwrite d1/f10 [0,4194304] 0 2024-04-12T05:55:58.265 INFO:tasks.workunit.client.1.smithi194.stdout:0/53: dwrite d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.267 INFO:tasks.workunit.client.1.smithi194.stdout:2/36: mknod d7/ce 0 2024-04-12T05:55:58.269 INFO:tasks.workunit.client.1.smithi194.stdout:8/98: symlink d2/d8/d9/d1a/l22 0 2024-04-12T05:55:58.270 INFO:tasks.workunit.client.1.smithi194.stdout:4/61: write d2/d3/ff [770513,121082] 0 2024-04-12T05:55:58.272 INFO:tasks.workunit.client.1.smithi194.stdout:9/58: mkdir d1/d16 0 2024-04-12T05:55:58.274 INFO:tasks.workunit.client.1.smithi194.stdout:1/166: creat d0/d11/d18/d19/d1a/d23/f29 x:0 0 0 2024-04-12T05:55:58.276 INFO:tasks.workunit.client.1.smithi194.stdout:0/54: mknod d0/d5/cc 0 2024-04-12T05:55:58.277 INFO:tasks.workunit.client.1.smithi194.stdout:5/79: dwrite d1/fa [0,4194304] 0 2024-04-12T05:55:58.281 INFO:tasks.workunit.client.1.smithi194.stdout:5/80: dread d1/f11 [0,4194304] 0 2024-04-12T05:55:58.283 INFO:tasks.workunit.client.1.smithi194.stdout:6/98: dread d3/d15/f13 [0,4194304] 0 2024-04-12T05:55:58.284 INFO:tasks.workunit.client.1.smithi194.stdout:6/99: fsync d3/f18 0 2024-04-12T05:55:58.285 INFO:tasks.workunit.client.1.smithi194.stdout:9/59: symlink d1/d16/l17 0 2024-04-12T05:55:58.285 INFO:tasks.workunit.client.1.smithi194.stdout:6/100: fsync d3/f16 0 2024-04-12T05:55:58.286 INFO:tasks.workunit.client.1.smithi194.stdout:9/60: fsync d1/fe 0 2024-04-12T05:55:58.287 INFO:tasks.workunit.client.1.smithi194.stdout:0/55: unlink d0/lb 0 2024-04-12T05:55:58.288 INFO:tasks.workunit.client.1.smithi194.stdout:9/61: write d1/fb [571510,102199] 0 2024-04-12T05:55:58.289 INFO:tasks.workunit.client.1.smithi194.stdout:2/37: truncate f6 3290382 0 2024-04-12T05:55:58.290 INFO:tasks.workunit.client.1.smithi194.stdout:1/167: creat d0/d11/d18/d19/d1c/f2a x:0 0 0 2024-04-12T05:55:58.290 INFO:tasks.workunit.client.1.smithi194.stdout:3/50: link d0/l13 d0/dc/l14 0 2024-04-12T05:55:58.293 INFO:tasks.workunit.client.1.smithi194.stdout:5/81: creat d1/d4/d6/db/f19 x:0 0 0 2024-04-12T05:55:58.300 INFO:tasks.workunit.client.1.smithi194.stdout:6/101: rmdir d3/d15 39 2024-04-12T05:55:58.300 INFO:tasks.workunit.client.1.smithi194.stdout:3/51: dread d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:58.300 INFO:tasks.workunit.client.1.smithi194.stdout:9/62: creat d1/f18 x:0 0 0 2024-04-12T05:55:58.300 INFO:tasks.workunit.client.1.smithi194.stdout:6/102: write d3/fe [397276,111122] 0 2024-04-12T05:55:58.300 INFO:tasks.workunit.client.1.smithi194.stdout:3/52: write d0/d1/fd [3911224,1606] 0 2024-04-12T05:55:58.302 INFO:tasks.workunit.client.1.smithi194.stdout:1/168: symlink d0/d11/d18/d19/d1c/l2b 0 2024-04-12T05:55:58.302 INFO:tasks.workunit.client.1.smithi194.stdout:3/53: chown d0/d1/f9 19 1 2024-04-12T05:55:58.302 INFO:tasks.workunit.client.1.smithi194.stdout:5/82: dread d1/fd [0,4194304] 0 2024-04-12T05:55:58.303 INFO:tasks.workunit.client.1.smithi194.stdout:4/62: getdents d2/d3 0 2024-04-12T05:55:58.305 INFO:tasks.workunit.client.1.smithi194.stdout:3/54: chown d0/d1/l5 8243812 1 2024-04-12T05:55:58.306 INFO:tasks.workunit.client.1.smithi194.stdout:0/56: symlink d0/da/ld 0 2024-04-12T05:55:58.306 INFO:tasks.workunit.client.1.smithi194.stdout:5/83: fdatasync d1/d4/d6/db/f19 0 2024-04-12T05:55:58.311 INFO:tasks.workunit.client.1.smithi194.stdout:1/169: dread d0/d11/d18/d19/f1f [0,4194304] 0 2024-04-12T05:55:58.315 INFO:tasks.workunit.client.1.smithi194.stdout:4/63: creat d2/d3/f17 x:0 0 0 2024-04-12T05:55:58.315 INFO:tasks.workunit.client.1.smithi194.stdout:6/103: dread - d3/d15/f14 zero size 2024-04-12T05:55:58.318 INFO:tasks.workunit.client.1.smithi194.stdout:3/55: dwrite d0/d1/fd [4194304,4194304] 0 2024-04-12T05:55:58.322 INFO:tasks.workunit.client.1.smithi194.stdout:0/57: dread d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.330 INFO:tasks.workunit.client.1.smithi194.stdout:4/64: unlink d2/d3/l10 0 2024-04-12T05:55:58.330 INFO:tasks.workunit.client.1.smithi194.stdout:2/38: link d7/cc d7/cf 0 2024-04-12T05:55:58.330 INFO:tasks.workunit.client.1.smithi194.stdout:0/58: creat d0/d5/fe x:0 0 0 2024-04-12T05:55:58.334 INFO:tasks.workunit.client.1.smithi194.stdout:6/104: dwrite d3/f18 [4194304,4194304] 0 2024-04-12T05:55:58.337 INFO:tasks.workunit.client.1.smithi194.stdout:1/170: dwrite d0/df/f25 [4194304,4194304] 0 2024-04-12T05:55:58.337 INFO:tasks.workunit.client.1.smithi194.stdout:6/105: fsync d3/d15/f8 0 2024-04-12T05:55:58.338 INFO:tasks.workunit.client.1.smithi194.stdout:4/65: rename d2/f9 to d2/d3/f18 0 2024-04-12T05:55:58.338 INFO:tasks.workunit.client.1.smithi194.stdout:5/84: write d1/d4/f14 [350379,17349] 0 2024-04-12T05:55:58.339 INFO:tasks.workunit.client.1.smithi194.stdout:6/106: chown d3/d15 942038 1 2024-04-12T05:55:58.340 INFO:tasks.workunit.client.1.smithi194.stdout:6/107: stat d3/f17 0 2024-04-12T05:55:58.342 INFO:tasks.workunit.client.1.smithi194.stdout:0/59: dread d0/f2 [0,4194304] 0 2024-04-12T05:55:58.342 INFO:tasks.workunit.client.1.smithi194.stdout:5/85: truncate d1/d4/d6/db/f19 387304 0 2024-04-12T05:55:58.346 INFO:tasks.workunit.client.1.smithi194.stdout:5/86: write d1/d4/f14 [1028519,96023] 0 2024-04-12T05:55:58.349 INFO:tasks.workunit.client.1.smithi194.stdout:9/63: truncate d1/f5 576684 0 2024-04-12T05:55:58.349 INFO:tasks.workunit.client.1.smithi194.stdout:4/66: mknod d2/d11/c19 0 2024-04-12T05:55:58.350 INFO:tasks.workunit.client.1.smithi194.stdout:1/171: mknod d0/d11/d18/d19/c2c 0 2024-04-12T05:55:58.352 INFO:tasks.workunit.client.1.smithi194.stdout:5/87: creat d1/d4/f1a x:0 0 0 2024-04-12T05:55:58.354 INFO:tasks.workunit.client.1.smithi194.stdout:6/108: mknod d3/c19 0 2024-04-12T05:55:58.355 INFO:tasks.workunit.client.1.smithi194.stdout:1/172: chown d0/d11/f26 1679459805 1 2024-04-12T05:55:58.357 INFO:tasks.workunit.client.1.smithi194.stdout:5/88: chown f0 28811 1 2024-04-12T05:55:58.360 INFO:tasks.workunit.client.1.smithi194.stdout:6/109: mkdir d3/d1a 0 2024-04-12T05:55:58.365 INFO:tasks.workunit.client.1.smithi194.stdout:0/60: dwrite d0/d5/f6 [0,4194304] 0 2024-04-12T05:55:58.368 INFO:tasks.workunit.client.1.smithi194.stdout:5/89: write d1/fa [2427661,24267] 0 2024-04-12T05:55:58.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/173: creat d0/d11/d18/f2d x:0 0 0 2024-04-12T05:55:58.370 INFO:tasks.workunit.client.1.smithi194.stdout:6/110: fdatasync d3/f17 0 2024-04-12T05:55:58.374 INFO:tasks.workunit.client.1.smithi194.stdout:2/39: dwrite f5 [4194304,4194304] 0 2024-04-12T05:55:58.376 INFO:tasks.workunit.client.1.smithi194.stdout:4/67: dwrite d2/d3/f12 [0,4194304] 0 2024-04-12T05:55:58.379 INFO:tasks.workunit.client.1.smithi194.stdout:3/56: dwrite d0/fb [0,4194304] 0 2024-04-12T05:55:58.379 INFO:tasks.workunit.client.1.smithi194.stdout:2/40: fdatasync d7/fa 0 2024-04-12T05:55:58.380 INFO:tasks.workunit.client.1.smithi194.stdout:5/90: symlink d1/d4/l1b 0 2024-04-12T05:55:58.386 INFO:tasks.workunit.client.1.smithi194.stdout:3/57: readlink d0/d1/l8 0 2024-04-12T05:55:58.386 INFO:tasks.workunit.client.1.smithi194.stdout:5/91: readlink d1/le 0 2024-04-12T05:55:58.387 INFO:tasks.workunit.client.1.smithi194.stdout:1/174: mkdir d0/d11/d18/d19/d1a/d23/d27/d2e 0 2024-04-12T05:55:58.388 INFO:tasks.workunit.client.1.smithi194.stdout:0/61: write d0/d5/f6 [871875,93715] 0 2024-04-12T05:55:58.390 INFO:tasks.workunit.client.1.smithi194.stdout:2/41: mknod d7/c10 0 2024-04-12T05:55:58.392 INFO:tasks.workunit.client.1.smithi194.stdout:3/58: unlink d0/l13 0 2024-04-12T05:55:58.392 INFO:tasks.workunit.client.1.smithi194.stdout:5/92: creat d1/d4/f1c x:0 0 0 2024-04-12T05:55:58.393 INFO:tasks.workunit.client.1.smithi194.stdout:2/42: dread - d7/fa zero size 2024-04-12T05:55:58.394 INFO:tasks.workunit.client.1.smithi194.stdout:3/59: readlink d0/d1/l8 0 2024-04-12T05:55:58.400 INFO:tasks.workunit.client.1.smithi194.stdout:9/64: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:55:58.401 INFO:tasks.workunit.client.1.smithi194.stdout:3/60: dread - d0/dc/f12 zero size 2024-04-12T05:55:58.402 INFO:tasks.workunit.client.1.smithi194.stdout:9/65: chown d1/f18 2667119 1 2024-04-12T05:55:58.405 INFO:tasks.workunit.client.1.smithi194.stdout:9/66: readlink d1/lc 0 2024-04-12T05:55:58.406 INFO:tasks.workunit.client.1.smithi194.stdout:9/67: read d1/fb [2196874,100390] 0 2024-04-12T05:55:58.407 INFO:tasks.workunit.client.1.smithi194.stdout:9/68: dread - d1/f18 zero size 2024-04-12T05:55:58.410 INFO:tasks.workunit.client.1.smithi194.stdout:2/43: dwrite d7/fd [0,4194304] 0 2024-04-12T05:55:58.475 INFO:tasks.workunit.client.1.smithi194.stdout:1/175: unlink d0/d11/f21 0 2024-04-12T05:55:58.478 INFO:tasks.workunit.client.1.smithi194.stdout:7/15: sync 2024-04-12T05:55:58.479 INFO:tasks.workunit.client.1.smithi194.stdout:3/61: unlink d0/d1/l5 0 2024-04-12T05:55:58.482 INFO:tasks.workunit.client.1.smithi194.stdout:9/69: rename d1/d16 to d1/d19 0 2024-04-12T05:55:58.484 INFO:tasks.workunit.client.1.smithi194.stdout:2/44: creat d7/f11 x:0 0 0 2024-04-12T05:55:58.486 INFO:tasks.workunit.client.1.smithi194.stdout:1/176: symlink d0/d11/d18/d19/l2f 0 2024-04-12T05:55:58.490 INFO:tasks.workunit.client.1.smithi194.stdout:5/93: creat d1/d4/d6/db/d15/f1d x:0 0 0 2024-04-12T05:55:58.490 INFO:tasks.workunit.client.1.smithi194.stdout:7/16: mkdir d3 0 2024-04-12T05:55:58.492 INFO:tasks.workunit.client.1.smithi194.stdout:9/70: mknod d1/c1a 0 2024-04-12T05:55:58.493 INFO:tasks.workunit.client.1.smithi194.stdout:2/45: mkdir d7/d12 0 2024-04-12T05:55:58.497 INFO:tasks.workunit.client.1.smithi194.stdout:3/62: symlink d0/dc/l15 0 2024-04-12T05:55:58.498 INFO:tasks.workunit.client.1.smithi194.stdout:1/177: dwrite d0/d11/d18/d19/d1c/f24 [0,4194304] 0 2024-04-12T05:55:58.499 INFO:tasks.workunit.client.1.smithi194.stdout:1/178: dread - d0/d11/d18/d19/d1c/f20 zero size 2024-04-12T05:55:58.500 INFO:tasks.workunit.client.1.smithi194.stdout:0/62: link d0/d5/c7 d0/da/cf 0 2024-04-12T05:55:58.503 INFO:tasks.workunit.client.1.smithi194.stdout:9/71: mknod d1/c1b 0 2024-04-12T05:55:58.504 INFO:tasks.workunit.client.1.smithi194.stdout:9/72: dread - d1/f14 zero size 2024-04-12T05:55:58.505 INFO:tasks.workunit.client.1.smithi194.stdout:5/94: mkdir d1/d4/d6/db/d1e 0 2024-04-12T05:55:58.506 INFO:tasks.workunit.client.1.smithi194.stdout:0/63: dread d0/d5/f8 [0,4194304] 0 2024-04-12T05:55:58.506 INFO:tasks.workunit.client.1.smithi194.stdout:5/95: chown d1/d4/f1a 501734 1 2024-04-12T05:55:58.510 INFO:tasks.workunit.client.1.smithi194.stdout:7/17: mkdir d3/d4 0 2024-04-12T05:55:58.511 INFO:tasks.workunit.client.1.smithi194.stdout:1/179: truncate d0/d11/d18/d19/f1f 878743 0 2024-04-12T05:55:58.512 INFO:tasks.workunit.client.1.smithi194.stdout:2/46: symlink d7/d12/l13 0 2024-04-12T05:55:58.514 INFO:tasks.workunit.client.1.smithi194.stdout:3/63: mknod d0/d1/d10/c16 0 2024-04-12T05:55:58.515 INFO:tasks.workunit.client.1.smithi194.stdout:7/18: mkdir d3/d5 0 2024-04-12T05:55:58.515 INFO:tasks.workunit.client.1.smithi194.stdout:2/47: fdatasync f5 0 2024-04-12T05:55:58.528 INFO:tasks.workunit.client.1.smithi194.stdout:1/180: dread d0/f4 [4194304,4194304] 0 2024-04-12T05:55:58.529 INFO:tasks.workunit.client.1.smithi194.stdout:9/73: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:55:58.530 INFO:tasks.workunit.client.1.smithi194.stdout:3/64: chown d0/dc/l15 11645436 1 2024-04-12T05:55:58.532 INFO:tasks.workunit.client.1.smithi194.stdout:5/96: dwrite d1/d4/f16 [0,4194304] 0 2024-04-12T05:55:58.533 INFO:tasks.workunit.client.1.smithi194.stdout:7/19: mkdir d3/d5/d6 0 2024-04-12T05:55:58.535 INFO:tasks.workunit.client.1.smithi194.stdout:0/64: dwrite d0/d5/fe [0,4194304] 0 2024-04-12T05:55:58.538 INFO:tasks.workunit.client.1.smithi194.stdout:5/97: write d1/fa [5002707,91951] 0 2024-04-12T05:55:58.538 INFO:tasks.workunit.client.1.smithi194.stdout:1/181: creat d0/d11/d18/d19/d1a/d23/f30 x:0 0 0 2024-04-12T05:55:58.540 INFO:tasks.workunit.client.1.smithi194.stdout:0/65: read d0/d5/f6 [939828,72833] 0 2024-04-12T05:55:58.540 INFO:tasks.workunit.client.1.smithi194.stdout:3/65: mkdir d0/dc/de/d17 0 2024-04-12T05:55:58.544 INFO:tasks.workunit.client.1.smithi194.stdout:2/48: dwrite d7/fa [0,4194304] 0 2024-04-12T05:55:58.548 INFO:tasks.workunit.client.1.smithi194.stdout:0/66: write d0/f4 [289080,68345] 0 2024-04-12T05:55:58.548 INFO:tasks.workunit.client.1.smithi194.stdout:5/98: creat d1/d4/df/f1f x:0 0 0 2024-04-12T05:55:58.556 INFO:tasks.workunit.client.1.smithi194.stdout:2/49: creat d7/d12/f14 x:0 0 0 2024-04-12T05:55:58.560 INFO:tasks.workunit.client.1.smithi194.stdout:2/50: chown d7/fa 0 1 2024-04-12T05:55:58.560 INFO:tasks.workunit.client.1.smithi194.stdout:9/74: dread d1/fe [0,4194304] 0 2024-04-12T05:55:58.561 INFO:tasks.workunit.client.1.smithi194.stdout:2/51: fdatasync f5 0 2024-04-12T05:55:58.561 INFO:tasks.workunit.client.1.smithi194.stdout:9/75: dread - d1/f14 zero size 2024-04-12T05:55:58.562 INFO:tasks.workunit.client.1.smithi194.stdout:3/66: link d0/d1/l2 d0/d1/d10/l18 0 2024-04-12T05:55:58.562 INFO:tasks.workunit.client.1.smithi194.stdout:2/52: read f5 [4485000,110439] 0 2024-04-12T05:55:58.564 INFO:tasks.workunit.client.1.smithi194.stdout:2/53: rename d7 to d7/d12/d15 22 2024-04-12T05:55:58.565 INFO:tasks.workunit.client.1.smithi194.stdout:5/99: dread d1/f3 [4194304,4194304] 0 2024-04-12T05:55:58.565 INFO:tasks.workunit.client.1.smithi194.stdout:9/76: symlink d1/l1c 0 2024-04-12T05:55:58.567 INFO:tasks.workunit.client.1.smithi194.stdout:2/54: creat d7/f16 x:0 0 0 2024-04-12T05:55:58.572 INFO:tasks.workunit.client.1.smithi194.stdout:0/67: dread d0/d5/f6 [0,4194304] 0 2024-04-12T05:55:58.573 INFO:tasks.workunit.client.1.smithi194.stdout:2/55: mknod d7/d12/c17 0 2024-04-12T05:55:58.575 INFO:tasks.workunit.client.1.smithi194.stdout:5/100: mknod d1/d4/d6/c20 0 2024-04-12T05:55:58.579 INFO:tasks.workunit.client.1.smithi194.stdout:0/68: creat d0/f10 x:0 0 0 2024-04-12T05:55:58.579 INFO:tasks.workunit.client.1.smithi194.stdout:2/56: truncate f6 4104627 0 2024-04-12T05:55:58.582 INFO:tasks.workunit.client.1.smithi194.stdout:0/69: creat d0/f11 x:0 0 0 2024-04-12T05:55:58.583 INFO:tasks.workunit.client.1.smithi194.stdout:9/77: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:55:58.585 INFO:tasks.workunit.client.1.smithi194.stdout:5/101: dread d1/d4/f5 [0,4194304] 0 2024-04-12T05:55:58.588 INFO:tasks.workunit.client.1.smithi194.stdout:0/70: symlink d0/d5/l12 0 2024-04-12T05:55:58.590 INFO:tasks.workunit.client.1.smithi194.stdout:2/57: link d7/c8 d7/c18 0 2024-04-12T05:55:58.592 INFO:tasks.workunit.client.1.smithi194.stdout:3/67: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:58.592 INFO:tasks.workunit.client.1.smithi194.stdout:2/58: fdatasync d7/f11 0 2024-04-12T05:55:58.593 INFO:tasks.workunit.client.1.smithi194.stdout:0/71: creat d0/da/f13 x:0 0 0 2024-04-12T05:55:58.598 INFO:tasks.workunit.client.1.smithi194.stdout:0/72: creat d0/f14 x:0 0 0 2024-04-12T05:55:58.598 INFO:tasks.workunit.client.1.smithi194.stdout:9/78: dwrite f0 [0,4194304] 0 2024-04-12T05:55:58.599 INFO:tasks.workunit.client.1.smithi194.stdout:3/68: truncate d0/dc/f12 751662 0 2024-04-12T05:55:58.599 INFO:tasks.workunit.client.1.smithi194.stdout:2/59: rename d7/cb to d7/c19 0 2024-04-12T05:55:58.606 INFO:tasks.workunit.client.1.smithi194.stdout:9/79: creat d1/f1d x:0 0 0 2024-04-12T05:55:58.606 INFO:tasks.workunit.client.1.smithi194.stdout:0/73: unlink d0/d5/f6 0 2024-04-12T05:55:58.606 INFO:tasks.workunit.client.1.smithi194.stdout:2/60: mkdir d7/d12/d1a 0 2024-04-12T05:55:58.607 INFO:tasks.workunit.client.1.smithi194.stdout:5/102: dwrite d1/fa [0,4194304] 0 2024-04-12T05:55:58.609 INFO:tasks.workunit.client.1.smithi194.stdout:0/74: write d0/f11 [660228,116457] 0 2024-04-12T05:55:58.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/80: mknod d1/c1e 0 2024-04-12T05:55:58.611 INFO:tasks.workunit.client.1.smithi194.stdout:2/61: mkdir d7/d12/d1b 0 2024-04-12T05:55:58.614 INFO:tasks.workunit.client.1.smithi194.stdout:5/103: write d1/f11 [451832,28364] 0 2024-04-12T05:55:58.615 INFO:tasks.workunit.client.1.smithi194.stdout:3/69: dwrite d0/fb [4194304,4194304] 0 2024-04-12T05:55:58.616 INFO:tasks.workunit.client.1.smithi194.stdout:2/62: write d7/d12/f14 [802811,92584] 0 2024-04-12T05:55:58.627 INFO:tasks.workunit.client.1.smithi194.stdout:3/70: symlink d0/dc/l19 0 2024-04-12T05:55:58.629 INFO:tasks.workunit.client.1.smithi194.stdout:2/63: link d7/cf d7/d12/c1c 0 2024-04-12T05:55:58.633 INFO:tasks.workunit.client.1.smithi194.stdout:3/71: mknod d0/dc/c1a 0 2024-04-12T05:55:58.635 INFO:tasks.workunit.client.1.smithi194.stdout:9/81: dwrite d1/f1d [0,4194304] 0 2024-04-12T05:55:58.636 INFO:tasks.workunit.client.1.smithi194.stdout:0/75: dwrite d0/f2 [4194304,4194304] 0 2024-04-12T05:55:58.640 INFO:tasks.workunit.client.1.smithi194.stdout:2/64: creat d7/d12/d1b/f1d x:0 0 0 2024-04-12T05:55:58.646 INFO:tasks.workunit.client.1.smithi194.stdout:5/104: dwrite d1/d4/f14 [0,4194304] 0 2024-04-12T05:55:58.651 INFO:tasks.workunit.client.1.smithi194.stdout:3/72: dwrite d0/d1/fd [4194304,4194304] 0 2024-04-12T05:55:58.651 INFO:tasks.workunit.client.1.smithi194.stdout:2/65: truncate d7/d12/d1b/f1d 773634 0 2024-04-12T05:55:58.653 INFO:tasks.workunit.client.1.smithi194.stdout:9/82: fsync d1/f1d 0 2024-04-12T05:55:58.654 INFO:tasks.workunit.client.1.smithi194.stdout:2/66: write d7/f16 [857735,318] 0 2024-04-12T05:55:58.656 INFO:tasks.workunit.client.1.smithi194.stdout:3/73: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:55:58.656 INFO:tasks.workunit.client.1.smithi194.stdout:9/83: dread - d1/f11 zero size 2024-04-12T05:55:58.656 INFO:tasks.workunit.client.1.smithi194.stdout:2/67: fdatasync d7/d12/d1b/f1d 0 2024-04-12T05:55:58.657 INFO:tasks.workunit.client.1.smithi194.stdout:5/105: fdatasync d1/d4/f5 0 2024-04-12T05:55:58.660 INFO:tasks.workunit.client.1.smithi194.stdout:5/106: stat d1/le 0 2024-04-12T05:55:58.660 INFO:tasks.workunit.client.1.smithi194.stdout:0/76: truncate d0/f2 8260843 0 2024-04-12T05:55:58.661 INFO:tasks.workunit.client.1.smithi194.stdout:9/84: read d1/f1d [3899354,26018] 0 2024-04-12T05:55:58.662 INFO:tasks.workunit.client.1.smithi194.stdout:2/68: rmdir d7 39 2024-04-12T05:55:58.662 INFO:tasks.workunit.client.1.smithi194.stdout:5/107: stat d1/d4 0 2024-04-12T05:55:58.663 INFO:tasks.workunit.client.1.smithi194.stdout:9/85: chown d1/c1a 7 1 2024-04-12T05:55:58.664 INFO:tasks.workunit.client.1.smithi194.stdout:5/108: fdatasync d1/d4/df/f1f 0 2024-04-12T05:55:58.666 INFO:tasks.workunit.client.1.smithi194.stdout:3/74: dread d0/d1/fd [4194304,4194304] 0 2024-04-12T05:55:58.667 INFO:tasks.workunit.client.1.smithi194.stdout:2/69: creat d7/d12/d1b/f1e x:0 0 0 2024-04-12T05:55:58.668 INFO:tasks.workunit.client.1.smithi194.stdout:3/75: readlink d0/l3 0 2024-04-12T05:55:58.669 INFO:tasks.workunit.client.1.smithi194.stdout:5/109: getdents d1 0 2024-04-12T05:55:58.670 INFO:tasks.workunit.client.1.smithi194.stdout:2/70: write d7/d12/d1b/f1e [819592,93477] 0 2024-04-12T05:55:58.672 INFO:tasks.workunit.client.1.smithi194.stdout:5/110: write d1/d4/d6/db/f19 [340252,123715] 0 2024-04-12T05:55:58.676 INFO:tasks.workunit.client.1.smithi194.stdout:3/76: rename d0/dc/l14 to d0/d1/d10/l1b 0 2024-04-12T05:55:58.677 INFO:tasks.workunit.client.1.smithi194.stdout:9/86: dwrite f0 [0,4194304] 0 2024-04-12T05:55:58.678 INFO:tasks.workunit.client.1.smithi194.stdout:0/77: fsync d0/f2 0 2024-04-12T05:55:58.682 INFO:tasks.workunit.client.1.smithi194.stdout:3/77: fdatasync d0/d1/fd 0 2024-04-12T05:55:58.683 INFO:tasks.workunit.client.1.smithi194.stdout:0/78: fdatasync d0/f14 0 2024-04-12T05:55:58.685 INFO:tasks.workunit.client.1.smithi194.stdout:3/78: creat d0/d1/f1c x:0 0 0 2024-04-12T05:55:58.687 INFO:tasks.workunit.client.1.smithi194.stdout:2/71: dread d7/fd [0,4194304] 0 2024-04-12T05:55:58.689 INFO:tasks.workunit.client.1.smithi194.stdout:3/79: write d0/d1/f1c [603933,23721] 0 2024-04-12T05:55:58.690 INFO:tasks.workunit.client.1.smithi194.stdout:5/111: dread d1/fd [0,4194304] 0 2024-04-12T05:55:58.691 INFO:tasks.workunit.client.1.smithi194.stdout:0/79: link d0/f11 d0/d5/f15 0 2024-04-12T05:55:58.692 INFO:tasks.workunit.client.1.smithi194.stdout:9/87: dwrite d1/f5 [4194304,4194304] 0 2024-04-12T05:55:58.693 INFO:tasks.workunit.client.1.smithi194.stdout:0/80: chown d0/f10 318 1 2024-04-12T05:55:58.698 INFO:tasks.workunit.client.1.smithi194.stdout:3/80: symlink d0/dc/de/l1d 0 2024-04-12T05:55:58.698 INFO:tasks.workunit.client.1.smithi194.stdout:2/72: rename d7/d12/c1c to d7/c1f 0 2024-04-12T05:55:58.699 INFO:tasks.workunit.client.1.smithi194.stdout:0/81: mknod d0/da/c16 0 2024-04-12T05:55:58.700 INFO:tasks.workunit.client.1.smithi194.stdout:2/73: dread - d7/f11 zero size 2024-04-12T05:55:58.706 INFO:tasks.workunit.client.1.smithi194.stdout:2/74: write d7/d12/d1b/f1d [437870,37250] 0 2024-04-12T05:55:58.706 INFO:tasks.workunit.client.1.smithi194.stdout:5/112: dread d1/d4/d6/db/fc [0,4194304] 0 2024-04-12T05:55:58.706 INFO:tasks.workunit.client.1.smithi194.stdout:0/82: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:58.708 INFO:tasks.workunit.client.1.smithi194.stdout:3/81: dwrite d0/d1/fd [4194304,4194304] 0 2024-04-12T05:55:58.708 INFO:tasks.workunit.client.1.smithi194.stdout:2/75: write d7/d12/d1b/f1d [1081747,102274] 0 2024-04-12T05:55:58.708 INFO:tasks.workunit.client.1.smithi194.stdout:5/113: symlink d1/d4/df/l21 0 2024-04-12T05:55:58.712 INFO:tasks.workunit.client.1.smithi194.stdout:0/83: creat d0/d5/f17 x:0 0 0 2024-04-12T05:55:58.713 INFO:tasks.workunit.client.1.smithi194.stdout:2/76: write d7/d12/f14 [1251703,24259] 0 2024-04-12T05:55:58.719 INFO:tasks.workunit.client.1.smithi194.stdout:2/77: rename d7/cf to d7/d12/d1b/c20 0 2024-04-12T05:55:58.722 INFO:tasks.workunit.client.1.smithi194.stdout:0/84: link d0/da/c16 d0/da/c18 0 2024-04-12T05:55:58.722 INFO:tasks.workunit.client.1.smithi194.stdout:3/82: dwrite d0/d1/fd [4194304,4194304] 0 2024-04-12T05:55:58.722 INFO:tasks.workunit.client.1.smithi194.stdout:5/114: getdents d1/d4/d6/db/d15 0 2024-04-12T05:55:58.723 INFO:tasks.workunit.client.1.smithi194.stdout:5/115: read d1/d4/f16 [2164587,90794] 0 2024-04-12T05:55:58.728 INFO:tasks.workunit.client.1.smithi194.stdout:2/78: dwrite d7/f11 [0,4194304] 0 2024-04-12T05:55:58.733 INFO:tasks.workunit.client.1.smithi194.stdout:2/79: dread d7/fd [0,4194304] 0 2024-04-12T05:55:58.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 -- 172.21.15.8:0/1752428672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 msgr2=0x7f2450106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 --2- 172.21.15.8:0/1752428672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f2450106470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2440009910 tx=0x7f2440009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 -- 172.21.15.8:0/1752428672 shutdown_connections 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 --2- 172.21.15.8:0/1752428672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f2450106470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 --2- 172.21.15.8:0/1752428672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f2450103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 -- 172.21.15.8:0/1752428672 >> 172.21.15.8:0/1752428672 conn(0x7f24500fb160 msgr2=0x7f24500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 -- 172.21.15.8:0/1752428672 shutdown_connections 2024-04-12T05:55:58.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.751+0000 7f2457d69700 1 -- 172.21.15.8:0/1752428672 wait complete. 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 Processor -- start 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 -- start start 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f24500ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500ffc40 con 0x7f2450104080 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.752+0000 7f2457d69700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500ffd80 con 0x7f2450101750 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455304700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f24500ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455304700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f24500ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39666/0 (socket says 172.21.15.8:39666) 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:56370/0 (socket says 172.21.15.8:56370) 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455304700 1 -- 172.21.15.8:0/1710991001 learned_addr learned my addr 172.21.15.8:0/1710991001 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 -- 172.21.15.8:0/1710991001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 msgr2=0x7f24500ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:58.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f24500ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 -- 172.21.15.8:0/1710991001 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400095f0 con 0x7f2450101750 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2455b05700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e970 tx=0x7f244c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c00c9d0 con 0x7f2450101750 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2450100060 con 0x7f2450101750 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f244c010430 con 0x7f2450101750 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f244c018630 con 0x7f2450101750 2024-04-12T05:55:58.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.753+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245019e710 con 0x7f2450101750 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.754+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24501904f0 con 0x7f2450101750 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.755+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f244c0105a0 con 0x7f2450101750 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.755+0000 7f2446ffd700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 0x7f243c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.756+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f244c014070 con 0x7f2450101750 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.756+0000 7f2455304700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 0x7f243c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:58.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.756+0000 7f2455304700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 0x7f243c06eed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2440006010 tx=0x7f244000b5a0 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:58.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:58.760+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f244c0529b0 con 0x7f2450101750 2024-04-12T05:55:58.845 INFO:tasks.workunit.client.1.smithi194.stdout:8/99: sync 2024-04-12T05:55:58.850 INFO:tasks.workunit.client.1.smithi194.stdout:5/116: mknod d1/d4/c22 0 2024-04-12T05:55:58.854 INFO:tasks.workunit.client.1.smithi194.stdout:2/80: rename d7/c10 to d7/c21 0 2024-04-12T05:55:58.855 INFO:tasks.workunit.client.1.smithi194.stdout:3/83: mkdir d0/dc/d1e 0 2024-04-12T05:55:58.858 INFO:tasks.workunit.client.1.smithi194.stdout:8/100: mknod d2/d8/d9/de/c23 0 2024-04-12T05:55:58.865 INFO:tasks.workunit.client.1.smithi194.stdout:0/85: link d0/d5/fe d0/d5/f19 0 2024-04-12T05:55:58.868 INFO:tasks.workunit.client.1.smithi194.stdout:0/86: write d0/d5/f17 [176434,18289] 0 2024-04-12T05:55:58.870 INFO:tasks.workunit.client.1.smithi194.stdout:8/101: creat d2/d8/d9/de/d10/f24 x:0 0 0 2024-04-12T05:55:58.871 INFO:tasks.workunit.client.1.smithi194.stdout:0/87: readlink d0/da/ld 0 2024-04-12T05:55:58.871 INFO:tasks.workunit.client.1.smithi194.stdout:2/81: dwrite d7/fa [0,4194304] 0 2024-04-12T05:55:58.872 INFO:tasks.workunit.client.1.smithi194.stdout:3/84: creat d0/dc/f1f x:0 0 0 2024-04-12T05:55:58.874 INFO:tasks.workunit.client.1.smithi194.stdout:2/82: mkdir d7/d22 0 2024-04-12T05:55:58.875 INFO:tasks.workunit.client.1.smithi194.stdout:2/83: chown d7/d12/d1b 8338626 1 2024-04-12T05:55:58.875 INFO:tasks.workunit.client.1.smithi194.stdout:3/85: chown d0/f11 61 1 2024-04-12T05:55:58.876 INFO:tasks.workunit.client.1.smithi194.stdout:0/88: mkdir d0/d5/d1a 0 2024-04-12T05:55:58.878 INFO:tasks.workunit.client.1.smithi194.stdout:8/102: dread d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:55:58.879 INFO:tasks.workunit.client.1.smithi194.stdout:5/117: write f0 [2903069,19170] 0 2024-04-12T05:55:58.881 INFO:tasks.workunit.client.1.smithi194.stdout:8/103: write d2/d8/d9/de/d10/f24 [9388,83529] 0 2024-04-12T05:55:58.883 INFO:tasks.workunit.client.1.smithi194.stdout:0/89: rename d0/da/cf to d0/d5/c1b 0 2024-04-12T05:55:58.883 INFO:tasks.workunit.client.1.smithi194.stdout:2/84: fsync f6 0 2024-04-12T05:55:58.885 INFO:tasks.workunit.client.1.smithi194.stdout:3/86: symlink d0/dc/l20 0 2024-04-12T05:55:58.889 INFO:tasks.workunit.client.1.smithi194.stdout:3/87: readlink d0/d1/l2 0 2024-04-12T05:55:58.890 INFO:tasks.workunit.client.1.smithi194.stdout:3/88: fdatasync d0/d1/f9 0 2024-04-12T05:55:58.893 INFO:tasks.workunit.client.1.smithi194.stdout:8/104: unlink d2/c5 0 2024-04-12T05:55:58.897 INFO:tasks.workunit.client.1.smithi194.stdout:5/118: rmdir d1/d4/d6/db/d1e 0 2024-04-12T05:55:58.899 INFO:tasks.workunit.client.1.smithi194.stdout:8/105: creat d2/d8/d9/d1a/f25 x:0 0 0 2024-04-12T05:55:58.900 INFO:tasks.workunit.client.1.smithi194.stdout:5/119: chown d1/d4/d6/db/f12 1849 1 2024-04-12T05:55:58.900 INFO:tasks.workunit.client.1.smithi194.stdout:3/89: symlink d0/dc/de/d17/l21 0 2024-04-12T05:55:58.902 INFO:tasks.workunit.client.1.smithi194.stdout:8/106: dread d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:55:58.903 INFO:tasks.workunit.client.1.smithi194.stdout:5/120: mkdir d1/d4/df/d23 0 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: pgmap v25: 65 pgs: 65 active+clean; 1.5 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 6.2 MiB/s rd, 35 MiB/s wr, 267 op/s 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: Upgrade: Updating mgr.smithi008.ixmnlp 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:55:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: Deploying daemon mgr.smithi008.ixmnlp on smithi008 2024-04-12T05:55:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:58 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:58.905 INFO:tasks.workunit.client.1.smithi194.stdout:3/90: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:55:58.908 INFO:tasks.workunit.client.1.smithi194.stdout:5/121: truncate d1/d4/d6/db/d15/f1d 364187 0 2024-04-12T05:55:58.909 INFO:tasks.workunit.client.1.smithi194.stdout:3/91: mknod d0/dc/de/c22 0 2024-04-12T05:55:58.909 INFO:tasks.workunit.client.1.smithi194.stdout:2/85: dwrite f6 [0,4194304] 0 2024-04-12T05:55:58.911 INFO:tasks.workunit.client.1.smithi194.stdout:5/122: symlink d1/l24 0 2024-04-12T05:55:58.914 INFO:tasks.workunit.client.1.smithi194.stdout:2/86: fdatasync d7/fd 0 2024-04-12T05:55:58.917 INFO:tasks.workunit.client.1.smithi194.stdout:3/92: mknod d0/dc/c23 0 2024-04-12T05:55:58.923 INFO:tasks.workunit.client.1.smithi194.stdout:2/87: rename d7/fd to d7/d12/d1a/f23 0 2024-04-12T05:55:58.924 INFO:tasks.workunit.client.1.smithi194.stdout:0/90: dwrite d0/d5/f15 [0,4194304] 0 2024-04-12T05:55:58.926 INFO:tasks.workunit.client.1.smithi194.stdout:3/93: dread d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:58.927 INFO:tasks.workunit.client.1.smithi194.stdout:2/88: write d7/fa [5216382,6636] 0 2024-04-12T05:55:58.939 INFO:tasks.workunit.client.1.smithi194.stdout:5/123: link d1/d4/f14 d1/d4/f25 0 2024-04-12T05:55:58.942 INFO:tasks.workunit.client.1.smithi194.stdout:0/91: dwrite d0/f14 [0,4194304] 0 2024-04-12T05:55:58.943 INFO:tasks.workunit.client.1.smithi194.stdout:0/92: dread - d0/da/f13 zero size 2024-04-12T05:55:58.943 INFO:tasks.workunit.client.1.smithi194.stdout:5/124: mkdir d1/d4/d6/db/d15/d26 0 2024-04-12T05:55:58.944 INFO:tasks.workunit.client.1.smithi194.stdout:5/125: stat d1/l7 0 2024-04-12T05:55:58.952 INFO:tasks.workunit.client.1.smithi194.stdout:3/94: write d0/dc/f12 [452558,30353] 0 2024-04-12T05:55:58.953 INFO:tasks.workunit.client.1.smithi194.stdout:2/89: truncate d7/f16 530809 0 2024-04-12T05:55:58.953 INFO:tasks.workunit.client.1.smithi194.stdout:5/126: dread d1/d4/d6/db/f19 [0,4194304] 0 2024-04-12T05:55:58.958 INFO:tasks.workunit.client.1.smithi194.stdout:5/127: mknod d1/d4/d6/db/c27 0 2024-04-12T05:55:58.959 INFO:tasks.workunit.client.1.smithi194.stdout:3/95: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:55:58.961 INFO:tasks.workunit.client.1.smithi194.stdout:5/128: write d1/d4/d6/db/d15/f1d [1266427,32609] 0 2024-04-12T05:55:58.967 INFO:tasks.workunit.client.1.smithi194.stdout:3/96: mkdir d0/dc/d24 0 2024-04-12T05:55:58.967 INFO:tasks.workunit.client.1.smithi194.stdout:2/90: rename d7/f16 to d7/d12/d1a/f24 0 2024-04-12T05:55:58.970 INFO:tasks.workunit.client.1.smithi194.stdout:5/129: creat d1/d4/d6/db/d15/d26/f28 x:0 0 0 2024-04-12T05:55:58.972 INFO:tasks.workunit.client.1.smithi194.stdout:2/91: unlink f6 0 2024-04-12T05:55:58.973 INFO:tasks.workunit.client.1.smithi194.stdout:3/97: symlink d0/dc/d1e/l25 0 2024-04-12T05:55:58.976 INFO:tasks.workunit.client.1.smithi194.stdout:3/98: truncate d0/dc/f12 1201572 0 2024-04-12T05:55:58.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/92: write d7/d12/f14 [1992662,85182] 0 2024-04-12T05:55:58.978 INFO:tasks.workunit.client.1.smithi194.stdout:5/130: truncate d1/fa 5121376 0 2024-04-12T05:55:58.982 INFO:tasks.workunit.client.1.smithi194.stdout:3/99: mknod d0/dc/de/d17/c26 0 2024-04-12T05:55:58.983 INFO:tasks.workunit.client.1.smithi194.stdout:5/131: mknod d1/d4/d6/db/d15/d26/c29 0 2024-04-12T05:55:58.986 INFO:tasks.workunit.client.1.smithi194.stdout:3/100: unlink d0/dc/c23 0 2024-04-12T05:55:58.987 INFO:tasks.workunit.client.1.smithi194.stdout:3/101: write d0/d1/fd [5105430,5250] 0 2024-04-12T05:55:58.989 INFO:tasks.workunit.client.1.smithi194.stdout:5/132: rename d1/d4/d6/db/f12 to d1/d4/df/f2a 0 2024-04-12T05:55:58.997 INFO:tasks.workunit.client.1.smithi194.stdout:2/93: truncate d7/d12/f14 383455 0 2024-04-12T05:55:58.999 INFO:tasks.workunit.client.1.smithi194.stdout:3/102: rmdir d0/dc/d24 0 2024-04-12T05:55:58.999 INFO:tasks.workunit.client.1.smithi194.stdout:2/94: write d7/d12/d1b/f1d [1086564,102138] 0 2024-04-12T05:55:59.005 INFO:tasks.workunit.client.1.smithi194.stdout:5/133: dwrite d1/f11 [0,4194304] 0 2024-04-12T05:55:59.006 INFO:tasks.workunit.client.1.smithi194.stdout:3/103: creat d0/f27 x:0 0 0 2024-04-12T05:55:59.006 INFO:tasks.workunit.client.1.smithi194.stdout:2/95: mkdir d7/d12/d25 0 2024-04-12T05:55:59.009 INFO:tasks.workunit.client.1.smithi194.stdout:3/104: truncate d0/fb 8422861 0 2024-04-12T05:55:59.009 INFO:tasks.workunit.client.1.smithi194.stdout:5/134: mkdir d1/d4/d6/db/d2b 0 2024-04-12T05:55:59.010 INFO:tasks.workunit.client.1.smithi194.stdout:3/105: read d0/d1/f1c [611619,118501] 0 2024-04-12T05:55:59.012 INFO:tasks.workunit.client.1.smithi194.stdout:2/96: creat d7/d12/f26 x:0 0 0 2024-04-12T05:55:59.013 INFO:tasks.workunit.client.1.smithi194.stdout:5/135: creat d1/d4/df/d23/f2c x:0 0 0 2024-04-12T05:55:59.016 INFO:tasks.workunit.client.1.smithi194.stdout:2/97: creat d7/d22/f27 x:0 0 0 2024-04-12T05:55:59.019 INFO:tasks.workunit.client.1.smithi194.stdout:5/136: rename d1/f11 to d1/d4/d6/f2d 0 2024-04-12T05:55:59.020 INFO:tasks.workunit.client.1.smithi194.stdout:5/137: chown d1/d4/c22 6 1 2024-04-12T05:55:59.021 INFO:tasks.workunit.client.1.smithi194.stdout:5/138: fdatasync d1/d4/f1c 0 2024-04-12T05:55:59.024 INFO:tasks.workunit.client.1.smithi194.stdout:2/98: rename d7/cc to d7/d12/d1b/c28 0 2024-04-12T05:55:59.025 INFO:tasks.workunit.client.1.smithi194.stdout:5/139: dread d1/d4/d6/f2d [0,4194304] 0 2024-04-12T05:55:59.026 INFO:tasks.workunit.client.1.smithi194.stdout:2/99: symlink d7/d22/l29 0 2024-04-12T05:55:59.029 INFO:tasks.workunit.client.1.smithi194.stdout:5/140: creat d1/d4/df/f2e x:0 0 0 2024-04-12T05:55:59.029 INFO:tasks.workunit.client.1.smithi194.stdout:2/100: creat d7/d12/d1b/f2a x:0 0 0 2024-04-12T05:55:59.031 INFO:tasks.workunit.client.1.smithi194.stdout:5/141: truncate d1/d4/d6/db/d15/f1d 2240506 0 2024-04-12T05:55:59.031 INFO:tasks.workunit.client.1.smithi194.stdout:5/142: stat d1/fa 0 2024-04-12T05:55:59.032 INFO:tasks.workunit.client.1.smithi194.stdout:2/101: mkdir d7/d22/d2b 0 2024-04-12T05:55:59.032 INFO:tasks.workunit.client.1.smithi194.stdout:3/106: rmdir d0 39 2024-04-12T05:55:59.034 INFO:tasks.workunit.client.1.smithi194.stdout:5/143: mknod d1/d4/d6/db/d15/d26/c2f 0 2024-04-12T05:55:59.036 INFO:tasks.workunit.client.1.smithi194.stdout:3/107: mknod d0/d1/c28 0 2024-04-12T05:55:59.038 INFO:tasks.workunit.client.1.smithi194.stdout:5/144: mknod d1/d4/c30 0 2024-04-12T05:55:59.040 INFO:tasks.workunit.client.1.smithi194.stdout:2/102: rename d7/d12/f26 to d7/d12/d1b/f2c 0 2024-04-12T05:55:59.043 INFO:tasks.workunit.client.1.smithi194.stdout:3/108: rename d0/d1/c28 to d0/dc/de/d17/c29 0 2024-04-12T05:55:59.044 INFO:tasks.workunit.client.1.smithi194.stdout:2/103: creat d7/d12/d1a/f2d x:0 0 0 2024-04-12T05:55:59.047 INFO:tasks.workunit.client.1.smithi194.stdout:5/145: dwrite d1/d4/d6/db/d15/f1d [0,4194304] 0 2024-04-12T05:55:59.047 INFO:tasks.workunit.client.1.smithi194.stdout:2/104: creat d7/d22/f2e x:0 0 0 2024-04-12T05:55:59.048 INFO:tasks.workunit.client.1.smithi194.stdout:2/105: chown d7 0 1 2024-04-12T05:55:59.049 INFO:tasks.workunit.client.1.smithi194.stdout:2/106: chown d7/ce 0 1 2024-04-12T05:55:59.051 INFO:tasks.workunit.client.1.smithi194.stdout:5/146: mknod d1/d4/df/c31 0 2024-04-12T05:55:59.051 INFO:tasks.workunit.client.1.smithi194.stdout:3/109: dread d0/d1/f1c [0,4194304] 0 2024-04-12T05:55:59.054 INFO:tasks.workunit.client.1.smithi194.stdout:5/147: mkdir d1/d4/df/d23/d32 0 2024-04-12T05:55:59.058 INFO:tasks.workunit.client.1.smithi194.stdout:2/107: dwrite f5 [4194304,4194304] 0 2024-04-12T05:55:59.059 INFO:tasks.workunit.client.1.smithi194.stdout:2/108: chown d7/d22/d2b 37691411 1 2024-04-12T05:55:59.059 INFO:tasks.workunit.client.1.smithi194.stdout:2/109: readlink d7/d22/l29 0 2024-04-12T05:55:59.060 INFO:tasks.workunit.client.1.smithi194.stdout:2/110: chown d7/d22 444 1 2024-04-12T05:55:59.067 INFO:tasks.workunit.client.1.smithi194.stdout:5/148: creat d1/d4/d6/db/d2b/f33 x:0 0 0 2024-04-12T05:55:59.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: pgmap v25: 65 pgs: 65 active+clean; 1.5 GiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 6.2 MiB/s rd, 35 MiB/s wr, 267 op/s 2024-04-12T05:55:59.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: Upgrade: Updating mgr.smithi008.ixmnlp 2024-04-12T05:55:59.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi008.ixmnlp", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: Deploying daemon mgr.smithi008.ixmnlp on smithi008 2024-04-12T05:55:59.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:58 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:55:59.073 INFO:tasks.workunit.client.1.smithi194.stdout:5/149: rename d1/d4/df/d23/d32 to d1/d4/d6/d34 0 2024-04-12T05:55:59.075 INFO:tasks.workunit.client.1.smithi194.stdout:3/110: dwrite d0/d1/f1c [0,4194304] 0 2024-04-12T05:55:59.081 INFO:tasks.workunit.client.1.smithi194.stdout:5/150: rename d1/d4/f14 to d1/d4/d6/f35 0 2024-04-12T05:55:59.083 INFO:tasks.workunit.client.1.smithi194.stdout:6/111: sync 2024-04-12T05:55:59.083 INFO:tasks.workunit.client.1.smithi194.stdout:4/68: sync 2024-04-12T05:55:59.083 INFO:tasks.workunit.client.1.smithi194.stdout:3/111: truncate d0/d1/f9 3294428 0 2024-04-12T05:55:59.085 INFO:tasks.workunit.client.1.smithi194.stdout:5/151: rename d1/l24 to d1/d4/d6/db/d15/l36 0 2024-04-12T05:55:59.086 INFO:tasks.workunit.client.1.smithi194.stdout:6/112: stat d3/d15/cf 0 2024-04-12T05:55:59.087 INFO:tasks.workunit.client.1.smithi194.stdout:6/113: write d3/f17 [475546,23553] 0 2024-04-12T05:55:59.088 INFO:tasks.workunit.client.1.smithi194.stdout:3/112: creat d0/dc/d1e/f2a x:0 0 0 2024-04-12T05:55:59.090 INFO:tasks.workunit.client.1.smithi194.stdout:3/113: chown d0/dc 178959941 1 2024-04-12T05:55:59.093 INFO:tasks.workunit.client.1.smithi194.stdout:3/114: fdatasync d0/dc/f12 0 2024-04-12T05:55:59.093 INFO:tasks.workunit.client.1.smithi194.stdout:4/69: getdents d2/d11 0 2024-04-12T05:55:59.095 INFO:tasks.workunit.client.1.smithi194.stdout:6/114: getdents d3/d15 0 2024-04-12T05:55:59.097 INFO:tasks.workunit.client.1.smithi194.stdout:3/115: chown d0/d1/f1c 16182 1 2024-04-12T05:55:59.097 INFO:tasks.workunit.client.1.smithi194.stdout:5/152: dwrite d1/d4/f1c [0,4194304] 0 2024-04-12T05:55:59.101 INFO:tasks.workunit.client.1.smithi194.stdout:6/115: write d3/f18 [3053909,122985] 0 2024-04-12T05:55:59.107 INFO:tasks.workunit.client.1.smithi194.stdout:4/70: truncate f0 1735911 0 2024-04-12T05:55:59.108 INFO:tasks.workunit.client.1.smithi194.stdout:4/71: stat d2/fc 0 2024-04-12T05:55:59.109 INFO:tasks.workunit.client.1.smithi194.stdout:3/116: dread d0/fb [0,4194304] 0 2024-04-12T05:55:59.112 INFO:tasks.workunit.client.1.smithi194.stdout:4/72: write d2/d3/f18 [2256643,7758] 0 2024-04-12T05:55:59.112 INFO:tasks.workunit.client.1.smithi194.stdout:6/116: dwrite d3/fe [0,4194304] 0 2024-04-12T05:55:59.112 INFO:tasks.workunit.client.1.smithi194.stdout:3/117: write d0/d1/f1c [5090555,52779] 0 2024-04-12T05:55:59.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.118+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f245002ce60 con 0x7f243c06ca10 2024-04-12T05:55:59.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.120+0000 7f2446ffd700 1 -- 172.21.15.8:0/1710991001 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f245002ce60 con 0x7f243c06ca10 2024-04-12T05:55:59.116 INFO:tasks.workunit.client.1.smithi194.stdout:4/73: write d2/fc [254441,10917] 0 2024-04-12T05:55:59.118 INFO:tasks.workunit.client.1.smithi194.stdout:3/118: rmdir d0/dc/de/d17 39 2024-04-12T05:55:59.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 msgr2=0x7f243c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 0x7f243c06eed0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2440006010 tx=0x7f244000b5a0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 msgr2=0x7f24501010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f244c00e970 tx=0x7f244c00ec80 comp rx=0 tx=0).stop 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 shutdown_connections 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f243c06ca10 0x7f243c06eed0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2450101750 0x7f24501010b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 --2- 172.21.15.8:0/1710991001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2450104080 0x7f24500ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 >> 172.21.15.8:0/1710991001 conn(0x7f24500fb160 msgr2=0x7f24500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 shutdown_connections 2024-04-12T05:55:59.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.124+0000 7f2457d69700 1 -- 172.21.15.8:0/1710991001 wait complete. 2024-04-12T05:55:59.121 INFO:tasks.workunit.client.1.smithi194.stdout:5/153: write d1/d4/d6/f35 [4315097,103134] 0 2024-04-12T05:55:59.121 INFO:tasks.workunit.client.1.smithi194.stdout:3/119: fdatasync d0/f27 0 2024-04-12T05:55:59.122 INFO:tasks.workunit.client.1.smithi194.stdout:4/74: read f0 [771764,8109] 0 2024-04-12T05:55:59.125 INFO:tasks.workunit.client.1.smithi194.stdout:5/154: getdents d1/d4/d6/db/d15/d26 0 2024-04-12T05:55:59.129 INFO:tasks.workunit.client.1.smithi194.stdout:4/75: dread d2/d3/ff [0,4194304] 0 2024-04-12T05:55:59.129 INFO:tasks.workunit.client.1.smithi194.stdout:5/155: truncate d1/d4/f1a 459427 0 2024-04-12T05:55:59.131 INFO:tasks.workunit.client.1.smithi194.stdout:3/120: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:55:59.134 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:55:59.136 INFO:tasks.workunit.client.1.smithi194.stdout:3/121: fdatasync d0/d1/f9 0 2024-04-12T05:55:59.138 INFO:tasks.workunit.client.1.smithi194.stdout:5/156: dread d1/d4/f5 [4194304,4194304] 0 2024-04-12T05:55:59.138 INFO:tasks.workunit.client.1.smithi194.stdout:3/122: read d0/d1/fd [2217960,22820] 0 2024-04-12T05:55:59.139 INFO:tasks.workunit.client.1.smithi194.stdout:4/76: dwrite d2/f6 [0,4194304] 0 2024-04-12T05:55:59.147 INFO:tasks.workunit.client.1.smithi194.stdout:5/157: link d1/d4/l1b d1/d4/df/d23/l37 0 2024-04-12T05:55:59.147 INFO:tasks.workunit.client.1.smithi194.stdout:3/123: truncate d0/d1/f9 2027155 0 2024-04-12T05:55:59.150 INFO:tasks.workunit.client.1.smithi194.stdout:4/77: dwrite d2/d3/f17 [0,4194304] 0 2024-04-12T05:55:59.151 INFO:tasks.workunit.client.1.smithi194.stdout:4/78: stat d2/l5 0 2024-04-12T05:55:59.159 INFO:tasks.workunit.client.1.smithi194.stdout:3/124: dread d0/d1/f9 [0,4194304] 0 2024-04-12T05:55:59.162 INFO:tasks.workunit.client.1.smithi194.stdout:4/79: rename d2/d11/f14 to d2/d3/f1a 0 2024-04-12T05:55:59.162 INFO:tasks.workunit.client.1.smithi194.stdout:3/125: chown d0/d1/l2 0 1 2024-04-12T05:55:59.163 INFO:tasks.workunit.client.1.smithi194.stdout:5/158: dwrite d1/fd [4194304,4194304] 0 2024-04-12T05:55:59.167 INFO:tasks.workunit.client.1.smithi194.stdout:3/126: creat d0/d1/d10/f2b x:0 0 0 2024-04-12T05:55:59.171 INFO:tasks.workunit.client.1.smithi194.stdout:5/159: dwrite f0 [4194304,4194304] 0 2024-04-12T05:55:59.174 INFO:tasks.workunit.client.1.smithi194.stdout:5/160: unlink d1/d4/d6/db/d15/f1d 0 2024-04-12T05:55:59.176 INFO:tasks.workunit.client.1.smithi194.stdout:5/161: fdatasync d1/d4/df/f2a 0 2024-04-12T05:55:59.177 INFO:tasks.workunit.client.1.smithi194.stdout:5/162: stat d1/d4 0 2024-04-12T05:55:59.180 INFO:tasks.workunit.client.1.smithi194.stdout:5/163: mkdir d1/d4/d6/db/d38 0 2024-04-12T05:55:59.181 INFO:tasks.workunit.client.1.smithi194.stdout:5/164: rename d1 to d1/d4/d6/d34/d39 22 2024-04-12T05:55:59.181 INFO:tasks.workunit.client.1.smithi194.stdout:5/165: chown d1/d4/d6/db/d15/d26/c2f 6 1 2024-04-12T05:55:59.183 INFO:tasks.workunit.client.1.smithi194.stdout:5/166: mknod d1/d4/d6/d34/c3a 0 2024-04-12T05:55:59.184 INFO:tasks.workunit.client.1.smithi194.stdout:5/167: chown d1/d4/d6/db/d15 3092 1 2024-04-12T05:55:59.184 INFO:tasks.workunit.client.1.smithi194.stdout:5/168: chown d1/d4/f5 4 1 2024-04-12T05:55:59.186 INFO:tasks.workunit.client.1.smithi194.stdout:5/169: mkdir d1/d4/d6/db/d3b 0 2024-04-12T05:55:59.191 INFO:tasks.workunit.client.1.smithi194.stdout:5/170: dread d1/d4/f1c [0,4194304] 0 2024-04-12T05:55:59.192 INFO:tasks.workunit.client.1.smithi194.stdout:5/171: write d1/d4/df/f1f [135548,8982] 0 2024-04-12T05:55:59.200 INFO:tasks.workunit.client.1.smithi194.stdout:5/172: dwrite d1/d4/d6/db/f19 [0,4194304] 0 2024-04-12T05:55:59.203 INFO:tasks.workunit.client.1.smithi194.stdout:5/173: creat d1/d4/df/d23/f3c x:0 0 0 2024-04-12T05:55:59.204 INFO:tasks.workunit.client.1.smithi194.stdout:5/174: write d1/d4/d6/db/d15/d26/f28 [613865,57595] 0 2024-04-12T05:55:59.205 INFO:tasks.workunit.client.1.smithi194.stdout:5/175: chown d1/d4/df/d23/f3c 1862460297 1 2024-04-12T05:55:59.205 INFO:tasks.workunit.client.1.smithi194.stdout:5/176: fsync d1/d4/d6/f13 0 2024-04-12T05:55:59.207 INFO:tasks.workunit.client.1.smithi194.stdout:5/177: read d1/d4/df/f1f [52632,116929] 0 2024-04-12T05:55:59.209 INFO:tasks.workunit.client.1.smithi194.stdout:5/178: write d1/d4/d6/f2d [2927713,123860] 0 2024-04-12T05:55:59.210 INFO:tasks.workunit.client.1.smithi194.stdout:5/179: truncate d1/d4/df/f1f 169874 0 2024-04-12T05:55:59.211 INFO:tasks.workunit.client.1.smithi194.stdout:5/180: fsync d1/d4/df/f2e 0 2024-04-12T05:55:59.214 INFO:tasks.workunit.client.1.smithi194.stdout:5/181: creat d1/d4/d6/f3d x:0 0 0 2024-04-12T05:55:59.219 INFO:tasks.workunit.client.1.smithi194.stdout:5/182: truncate d1/d4/d6/f2d 2027047 0 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 -- 172.21.15.8:0/312288568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 msgr2=0x7fd9d0103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/312288568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0103b80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009910 tx=0x7fd9cc009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 -- 172.21.15.8:0/312288568 shutdown_connections 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/312288568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 0x7fd9d01064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/312288568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0103b80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.283+0000 7fd9d7f21700 1 -- 172.21.15.8:0/312288568 >> 172.21.15.8:0/312288568 conn(0x7fd9d00fb120 msgr2=0x7fd9d00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 -- 172.21.15.8:0/312288568 shutdown_connections 2024-04-12T05:55:59.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 -- 172.21.15.8:0/312288568 wait complete. 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 Processor -- start 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 -- start start 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 0x7fd9d01968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0196ef0 con 0x7fd9d0101790 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.284+0000 7fd9d7f21700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d0197030 con 0x7fd9d01040c0 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39688/0 (socket says 172.21.15.8:39688) 2024-04-12T05:55:59.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 -- 172.21.15.8:0/3762881600 learned_addr learned my addr 172.21.15.8:0/3762881600 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d54bc700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 0x7fd9d01968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 -- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 msgr2=0x7fd9d01968d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 0x7fd9d01968d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 -- 172.21.15.8:0/3762881600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0095f0 con 0x7fd9d0101790 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d5cbd700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b5c0 tx=0x7fd9cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9cc01d070 con 0x7fd9d0101790 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fd9cc00f460 con 0x7fd9d0101790 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9cc021640 con 0x7fd9d0101790 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.285+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d019ba80 con 0x7fd9d0101790 2024-04-12T05:55:59.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.286+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d019bed0 con 0x7fd9d0101790 2024-04-12T05:55:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.287+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d01904d0 con 0x7fd9d0101790 2024-04-12T05:55:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.287+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7fd9cc00fa20 con 0x7fd9d0101790 2024-04-12T05:55:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.288+0000 7fd9c6ffd700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 0x7fd9bc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.288+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd9cc08d3c0 con 0x7fd9d0101790 2024-04-12T05:55:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.288+0000 7fd9d54bc700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 0x7fd9bc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.289+0000 7fd9d54bc700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 0x7fd9bc06eed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005950 tx=0x7fd9c000b410 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:59.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.293+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fd9cc05aea0 con 0x7fd9d0101790 2024-04-12T05:55:59.312 INFO:tasks.workunit.client.1.smithi194.stdout:1/182: sync 2024-04-12T05:55:59.312 INFO:tasks.workunit.client.1.smithi194.stdout:9/88: sync 2024-04-12T05:55:59.312 INFO:tasks.workunit.client.1.smithi194.stdout:7/20: sync 2024-04-12T05:55:59.314 INFO:tasks.workunit.client.1.smithi194.stdout:1/183: chown d0/d11/c28 229761844 1 2024-04-12T05:55:59.314 INFO:tasks.workunit.client.1.smithi194.stdout:9/89: mkdir d1/d19/d1f 0 2024-04-12T05:55:59.315 INFO:tasks.workunit.client.1.smithi194.stdout:1/184: chown d0/d11/d18/d19/d1c/f2a 9672 1 2024-04-12T05:55:59.315 INFO:tasks.workunit.client.1.smithi194.stdout:9/90: dread - d1/f13 zero size 2024-04-12T05:55:59.315 INFO:tasks.workunit.client.1.smithi194.stdout:7/21: creat d3/d4/f7 x:0 0 0 2024-04-12T05:55:59.318 INFO:tasks.workunit.client.1.smithi194.stdout:7/22: creat d3/d5/f8 x:0 0 0 2024-04-12T05:55:59.320 INFO:tasks.workunit.client.1.smithi194.stdout:7/23: dread - d3/d4/f7 zero size 2024-04-12T05:55:59.324 INFO:tasks.workunit.client.1.smithi194.stdout:9/91: dwrite d1/fe [0,4194304] 0 2024-04-12T05:55:59.324 INFO:tasks.workunit.client.1.smithi194.stdout:1/185: dwrite d0/d11/d18/d19/d1c/f24 [4194304,4194304] 0 2024-04-12T05:55:59.325 INFO:tasks.workunit.client.1.smithi194.stdout:7/24: rmdir d3/d5/d6 0 2024-04-12T05:55:59.328 INFO:tasks.workunit.client.1.smithi194.stdout:1/186: mknod d0/d11/d18/d19/d1c/c31 0 2024-04-12T05:55:59.329 INFO:tasks.workunit.client.1.smithi194.stdout:7/25: rename d3/d5/f8 to d3/d5/f9 0 2024-04-12T05:55:59.330 INFO:tasks.workunit.client.1.smithi194.stdout:1/187: chown d0/f12 37 1 2024-04-12T05:55:59.331 INFO:tasks.workunit.client.1.smithi194.stdout:7/26: rmdir d3 39 2024-04-12T05:55:59.333 INFO:tasks.workunit.client.1.smithi194.stdout:9/92: dwrite d1/f14 [0,4194304] 0 2024-04-12T05:55:59.334 INFO:tasks.workunit.client.1.smithi194.stdout:9/93: fdatasync d1/f1d 0 2024-04-12T05:55:59.336 INFO:tasks.workunit.client.1.smithi194.stdout:7/27: creat d3/d4/fa x:0 0 0 2024-04-12T05:55:59.336 INFO:tasks.workunit.client.1.smithi194.stdout:7/28: dread - f2 zero size 2024-04-12T05:55:59.339 INFO:tasks.workunit.client.1.smithi194.stdout:7/29: write d3/d4/f7 [143210,100013] 0 2024-04-12T05:55:59.340 INFO:tasks.workunit.client.1.smithi194.stdout:9/94: dread d1/fe [0,4194304] 0 2024-04-12T05:55:59.344 INFO:tasks.workunit.client.1.smithi194.stdout:7/30: creat d3/d5/fb x:0 0 0 2024-04-12T05:55:59.347 INFO:tasks.workunit.client.1.smithi194.stdout:1/188: getdents d0/d11/d18/d19/d1a/d23/d27 0 2024-04-12T05:55:59.348 INFO:tasks.workunit.client.1.smithi194.stdout:1/189: fsync d0/d11/d18/d19/d1a/d23/f29 0 2024-04-12T05:55:59.350 INFO:tasks.workunit.client.1.smithi194.stdout:7/31: unlink d3/d5/fb 0 2024-04-12T05:55:59.356 INFO:tasks.workunit.client.1.smithi194.stdout:7/32: link d3/d4/fa d3/fc 0 2024-04-12T05:55:59.357 INFO:tasks.workunit.client.1.smithi194.stdout:1/190: dwrite d0/d11/f26 [0,4194304] 0 2024-04-12T05:55:59.358 INFO:tasks.workunit.client.1.smithi194.stdout:7/33: dread - d3/fc zero size 2024-04-12T05:55:59.360 INFO:tasks.workunit.client.1.smithi194.stdout:9/95: truncate d1/f18 4174414 0 2024-04-12T05:55:59.363 INFO:tasks.workunit.client.1.smithi194.stdout:7/34: write d3/fc [627340,64869] 0 2024-04-12T05:55:59.364 INFO:tasks.workunit.client.1.smithi194.stdout:9/96: mkdir d1/d19/d1f/d20 0 2024-04-12T05:55:59.366 INFO:tasks.workunit.client.1.smithi194.stdout:9/97: mkdir d1/d21 0 2024-04-12T05:55:59.368 INFO:tasks.workunit.client.1.smithi194.stdout:1/191: link d0/d11/d18/f2d d0/f32 0 2024-04-12T05:55:59.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/192: dread - d0/d11/d18/d19/d1a/d23/f29 zero size 2024-04-12T05:55:59.370 INFO:tasks.workunit.client.1.smithi194.stdout:1/193: dread - d0/d11/d18/d19/d1c/f2a zero size 2024-04-12T05:55:59.371 INFO:tasks.workunit.client.1.smithi194.stdout:9/98: stat d1/f18 0 2024-04-12T05:55:59.371 INFO:tasks.workunit.client.1.smithi194.stdout:1/194: chown d0/d11/c14 25 1 2024-04-12T05:55:59.375 INFO:tasks.workunit.client.1.smithi194.stdout:7/35: getdents d3/d4 0 2024-04-12T05:55:59.377 INFO:tasks.workunit.client.1.smithi194.stdout:1/195: creat d0/d11/d18/d19/d1a/d23/d27/d2e/f33 x:0 0 0 2024-04-12T05:55:59.379 INFO:tasks.workunit.client.1.smithi194.stdout:7/36: creat d3/fd x:0 0 0 2024-04-12T05:55:59.380 INFO:tasks.workunit.client.1.smithi194.stdout:7/37: write d3/fc [295986,38462] 0 2024-04-12T05:55:59.383 INFO:tasks.workunit.client.1.smithi194.stdout:7/38: mknod d3/d4/ce 0 2024-04-12T05:55:59.385 INFO:tasks.workunit.client.1.smithi194.stdout:7/39: chown d3/d4 3194769 1 2024-04-12T05:55:59.387 INFO:tasks.workunit.client.1.smithi194.stdout:1/196: dwrite d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [0,4194304] 0 2024-04-12T05:55:59.387 INFO:tasks.workunit.client.1.smithi194.stdout:7/40: symlink d3/lf 0 2024-04-12T05:55:59.390 INFO:tasks.workunit.client.1.smithi194.stdout:7/41: mknod d3/d5/c10 0 2024-04-12T05:55:59.393 INFO:tasks.workunit.client.1.smithi194.stdout:7/42: unlink d3/fc 0 2024-04-12T05:55:59.395 INFO:tasks.workunit.client.1.smithi194.stdout:7/43: dread d3/d4/f7 [0,4194304] 0 2024-04-12T05:55:59.396 INFO:tasks.workunit.client.1.smithi194.stdout:7/44: dread - f2 zero size 2024-04-12T05:55:59.398 INFO:tasks.workunit.client.1.smithi194.stdout:7/45: creat d3/d4/f11 x:0 0 0 2024-04-12T05:55:59.401 INFO:tasks.workunit.client.1.smithi194.stdout:9/99: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:55:59.405 INFO:tasks.workunit.client.1.smithi194.stdout:7/46: rename d3/d5 to d3/d4/d12 0 2024-04-12T05:55:59.406 INFO:tasks.workunit.client.1.smithi194.stdout:9/100: mkdir d1/d19/d1f/d20/d22 0 2024-04-12T05:55:59.406 INFO:tasks.workunit.client.1.smithi194.stdout:9/101: fdatasync d1/f14 0 2024-04-12T05:55:59.407 INFO:tasks.workunit.client.1.smithi194.stdout:9/102: write d1/f1d [3889874,6841] 0 2024-04-12T05:55:59.408 INFO:tasks.workunit.client.1.smithi194.stdout:9/103: readlink d1/l1c 0 2024-04-12T05:55:59.408 INFO:tasks.workunit.client.1.smithi194.stdout:7/47: symlink d3/l13 0 2024-04-12T05:55:59.408 INFO:tasks.workunit.client.1.smithi194.stdout:7/48: dread - d3/fd zero size 2024-04-12T05:55:59.411 INFO:tasks.workunit.client.1.smithi194.stdout:7/49: unlink d3/lf 0 2024-04-12T05:55:59.412 INFO:tasks.workunit.client.1.smithi194.stdout:7/50: write d3/fd [980250,107413] 0 2024-04-12T05:55:59.416 INFO:tasks.workunit.client.1.smithi194.stdout:7/51: rmdir d3/d4 39 2024-04-12T05:55:59.420 INFO:tasks.workunit.client.1.smithi194.stdout:9/104: truncate f0 2694492 0 2024-04-12T05:55:59.421 INFO:tasks.workunit.client.1.smithi194.stdout:7/52: write d3/d4/fa [1078182,115222] 0 2024-04-12T05:55:59.421 INFO:tasks.workunit.client.1.smithi194.stdout:7/53: chown d3/fd 1418601 1 2024-04-12T05:55:59.425 INFO:tasks.workunit.client.1.smithi194.stdout:9/105: dwrite d1/f14 [0,4194304] 0 2024-04-12T05:55:59.426 INFO:tasks.workunit.client.1.smithi194.stdout:9/106: stat d1/d19 0 2024-04-12T05:55:59.427 INFO:tasks.workunit.client.1.smithi194.stdout:9/107: write d1/f14 [3410980,87062] 0 2024-04-12T05:55:59.429 INFO:tasks.workunit.client.1.smithi194.stdout:9/108: symlink d1/d19/d1f/d20/l23 0 2024-04-12T05:55:59.429 INFO:tasks.workunit.client.1.smithi194.stdout:9/109: fdatasync d1/f1d 0 2024-04-12T05:55:59.431 INFO:tasks.workunit.client.1.smithi194.stdout:9/110: creat d1/f24 x:0 0 0 2024-04-12T05:55:59.437 INFO:tasks.workunit.client.1.smithi194.stdout:9/111: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:55:59.441 INFO:tasks.workunit.client.1.smithi194.stdout:9/112: creat d1/d19/d1f/d20/f25 x:0 0 0 2024-04-12T05:55:59.443 INFO:tasks.workunit.client.1.smithi194.stdout:9/113: mknod d1/d21/c26 0 2024-04-12T05:55:59.444 INFO:tasks.workunit.client.1.smithi194.stdout:9/114: fdatasync d1/f11 0 2024-04-12T05:55:59.445 INFO:tasks.workunit.client.1.smithi194.stdout:9/115: mkdir d1/d27 0 2024-04-12T05:55:59.462 INFO:tasks.workunit.client.1.smithi194.stdout:9/116: fsync d1/f14 0 2024-04-12T05:55:59.465 INFO:tasks.workunit.client.1.smithi194.stdout:9/117: dread d1/f5 [4194304,4194304] 0 2024-04-12T05:55:59.465 INFO:tasks.workunit.client.1.smithi194.stdout:9/118: chown d1/c1b 68628 1 2024-04-12T05:55:59.467 INFO:tasks.workunit.client.1.smithi194.stdout:9/119: mknod d1/d19/c28 0 2024-04-12T05:55:59.468 INFO:tasks.workunit.client.1.smithi194.stdout:9/120: getdents d1/d19/d1f/d20/d22 0 2024-04-12T05:55:59.471 INFO:tasks.workunit.client.1.smithi194.stdout:9/121: rename d1/f11 to d1/d19/d1f/d20/d22/f29 0 2024-04-12T05:55:59.473 INFO:tasks.workunit.client.1.smithi194.stdout:9/122: rmdir d1/d27 0 2024-04-12T05:55:59.474 INFO:tasks.workunit.client.1.smithi194.stdout:9/123: write d1/f13 [143054,48068] 0 2024-04-12T05:55:59.479 INFO:tasks.workunit.client.1.smithi194.stdout:9/124: dwrite d1/f13 [0,4194304] 0 2024-04-12T05:55:59.480 INFO:tasks.workunit.client.1.smithi194.stdout:9/125: read d1/f5 [6378642,67528] 0 2024-04-12T05:55:59.480 INFO:tasks.workunit.client.1.smithi194.stdout:9/126: stat d1/lc 0 2024-04-12T05:55:59.481 INFO:tasks.workunit.client.1.smithi194.stdout:9/127: write d1/f1d [725963,55264] 0 2024-04-12T05:55:59.484 INFO:tasks.workunit.client.1.smithi194.stdout:9/128: mknod d1/c2a 0 2024-04-12T05:55:59.487 INFO:tasks.workunit.client.1.smithi194.stdout:9/129: mknod d1/d19/d1f/d20/c2b 0 2024-04-12T05:55:59.489 INFO:tasks.workunit.client.1.smithi194.stdout:9/130: creat d1/d19/d1f/f2c x:0 0 0 2024-04-12T05:55:59.490 INFO:tasks.workunit.client.1.smithi194.stdout:9/131: write d1/f18 [3716576,22897] 0 2024-04-12T05:55:59.494 INFO:tasks.workunit.client.1.smithi194.stdout:9/132: dread d1/f18 [0,4194304] 0 2024-04-12T05:55:59.496 INFO:tasks.workunit.client.1.smithi194.stdout:9/133: truncate d1/d19/d1f/d20/d22/f29 546410 0 2024-04-12T05:55:59.497 INFO:tasks.workunit.client.1.smithi194.stdout:9/134: chown d1/f1d 109308 1 2024-04-12T05:55:59.497 INFO:tasks.workunit.client.1.smithi194.stdout:9/135: chown d1/c1b 16943818 1 2024-04-12T05:55:59.500 INFO:tasks.workunit.client.1.smithi194.stdout:9/136: mknod d1/d19/d1f/d20/c2d 0 2024-04-12T05:55:59.502 INFO:tasks.workunit.client.1.smithi194.stdout:9/137: rename d1/d19/d1f/d20/d22/f29 to d1/d19/f2e 0 2024-04-12T05:55:59.504 INFO:tasks.workunit.client.1.smithi194.stdout:9/138: write d1/fe [1625207,18684] 0 2024-04-12T05:55:59.510 INFO:tasks.workunit.client.1.smithi194.stdout:9/139: dwrite d1/d19/f2e [0,4194304] 0 2024-04-12T05:55:59.515 INFO:tasks.workunit.client.1.smithi194.stdout:9/140: rmdir d1/d19/d1f/d20/d22 0 2024-04-12T05:55:59.516 INFO:tasks.workunit.client.1.smithi194.stdout:9/141: mknod d1/d19/d1f/d20/c2f 0 2024-04-12T05:55:59.522 INFO:tasks.workunit.client.1.smithi194.stdout:9/142: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:55:59.523 INFO:tasks.workunit.client.1.smithi194.stdout:9/143: write d1/f1d [57412,20153] 0 2024-04-12T05:55:59.530 INFO:tasks.workunit.client.1.smithi194.stdout:9/144: dread f0 [0,4194304] 0 2024-04-12T05:55:59.531 INFO:tasks.workunit.client.1.smithi194.stdout:9/145: chown d1/d19/d1f 223 1 2024-04-12T05:55:59.535 INFO:tasks.workunit.client.1.smithi194.stdout:9/146: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:55:59.536 INFO:tasks.workunit.client.1.smithi194.stdout:9/147: symlink d1/l30 0 2024-04-12T05:55:59.538 INFO:tasks.workunit.client.1.smithi194.stdout:9/148: creat d1/d19/d1f/d20/f31 x:0 0 0 2024-04-12T05:55:59.563 INFO:tasks.workunit.client.1.smithi194.stdout:8/107: sync 2024-04-12T05:55:59.569 INFO:tasks.workunit.client.1.smithi194.stdout:8/108: write d2/d8/d9/de/d10/f24 [790130,105682] 0 2024-04-12T05:55:59.574 INFO:tasks.workunit.client.1.smithi194.stdout:9/149: dwrite d1/f24 [0,4194304] 0 2024-04-12T05:55:59.576 INFO:tasks.workunit.client.1.smithi194.stdout:8/109: truncate d2/fb 1076072 0 2024-04-12T05:55:59.585 INFO:tasks.workunit.client.1.smithi194.stdout:9/150: truncate d1/fe 3610614 0 2024-04-12T05:55:59.586 INFO:tasks.workunit.client.1.smithi194.stdout:9/151: chown d1/d19/d1f/f2c 1 1 2024-04-12T05:55:59.587 INFO:tasks.workunit.client.1.smithi194.stdout:9/152: write d1/d19/d1f/f2c [674556,6151] 0 2024-04-12T05:55:59.588 INFO:tasks.workunit.client.1.smithi194.stdout:9/153: mknod d1/d21/c32 0 2024-04-12T05:55:59.592 INFO:tasks.workunit.client.1.smithi194.stdout:9/154: dread d1/f13 [0,4194304] 0 2024-04-12T05:55:59.594 INFO:tasks.workunit.client.1.smithi194.stdout:9/155: symlink d1/l33 0 2024-04-12T05:55:59.599 INFO:tasks.workunit.client.1.smithi194.stdout:9/156: dwrite d1/f13 [0,4194304] 0 2024-04-12T05:55:59.605 INFO:tasks.workunit.client.1.smithi194.stdout:9/157: dread d1/fb [0,4194304] 0 2024-04-12T05:55:59.606 INFO:tasks.workunit.client.1.smithi194.stdout:9/158: write d1/f1d [3071869,24557] 0 2024-04-12T05:55:59.608 INFO:tasks.workunit.client.1.smithi194.stdout:9/159: unlink d1/f14 0 2024-04-12T05:55:59.609 INFO:tasks.workunit.client.1.smithi194.stdout:9/160: write d1/d19/d1f/d20/f31 [802523,125876] 0 2024-04-12T05:55:59.612 INFO:tasks.workunit.client.1.smithi194.stdout:9/161: unlink d1/c6 0 2024-04-12T05:55:59.612 INFO:tasks.workunit.client.1.smithi194.stdout:9/162: chown d1/c1a 60080731 1 2024-04-12T05:55:59.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.617+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9d00611d0 con 0x7fd9bc06ca10 2024-04-12T05:55:59.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.619+0000 7fd9c6ffd700 1 -- 172.21.15.8:0/3762881600 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fd9d00611d0 con 0x7fd9bc06ca10 2024-04-12T05:55:59.614 INFO:tasks.workunit.client.1.smithi194.stdout:9/163: mknod d1/d19/c34 0 2024-04-12T05:55:59.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 msgr2=0x7fd9bc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 0x7fd9bc06eed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0005950 tx=0x7fd9c000b410 comp rx=0 tx=0).stop 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 msgr2=0x7fd9d0196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b5c0 tx=0x7fd9cc00bd70 comp rx=0 tx=0).stop 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 shutdown_connections 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7fd9bc06ca10 0x7fd9bc06eed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd9d0101790 0x7fd9d0196390 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b5c0 tx=0x7fd9cc00bd70 comp rx=0 tx=0).stop 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 --2- 172.21.15.8:0/3762881600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd9d01040c0 0x7fd9d01968d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.621+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 >> 172.21.15.8:0/3762881600 conn(0x7fd9d00fb120 msgr2=0x7fd9d00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.622+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 shutdown_connections 2024-04-12T05:55:59.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.622+0000 7fd9d7f21700 1 -- 172.21.15.8:0/3762881600 wait complete. 2024-04-12T05:55:59.618 INFO:tasks.workunit.client.1.smithi194.stdout:9/164: rename d1/f24 to d1/d19/d1f/d20/f35 0 2024-04-12T05:55:59.619 INFO:tasks.workunit.client.1.smithi194.stdout:9/165: unlink d1/d19/l17 0 2024-04-12T05:55:59.620 INFO:tasks.workunit.client.1.smithi194.stdout:9/166: chown d1/d19/d1f/d20 56660733 1 2024-04-12T05:55:59.622 INFO:tasks.workunit.client.1.smithi194.stdout:9/167: creat d1/d19/f36 x:0 0 0 2024-04-12T05:55:59.625 INFO:tasks.workunit.client.1.smithi194.stdout:9/168: mkdir d1/d19/d1f/d20/d37 0 2024-04-12T05:55:59.633 INFO:tasks.workunit.client.1.smithi194.stdout:9/169: mkdir d1/d19/d1f/d20/d37/d38 0 2024-04-12T05:55:59.639 INFO:tasks.workunit.client.1.smithi194.stdout:9/170: truncate d1/f10 50519 0 2024-04-12T05:55:59.647 INFO:tasks.workunit.client.1.smithi194.stdout:9/171: read d1/f10 [23,115789] 0 2024-04-12T05:55:59.650 INFO:tasks.workunit.client.1.smithi194.stdout:9/172: mkdir d1/d21/d39 0 2024-04-12T05:55:59.652 INFO:tasks.workunit.client.1.smithi194.stdout:9/173: mknod d1/c3a 0 2024-04-12T05:55:59.653 INFO:tasks.workunit.client.1.smithi194.stdout:9/174: mknod d1/d21/d39/c3b 0 2024-04-12T05:55:59.654 INFO:tasks.workunit.client.1.smithi194.stdout:9/175: write d1/d19/d1f/f2c [311054,125839] 0 2024-04-12T05:55:59.655 INFO:tasks.workunit.client.1.smithi194.stdout:9/176: write d1/d19/d1f/f2c [453721,36168] 0 2024-04-12T05:55:59.656 INFO:tasks.workunit.client.1.smithi194.stdout:9/177: write d1/d19/d1f/d20/f25 [709521,101764] 0 2024-04-12T05:55:59.658 INFO:tasks.workunit.client.1.smithi194.stdout:9/178: mknod d1/d19/d1f/d20/c3c 0 2024-04-12T05:55:59.661 INFO:tasks.workunit.client.1.smithi194.stdout:9/179: rename d1/d19/f36 to d1/d19/d1f/d20/f3d 0 2024-04-12T05:55:59.664 INFO:tasks.workunit.client.1.smithi194.stdout:9/180: dread d1/d19/f2e [0,4194304] 0 2024-04-12T05:55:59.666 INFO:tasks.workunit.client.1.smithi194.stdout:9/181: write d1/d19/d1f/d20/f3d [397427,61151] 0 2024-04-12T05:55:59.671 INFO:tasks.workunit.client.1.smithi194.stdout:9/182: dread d1/fe [0,4194304] 0 2024-04-12T05:55:59.673 INFO:tasks.workunit.client.1.smithi194.stdout:9/183: unlink d1/l30 0 2024-04-12T05:55:59.674 INFO:tasks.workunit.client.1.smithi194.stdout:9/184: creat d1/d21/f3e x:0 0 0 2024-04-12T05:55:59.676 INFO:tasks.workunit.client.1.smithi194.stdout:9/185: creat d1/f3f x:0 0 0 2024-04-12T05:55:59.678 INFO:tasks.workunit.client.1.smithi194.stdout:9/186: rmdir d1/d19/d1f 39 2024-04-12T05:55:59.682 INFO:tasks.workunit.client.1.smithi194.stdout:9/187: dread d1/d19/d1f/d20/f31 [0,4194304] 0 2024-04-12T05:55:59.684 INFO:tasks.workunit.client.1.smithi194.stdout:9/188: symlink d1/d19/d1f/d20/d37/d38/l40 0 2024-04-12T05:55:59.685 INFO:tasks.workunit.client.1.smithi194.stdout:9/189: stat d1 0 2024-04-12T05:55:59.685 INFO:tasks.workunit.client.1.smithi194.stdout:9/190: chown d1/d21/d39 32051 1 2024-04-12T05:55:59.687 INFO:tasks.workunit.client.1.smithi194.stdout:9/191: dread f0 [0,4194304] 0 2024-04-12T05:55:59.690 INFO:tasks.workunit.client.1.smithi194.stdout:9/192: dread d1/d19/f2e [0,4194304] 0 2024-04-12T05:55:59.690 INFO:tasks.workunit.client.1.smithi194.stdout:9/193: dread - d1/d21/f3e zero size 2024-04-12T05:55:59.694 INFO:tasks.workunit.client.1.smithi194.stdout:9/194: write d1/fe [2230217,4030] 0 2024-04-12T05:55:59.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 -- 172.21.15.8:0/565627910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c4101eb0 msgr2=0x7f63c4104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 --2- 172.21.15.8:0/565627910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c4101eb0 0x7f63c4104340 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009910 tx=0x7f63b4009c20 comp rx=0 tx=0).stop 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 -- 172.21.15.8:0/565627910 shutdown_connections 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 --2- 172.21.15.8:0/565627910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c4101eb0 0x7f63c4104340 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 --2- 172.21.15.8:0/565627910 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c40ff540 0x7f63c4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 -- 172.21.15.8:0/565627910 >> 172.21.15.8:0/565627910 conn(0x7f63c40fb160 msgr2=0x7f63c40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 -- 172.21.15.8:0/565627910 shutdown_connections 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.769+0000 7f63cbda5700 1 -- 172.21.15.8:0/565627910 wait complete. 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 Processor -- start 2024-04-12T05:55:59.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 -- start start 2024-04-12T05:55:59.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c40ff540 0x7f63c419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c419d850 con 0x7f63c4101eb0 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63cbda5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c41a2260 con 0x7f63c40ff540 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9340700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9340700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39708/0 (socket says 172.21.15.8:39708) 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9340700 1 -- 172.21.15.8:0/1473703603 learned_addr learned my addr 172.21.15.8:0/1473703603 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9b41700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c40ff540 0x7f63c419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9340700 1 -- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c40ff540 msgr2=0x7f63c419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.770+0000 7f63c9340700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c40ff540 0x7f63c419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63c9340700 1 -- 172.21.15.8:0/1473703603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b40095f0 con 0x7f63c4101eb0 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63c9340700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:59.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b401d070 con 0x7f63c4101eb0 2024-04-12T05:55:59.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f63b400f460 con 0x7f63c4101eb0 2024-04-12T05:55:59.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b4021640 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c41a2400 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.771+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c41a28c0 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.773+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c4196ae0 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.773+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f63b400f9b0 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.774+0000 7f63baffd700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 0x7f63b006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.774+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f63b408d340 con 0x7f63c4101eb0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.774+0000 7f63c9b41700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 0x7f63b006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:55:59.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.775+0000 7f63c9b41700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 0x7f63b006eee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63c419e1d0 tx=0x7f63c000b410 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:55:59.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:55:59.783+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f63b4057370 con 0x7f63c4101eb0 2024-04-12T05:55:59.780 INFO:tasks.workunit.client.1.smithi194.stdout:0/93: sync 2024-04-12T05:55:59.780 INFO:tasks.workunit.client.1.smithi194.stdout:2/111: sync 2024-04-12T05:55:59.783 INFO:tasks.workunit.client.1.smithi194.stdout:2/112: fsync d7/d12/f14 0 2024-04-12T05:55:59.784 INFO:tasks.workunit.client.1.smithi194.stdout:2/113: write d7/d22/f2e [282938,29986] 0 2024-04-12T05:55:59.790 INFO:tasks.workunit.client.1.smithi194.stdout:2/114: dread d7/d12/d1b/f1d [0,4194304] 0 2024-04-12T05:55:59.791 INFO:tasks.workunit.client.1.smithi194.stdout:0/94: dwrite d0/d5/fe [0,4194304] 0 2024-04-12T05:55:59.794 INFO:tasks.workunit.client.1.smithi194.stdout:2/115: rename d7/d12/d1a/f23 to d7/d22/d2b/f2f 0 2024-04-12T05:55:59.796 INFO:tasks.workunit.client.1.smithi194.stdout:0/95: unlink d0/d5/c7 0 2024-04-12T05:55:59.796 INFO:tasks.workunit.client.1.smithi194.stdout:0/96: fdatasync d0/f2 0 2024-04-12T05:55:59.800 INFO:tasks.workunit.client.1.smithi194.stdout:0/97: creat d0/d5/d1a/f1c x:0 0 0 2024-04-12T05:55:59.802 INFO:tasks.workunit.client.1.smithi194.stdout:2/116: dwrite d7/d12/d1b/f1e [0,4194304] 0 2024-04-12T05:55:59.811 INFO:tasks.workunit.client.1.smithi194.stdout:0/98: dwrite d0/d5/f19 [0,4194304] 0 2024-04-12T05:55:59.812 INFO:tasks.workunit.client.1.smithi194.stdout:0/99: write d0/f10 [411088,43079] 0 2024-04-12T05:55:59.815 INFO:tasks.workunit.client.1.smithi194.stdout:0/100: creat d0/f1d x:0 0 0 2024-04-12T05:55:59.831 INFO:tasks.workunit.client.1.smithi194.stdout:0/101: rename d0/d5 to d0/da/d1e 0 2024-04-12T05:55:59.837 INFO:tasks.workunit.client.1.smithi194.stdout:2/117: dwrite d7/d12/d1a/f24 [0,4194304] 0 2024-04-12T05:55:59.842 INFO:tasks.workunit.client.1.smithi194.stdout:2/118: symlink d7/l30 0 2024-04-12T05:55:59.844 INFO:tasks.workunit.client.1.smithi194.stdout:0/102: dwrite d0/f2 [0,4194304] 0 2024-04-12T05:55:59.845 INFO:tasks.workunit.client.1.smithi194.stdout:2/119: chown d7/c8 99 1 2024-04-12T05:55:59.852 INFO:tasks.workunit.client.1.smithi194.stdout:2/120: creat d7/d12/d25/f31 x:0 0 0 2024-04-12T05:55:59.856 INFO:tasks.workunit.client.1.smithi194.stdout:2/121: symlink d7/d12/d1a/l32 0 2024-04-12T05:55:59.858 INFO:tasks.workunit.client.1.smithi194.stdout:0/103: read d0/da/d1e/f17 [72172,1127] 0 2024-04-12T05:55:59.866 INFO:tasks.workunit.client.1.smithi194.stdout:2/122: creat d7/d12/f33 x:0 0 0 2024-04-12T05:55:59.867 INFO:tasks.workunit.client.1.smithi194.stdout:0/104: dwrite d0/f1d [0,4194304] 0 2024-04-12T05:55:59.870 INFO:tasks.workunit.client.1.smithi194.stdout:2/123: rmdir d7/d22/d2b 39 2024-04-12T05:55:59.871 INFO:tasks.workunit.client.1.smithi194.stdout:2/124: chown d7/d22/f27 0 1 2024-04-12T05:55:59.872 INFO:tasks.workunit.client.1.smithi194.stdout:0/105: mknod d0/da/c1f 0 2024-04-12T05:55:59.875 INFO:tasks.workunit.client.1.smithi194.stdout:2/125: unlink d7/ce 0 2024-04-12T05:55:59.880 INFO:tasks.workunit.client.1.smithi194.stdout:2/126: mkdir d7/d12/d34 0 2024-04-12T05:55:59.881 INFO:tasks.workunit.client.1.smithi194.stdout:2/127: fdatasync d7/fa 0 2024-04-12T05:55:59.882 INFO:tasks.workunit.client.1.smithi194.stdout:0/106: dwrite d0/f1d [0,4194304] 0 2024-04-12T05:55:59.886 INFO:tasks.workunit.client.1.smithi194.stdout:0/107: symlink d0/da/l20 0 2024-04-12T05:55:59.893 INFO:tasks.workunit.client.1.smithi194.stdout:2/128: link d7/d12/d1b/c28 d7/d12/d25/c35 0 2024-04-12T05:55:59.894 INFO:tasks.workunit.client.1.smithi194.stdout:2/129: dread - d7/d12/d1a/f2d zero size 2024-04-12T05:55:59.897 INFO:tasks.workunit.client.1.smithi194.stdout:2/130: mknod d7/d22/d2b/c36 0 2024-04-12T05:55:59.900 INFO:tasks.workunit.client.1.smithi194.stdout:2/131: mknod d7/c37 0 2024-04-12T05:55:59.900 INFO:tasks.workunit.client.1.smithi194.stdout:0/108: dwrite d0/da/d1e/fe [0,4194304] 0 2024-04-12T05:55:59.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:55:59 smithi008 ceph-mon[39722]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:55:59.910 INFO:tasks.workunit.client.1.smithi194.stdout:2/132: dwrite d7/d22/f2e [0,4194304] 0 2024-04-12T05:55:59.916 INFO:tasks.workunit.client.1.smithi194.stdout:2/133: dread d7/f11 [0,4194304] 0 2024-04-12T05:55:59.919 INFO:tasks.workunit.client.1.smithi194.stdout:2/134: creat d7/d12/f38 x:0 0 0 2024-04-12T05:55:59.923 INFO:tasks.workunit.client.1.smithi194.stdout:0/109: truncate d0/da/d1e/fe 3666193 0 2024-04-12T05:55:59.923 INFO:tasks.workunit.client.1.smithi194.stdout:0/110: rename d0/da to d0/da/d21 22 2024-04-12T05:55:59.926 INFO:tasks.workunit.client.1.smithi194.stdout:2/135: rename d7/d22/d2b/c36 to d7/d12/d34/c39 0 2024-04-12T05:55:59.927 INFO:tasks.workunit.client.1.smithi194.stdout:2/136: chown d7/d22/f27 316 1 2024-04-12T05:55:59.930 INFO:tasks.workunit.client.1.smithi194.stdout:0/111: unlink d0/da/d1e/l12 0 2024-04-12T05:55:59.942 INFO:tasks.workunit.client.1.smithi194.stdout:0/112: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:55:59.942 INFO:tasks.workunit.client.1.smithi194.stdout:2/137: write d7/d22/f2e [4677045,85346] 0 2024-04-12T05:55:59.951 INFO:tasks.workunit.client.1.smithi194.stdout:2/138: creat d7/d12/d34/f3a x:0 0 0 2024-04-12T05:55:59.952 INFO:tasks.workunit.client.1.smithi194.stdout:2/139: stat d7/d12/d25 0 2024-04-12T05:55:59.955 INFO:tasks.workunit.client.1.smithi194.stdout:0/113: link d0/da/c18 d0/da/d1e/d1a/c22 0 2024-04-12T05:55:59.956 INFO:tasks.workunit.client.1.smithi194.stdout:6/117: sync 2024-04-12T05:55:59.965 INFO:tasks.workunit.client.1.smithi194.stdout:2/140: dwrite d7/d12/d1b/f1d [0,4194304] 0 2024-04-12T05:55:59.970 INFO:tasks.workunit.client.1.smithi194.stdout:6/118: dwrite d3/d15/f8 [0,4194304] 0 2024-04-12T05:55:59.970 INFO:tasks.workunit.client.1.smithi194.stdout:0/114: dread d0/f4 [0,4194304] 0 2024-04-12T05:55:59.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/141: dread d7/d22/f2e [0,4194304] 0 2024-04-12T05:55:59.976 INFO:tasks.workunit.client.1.smithi194.stdout:6/119: mknod d3/c1b 0 2024-04-12T05:55:59.981 INFO:tasks.workunit.client.1.smithi194.stdout:2/142: unlink d7/d22/f27 0 2024-04-12T05:55:59.981 INFO:tasks.workunit.client.1.smithi194.stdout:6/120: dread d3/f17 [0,4194304] 0 2024-04-12T05:55:59.983 INFO:tasks.workunit.client.1.smithi194.stdout:2/143: creat d7/d12/f3b x:0 0 0 2024-04-12T05:55:59.983 INFO:tasks.workunit.client.1.smithi194.stdout:6/121: truncate d3/d15/f14 316206 0 2024-04-12T05:55:59.985 INFO:tasks.workunit.client.1.smithi194.stdout:6/122: read d3/d15/f8 [967783,29972] 0 2024-04-12T05:55:59.985 INFO:tasks.workunit.client.1.smithi194.stdout:2/144: dread - d7/d12/d1a/f2d zero size 2024-04-12T05:55:59.988 INFO:tasks.workunit.client.1.smithi194.stdout:2/145: read - d7/d12/d1b/f2a zero size 2024-04-12T05:55:59.989 INFO:tasks.workunit.client.1.smithi194.stdout:6/123: getdents d3/d1a 0 2024-04-12T05:55:59.991 INFO:tasks.workunit.client.1.smithi194.stdout:2/146: mknod d7/c3c 0 2024-04-12T05:55:59.993 INFO:tasks.workunit.client.1.smithi194.stdout:2/147: creat d7/d12/f3d x:0 0 0 2024-04-12T05:55:59.993 INFO:tasks.workunit.client.1.smithi194.stdout:2/148: chown d7/d12 139108 1 2024-04-12T05:55:59.994 INFO:tasks.workunit.client.1.smithi194.stdout:2/149: dread - d7/d12/f3b zero size 2024-04-12T05:55:59.994 INFO:tasks.workunit.client.1.smithi194.stdout:6/124: link d3/d15/f13 d3/f1c 0 2024-04-12T05:55:59.995 INFO:tasks.workunit.client.1.smithi194.stdout:6/125: chown d3/d15/c10 366070881 1 2024-04-12T05:56:00.001 INFO:tasks.workunit.client.1.smithi194.stdout:6/126: dwrite d3/f18 [4194304,4194304] 0 2024-04-12T05:56:00.013 INFO:tasks.workunit.client.1.smithi194.stdout:6/127: dwrite d3/f1c [0,4194304] 0 2024-04-12T05:56:00.015 INFO:tasks.workunit.client.1.smithi194.stdout:6/128: dread d3/f17 [0,4194304] 0 2024-04-12T05:56:00.023 INFO:tasks.workunit.client.1.smithi194.stdout:6/129: dread d3/fe [0,4194304] 0 2024-04-12T05:56:00.025 INFO:tasks.workunit.client.1.smithi194.stdout:6/130: mknod d3/c1d 0 2024-04-12T05:56:00.026 INFO:tasks.workunit.client.1.smithi194.stdout:6/131: write d3/d15/f8 [2303728,38797] 0 2024-04-12T05:56:00.027 INFO:tasks.workunit.client.1.smithi194.stdout:6/132: rmdir d3/d15 39 2024-04-12T05:56:00.030 INFO:tasks.workunit.client.1.smithi194.stdout:6/133: creat d3/f1e x:0 0 0 2024-04-12T05:56:00.044 INFO:tasks.workunit.client.1.smithi194.stdout:6/134: dread d3/f16 [0,4194304] 0 2024-04-12T05:56:00.045 INFO:tasks.workunit.client.1.smithi194.stdout:6/135: write d3/f17 [1306927,100939] 0 2024-04-12T05:56:00.050 INFO:tasks.workunit.client.1.smithi194.stdout:6/136: rename d3/d15/f8 to d3/f1f 0 2024-04-12T05:56:00.050 INFO:tasks.workunit.client.1.smithi194.stdout:6/137: stat d3/c1b 0 2024-04-12T05:56:00.052 INFO:tasks.workunit.client.1.smithi194.stdout:6/138: creat d3/f20 x:0 0 0 2024-04-12T05:56:00.053 INFO:tasks.workunit.client.1.smithi194.stdout:6/139: getdents d3/d1a 0 2024-04-12T05:56:00.053 INFO:tasks.workunit.client.1.smithi194.stdout:6/140: fdatasync d3/f16 0 2024-04-12T05:56:00.054 INFO:tasks.workunit.client.1.smithi194.stdout:6/141: chown d3/d15/f13 102272 1 2024-04-12T05:56:00.054 INFO:tasks.workunit.client.1.smithi194.stdout:6/142: chown d3 2 1 2024-04-12T05:56:00.059 INFO:tasks.workunit.client.1.smithi194.stdout:6/143: dwrite d3/fe [0,4194304] 0 2024-04-12T05:56:00.061 INFO:tasks.workunit.client.1.smithi194.stdout:6/144: write d3/d15/f13 [1581030,113719] 0 2024-04-12T05:56:00.065 INFO:tasks.workunit.client.1.smithi194.stdout:6/145: creat d3/d15/f21 x:0 0 0 2024-04-12T05:56:00.065 INFO:tasks.workunit.client.1.smithi194.stdout:6/146: stat d3/d15 0 2024-04-12T05:56:00.067 INFO:tasks.workunit.client.1.smithi194.stdout:6/147: write d3/f16 [2308394,26043] 0 2024-04-12T05:56:00.067 INFO:tasks.workunit.client.1.smithi194.stdout:6/148: chown d3/f1c 451705 1 2024-04-12T05:56:00.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:55:59 smithi194 ceph-mon[49771]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:00.070 INFO:tasks.workunit.client.1.smithi194.stdout:6/149: creat d3/d15/f22 x:0 0 0 2024-04-12T05:56:00.071 INFO:tasks.workunit.client.1.smithi194.stdout:6/150: creat d3/d1a/f23 x:0 0 0 2024-04-12T05:56:00.075 INFO:tasks.workunit.client.1.smithi194.stdout:6/151: dread d3/fe [0,4194304] 0 2024-04-12T05:56:00.076 INFO:tasks.workunit.client.1.smithi194.stdout:6/152: write d3/f1f [2418834,117462] 0 2024-04-12T05:56:00.076 INFO:tasks.workunit.client.1.smithi194.stdout:6/153: stat d3/f20 0 2024-04-12T05:56:00.079 INFO:tasks.workunit.client.1.smithi194.stdout:6/154: rename d3/f1c to d3/d15/f24 0 2024-04-12T05:56:00.081 INFO:tasks.workunit.client.1.smithi194.stdout:6/155: symlink d3/d1a/l25 0 2024-04-12T05:56:00.081 INFO:tasks.workunit.client.1.smithi194.stdout:6/156: stat d3/d15/f13 0 2024-04-12T05:56:00.083 INFO:tasks.workunit.client.1.smithi194.stdout:6/157: symlink d3/l26 0 2024-04-12T05:56:00.084 INFO:tasks.workunit.client.1.smithi194.stdout:6/158: symlink d3/d15/l27 0 2024-04-12T05:56:00.087 INFO:tasks.workunit.client.1.smithi194.stdout:6/159: dread d3/f1f [0,4194304] 0 2024-04-12T05:56:00.089 INFO:tasks.workunit.client.1.smithi194.stdout:6/160: creat d3/d15/f28 x:0 0 0 2024-04-12T05:56:00.091 INFO:tasks.workunit.client.1.smithi194.stdout:6/161: rename d3/f16 to d3/d1a/f29 0 2024-04-12T05:56:00.092 INFO:tasks.workunit.client.1.smithi194.stdout:6/162: readlink d3/l26 0 2024-04-12T05:56:00.095 INFO:tasks.workunit.client.1.smithi194.stdout:6/163: dread d3/fe [0,4194304] 0 2024-04-12T05:56:00.097 INFO:tasks.workunit.client.1.smithi194.stdout:6/164: symlink d3/d15/l2a 0 2024-04-12T05:56:00.098 INFO:tasks.workunit.client.1.smithi194.stdout:6/165: mkdir d3/d15/d2b 0 2024-04-12T05:56:00.099 INFO:tasks.workunit.client.1.smithi194.stdout:6/166: dread - d3/d1a/f23 zero size 2024-04-12T05:56:00.099 INFO:tasks.workunit.client.1.smithi194.stdout:6/167: chown d3/d15/c11 2101 1 2024-04-12T05:56:00.100 INFO:tasks.workunit.client.1.smithi194.stdout:6/168: dread - d3/d1a/f23 zero size 2024-04-12T05:56:00.103 INFO:tasks.workunit.client.1.smithi194.stdout:6/169: link c2 d3/d1a/c2c 0 2024-04-12T05:56:00.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.110+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f63c402d090 con 0x7f63b006ca20 2024-04-12T05:56:00.108 INFO:tasks.workunit.client.1.smithi194.stdout:6/170: dwrite d3/f18 [0,4194304] 0 2024-04-12T05:56:00.109 INFO:tasks.workunit.client.1.smithi194.stdout:6/171: read d3/f18 [870777,118989] 0 2024-04-12T05:56:00.112 INFO:tasks.workunit.client.1.smithi194.stdout:6/172: write d3/fe [4642266,47603] 0 2024-04-12T05:56:00.114 INFO:tasks.workunit.client.1.smithi194.stdout:6/173: mkdir d3/d2d 0 2024-04-12T05:56:00.121 INFO:tasks.workunit.client.1.smithi194.stdout:6/174: dwrite d3/f18 [0,4194304] 0 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.128+0000 7f63baffd700 1 -- 172.21.15.8:0/1473703603 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f63c402d090 con 0x7f63b006ca20 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (7m) 13s ago 9m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (9m) 13s ago 9m 7721k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (8m) 39s ago 8m 7662k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:56:00.123 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (9m) 13s ago 9m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (8m) 39s ago 8m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (7m) 13s ago 8m 87.5M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (3m) 13s ago 3m 98.0M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (3m) 13s ago 3m 55.9M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (3m) 39s ago 3m 45.9M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (3m) 39s ago 3m 65.7M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:9283,8765,8443 running (10m) 13s ago 10m 156M - 18.2.1 5be31c24972a 545bb7b78dec 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (61s) 39s ago 8m 476M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b06c49f54d22 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (10m) 13s ago 11m 50.6M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (8m) 39s ago 8m 40.3M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (9m) 13s ago 9m 17.0M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (8m) 39s ago 8m 18.2M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 13s ago 7m 102M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (6m) 13s ago 6m 105M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:56:00.124 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (6m) 13s ago 6m 88.3M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:56:00.125 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (5m) 39s ago 5m 105M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:56:00.125 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (5m) 39s ago 5m 103M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:56:00.125 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (4m) 39s ago 4m 93.4M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:56:00.125 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 starting - - - - 2024-04-12T05:56:00.125 INFO:tasks.workunit.client.1.smithi194.stdout:6/175: creat d3/d15/f2e x:0 0 0 2024-04-12T05:56:00.127 INFO:tasks.workunit.client.1.smithi194.stdout:6/176: unlink d3/d15/c10 0 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 msgr2=0x7f63b006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 0x7f63b006eee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63c419e1d0 tx=0x7f63c000b410 comp rx=0 tx=0).stop 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 msgr2=0x7f63c419d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b400bd70 comp rx=0 tx=0).stop 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 shutdown_connections 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f63b006ca20 0x7f63b006eee0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f63c419e1d0 tx=0x7f63c000b410 comp rx=0 tx=0).stop 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63c40ff540 0x7f63c419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 --2- 172.21.15.8:0/1473703603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63c4101eb0 0x7f63c419d230 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 >> 172.21.15.8:0/1473703603 conn(0x7f63c40fb160 msgr2=0x7f63c40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 shutdown_connections 2024-04-12T05:56:00.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.132+0000 7f63cbda5700 1 -- 172.21.15.8:0/1473703603 wait complete. 2024-04-12T05:56:00.129 INFO:tasks.workunit.client.1.smithi194.stdout:6/177: write d3/f17 [1030220,16952] 0 2024-04-12T05:56:00.131 INFO:tasks.workunit.client.1.smithi194.stdout:6/178: unlink c2 0 2024-04-12T05:56:00.133 INFO:tasks.workunit.client.1.smithi194.stdout:6/179: mkdir d3/d2f 0 2024-04-12T05:56:00.153 INFO:tasks.workunit.client.1.smithi194.stdout:6/180: getdents d3/d15 0 2024-04-12T05:56:00.157 INFO:tasks.workunit.client.1.smithi194.stdout:6/181: getdents d3/d2f 0 2024-04-12T05:56:00.158 INFO:tasks.workunit.client.1.smithi194.stdout:6/182: chown d3/d15/f22 158575 1 2024-04-12T05:56:00.161 INFO:tasks.workunit.client.1.smithi194.stdout:6/183: symlink d3/d15/l30 0 2024-04-12T05:56:00.163 INFO:tasks.workunit.client.1.smithi194.stdout:6/184: dread d3/d15/f14 [0,4194304] 0 2024-04-12T05:56:00.163 INFO:tasks.workunit.client.1.smithi194.stdout:6/185: chown d3/d15/f14 73 1 2024-04-12T05:56:00.165 INFO:tasks.workunit.client.1.smithi194.stdout:6/186: mknod d3/d15/d2b/c31 0 2024-04-12T05:56:00.167 INFO:tasks.workunit.client.1.smithi194.stdout:6/187: creat d3/d2d/f32 x:0 0 0 2024-04-12T05:56:00.169 INFO:tasks.workunit.client.1.smithi194.stdout:6/188: mkdir d3/d2d/d33 0 2024-04-12T05:56:00.170 INFO:tasks.workunit.client.1.smithi194.stdout:6/189: write d3/d15/f14 [531537,90] 0 2024-04-12T05:56:00.172 INFO:tasks.workunit.client.1.smithi194.stdout:6/190: mkdir d3/d34 0 2024-04-12T05:56:00.178 INFO:tasks.workunit.client.1.smithi194.stdout:6/191: dwrite d3/d15/f2e [0,4194304] 0 2024-04-12T05:56:00.179 INFO:tasks.workunit.client.1.smithi194.stdout:6/192: write d3/d15/f28 [864850,36348] 0 2024-04-12T05:56:00.180 INFO:tasks.workunit.client.1.smithi194.stdout:6/193: chown d3/d2f 501788553 1 2024-04-12T05:56:00.187 INFO:tasks.workunit.client.1.smithi194.stdout:6/194: creat d3/d1a/f35 x:0 0 0 2024-04-12T05:56:00.190 INFO:tasks.workunit.client.1.smithi194.stdout:6/195: rename d3/d15/l30 to d3/d2f/l36 0 2024-04-12T05:56:00.190 INFO:tasks.workunit.client.1.smithi194.stdout:6/196: read - d3/d2d/f32 zero size 2024-04-12T05:56:00.192 INFO:tasks.workunit.client.1.smithi194.stdout:6/197: unlink d3/d15/l27 0 2024-04-12T05:56:00.193 INFO:tasks.workunit.client.1.smithi194.stdout:6/198: mkdir d3/d2f/d37 0 2024-04-12T05:56:00.194 INFO:tasks.workunit.client.1.smithi194.stdout:6/199: write d3/d15/f22 [329286,19756] 0 2024-04-12T05:56:00.196 INFO:tasks.workunit.client.1.smithi194.stdout:6/200: unlink d3/c1d 0 2024-04-12T05:56:00.198 INFO:tasks.workunit.client.1.smithi194.stdout:6/201: mknod d3/d2d/d33/c38 0 2024-04-12T05:56:00.199 INFO:tasks.workunit.client.1.smithi194.stdout:6/202: creat d3/d2f/f39 x:0 0 0 2024-04-12T05:56:00.203 INFO:tasks.workunit.client.1.smithi194.stdout:6/203: rename d3/d15/f13 to d3/f3a 0 2024-04-12T05:56:00.203 INFO:tasks.workunit.client.1.smithi194.stdout:6/204: chown d3/d15/f2e 160172771 1 2024-04-12T05:56:00.204 INFO:tasks.workunit.client.1.smithi194.stdout:6/205: dread - d3/d2f/f39 zero size 2024-04-12T05:56:00.207 INFO:tasks.workunit.client.1.smithi194.stdout:6/206: dread d3/f18 [4194304,4194304] 0 2024-04-12T05:56:00.215 INFO:tasks.workunit.client.1.smithi194.stdout:6/207: dread d3/d1a/f29 [0,4194304] 0 2024-04-12T05:56:00.218 INFO:tasks.workunit.client.1.smithi194.stdout:6/208: creat d3/f3b x:0 0 0 2024-04-12T05:56:00.221 INFO:tasks.workunit.client.1.smithi194.stdout:6/209: rename d3/c19 to d3/d2f/c3c 0 2024-04-12T05:56:00.226 INFO:tasks.workunit.client.1.smithi194.stdout:6/210: getdents d3/d2f 0 2024-04-12T05:56:00.226 INFO:tasks.workunit.client.1.smithi194.stdout:6/211: fdatasync d3/fe 0 2024-04-12T05:56:00.227 INFO:tasks.workunit.client.1.smithi194.stdout:6/212: truncate d3/f1f 4818140 0 2024-04-12T05:56:00.227 INFO:tasks.workunit.client.1.smithi194.stdout:4/80: sync 2024-04-12T05:56:00.228 INFO:tasks.workunit.client.1.smithi194.stdout:6/213: creat d3/d2d/f3d x:0 0 0 2024-04-12T05:56:00.230 INFO:tasks.workunit.client.1.smithi194.stdout:4/81: fdatasync f0 0 2024-04-12T05:56:00.230 INFO:tasks.workunit.client.1.smithi194.stdout:6/214: dread - d3/d1a/f23 zero size 2024-04-12T05:56:00.232 INFO:tasks.workunit.client.1.smithi194.stdout:6/215: symlink d3/d15/d2b/l3e 0 2024-04-12T05:56:00.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 -- 172.21.15.8:0/1422235586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 msgr2=0x7f1b200ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 --2- 172.21.15.8:0/1422235586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b200ff3a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009910 tx=0x7f1b10009c20 comp rx=0 tx=0).stop 2024-04-12T05:56:00.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 -- 172.21.15.8:0/1422235586 shutdown_connections 2024-04-12T05:56:00.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 --2- 172.21.15.8:0/1422235586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 0x7f1b20102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 --2- 172.21.15.8:0/1422235586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b200ff3a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 -- 172.21.15.8:0/1422235586 >> 172.21.15.8:0/1422235586 conn(0x7f1b200fa8c0 msgr2=0x7f1b200fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 -- 172.21.15.8:0/1422235586 shutdown_connections 2024-04-12T05:56:00.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.259+0000 7f1b26a24700 1 -- 172.21.15.8:0/1422235586 wait complete. 2024-04-12T05:56:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 Processor -- start 2024-04-12T05:56:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 -- start start 2024-04-12T05:56:00.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 0x7f1b200fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b201004f0 con 0x7f1b200fef80 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b26a24700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20100630 con 0x7f1b20102000 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b1ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b1ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39720/0 (socket says 172.21.15.8:39720) 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.260+0000 7f1b1ffff700 1 -- 172.21.15.8:0/362540189 learned_addr learned my addr 172.21.15.8:0/362540189 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1f7fe700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 0x7f1b200fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1ffff700 1 -- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 msgr2=0x7f1b200fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1ffff700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 0x7f1b200fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1ffff700 1 -- 172.21.15.8:0/362540189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b100095f0 con 0x7f1b200fef80 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1ffff700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b10000c00 tx=0x7f1b1000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b1001d070 con 0x7f1b200fef80 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b201008b0 con 0x7f1b200fef80 2024-04-12T05:56:00.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f1b100059e0 con 0x7f1b200fef80 2024-04-12T05:56:00.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b10003980 con 0x7f1b200fef80 2024-04-12T05:56:00.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.261+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b20100d40 con 0x7f1b200fef80 2024-04-12T05:56:00.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.262+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b201096c0 con 0x7f1b200fef80 2024-04-12T05:56:00.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.263+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f1b100041f0 con 0x7f1b200fef80 2024-04-12T05:56:00.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.263+0000 7f1b1d7fa700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 0x7f1b0806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.263+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1b1008d810 con 0x7f1b200fef80 2024-04-12T05:56:00.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.264+0000 7f1b1f7fe700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 0x7f1b0806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:00.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.264+0000 7f1b1f7fe700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 0x7f1b0806eee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009c60 tx=0x7f1b14009260 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:00.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.272+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f1b1005b2f0 con 0x7f1b200fef80 2024-04-12T05:56:00.270 INFO:tasks.workunit.client.1.smithi194.stdout:6/216: rename d3/d15/f14 to d3/d2f/d37/f3f 0 2024-04-12T05:56:00.275 INFO:tasks.workunit.client.1.smithi194.stdout:6/217: dread d3/d1a/f29 [0,4194304] 0 2024-04-12T05:56:00.276 INFO:tasks.workunit.client.1.smithi194.stdout:6/218: write d3/d2f/f39 [120681,92292] 0 2024-04-12T05:56:00.290 INFO:tasks.workunit.client.1.smithi194.stdout:4/82: truncate d2/d3/f1a 87767 0 2024-04-12T05:56:00.294 INFO:tasks.workunit.client.1.smithi194.stdout:4/83: mkdir d2/d3/d1b 0 2024-04-12T05:56:00.312 INFO:tasks.workunit.client.1.smithi194.stdout:6/219: symlink d3/l40 0 2024-04-12T05:56:00.317 INFO:tasks.workunit.client.1.smithi194.stdout:6/220: dread d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:00.317 INFO:tasks.workunit.client.1.smithi194.stdout:4/84: write f0 [2570533,105370] 0 2024-04-12T05:56:00.321 INFO:tasks.workunit.client.1.smithi194.stdout:4/85: mknod d2/d3/c1c 0 2024-04-12T05:56:00.324 INFO:tasks.workunit.client.1.smithi194.stdout:6/221: link d3/c1b d3/d2d/c41 0 2024-04-12T05:56:00.331 INFO:tasks.workunit.client.1.smithi194.stdout:4/86: dwrite d2/d3/f13 [0,4194304] 0 2024-04-12T05:56:00.334 INFO:tasks.workunit.client.1.smithi194.stdout:4/87: write d2/f16 [597852,14806] 0 2024-04-12T05:56:00.338 INFO:tasks.workunit.client.1.smithi194.stdout:4/88: mknod d2/d3/d1b/c1d 0 2024-04-12T05:56:00.338 INFO:tasks.workunit.client.1.smithi194.stdout:6/222: dwrite d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:00.342 INFO:tasks.workunit.client.1.smithi194.stdout:6/223: mknod d3/d15/c42 0 2024-04-12T05:56:00.343 INFO:tasks.workunit.client.1.smithi194.stdout:4/89: mkdir d2/d11/d1e 0 2024-04-12T05:56:00.343 INFO:tasks.workunit.client.1.smithi194.stdout:6/224: write d3/f17 [993509,1695] 0 2024-04-12T05:56:00.345 INFO:tasks.workunit.client.1.smithi194.stdout:4/90: mkdir d2/d11/d1f 0 2024-04-12T05:56:00.346 INFO:tasks.workunit.client.1.smithi194.stdout:4/91: truncate d2/f16 785243 0 2024-04-12T05:56:00.346 INFO:tasks.workunit.client.1.smithi194.stdout:6/225: creat d3/d2f/d37/f43 x:0 0 0 2024-04-12T05:56:00.347 INFO:tasks.workunit.client.1.smithi194.stdout:6/226: chown d3/d15/l2a 256086 1 2024-04-12T05:56:00.350 INFO:tasks.workunit.client.1.smithi194.stdout:6/227: mknod d3/c44 0 2024-04-12T05:56:00.350 INFO:tasks.workunit.client.1.smithi194.stdout:6/228: readlink d3/d15/d2b/l3e 0 2024-04-12T05:56:00.353 INFO:tasks.workunit.client.1.smithi194.stdout:6/229: getdents d3/d2d/d33 0 2024-04-12T05:56:00.356 INFO:tasks.workunit.client.1.smithi194.stdout:6/230: link d3/f17 d3/d2f/d37/f45 0 2024-04-12T05:56:00.358 INFO:tasks.workunit.client.1.smithi194.stdout:4/92: write d2/d3/f1a [419969,34580] 0 2024-04-12T05:56:00.359 INFO:tasks.workunit.client.1.smithi194.stdout:4/93: symlink d2/d3/l20 0 2024-04-12T05:56:00.372 INFO:tasks.workunit.client.1.smithi194.stdout:6/231: dread d3/d15/f24 [4194304,4194304] 0 2024-04-12T05:56:00.378 INFO:tasks.workunit.client.1.smithi194.stdout:6/232: dread d3/d15/f2e [0,4194304] 0 2024-04-12T05:56:00.378 INFO:tasks.workunit.client.1.smithi194.stdout:6/233: readlink d3/l40 0 2024-04-12T05:56:00.380 INFO:tasks.workunit.client.1.smithi194.stdout:6/234: link d3/d15/d2b/c31 d3/c46 0 2024-04-12T05:56:00.383 INFO:tasks.workunit.client.1.smithi194.stdout:6/235: getdents d3/d15 0 2024-04-12T05:56:00.385 INFO:tasks.workunit.client.1.smithi194.stdout:6/236: creat d3/d1a/f47 x:0 0 0 2024-04-12T05:56:00.386 INFO:tasks.workunit.client.1.smithi194.stdout:6/237: chown d3/d15/f24 548568708 1 2024-04-12T05:56:00.387 INFO:tasks.workunit.client.1.smithi194.stdout:6/238: chown d3/d1a/f23 1 1 2024-04-12T05:56:00.388 INFO:tasks.workunit.client.1.smithi194.stdout:6/239: mkdir d3/d15/d2b/d48 0 2024-04-12T05:56:00.394 INFO:tasks.workunit.client.1.smithi194.stdout:6/240: dwrite d3/d15/f28 [0,4194304] 0 2024-04-12T05:56:00.394 INFO:tasks.workunit.client.1.smithi194.stdout:6/241: truncate d3/f1e 920334 0 2024-04-12T05:56:00.395 INFO:tasks.workunit.client.1.smithi194.stdout:6/242: write d3/d2d/f32 [956042,47348] 0 2024-04-12T05:56:00.396 INFO:tasks.workunit.client.1.smithi194.stdout:6/243: stat d3/d15 0 2024-04-12T05:56:00.399 INFO:tasks.workunit.client.1.smithi194.stdout:6/244: unlink d3/d2f/l36 0 2024-04-12T05:56:00.401 INFO:tasks.workunit.client.1.smithi194.stdout:6/245: creat d3/d15/d2b/d48/f49 x:0 0 0 2024-04-12T05:56:00.401 INFO:tasks.workunit.client.1.smithi194.stdout:3/127: sync 2024-04-12T05:56:00.408 INFO:tasks.workunit.client.1.smithi194.stdout:6/246: dwrite d3/d15/d2b/d48/f49 [0,4194304] 0 2024-04-12T05:56:00.408 INFO:tasks.workunit.client.1.smithi194.stdout:3/128: link d0/d1/d10/l1b d0/dc/de/l2c 0 2024-04-12T05:56:00.413 INFO:tasks.workunit.client.1.smithi194.stdout:3/129: fsync d0/d1/fd 0 2024-04-12T05:56:00.415 INFO:tasks.workunit.client.1.smithi194.stdout:6/247: dread d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:00.417 INFO:tasks.workunit.client.1.smithi194.stdout:6/248: rmdir d3/d2f/d37 39 2024-04-12T05:56:00.421 INFO:tasks.workunit.client.1.smithi194.stdout:3/130: dwrite d0/dc/f1f [0,4194304] 0 2024-04-12T05:56:00.425 INFO:tasks.workunit.client.1.smithi194.stdout:3/131: mkdir d0/dc/de/d17/d2d 0 2024-04-12T05:56:00.427 INFO:tasks.workunit.client.1.smithi194.stdout:3/132: dread d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:00.428 INFO:tasks.workunit.client.1.smithi194.stdout:3/133: fsync d0/dc/f12 0 2024-04-12T05:56:00.441 INFO:tasks.workunit.client.1.smithi194.stdout:3/134: dwrite d0/fb [0,4194304] 0 2024-04-12T05:56:00.443 INFO:tasks.workunit.client.1.smithi194.stdout:3/135: write d0/dc/d1e/f2a [752448,48199] 0 2024-04-12T05:56:00.445 INFO:tasks.workunit.client.1.smithi194.stdout:3/136: write d0/dc/d1e/f2a [1135045,71773] 0 2024-04-12T05:56:00.445 INFO:tasks.workunit.client.1.smithi194.stdout:3/137: chown d0/d1 193453439 1 2024-04-12T05:56:00.446 INFO:tasks.workunit.client.1.smithi194.stdout:3/138: read d0/f11 [2918074,48596] 0 2024-04-12T05:56:00.449 INFO:tasks.workunit.client.1.smithi194.stdout:3/139: creat d0/d1/d10/f2e x:0 0 0 2024-04-12T05:56:00.453 INFO:tasks.workunit.client.1.smithi194.stdout:3/140: dread d0/fb [4194304,4194304] 0 2024-04-12T05:56:00.454 INFO:tasks.workunit.client.1.smithi194.stdout:3/141: mknod d0/c2f 0 2024-04-12T05:56:00.460 INFO:tasks.workunit.client.1.smithi194.stdout:3/142: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:00.461 INFO:tasks.workunit.client.1.smithi194.stdout:3/143: stat d0/dc/de/d17 0 2024-04-12T05:56:00.462 INFO:tasks.workunit.client.1.smithi194.stdout:3/144: fdatasync d0/d1/f1c 0 2024-04-12T05:56:00.467 INFO:tasks.workunit.client.1.smithi194.stdout:3/145: rename d0/d1/d10/l18 to d0/dc/de/d17/d2d/l30 0 2024-04-12T05:56:00.468 INFO:tasks.workunit.client.1.smithi194.stdout:3/146: fsync d0/dc/f12 0 2024-04-12T05:56:00.471 INFO:tasks.workunit.client.1.smithi194.stdout:3/147: dread d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:00.472 INFO:tasks.workunit.client.1.smithi194.stdout:3/148: fdatasync d0/d1/d10/f2e 0 2024-04-12T05:56:00.474 INFO:tasks.workunit.client.1.smithi194.stdout:3/149: mkdir d0/dc/de/d31 0 2024-04-12T05:56:00.475 INFO:tasks.workunit.client.1.smithi194.stdout:3/150: fdatasync d0/d1/f1c 0 2024-04-12T05:56:00.479 INFO:tasks.workunit.client.1.smithi194.stdout:3/151: dread d0/dc/d1e/f2a [0,4194304] 0 2024-04-12T05:56:00.484 INFO:tasks.workunit.client.1.smithi194.stdout:3/152: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:00.492 INFO:tasks.workunit.client.1.smithi194.stdout:3/153: dwrite d0/f27 [0,4194304] 0 2024-04-12T05:56:00.493 INFO:tasks.workunit.client.1.smithi194.stdout:3/154: chown d0/l7 42576 1 2024-04-12T05:56:00.511 INFO:tasks.workunit.client.1.smithi194.stdout:3/155: dread d0/d1/fd [4194304,4194304] 0 2024-04-12T05:56:00.513 INFO:tasks.workunit.client.1.smithi194.stdout:3/156: truncate d0/d1/fd 5782419 0 2024-04-12T05:56:00.522 INFO:tasks.workunit.client.1.smithi194.stdout:3/157: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:00.523 INFO:tasks.workunit.client.1.smithi194.stdout:3/158: write d0/d1/d10/f2b [581205,36788] 0 2024-04-12T05:56:00.541 INFO:tasks.workunit.client.1.smithi194.stdout:3/159: dwrite d0/d1/fd [4194304,4194304] 0 2024-04-12T05:56:00.542 INFO:tasks.workunit.client.1.smithi194.stdout:3/160: write d0/d1/f9 [1831617,37801] 0 2024-04-12T05:56:00.550 INFO:tasks.workunit.client.1.smithi194.stdout:3/161: dread d0/d1/d10/f2b [0,4194304] 0 2024-04-12T05:56:00.552 INFO:tasks.workunit.client.1.smithi194.stdout:3/162: unlink d0/dc/d1e/l25 0 2024-04-12T05:56:00.554 INFO:tasks.workunit.client.1.smithi194.stdout:3/163: creat d0/dc/de/f32 x:0 0 0 2024-04-12T05:56:00.556 INFO:tasks.workunit.client.1.smithi194.stdout:3/164: dread d0/dc/d1e/f2a [0,4194304] 0 2024-04-12T05:56:00.557 INFO:tasks.workunit.client.1.smithi194.stdout:3/165: chown d0/dc 773 1 2024-04-12T05:56:00.560 INFO:tasks.workunit.client.1.smithi194.stdout:3/166: dread d0/d1/fd [0,4194304] 0 2024-04-12T05:56:00.566 INFO:tasks.workunit.client.1.smithi194.stdout:3/167: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:00.568 INFO:tasks.workunit.client.1.smithi194.stdout:3/168: mkdir d0/dc/d1e/d33 0 2024-04-12T05:56:00.570 INFO:tasks.workunit.client.1.smithi194.stdout:3/169: rename d0/dc/de/d17 to d0/dc/de/d31/d34 0 2024-04-12T05:56:00.575 INFO:tasks.workunit.client.1.smithi194.stdout:3/170: truncate d0/f27 443213 0 2024-04-12T05:56:00.576 INFO:tasks.workunit.client.1.smithi194.stdout:3/171: chown d0/dc/de/l1d 14 1 2024-04-12T05:56:00.582 INFO:tasks.workunit.client.1.smithi194.stdout:3/172: dwrite d0/dc/f1f [4194304,4194304] 0 2024-04-12T05:56:00.583 INFO:tasks.workunit.client.1.smithi194.stdout:3/173: write d0/d1/f9 [3269855,119503] 0 2024-04-12T05:56:00.584 INFO:tasks.workunit.client.1.smithi194.stdout:3/174: stat d0/dc/l19 0 2024-04-12T05:56:00.588 INFO:tasks.workunit.client.1.smithi194.stdout:3/175: stat d0/d1/l2 0 2024-04-12T05:56:00.589 INFO:tasks.workunit.client.1.smithi194.stdout:3/176: write d0/dc/d1e/f2a [528574,125408] 0 2024-04-12T05:56:00.597 INFO:tasks.workunit.client.1.smithi194.stdout:3/177: truncate d0/d1/fd 330447 0 2024-04-12T05:56:00.603 INFO:tasks.workunit.client.1.smithi194.stdout:3/178: dwrite d0/dc/de/f32 [0,4194304] 0 2024-04-12T05:56:00.607 INFO:tasks.workunit.client.1.smithi194.stdout:3/179: chown d0/dc/de/l2c 98 1 2024-04-12T05:56:00.610 INFO:tasks.workunit.client.1.smithi194.stdout:3/180: dread d0/fb [0,4194304] 0 2024-04-12T05:56:00.618 INFO:tasks.workunit.client.1.smithi194.stdout:3/181: write d0/f27 [890582,65807] 0 2024-04-12T05:56:00.619 INFO:tasks.workunit.client.1.smithi194.stdout:3/182: write d0/dc/f1f [4869676,104251] 0 2024-04-12T05:56:00.622 INFO:tasks.workunit.client.1.smithi194.stdout:3/183: mknod d0/dc/c35 0 2024-04-12T05:56:00.623 INFO:tasks.workunit.client.1.smithi194.stdout:3/184: mknod d0/dc/d1e/c36 0 2024-04-12T05:56:00.624 INFO:tasks.workunit.client.1.smithi194.stdout:3/185: chown d0/dc/f12 519431171 1 2024-04-12T05:56:00.624 INFO:tasks.workunit.client.1.smithi194.stdout:3/186: read d0/f11 [2058655,91077] 0 2024-04-12T05:56:00.626 INFO:tasks.workunit.client.1.smithi194.stdout:3/187: creat d0/dc/de/d31/d34/f37 x:0 0 0 2024-04-12T05:56:00.630 INFO:tasks.workunit.client.1.smithi194.stdout:3/188: dread d0/f11 [0,4194304] 0 2024-04-12T05:56:00.631 INFO:tasks.workunit.client.1.smithi194.stdout:3/189: rmdir d0/dc 39 2024-04-12T05:56:00.634 INFO:tasks.workunit.client.1.smithi194.stdout:3/190: dread d0/fb [0,4194304] 0 2024-04-12T05:56:00.635 INFO:tasks.workunit.client.1.smithi194.stdout:5/183: sync 2024-04-12T05:56:00.635 INFO:tasks.workunit.client.1.smithi194.stdout:1/197: sync 2024-04-12T05:56:00.635 INFO:tasks.workunit.client.1.smithi194.stdout:7/54: sync 2024-04-12T05:56:00.640 INFO:tasks.workunit.client.1.smithi194.stdout:5/184: creat d1/d4/d6/db/d3b/f3e x:0 0 0 2024-04-12T05:56:00.640 INFO:tasks.workunit.client.1.smithi194.stdout:7/55: getdents d3/d4 0 2024-04-12T05:56:00.641 INFO:tasks.workunit.client.1.smithi194.stdout:7/56: dread - d3/d4/f11 zero size 2024-04-12T05:56:00.642 INFO:tasks.workunit.client.1.smithi194.stdout:3/191: link d0/dc/de/d31/d34/c26 d0/dc/de/c38 0 2024-04-12T05:56:00.643 INFO:tasks.workunit.client.1.smithi194.stdout:5/185: mknod d1/d4/d6/db/d15/d26/c3f 0 2024-04-12T05:56:00.644 INFO:tasks.workunit.client.1.smithi194.stdout:5/186: dread - d1/d4/d6/db/d3b/f3e zero size 2024-04-12T05:56:00.645 INFO:tasks.workunit.client.1.smithi194.stdout:7/57: symlink d3/l14 0 2024-04-12T05:56:00.646 INFO:tasks.workunit.client.1.smithi194.stdout:5/187: write d1/d4/df/f2e [912245,11419] 0 2024-04-12T05:56:00.646 INFO:tasks.workunit.client.1.smithi194.stdout:3/192: mkdir d0/dc/de/d39 0 2024-04-12T05:56:00.647 INFO:tasks.workunit.client.1.smithi194.stdout:7/58: symlink d3/d4/l15 0 2024-04-12T05:56:00.649 INFO:tasks.workunit.client.1.smithi194.stdout:5/188: creat d1/d4/d6/db/d38/f40 x:0 0 0 2024-04-12T05:56:00.650 INFO:tasks.workunit.client.1.smithi194.stdout:5/189: fdatasync d1/d4/d6/f35 0 2024-04-12T05:56:00.651 INFO:tasks.workunit.client.1.smithi194.stdout:3/193: creat d0/dc/de/d31/d34/d2d/f3a x:0 0 0 2024-04-12T05:56:00.651 INFO:tasks.workunit.client.1.smithi194.stdout:7/59: mkdir d3/d16 0 2024-04-12T05:56:00.652 INFO:tasks.workunit.client.1.smithi194.stdout:7/60: chown d3/d16 420524722 1 2024-04-12T05:56:00.653 INFO:tasks.workunit.client.1.smithi194.stdout:3/194: creat d0/dc/de/d39/f3b x:0 0 0 2024-04-12T05:56:00.654 INFO:tasks.workunit.client.1.smithi194.stdout:7/61: truncate d3/d4/f11 144085 0 2024-04-12T05:56:00.656 INFO:tasks.workunit.client.1.smithi194.stdout:7/62: creat d3/d4/f17 x:0 0 0 2024-04-12T05:56:00.657 INFO:tasks.workunit.client.1.smithi194.stdout:5/190: dwrite d1/d4/f5 [4194304,4194304] 0 2024-04-12T05:56:00.658 INFO:tasks.workunit.client.1.smithi194.stdout:5/191: readlink d1/le 0 2024-04-12T05:56:00.659 INFO:tasks.workunit.client.1.smithi194.stdout:7/63: rename d3/d4/d12/f9 to d3/d4/d12/f18 0 2024-04-12T05:56:00.660 INFO:tasks.workunit.client.1.smithi194.stdout:7/64: fdatasync d3/d4/f11 0 2024-04-12T05:56:00.660 INFO:tasks.workunit.client.1.smithi194.stdout:5/192: creat d1/d4/d6/db/d38/f41 x:0 0 0 2024-04-12T05:56:00.662 INFO:tasks.workunit.client.1.smithi194.stdout:7/65: creat d3/d16/f19 x:0 0 0 2024-04-12T05:56:00.663 INFO:tasks.workunit.client.1.smithi194.stdout:7/66: chown d3/l14 2854 1 2024-04-12T05:56:00.663 INFO:tasks.workunit.client.1.smithi194.stdout:5/193: unlink d1/d4/d6/db/d38/f40 0 2024-04-12T05:56:00.683 INFO:tasks.workunit.client.1.smithi194.stdout:5/194: dread f0 [0,4194304] 0 2024-04-12T05:56:00.685 INFO:tasks.workunit.client.1.smithi194.stdout:5/195: creat d1/d4/f42 x:0 0 0 2024-04-12T05:56:00.688 INFO:tasks.workunit.client.1.smithi194.stdout:5/196: symlink d1/d4/d6/l43 0 2024-04-12T05:56:00.696 INFO:tasks.workunit.client.1.smithi194.stdout:5/197: getdents d1/d4/d6/db/d15/d26 0 2024-04-12T05:56:00.698 INFO:tasks.workunit.client.1.smithi194.stdout:5/198: creat d1/d4/d6/db/d2b/f44 x:0 0 0 2024-04-12T05:56:00.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.706+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b2019e880 con 0x7f1b200fef80 2024-04-12T05:56:00.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.707+0000 7f1b1d7fa700 1 -- 172.21.15.8:0/362540189 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f1b1005ae80 con 0x7f1b200fef80 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:56:00.703 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:56:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-04-12T05:56:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T05:56:00.704 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:56:00.704 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:56:00.705 INFO:tasks.workunit.client.1.smithi194.stdout:5/199: dwrite d1/d4/d6/db/d38/f41 [0,4194304] 0 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 msgr2=0x7f1b0806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 0x7f1b0806eee0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009c60 tx=0x7f1b14009260 comp rx=0 tx=0).stop 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 msgr2=0x7f1b20101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b10000c00 tx=0x7f1b1000bde0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 shutdown_connections 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f1b0806ca20 0x7f1b0806eee0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b200fef80 0x7f1b20101960 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 --2- 172.21.15.8:0/362540189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b20102000 0x7f1b200fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.710+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 >> 172.21.15.8:0/362540189 conn(0x7f1b200fa8c0 msgr2=0x7f1b20105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.711+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 shutdown_connections 2024-04-12T05:56:00.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.711+0000 7f1b26a24700 1 -- 172.21.15.8:0/362540189 wait complete. 2024-04-12T05:56:00.707 INFO:tasks.workunit.client.1.smithi194.stdout:5/200: dread - d1/d4/df/d23/f3c zero size 2024-04-12T05:56:00.708 INFO:tasks.workunit.client.1.smithi194.stdout:5/201: chown d1/d4/df/d23/f3c 2840226 1 2024-04-12T05:56:00.713 INFO:tasks.workunit.client.1.smithi194.stdout:5/202: dread d1/d4/d6/db/d38/f41 [0,4194304] 0 2024-04-12T05:56:00.714 INFO:tasks.workunit.client.1.smithi194.stdout:5/203: dread - d1/d4/d6/db/d2b/f33 zero size 2024-04-12T05:56:00.717 INFO:tasks.workunit.client.1.smithi194.stdout:5/204: creat d1/d4/d6/db/d15/d26/f45 x:0 0 0 2024-04-12T05:56:00.724 INFO:tasks.workunit.client.1.smithi194.stdout:5/205: dwrite d1/d4/df/f1f [0,4194304] 0 2024-04-12T05:56:00.727 INFO:tasks.workunit.client.1.smithi194.stdout:5/206: unlink d1/d4/f16 0 2024-04-12T05:56:00.737 INFO:tasks.workunit.client.1.smithi194.stdout:5/207: write d1/d4/d6/f2d [1195961,12535] 0 2024-04-12T05:56:00.740 INFO:tasks.workunit.client.1.smithi194.stdout:5/208: mknod d1/d4/df/d23/c46 0 2024-04-12T05:56:00.746 INFO:tasks.workunit.client.1.smithi194.stdout:5/209: dwrite d1/d4/d6/db/d3b/f3e [0,4194304] 0 2024-04-12T05:56:00.750 INFO:tasks.workunit.client.1.smithi194.stdout:5/210: getdents d1/d4 0 2024-04-12T05:56:00.752 INFO:tasks.workunit.client.1.smithi194.stdout:5/211: fdatasync d1/d4/d6/f35 0 2024-04-12T05:56:00.754 INFO:tasks.workunit.client.1.smithi194.stdout:5/212: mknod d1/d4/d6/db/d15/d26/c47 0 2024-04-12T05:56:00.755 INFO:tasks.workunit.client.1.smithi194.stdout:5/213: chown d1/d4/d6/c20 104341 1 2024-04-12T05:56:00.763 INFO:tasks.workunit.client.1.smithi194.stdout:5/214: truncate d1/d4/d6/db/fc 5037148 0 2024-04-12T05:56:00.764 INFO:tasks.workunit.client.1.smithi194.stdout:5/215: truncate d1/d4/f42 592981 0 2024-04-12T05:56:00.765 INFO:tasks.workunit.client.1.smithi194.stdout:5/216: creat d1/d4/d6/db/f48 x:0 0 0 2024-04-12T05:56:00.767 INFO:tasks.workunit.client.1.smithi194.stdout:5/217: write d1/d4/d6/f2d [65716,79911] 0 2024-04-12T05:56:00.767 INFO:tasks.workunit.client.1.smithi194.stdout:5/218: stat d1/d4/d6/f3d 0 2024-04-12T05:56:00.774 INFO:tasks.workunit.client.1.smithi194.stdout:5/219: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:00.779 INFO:tasks.workunit.client.1.smithi194.stdout:5/220: dread d1/d4/d6/db/d3b/f3e [0,4194304] 0 2024-04-12T05:56:00.782 INFO:tasks.workunit.client.1.smithi194.stdout:5/221: mknod d1/d4/df/c49 0 2024-04-12T05:56:00.785 INFO:tasks.workunit.client.1.smithi194.stdout:5/222: rename d1/d4/d6/db/d15/d26/f45 to d1/d4/d6/db/f4a 0 2024-04-12T05:56:00.793 INFO:tasks.workunit.client.1.smithi194.stdout:5/223: truncate d1/fd 3617375 0 2024-04-12T05:56:00.807 INFO:tasks.workunit.client.1.smithi194.stdout:5/224: dread d1/fd [0,4194304] 0 2024-04-12T05:56:00.807 INFO:tasks.workunit.client.1.smithi194.stdout:5/225: chown d1/d4/d6/db/d15 2 1 2024-04-12T05:56:00.808 INFO:tasks.workunit.client.1.smithi194.stdout:8/110: sync 2024-04-12T05:56:00.808 INFO:tasks.workunit.client.1.smithi194.stdout:9/195: sync 2024-04-12T05:56:00.809 INFO:tasks.workunit.client.1.smithi194.stdout:0/115: sync 2024-04-12T05:56:00.809 INFO:tasks.workunit.client.1.smithi194.stdout:4/94: sync 2024-04-12T05:56:00.809 INFO:tasks.workunit.client.1.smithi194.stdout:6/249: sync 2024-04-12T05:56:00.809 INFO:tasks.workunit.client.1.smithi194.stdout:2/150: sync 2024-04-12T05:56:00.812 INFO:tasks.workunit.client.1.smithi194.stdout:8/111: fdatasync d2/d8/d9/de/f1d 0 2024-04-12T05:56:00.812 INFO:tasks.workunit.client.1.smithi194.stdout:9/196: chown d1/d21/c26 0 1 2024-04-12T05:56:00.812 INFO:tasks.workunit.client.1.smithi194.stdout:4/95: unlink d2/f16 0 2024-04-12T05:56:00.815 INFO:tasks.workunit.client.1.smithi194.stdout:0/116: creat d0/da/d1e/d1a/f23 x:0 0 0 2024-04-12T05:56:00.817 INFO:tasks.workunit.client.1.smithi194.stdout:2/151: rename d7/d12/f14 to d7/d12/d1a/f3e 0 2024-04-12T05:56:00.819 INFO:tasks.workunit.client.1.smithi194.stdout:5/226: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:00.822 INFO:tasks.workunit.client.1.smithi194.stdout:2/152: fsync d7/d12/f3b 0 2024-04-12T05:56:00.823 INFO:tasks.workunit.client.1.smithi194.stdout:0/117: mkdir d0/da/d1e/d1a/d24 0 2024-04-12T05:56:00.825 INFO:tasks.workunit.client.1.smithi194.stdout:9/197: mkdir d1/d41 0 2024-04-12T05:56:00.827 INFO:tasks.workunit.client.1.smithi194.stdout:6/250: dwrite d3/d2d/f32 [0,4194304] 0 2024-04-12T05:56:00.829 INFO:tasks.workunit.client.1.smithi194.stdout:5/227: mkdir d1/d4/d4b 0 2024-04-12T05:56:00.832 INFO:tasks.workunit.client.1.smithi194.stdout:4/96: dwrite d2/d3/f13 [0,4194304] 0 2024-04-12T05:56:00.832 INFO:tasks.workunit.client.1.smithi194.stdout:5/228: write d1/d4/f1a [1462636,71915] 0 2024-04-12T05:56:00.833 INFO:tasks.workunit.client.1.smithi194.stdout:9/198: write d1/f3f [963311,111508] 0 2024-04-12T05:56:00.835 INFO:tasks.workunit.client.1.smithi194.stdout:4/97: fsync d2/d3/f17 0 2024-04-12T05:56:00.837 INFO:tasks.workunit.client.1.smithi194.stdout:0/118: creat d0/da/d1e/f25 x:0 0 0 2024-04-12T05:56:00.840 INFO:tasks.workunit.client.1.smithi194.stdout:6/251: creat d3/d2f/d37/f4a x:0 0 0 2024-04-12T05:56:00.843 INFO:tasks.workunit.client.1.smithi194.stdout:0/119: write d0/da/d1e/f15 [1859971,62549] 0 2024-04-12T05:56:00.844 INFO:tasks.workunit.client.1.smithi194.stdout:6/252: rename d3/f17 to d3/d15/f4b 0 2024-04-12T05:56:00.845 INFO:tasks.workunit.client.1.smithi194.stdout:0/120: fdatasync d0/f10 0 2024-04-12T05:56:00.846 INFO:tasks.workunit.client.1.smithi194.stdout:0/121: fdatasync d0/f2 0 2024-04-12T05:56:00.847 INFO:tasks.workunit.client.1.smithi194.stdout:4/98: getdents d2/d11/d1f 0 2024-04-12T05:56:00.848 INFO:tasks.workunit.client.1.smithi194.stdout:9/199: unlink d1/l1c 0 2024-04-12T05:56:00.849 INFO:tasks.workunit.client.1.smithi194.stdout:9/200: chown d1/d21/c32 984459 1 2024-04-12T05:56:00.852 INFO:tasks.workunit.client.1.smithi194.stdout:6/253: write d3/d15/f22 [2264766,27976] 0 2024-04-12T05:56:00.853 INFO:tasks.workunit.client.1.smithi194.stdout:5/229: link d1/d4/d6/d34/c3a d1/d4/d6/db/d3b/c4c 0 2024-04-12T05:56:00.855 INFO:tasks.workunit.client.1.smithi194.stdout:6/254: readlink d3/d15/d2b/l3e 0 2024-04-12T05:56:00.857 INFO:tasks.workunit.client.1.smithi194.stdout:6/255: chown d3/d15/f28 0 1 2024-04-12T05:56:00.857 INFO:tasks.workunit.client.1.smithi194.stdout:4/99: dwrite d2/d3/f17 [0,4194304] 0 2024-04-12T05:56:00.859 INFO:tasks.workunit.client.1.smithi194.stdout:4/100: chown d2/d3/f18 5 1 2024-04-12T05:56:00.859 INFO:tasks.workunit.client.1.smithi194.stdout:6/256: truncate d3/d1a/f35 950554 0 2024-04-12T05:56:00.859 INFO:tasks.workunit.client.1.smithi194.stdout:9/201: rename d1/c1b to d1/d41/c42 0 2024-04-12T05:56:00.860 INFO:tasks.workunit.client.1.smithi194.stdout:4/101: readlink d2/l7 0 2024-04-12T05:56:00.861 INFO:tasks.workunit.client.1.smithi194.stdout:0/122: link d0/da/c1f d0/da/d1e/d1a/c26 0 2024-04-12T05:56:00.865 INFO:tasks.workunit.client.1.smithi194.stdout:4/102: creat d2/f21 x:0 0 0 2024-04-12T05:56:00.868 INFO:tasks.workunit.client.1.smithi194.stdout:9/202: creat d1/d41/f43 x:0 0 0 2024-04-12T05:56:00.876 INFO:tasks.workunit.client.1.smithi194.stdout:2/153: write d7/d12/d1a/f3e [920534,120515] 0 2024-04-12T05:56:00.877 INFO:tasks.workunit.client.1.smithi194.stdout:6/257: dwrite d3/d2f/d37/f43 [0,4194304] 0 2024-04-12T05:56:00.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 -- 172.21.15.8:0/2310573923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 msgr2=0x7f25e006b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 --2- 172.21.15.8:0/2310573923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 0x7f25e006b180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f25dc009910 tx=0x7f25dc009c20 comp rx=0 tx=0).stop 2024-04-12T05:56:00.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 -- 172.21.15.8:0/2310573923 shutdown_connections 2024-04-12T05:56:00.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 --2- 172.21.15.8:0/2310573923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e006c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 --2- 172.21.15.8:0/2310573923 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 0x7f25e006b180 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 -- 172.21.15.8:0/2310573923 >> 172.21.15.8:0/2310573923 conn(0x7f25e00fa490 msgr2=0x7f25e00fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:00.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 -- 172.21.15.8:0/2310573923 shutdown_connections 2024-04-12T05:56:00.879 INFO:tasks.workunit.client.1.smithi194.stdout:0/123: mkdir d0/da/d1e/d27 0 2024-04-12T05:56:00.879 INFO:tasks.workunit.client.1.smithi194.stdout:9/203: creat d1/d41/f44 x:0 0 0 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.883+0000 7f25e7897700 1 -- 172.21.15.8:0/2310573923 wait complete. 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 Processor -- start 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 -- start start 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 0x7f25e0111280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01a1f80 con 0x7f25e006ad60 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.884+0000 7f25e7897700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e01a20c0 con 0x7f25e006bfb0 2024-04-12T05:56:00.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:00.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:56456/0 (socket says 172.21.15.8:56456) 2024-04-12T05:56:00.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 -- 172.21.15.8:0/546791521 learned_addr learned my addr 172.21.15.8:0/546791521 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:00.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 -- 172.21.15.8:0/546791521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 msgr2=0x7f25e0111280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:00.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 0x7f25e0111280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:00.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 -- 172.21.15.8:0/546791521 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc0095f0 con 0x7f25e006bfb0 2024-04-12T05:56:00.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.885+0000 7f25e4e32700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b4b0 tx=0x7f25d000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:00.882 INFO:tasks.workunit.client.1.smithi194.stdout:8/112: dwrite d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:56:00.882 INFO:tasks.workunit.client.1.smithi194.stdout:9/204: stat d1/d41/f43 0 2024-04-12T05:56:00.882 INFO:tasks.workunit.client.1.smithi194.stdout:6/258: read d3/f3a [6933478,68797] 0 2024-04-12T05:56:00.884 INFO:tasks.workunit.client.1.smithi194.stdout:5/230: truncate f0 3398890 0 2024-04-12T05:56:00.884 INFO:tasks.workunit.client.1.smithi194.stdout:2/154: write f5 [8342694,129212] 0 2024-04-12T05:56:00.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.891+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25d0011800 con 0x7f25e006bfb0 2024-04-12T05:56:00.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.891+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f25d0011df0 con 0x7f25e006bfb0 2024-04-12T05:56:00.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.891+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25d0010e40 con 0x7f25e006bfb0 2024-04-12T05:56:00.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.891+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e01a23a0 con 0x7f25e006bfb0 2024-04-12T05:56:00.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.891+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e01a28a0 con 0x7f25e006bfb0 2024-04-12T05:56:00.887 INFO:tasks.workunit.client.1.smithi194.stdout:9/205: truncate d1/d41/f43 202662 0 2024-04-12T05:56:00.887 INFO:tasks.workunit.client.1.smithi194.stdout:8/113: unlink d2/l6 0 2024-04-12T05:56:00.887 INFO:tasks.workunit.client.1.smithi194.stdout:6/259: mkdir d3/d15/d2b/d48/d4c 0 2024-04-12T05:56:00.889 INFO:tasks.workunit.client.1.smithi194.stdout:5/231: readlink d1/d4/d6/db/d15/l36 0 2024-04-12T05:56:00.889 INFO:tasks.workunit.client.1.smithi194.stdout:6/260: chown d3/d2f/d37 7093 1 2024-04-12T05:56:00.891 INFO:tasks.workunit.client.1.smithi194.stdout:2/155: read d7/d12/d1b/f1d [875387,44060] 0 2024-04-12T05:56:00.894 INFO:tasks.workunit.client.1.smithi194.stdout:4/103: dread d2/d3/f17 [0,4194304] 0 2024-04-12T05:56:00.895 INFO:tasks.workunit.client.1.smithi194.stdout:5/232: chown d1/d4/d6/db/d15/d26/c3f 524339189 1 2024-04-12T05:56:00.895 INFO:tasks.workunit.client.1.smithi194.stdout:0/124: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:00.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.896+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0197c80 con 0x7f25e006bfb0 2024-04-12T05:56:00.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.897+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 25) v1 ==== 90681+0+0 (secure 0 0 0) 0x7f25d0010380 con 0x7f25e006bfb0 2024-04-12T05:56:00.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.897+0000 7f25d67fc700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 0x7f25cc06efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:00.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.897+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f25d008bad0 con 0x7f25e006bfb0 2024-04-12T05:56:00.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.899+0000 7f25e5633700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 0x7f25cc06efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:00.897 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:00 smithi008 ceph-mon[39722]: pgmap v26: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 7.4 MiB/s rd, 63 MiB/s wr, 355 op/s 2024-04-12T05:56:00.897 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:00 smithi008 ceph-mon[39722]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:00.897 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:00 smithi008 ceph-mon[39722]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:00.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.906+0000 7f25e5633700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 0x7f25cc06efe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b5c0 tx=0x7f25dc005fd0 comp rx=0 tx=0).ready entity=mgr.24385 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:00.903 INFO:tasks.workunit.client.1.smithi194.stdout:9/206: write d1/d19/d1f/d20/f35 [4488091,127733] 0 2024-04-12T05:56:00.903 INFO:tasks.workunit.client.1.smithi194.stdout:2/156: rmdir d7/d12/d34 39 2024-04-12T05:56:00.904 INFO:tasks.workunit.client.1.smithi194.stdout:5/233: rmdir d1/d4/d6 39 2024-04-12T05:56:00.905 INFO:tasks.workunit.client.1.smithi194.stdout:4/104: write d2/d3/f17 [698777,48768] 0 2024-04-12T05:56:00.905 INFO:tasks.workunit.client.1.smithi194.stdout:5/234: readlink d1/l7 0 2024-04-12T05:56:00.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:00.908+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f25d0059720 con 0x7f25e006bfb0 2024-04-12T05:56:00.908 INFO:tasks.workunit.client.1.smithi194.stdout:9/207: dread - d1/d41/f44 zero size 2024-04-12T05:56:00.910 INFO:tasks.workunit.client.1.smithi194.stdout:4/105: truncate d2/f21 636868 0 2024-04-12T05:56:00.912 INFO:tasks.workunit.client.1.smithi194.stdout:9/208: write d1/d41/f43 [984592,102454] 0 2024-04-12T05:56:00.914 INFO:tasks.workunit.client.1.smithi194.stdout:0/125: mknod d0/da/d1e/d1a/d24/c28 0 2024-04-12T05:56:00.916 INFO:tasks.workunit.client.1.smithi194.stdout:4/106: mkdir d2/d22 0 2024-04-12T05:56:00.916 INFO:tasks.workunit.client.1.smithi194.stdout:9/209: mknod d1/d19/d1f/c45 0 2024-04-12T05:56:00.918 INFO:tasks.workunit.client.1.smithi194.stdout:0/126: mknod d0/da/d1e/d1a/c29 0 2024-04-12T05:56:00.920 INFO:tasks.workunit.client.1.smithi194.stdout:9/210: fdatasync f0 0 2024-04-12T05:56:00.925 INFO:tasks.workunit.client.1.smithi194.stdout:2/157: dwrite d7/d12/d1b/f2a [0,4194304] 0 2024-04-12T05:56:00.930 INFO:tasks.workunit.client.1.smithi194.stdout:8/114: dwrite d2/fb [0,4194304] 0 2024-04-12T05:56:00.932 INFO:tasks.workunit.client.1.smithi194.stdout:2/158: truncate d7/d12/d1a/f24 4280741 0 2024-04-12T05:56:00.933 INFO:tasks.workunit.client.1.smithi194.stdout:2/159: fdatasync d7/d12/d1b/f1e 0 2024-04-12T05:56:00.933 INFO:tasks.workunit.client.1.smithi194.stdout:4/107: dread d2/d3/ff [0,4194304] 0 2024-04-12T05:56:00.938 INFO:tasks.workunit.client.1.smithi194.stdout:9/211: creat d1/f46 x:0 0 0 2024-04-12T05:56:00.940 INFO:tasks.workunit.client.1.smithi194.stdout:0/127: getdents d0/da/d1e/d1a 0 2024-04-12T05:56:00.943 INFO:tasks.workunit.client.1.smithi194.stdout:8/115: creat d2/d8/f26 x:0 0 0 2024-04-12T05:56:00.945 INFO:tasks.workunit.client.1.smithi194.stdout:9/212: write f0 [2642827,43985] 0 2024-04-12T05:56:00.945 INFO:tasks.workunit.client.1.smithi194.stdout:8/116: fsync d2/d8/d9/d1a/f25 0 2024-04-12T05:56:00.948 INFO:tasks.workunit.client.1.smithi194.stdout:5/235: stat d1/fd 0 2024-04-12T05:56:00.950 INFO:tasks.workunit.client.1.smithi194.stdout:9/213: rename d1/f10 to d1/d19/d1f/f47 0 2024-04-12T05:56:00.951 INFO:tasks.workunit.client.1.smithi194.stdout:4/108: dwrite d2/d3/f13 [4194304,4194304] 0 2024-04-12T05:56:00.955 INFO:tasks.workunit.client.1.smithi194.stdout:2/160: dread d7/d12/d1a/f24 [0,4194304] 0 2024-04-12T05:56:00.956 INFO:tasks.workunit.client.1.smithi194.stdout:9/214: dread d1/d19/d1f/d20/f31 [0,4194304] 0 2024-04-12T05:56:00.965 INFO:tasks.workunit.client.1.smithi194.stdout:0/128: read d0/da/d1e/f17 [143193,97304] 0 2024-04-12T05:56:00.968 INFO:tasks.workunit.client.1.smithi194.stdout:0/129: write d0/f2 [7632010,27131] 0 2024-04-12T05:56:00.970 INFO:tasks.workunit.client.1.smithi194.stdout:8/117: dread d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:56:00.971 INFO:tasks.workunit.client.1.smithi194.stdout:5/236: dwrite d1/d4/d6/db/f19 [4194304,4194304] 0 2024-04-12T05:56:00.972 INFO:tasks.workunit.client.1.smithi194.stdout:2/161: dwrite d7/d12/d1a/f3e [0,4194304] 0 2024-04-12T05:56:00.976 INFO:tasks.workunit.client.1.smithi194.stdout:9/215: truncate d1/f5 6403258 0 2024-04-12T05:56:00.976 INFO:tasks.workunit.client.1.smithi194.stdout:4/109: truncate f0 1734452 0 2024-04-12T05:56:00.982 INFO:tasks.workunit.client.1.smithi194.stdout:8/118: creat d2/d8/d15/d17/f27 x:0 0 0 2024-04-12T05:56:00.986 INFO:tasks.workunit.client.1.smithi194.stdout:8/119: chown d2/d8/d9/de/d10/l13 693 1 2024-04-12T05:56:00.987 INFO:tasks.workunit.client.1.smithi194.stdout:9/216: mknod d1/d19/d1f/d20/c48 0 2024-04-12T05:56:00.987 INFO:tasks.workunit.client.1.smithi194.stdout:8/120: dread - d2/d8/d15/d17/f27 zero size 2024-04-12T05:56:00.988 INFO:tasks.workunit.client.1.smithi194.stdout:0/130: creat d0/da/d1e/d27/f2a x:0 0 0 2024-04-12T05:56:00.988 INFO:tasks.workunit.client.1.smithi194.stdout:5/237: mknod d1/c4d 0 2024-04-12T05:56:00.990 INFO:tasks.workunit.client.1.smithi194.stdout:4/110: mknod d2/d22/c23 0 2024-04-12T05:56:00.990 INFO:tasks.workunit.client.1.smithi194.stdout:4/111: fsync d2/f21 0 2024-04-12T05:56:00.993 INFO:tasks.workunit.client.1.smithi194.stdout:9/217: mkdir d1/d21/d49 0 2024-04-12T05:56:00.994 INFO:tasks.workunit.client.1.smithi194.stdout:0/131: creat d0/da/d1e/d1a/f2b x:0 0 0 2024-04-12T05:56:00.999 INFO:tasks.workunit.client.1.smithi194.stdout:2/162: write d7/d12/d1b/f2c [416699,38256] 0 2024-04-12T05:56:00.999 INFO:tasks.workunit.client.1.smithi194.stdout:9/218: unlink d1/d19/d1f/c45 0 2024-04-12T05:56:01.001 INFO:tasks.workunit.client.1.smithi194.stdout:0/132: mknod d0/da/d1e/d27/c2c 0 2024-04-12T05:56:01.001 INFO:tasks.workunit.client.1.smithi194.stdout:5/238: dread d1/d4/d6/db/d15/d26/f28 [0,4194304] 0 2024-04-12T05:56:01.002 INFO:tasks.workunit.client.1.smithi194.stdout:8/121: creat d2/d8/d9/f28 x:0 0 0 2024-04-12T05:56:01.003 INFO:tasks.workunit.client.1.smithi194.stdout:2/163: fdatasync d7/d22/f2e 0 2024-04-12T05:56:01.003 INFO:tasks.workunit.client.1.smithi194.stdout:4/112: dwrite d2/f6 [4194304,4194304] 0 2024-04-12T05:56:01.005 INFO:tasks.workunit.client.1.smithi194.stdout:8/122: fdatasync d2/f12 0 2024-04-12T05:56:01.006 INFO:tasks.workunit.client.1.smithi194.stdout:0/133: stat d0/da/d1e/c1b 0 2024-04-12T05:56:01.010 INFO:tasks.workunit.client.1.smithi194.stdout:5/239: write d1/d4/d6/db/d3b/f3e [3135361,40994] 0 2024-04-12T05:56:01.011 INFO:tasks.workunit.client.1.smithi194.stdout:4/113: rmdir d2/d3/d1b 39 2024-04-12T05:56:01.012 INFO:tasks.workunit.client.1.smithi194.stdout:4/114: write d2/d3/f18 [4374502,36049] 0 2024-04-12T05:56:01.016 INFO:tasks.workunit.client.1.smithi194.stdout:4/115: write d2/d3/f1a [668996,101388] 0 2024-04-12T05:56:01.018 INFO:tasks.workunit.client.1.smithi194.stdout:2/164: creat d7/d12/f3f x:0 0 0 2024-04-12T05:56:01.020 INFO:tasks.workunit.client.1.smithi194.stdout:9/219: dread d1/d41/f43 [0,4194304] 0 2024-04-12T05:56:01.022 INFO:tasks.workunit.client.1.smithi194.stdout:2/165: truncate d7/d12/f38 473655 0 2024-04-12T05:56:01.029 INFO:tasks.workunit.client.1.smithi194.stdout:2/166: unlink d7/d12/d1a/f2d 0 2024-04-12T05:56:01.031 INFO:tasks.workunit.client.1.smithi194.stdout:4/116: dwrite d2/d3/f18 [4194304,4194304] 0 2024-04-12T05:56:01.033 INFO:tasks.workunit.client.1.smithi194.stdout:5/240: getdents d1/d4/df 0 2024-04-12T05:56:01.039 INFO:tasks.workunit.client.1.smithi194.stdout:4/117: symlink d2/l24 0 2024-04-12T05:56:01.041 INFO:tasks.workunit.client.1.smithi194.stdout:2/167: chown d7/c21 854 1 2024-04-12T05:56:01.043 INFO:tasks.workunit.client.1.smithi194.stdout:2/168: rmdir d7/d12 39 2024-04-12T05:56:01.044 INFO:tasks.workunit.client.1.smithi194.stdout:0/134: truncate d0/da/d1e/f19 283059 0 2024-04-12T05:56:01.046 INFO:tasks.workunit.client.1.smithi194.stdout:5/241: creat d1/d4/d6/db/d15/d26/f4e x:0 0 0 2024-04-12T05:56:01.049 INFO:tasks.workunit.client.1.smithi194.stdout:2/169: mknod d7/d12/d1a/c40 0 2024-04-12T05:56:01.052 INFO:tasks.workunit.client.1.smithi194.stdout:5/242: truncate d1/d4/d6/db/d2b/f44 671072 0 2024-04-12T05:56:01.054 INFO:tasks.workunit.client.1.smithi194.stdout:2/170: creat d7/d12/f41 x:0 0 0 2024-04-12T05:56:01.054 INFO:tasks.workunit.client.1.smithi194.stdout:0/135: dwrite d0/da/d1e/d1a/f1c [0,4194304] 0 2024-04-12T05:56:01.055 INFO:tasks.workunit.client.1.smithi194.stdout:1/198: sync 2024-04-12T05:56:01.060 INFO:tasks.workunit.client.1.smithi194.stdout:1/199: truncate d0/f32 112814 0 2024-04-12T05:56:01.060 INFO:tasks.workunit.client.1.smithi194.stdout:5/243: fdatasync d1/d4/d6/db/d15/d26/f4e 0 2024-04-12T05:56:01.063 INFO:tasks.workunit.client.1.smithi194.stdout:0/136: link d0/f1d d0/da/f2d 0 2024-04-12T05:56:01.065 INFO:tasks.workunit.client.1.smithi194.stdout:1/200: symlink d0/df/l34 0 2024-04-12T05:56:01.067 INFO:tasks.workunit.client.1.smithi194.stdout:2/171: dwrite d7/d12/f33 [0,4194304] 0 2024-04-12T05:56:01.069 INFO:tasks.workunit.client.1.smithi194.stdout:1/201: readlink d0/d11/d18/d19/d1a/l17 0 2024-04-12T05:56:01.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:00 smithi194 ceph-mon[49771]: pgmap v26: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 7.4 MiB/s rd, 63 MiB/s wr, 355 op/s 2024-04-12T05:56:01.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:00 smithi194 ceph-mon[49771]: from='client.14646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:01.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:00 smithi194 ceph-mon[49771]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:01.072 INFO:tasks.workunit.client.1.smithi194.stdout:4/118: dread d2/d3/f12 [0,4194304] 0 2024-04-12T05:56:01.077 INFO:tasks.workunit.client.1.smithi194.stdout:1/202: mkdir d0/d11/d18/d35 0 2024-04-12T05:56:01.077 INFO:tasks.workunit.client.1.smithi194.stdout:0/137: dread d0/da/d1e/f8 [0,4194304] 0 2024-04-12T05:56:01.078 INFO:tasks.workunit.client.1.smithi194.stdout:5/244: dwrite d1/d4/f5 [4194304,4194304] 0 2024-04-12T05:56:01.079 INFO:tasks.workunit.client.1.smithi194.stdout:4/119: write d2/d3/f12 [3661592,48748] 0 2024-04-12T05:56:01.079 INFO:tasks.workunit.client.1.smithi194.stdout:0/138: chown d0/da/d1e/d1a 123 1 2024-04-12T05:56:01.081 INFO:tasks.workunit.client.1.smithi194.stdout:2/172: rename d7/d12/d1b to d7/d22/d42 0 2024-04-12T05:56:01.081 INFO:tasks.workunit.client.1.smithi194.stdout:1/203: read d0/f4 [3986247,68531] 0 2024-04-12T05:56:01.083 INFO:tasks.workunit.client.1.smithi194.stdout:2/173: stat d7/d12/d1a/f24 0 2024-04-12T05:56:01.083 INFO:tasks.workunit.client.1.smithi194.stdout:0/139: dread d0/da/d1e/f17 [0,4194304] 0 2024-04-12T05:56:01.085 INFO:tasks.workunit.client.1.smithi194.stdout:4/120: mkdir d2/d11/d1e/d25 0 2024-04-12T05:56:01.088 INFO:tasks.workunit.client.1.smithi194.stdout:2/174: chown d7/d12/d1a/c40 799254535 1 2024-04-12T05:56:01.093 INFO:tasks.workunit.client.1.smithi194.stdout:4/121: getdents d2 0 2024-04-12T05:56:01.096 INFO:tasks.workunit.client.1.smithi194.stdout:1/204: dwrite d0/f12 [0,4194304] 0 2024-04-12T05:56:01.097 INFO:tasks.workunit.client.1.smithi194.stdout:2/175: write d7/d12/d25/f31 [300399,62461] 0 2024-04-12T05:56:01.098 INFO:tasks.workunit.client.1.smithi194.stdout:4/122: symlink d2/l26 0 2024-04-12T05:56:01.100 INFO:tasks.workunit.client.1.smithi194.stdout:0/140: dwrite d0/f4 [4194304,4194304] 0 2024-04-12T05:56:01.102 INFO:tasks.workunit.client.1.smithi194.stdout:1/205: creat d0/d11/d18/d19/f36 x:0 0 0 2024-04-12T05:56:01.105 INFO:tasks.workunit.client.1.smithi194.stdout:4/123: truncate d2/d3/f1a 1777500 0 2024-04-12T05:56:01.105 INFO:tasks.workunit.client.1.smithi194.stdout:5/245: dwrite d1/fd [0,4194304] 0 2024-04-12T05:56:01.106 INFO:tasks.workunit.client.1.smithi194.stdout:2/176: creat d7/d12/d1a/f43 x:0 0 0 2024-04-12T05:56:01.108 INFO:tasks.workunit.client.1.smithi194.stdout:2/177: chown d7/d12/f33 480 1 2024-04-12T05:56:01.113 INFO:tasks.workunit.client.1.smithi194.stdout:1/206: dread d0/f12 [0,4194304] 0 2024-04-12T05:56:01.115 INFO:tasks.workunit.client.1.smithi194.stdout:4/124: creat d2/d3/f27 x:0 0 0 2024-04-12T05:56:01.115 INFO:tasks.workunit.client.1.smithi194.stdout:5/246: unlink d1/d4/d6/db/f48 0 2024-04-12T05:56:01.115 INFO:tasks.workunit.client.1.smithi194.stdout:0/141: dwrite d0/f4 [4194304,4194304] 0 2024-04-12T05:56:01.117 INFO:tasks.workunit.client.1.smithi194.stdout:2/178: creat d7/d12/d1a/f44 x:0 0 0 2024-04-12T05:56:01.119 INFO:tasks.workunit.client.1.smithi194.stdout:4/125: creat d2/d11/f28 x:0 0 0 2024-04-12T05:56:01.119 INFO:tasks.workunit.client.1.smithi194.stdout:1/207: truncate d0/d11/d18/d19/f36 43316 0 2024-04-12T05:56:01.120 INFO:tasks.workunit.client.1.smithi194.stdout:5/247: mkdir d1/d4/d6/db/d38/d4f 0 2024-04-12T05:56:01.120 INFO:tasks.workunit.client.1.smithi194.stdout:4/126: chown d2/fc 62293 1 2024-04-12T05:56:01.121 INFO:tasks.workunit.client.1.smithi194.stdout:0/142: creat d0/da/d1e/d1a/f2e x:0 0 0 2024-04-12T05:56:01.123 INFO:tasks.workunit.client.1.smithi194.stdout:2/179: truncate d7/d12/d25/f31 943681 0 2024-04-12T05:56:01.124 INFO:tasks.workunit.client.1.smithi194.stdout:1/208: write d0/d11/d18/d19/d1c/f24 [6968703,78128] 0 2024-04-12T05:56:01.126 INFO:tasks.workunit.client.1.smithi194.stdout:4/127: symlink d2/d3/d1b/l29 0 2024-04-12T05:56:01.127 INFO:tasks.workunit.client.1.smithi194.stdout:0/143: truncate d0/da/d1e/d1a/f2e 1008318 0 2024-04-12T05:56:01.128 INFO:tasks.workunit.client.1.smithi194.stdout:5/248: creat d1/d4/d6/db/d15/d26/f50 x:0 0 0 2024-04-12T05:56:01.129 INFO:tasks.workunit.client.1.smithi194.stdout:2/180: fsync d7/d12/d34/f3a 0 2024-04-12T05:56:01.129 INFO:tasks.workunit.client.1.smithi194.stdout:1/209: symlink d0/d11/d18/d19/d1a/l37 0 2024-04-12T05:56:01.130 INFO:tasks.workunit.client.1.smithi194.stdout:1/210: chown d0/d11/d18/d19/d1c 1307 1 2024-04-12T05:56:01.131 INFO:tasks.workunit.client.1.smithi194.stdout:4/128: creat d2/d22/f2a x:0 0 0 2024-04-12T05:56:01.132 INFO:tasks.workunit.client.1.smithi194.stdout:4/129: chown d2/d3/f12 91 1 2024-04-12T05:56:01.132 INFO:tasks.workunit.client.1.smithi194.stdout:1/211: read - d0/d11/d18/d19/d1a/d23/f29 zero size 2024-04-12T05:56:01.136 INFO:tasks.workunit.client.1.smithi194.stdout:1/212: read d0/d11/d18/d19/d1c/f22 [787233,16519] 0 2024-04-12T05:56:01.137 INFO:tasks.workunit.client.1.smithi194.stdout:2/181: mknod d7/c45 0 2024-04-12T05:56:01.141 INFO:tasks.workunit.client.1.smithi194.stdout:1/213: creat d0/df/f38 x:0 0 0 2024-04-12T05:56:01.141 INFO:tasks.workunit.client.1.smithi194.stdout:4/130: dwrite d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:01.143 INFO:tasks.workunit.client.1.smithi194.stdout:4/131: truncate d2/d22/f2a 654038 0 2024-04-12T05:56:01.144 INFO:tasks.workunit.client.1.smithi194.stdout:2/182: mknod d7/c46 0 2024-04-12T05:56:01.146 INFO:tasks.workunit.client.1.smithi194.stdout:4/132: write d2/d3/f13 [8130006,37449] 0 2024-04-12T05:56:01.150 INFO:tasks.workunit.client.1.smithi194.stdout:4/133: mkdir d2/d11/d1e/d25/d2b 0 2024-04-12T05:56:01.152 INFO:tasks.workunit.client.1.smithi194.stdout:0/144: dwrite d0/da/f2d [4194304,4194304] 0 2024-04-12T05:56:01.156 INFO:tasks.workunit.client.1.smithi194.stdout:0/145: fsync d0/da/d1e/d1a/f23 0 2024-04-12T05:56:01.160 INFO:tasks.workunit.client.1.smithi194.stdout:2/183: dwrite d7/d22/d2b/f2f [0,4194304] 0 2024-04-12T05:56:01.160 INFO:tasks.workunit.client.1.smithi194.stdout:5/249: dread d1/fa [0,4194304] 0 2024-04-12T05:56:01.163 INFO:tasks.workunit.client.1.smithi194.stdout:4/134: getdents d2/d3/d1b 0 2024-04-12T05:56:01.164 INFO:tasks.workunit.client.1.smithi194.stdout:2/184: mkdir d7/d22/d2b/d47 0 2024-04-12T05:56:01.165 INFO:tasks.workunit.client.1.smithi194.stdout:1/214: truncate d0/d11/f26 657744 0 2024-04-12T05:56:01.168 INFO:tasks.workunit.client.1.smithi194.stdout:5/250: symlink d1/d4/df/d23/l51 0 2024-04-12T05:56:01.173 INFO:tasks.workunit.client.1.smithi194.stdout:5/251: chown d1/d4/df/d23/l37 70987838 1 2024-04-12T05:56:01.174 INFO:tasks.workunit.client.1.smithi194.stdout:0/146: dwrite d0/f2 [4194304,4194304] 0 2024-04-12T05:56:01.175 INFO:tasks.workunit.client.1.smithi194.stdout:2/185: truncate d7/d22/d42/f1d 4515402 0 2024-04-12T05:56:01.177 INFO:tasks.workunit.client.1.smithi194.stdout:1/215: truncate d0/d11/d18/d19/d1a/d23/f29 106162 0 2024-04-12T05:56:01.177 INFO:tasks.workunit.client.1.smithi194.stdout:2/186: fdatasync d7/d12/f3f 0 2024-04-12T05:56:01.179 INFO:tasks.workunit.client.1.smithi194.stdout:4/135: dwrite d2/d3/f12 [0,4194304] 0 2024-04-12T05:56:01.184 INFO:tasks.workunit.client.1.smithi194.stdout:4/136: fsync d2/d11/f28 0 2024-04-12T05:56:01.185 INFO:tasks.workunit.client.1.smithi194.stdout:1/216: unlink d0/d11/d18/d19/d1a/d23/f29 0 2024-04-12T05:56:01.185 INFO:tasks.workunit.client.1.smithi194.stdout:4/137: chown d2/d22 868517 1 2024-04-12T05:56:01.188 INFO:tasks.workunit.client.1.smithi194.stdout:2/187: link d7/fa d7/f48 0 2024-04-12T05:56:01.188 INFO:tasks.workunit.client.1.smithi194.stdout:1/217: write d0/d11/d18/d19/d1c/f20 [1011031,60321] 0 2024-04-12T05:56:01.190 INFO:tasks.workunit.client.1.smithi194.stdout:5/252: dwrite d1/d4/d6/db/f19 [0,4194304] 0 2024-04-12T05:56:01.193 INFO:tasks.workunit.client.1.smithi194.stdout:4/138: dread d2/fc [0,4194304] 0 2024-04-12T05:56:01.193 INFO:tasks.workunit.client.1.smithi194.stdout:2/188: unlink d7/d22/d42/f2c 0 2024-04-12T05:56:01.194 INFO:tasks.workunit.client.1.smithi194.stdout:4/139: chown d2/d22 124 1 2024-04-12T05:56:01.197 INFO:tasks.workunit.client.1.smithi194.stdout:4/140: write d2/d3/f1a [1132194,104757] 0 2024-04-12T05:56:01.208 INFO:tasks.workunit.client.1.smithi194.stdout:2/189: truncate d7/f48 1298190 0 2024-04-12T05:56:01.209 INFO:tasks.workunit.client.1.smithi194.stdout:0/147: dwrite d0/da/d1e/f19 [0,4194304] 0 2024-04-12T05:56:01.212 INFO:tasks.workunit.client.1.smithi194.stdout:4/141: truncate d2/d3/ff 174667 0 2024-04-12T05:56:01.214 INFO:tasks.workunit.client.1.smithi194.stdout:1/218: dread d0/d11/f26 [0,4194304] 0 2024-04-12T05:56:01.215 INFO:tasks.workunit.client.1.smithi194.stdout:0/148: mknod d0/da/d1e/d1a/c2f 0 2024-04-12T05:56:01.215 INFO:tasks.workunit.client.1.smithi194.stdout:2/190: chown d7/c9 620 1 2024-04-12T05:56:01.217 INFO:tasks.workunit.client.1.smithi194.stdout:1/219: creat d0/d11/d18/d19/f39 x:0 0 0 2024-04-12T05:56:01.217 INFO:tasks.workunit.client.1.smithi194.stdout:0/149: creat d0/f30 x:0 0 0 2024-04-12T05:56:01.218 INFO:tasks.workunit.client.1.smithi194.stdout:1/220: readlink d0/d11/d18/d19/d1a/l37 0 2024-04-12T05:56:01.219 INFO:tasks.workunit.client.1.smithi194.stdout:4/142: read f0 [1572840,47147] 0 2024-04-12T05:56:01.220 INFO:tasks.workunit.client.1.smithi194.stdout:1/221: stat d0/df/l34 0 2024-04-12T05:56:01.220 INFO:tasks.workunit.client.1.smithi194.stdout:2/191: creat d7/d22/f49 x:0 0 0 2024-04-12T05:56:01.221 INFO:tasks.workunit.client.1.smithi194.stdout:4/143: fsync d2/f6 0 2024-04-12T05:56:01.221 INFO:tasks.workunit.client.1.smithi194.stdout:0/150: creat d0/da/d1e/d1a/f31 x:0 0 0 2024-04-12T05:56:01.223 INFO:tasks.workunit.client.1.smithi194.stdout:2/192: mkdir d7/d12/d34/d4a 0 2024-04-12T05:56:01.226 INFO:tasks.workunit.client.1.smithi194.stdout:1/222: rename d0/df/f38 to d0/d11/d18/d35/f3a 0 2024-04-12T05:56:01.230 INFO:tasks.workunit.client.1.smithi194.stdout:4/144: getdents d2/d22 0 2024-04-12T05:56:01.230 INFO:tasks.workunit.client.1.smithi194.stdout:0/151: dread d0/da/d1e/f8 [0,4194304] 0 2024-04-12T05:56:01.232 INFO:tasks.workunit.client.1.smithi194.stdout:2/193: rename d7/c19 to d7/d12/d25/c4b 0 2024-04-12T05:56:01.233 INFO:tasks.workunit.client.1.smithi194.stdout:4/145: symlink d2/l2c 0 2024-04-12T05:56:01.234 INFO:tasks.workunit.client.1.smithi194.stdout:5/253: dread d1/f3 [0,4194304] 0 2024-04-12T05:56:01.235 INFO:tasks.workunit.client.1.smithi194.stdout:2/194: chown d7/l30 340234756 1 2024-04-12T05:56:01.236 INFO:tasks.workunit.client.1.smithi194.stdout:4/146: symlink d2/d11/d1e/l2d 0 2024-04-12T05:56:01.236 INFO:tasks.workunit.client.1.smithi194.stdout:2/195: stat d7/d12/d1a/f24 0 2024-04-12T05:56:01.239 INFO:tasks.workunit.client.1.smithi194.stdout:5/254: mkdir d1/d4/d6/d34/d52 0 2024-04-12T05:56:01.242 INFO:tasks.workunit.client.1.smithi194.stdout:0/152: rename d0/da/d1e/d1a/f2b to d0/da/d1e/f32 0 2024-04-12T05:56:01.244 INFO:tasks.workunit.client.1.smithi194.stdout:5/255: creat d1/d4/d6/db/f53 x:0 0 0 2024-04-12T05:56:01.244 INFO:tasks.workunit.client.1.smithi194.stdout:5/256: fdatasync d1/d4/f42 0 2024-04-12T05:56:01.245 INFO:tasks.workunit.client.1.smithi194.stdout:4/147: symlink d2/d11/d1e/d25/d2b/l2e 0 2024-04-12T05:56:01.246 INFO:tasks.workunit.client.1.smithi194.stdout:0/153: symlink d0/da/d1e/d1a/l33 0 2024-04-12T05:56:01.248 INFO:tasks.workunit.client.1.smithi194.stdout:2/196: symlink d7/d12/d34/d4a/l4c 0 2024-04-12T05:56:01.251 INFO:tasks.workunit.client.1.smithi194.stdout:5/257: rmdir d1/d4/d6/db/d15 39 2024-04-12T05:56:01.253 INFO:tasks.workunit.client.1.smithi194.stdout:0/154: dwrite d0/da/d1e/d1a/f1c [0,4194304] 0 2024-04-12T05:56:01.254 INFO:tasks.workunit.client.1.smithi194.stdout:4/148: mkdir d2/d11/d1e/d2f 0 2024-04-12T05:56:01.256 INFO:tasks.workunit.client.1.smithi194.stdout:1/223: truncate d0/d11/d18/d19/d1c/f24 6047945 0 2024-04-12T05:56:01.259 INFO:tasks.workunit.client.1.smithi194.stdout:5/258: dread d1/d4/f1a [0,4194304] 0 2024-04-12T05:56:01.261 INFO:tasks.workunit.client.1.smithi194.stdout:0/155: creat d0/da/d1e/d27/f34 x:0 0 0 2024-04-12T05:56:01.262 INFO:tasks.workunit.client.1.smithi194.stdout:0/156: stat d0/da/f13 0 2024-04-12T05:56:01.262 INFO:tasks.workunit.client.1.smithi194.stdout:1/224: creat d0/d11/d18/d19/d1a/d23/d27/d2e/f3b x:0 0 0 2024-04-12T05:56:01.262 INFO:tasks.workunit.client.1.smithi194.stdout:2/197: truncate d7/fa 426946 0 2024-04-12T05:56:01.264 INFO:tasks.workunit.client.1.smithi194.stdout:5/259: creat d1/d4/d6/d34/f54 x:0 0 0 2024-04-12T05:56:01.265 INFO:tasks.workunit.client.1.smithi194.stdout:4/149: mknod d2/d11/d1e/d2f/c30 0 2024-04-12T05:56:01.265 INFO:tasks.workunit.client.1.smithi194.stdout:2/198: rmdir d7/d22/d42 39 2024-04-12T05:56:01.273 INFO:tasks.workunit.client.1.smithi194.stdout:0/157: creat d0/da/d1e/f35 x:0 0 0 2024-04-12T05:56:01.273 INFO:tasks.workunit.client.1.smithi194.stdout:5/260: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:01.276 INFO:tasks.workunit.client.1.smithi194.stdout:2/199: dwrite d7/d12/f41 [0,4194304] 0 2024-04-12T05:56:01.276 INFO:tasks.workunit.client.1.smithi194.stdout:1/225: link d0/le d0/d11/d18/d19/d1a/d23/l3c 0 2024-04-12T05:56:01.283 INFO:tasks.workunit.client.1.smithi194.stdout:2/200: readlink d7/d12/l13 0 2024-04-12T05:56:01.283 INFO:tasks.workunit.client.1.smithi194.stdout:4/150: creat d2/d3/f31 x:0 0 0 2024-04-12T05:56:01.283 INFO:tasks.workunit.client.1.smithi194.stdout:4/151: write d2/d3/f13 [2895752,61386] 0 2024-04-12T05:56:01.287 INFO:tasks.workunit.client.1.smithi194.stdout:2/201: chown d7/d22/f49 3 1 2024-04-12T05:56:01.288 INFO:tasks.workunit.client.1.smithi194.stdout:1/226: mknod d0/df/c3d 0 2024-04-12T05:56:01.292 INFO:tasks.workunit.client.1.smithi194.stdout:1/227: readlink d0/d11/d18/d19/d1a/l13 0 2024-04-12T05:56:01.294 INFO:tasks.workunit.client.1.smithi194.stdout:5/261: dwrite d1/d4/d6/db/fc [4194304,4194304] 0 2024-04-12T05:56:01.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.299+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 --> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25e00611d0 con 0x7f25cc06cb20 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.302+0000 7f25d67fc700 1 -- 172.21.15.8:0/546791521 <== mgr.24385 v2:172.21.15.194:6828/2335168333 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f25e00611d0 con 0x7f25cc06cb20 2024-04-12T05:56:01.297 INFO:tasks.workunit.client.1.smithi194.stdout:5/262: readlink d1/l2 0 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "1/2 daemons upgraded", 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mgr daemons", 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:56:01.297 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:56:01.299 INFO:tasks.workunit.client.1.smithi194.stdout:5/263: chown d1/d4/l1b 14 1 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 msgr2=0x7f25cc06efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 0x7f25cc06efe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b5c0 tx=0x7f25dc005fd0 comp rx=0 tx=0).stop 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 msgr2=0x7f25e01a1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25d000b4b0 tx=0x7f25d000b870 comp rx=0 tx=0).stop 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 shutdown_connections 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:6828/2335168333,v1:172.21.15.194:6829/2335168333] conn(0x7f25cc06cb20 0x7f25cc06efe0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25e006ad60 0x7f25e0111280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 --2- 172.21.15.8:0/546791521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e006bfb0 0x7f25e01a1960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:01.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 >> 172.21.15.8:0/546791521 conn(0x7f25e00fa490 msgr2=0x7f25e00fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:01.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 shutdown_connections 2024-04-12T05:56:01.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:01.304+0000 7f25e7897700 1 -- 172.21.15.8:0/546791521 wait complete. 2024-04-12T05:56:01.302 INFO:tasks.workunit.client.1.smithi194.stdout:2/202: dwrite d7/f11 [4194304,4194304] 0 2024-04-12T05:56:01.304 INFO:tasks.workunit.client.1.smithi194.stdout:5/264: mknod d1/d4/d6/d34/c55 0 2024-04-12T05:56:01.306 INFO:tasks.workunit.client.1.smithi194.stdout:4/152: truncate d2/d22/f2a 97606 0 2024-04-12T05:56:01.310 INFO:tasks.workunit.client.1.smithi194.stdout:4/153: mkdir d2/d3/d1b/d32 0 2024-04-12T05:56:01.313 INFO:tasks.workunit.client.1.smithi194.stdout:5/265: rmdir d1/d4/d6/db/d38/d4f 0 2024-04-12T05:56:01.314 INFO:tasks.workunit.client.1.smithi194.stdout:0/158: dread d0/f14 [0,4194304] 0 2024-04-12T05:56:01.316 INFO:tasks.workunit.client.1.smithi194.stdout:0/159: symlink d0/da/l36 0 2024-04-12T05:56:01.318 INFO:tasks.workunit.client.1.smithi194.stdout:0/160: write d0/f14 [4668305,126851] 0 2024-04-12T05:56:01.320 INFO:tasks.workunit.client.1.smithi194.stdout:0/161: dread - d0/da/d1e/d27/f34 zero size 2024-04-12T05:56:01.323 INFO:tasks.workunit.client.1.smithi194.stdout:0/162: creat d0/f37 x:0 0 0 2024-04-12T05:56:01.325 INFO:tasks.workunit.client.1.smithi194.stdout:2/203: write d7/d12/d1a/f24 [15130,75776] 0 2024-04-12T05:56:01.327 INFO:tasks.workunit.client.1.smithi194.stdout:4/154: truncate d2/d3/f12 471121 0 2024-04-12T05:56:01.329 INFO:tasks.workunit.client.1.smithi194.stdout:4/155: chown d2/d11/d1e/d2f 29 1 2024-04-12T05:56:01.336 INFO:tasks.workunit.client.1.smithi194.stdout:0/163: dwrite d0/da/d1e/f25 [0,4194304] 0 2024-04-12T05:56:01.342 INFO:tasks.workunit.client.1.smithi194.stdout:2/204: dwrite d7/d12/f3f [0,4194304] 0 2024-04-12T05:56:01.345 INFO:tasks.workunit.client.1.smithi194.stdout:4/156: link d2/d22/c23 d2/d11/c33 0 2024-04-12T05:56:01.348 INFO:tasks.workunit.client.1.smithi194.stdout:5/266: dread d1/d4/f1c [0,4194304] 0 2024-04-12T05:56:01.349 INFO:tasks.workunit.client.1.smithi194.stdout:0/164: dwrite d0/f30 [0,4194304] 0 2024-04-12T05:56:01.351 INFO:tasks.workunit.client.1.smithi194.stdout:0/165: chown d0/da/d1e/fe 49407803 1 2024-04-12T05:56:01.357 INFO:tasks.workunit.client.1.smithi194.stdout:1/228: dread d0/d11/d18/d19/d1c/f24 [0,4194304] 0 2024-04-12T05:56:01.358 INFO:tasks.workunit.client.1.smithi194.stdout:1/229: read d0/d11/f26 [247358,66466] 0 2024-04-12T05:56:01.363 INFO:tasks.workunit.client.1.smithi194.stdout:5/267: write d1/d4/f25 [4413206,60926] 0 2024-04-12T05:56:01.365 INFO:tasks.workunit.client.1.smithi194.stdout:4/157: write d2/d3/f12 [1413992,64864] 0 2024-04-12T05:56:01.365 INFO:tasks.workunit.client.1.smithi194.stdout:1/230: write d0/d11/d18/d19/f1f [1885100,113987] 0 2024-04-12T05:56:01.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/231: fdatasync d0/d11/d18/d35/f3a 0 2024-04-12T05:56:01.372 INFO:tasks.workunit.client.1.smithi194.stdout:1/232: mkdir d0/d3e 0 2024-04-12T05:56:01.372 INFO:tasks.workunit.client.1.smithi194.stdout:4/158: truncate d2/d3/f13 6562103 0 2024-04-12T05:56:01.375 INFO:tasks.workunit.client.1.smithi194.stdout:4/159: rename d2/d3/f17 to d2/d22/f34 0 2024-04-12T05:56:01.376 INFO:tasks.workunit.client.1.smithi194.stdout:4/160: fsync d2/f6 0 2024-04-12T05:56:01.377 INFO:tasks.workunit.client.1.smithi194.stdout:4/161: dread d2/fc [0,4194304] 0 2024-04-12T05:56:01.378 INFO:tasks.workunit.client.1.smithi194.stdout:1/233: link d0/le d0/d11/d18/l3f 0 2024-04-12T05:56:01.380 INFO:tasks.workunit.client.1.smithi194.stdout:4/162: dread d2/d22/f2a [0,4194304] 0 2024-04-12T05:56:01.380 INFO:tasks.workunit.client.1.smithi194.stdout:1/234: mknod d0/d11/c40 0 2024-04-12T05:56:01.380 INFO:tasks.workunit.client.1.smithi194.stdout:4/163: stat d2/d11 0 2024-04-12T05:56:01.381 INFO:tasks.workunit.client.1.smithi194.stdout:1/235: chown d0/d11/d18/d19/d1a/l13 29 1 2024-04-12T05:56:01.382 INFO:tasks.workunit.client.1.smithi194.stdout:4/164: write d2/d3/f1a [4724818,59704] 0 2024-04-12T05:56:01.386 INFO:tasks.workunit.client.1.smithi194.stdout:1/236: unlink d0/d11/d18/d19/d1a/d23/d27/d2e/f3b 0 2024-04-12T05:56:01.389 INFO:tasks.workunit.client.1.smithi194.stdout:1/237: mknod d0/d11/d18/c41 0 2024-04-12T05:56:01.390 INFO:tasks.workunit.client.1.smithi194.stdout:4/165: dwrite d2/d3/f31 [0,4194304] 0 2024-04-12T05:56:01.392 INFO:tasks.workunit.client.1.smithi194.stdout:1/238: chown d0/d11/d18/d19/f36 1129 1 2024-04-12T05:56:01.394 INFO:tasks.workunit.client.1.smithi194.stdout:4/166: mknod d2/d22/c35 0 2024-04-12T05:56:01.398 INFO:tasks.workunit.client.1.smithi194.stdout:1/239: mkdir d0/d11/d18/d19/d1a/d23/d27/d2e/d42 0 2024-04-12T05:56:01.401 INFO:tasks.workunit.client.1.smithi194.stdout:1/240: creat d0/d11/d18/d19/d1a/f43 x:0 0 0 2024-04-12T05:56:01.404 INFO:tasks.workunit.client.1.smithi194.stdout:1/241: symlink d0/d11/d18/l44 0 2024-04-12T05:56:01.405 INFO:tasks.workunit.client.1.smithi194.stdout:4/167: dwrite d2/fc [0,4194304] 0 2024-04-12T05:56:01.407 INFO:tasks.workunit.client.1.smithi194.stdout:1/242: creat d0/d11/d18/f45 x:0 0 0 2024-04-12T05:56:01.413 INFO:tasks.workunit.client.1.smithi194.stdout:4/168: dread d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:01.417 INFO:tasks.workunit.client.1.smithi194.stdout:4/169: read d2/d3/f13 [611323,35861] 0 2024-04-12T05:56:01.417 INFO:tasks.workunit.client.1.smithi194.stdout:1/243: dwrite d0/d11/d18/d19/f39 [0,4194304] 0 2024-04-12T05:56:01.419 INFO:tasks.workunit.client.1.smithi194.stdout:4/170: symlink d2/d11/d1f/l36 0 2024-04-12T05:56:01.420 INFO:tasks.workunit.client.1.smithi194.stdout:1/244: creat d0/d11/d18/d19/d1c/f46 x:0 0 0 2024-04-12T05:56:01.421 INFO:tasks.workunit.client.1.smithi194.stdout:4/171: write d2/f21 [289163,127146] 0 2024-04-12T05:56:01.422 INFO:tasks.workunit.client.1.smithi194.stdout:4/172: rmdir d2 39 2024-04-12T05:56:01.428 INFO:tasks.workunit.client.1.smithi194.stdout:1/245: dwrite d0/d11/d18/d19/f36 [0,4194304] 0 2024-04-12T05:56:01.428 INFO:tasks.workunit.client.1.smithi194.stdout:4/173: getdents d2/d11/d1e/d25/d2b 0 2024-04-12T05:56:01.429 INFO:tasks.workunit.client.1.smithi194.stdout:1/246: chown d0/d11/d18/d19/d1a 368954 1 2024-04-12T05:56:01.430 INFO:tasks.workunit.client.1.smithi194.stdout:1/247: chown d0/d11/d18/d19/d1c 1972916 1 2024-04-12T05:56:01.431 INFO:tasks.workunit.client.1.smithi194.stdout:1/248: readlink d0/d11/d18/d19/d1a/l13 0 2024-04-12T05:56:01.434 INFO:tasks.workunit.client.1.smithi194.stdout:1/249: getdents d0/d3e 0 2024-04-12T05:56:01.435 INFO:tasks.workunit.client.1.smithi194.stdout:1/250: write d0/d11/d18/d19/d1a/f43 [160333,125102] 0 2024-04-12T05:56:01.436 INFO:tasks.workunit.client.1.smithi194.stdout:1/251: dread - d0/d11/d18/d19/d1c/f46 zero size 2024-04-12T05:56:01.438 INFO:tasks.workunit.client.1.smithi194.stdout:1/252: symlink d0/d11/d18/d19/d1a/l47 0 2024-04-12T05:56:01.439 INFO:tasks.workunit.client.1.smithi194.stdout:1/253: dread - d0/d11/d18/d19/d1a/d23/f30 zero size 2024-04-12T05:56:01.439 INFO:tasks.workunit.client.1.smithi194.stdout:1/254: stat d0/d11/d18/d19/d1a 0 2024-04-12T05:56:01.446 INFO:tasks.workunit.client.1.smithi194.stdout:1/255: dwrite d0/d11/d18/d19/d1a/d23/f30 [0,4194304] 0 2024-04-12T05:56:01.446 INFO:tasks.workunit.client.1.smithi194.stdout:3/195: sync 2024-04-12T05:56:01.446 INFO:tasks.workunit.client.1.smithi194.stdout:7/67: sync 2024-04-12T05:56:01.448 INFO:tasks.workunit.client.1.smithi194.stdout:7/68: truncate f2 1007626 0 2024-04-12T05:56:01.448 INFO:tasks.workunit.client.1.smithi194.stdout:1/256: fdatasync d0/d11/d18/d19/d1a/f43 0 2024-04-12T05:56:01.449 INFO:tasks.workunit.client.1.smithi194.stdout:1/257: chown d0/df/f25 14956555 1 2024-04-12T05:56:01.450 INFO:tasks.workunit.client.1.smithi194.stdout:3/196: mknod d0/dc/de/c3c 0 2024-04-12T05:56:01.451 INFO:tasks.workunit.client.1.smithi194.stdout:1/258: readlink d0/d11/d18/d19/l2f 0 2024-04-12T05:56:01.454 INFO:tasks.workunit.client.1.smithi194.stdout:7/69: getdents d3 0 2024-04-12T05:56:01.454 INFO:tasks.workunit.client.1.smithi194.stdout:1/259: unlink d0/df/l1b 0 2024-04-12T05:56:01.455 INFO:tasks.workunit.client.1.smithi194.stdout:7/70: truncate f2 1359099 0 2024-04-12T05:56:01.456 INFO:tasks.workunit.client.1.smithi194.stdout:3/197: mkdir d0/dc/d3d 0 2024-04-12T05:56:01.456 INFO:tasks.workunit.client.1.smithi194.stdout:3/198: chown d0 263273541 1 2024-04-12T05:56:01.456 INFO:tasks.workunit.client.1.smithi194.stdout:7/71: truncate d3/d4/f7 1197364 0 2024-04-12T05:56:01.457 INFO:tasks.workunit.client.1.smithi194.stdout:1/260: mkdir d0/d11/d18/d19/d1a/d23/d27/d48 0 2024-04-12T05:56:01.458 INFO:tasks.workunit.client.1.smithi194.stdout:1/261: fsync d0/d11/d18/d35/f3a 0 2024-04-12T05:56:01.459 INFO:tasks.workunit.client.1.smithi194.stdout:7/72: creat d3/d16/f1a x:0 0 0 2024-04-12T05:56:01.459 INFO:tasks.workunit.client.1.smithi194.stdout:1/262: chown d0/df/f25 175 1 2024-04-12T05:56:01.461 INFO:tasks.workunit.client.1.smithi194.stdout:3/199: rename d0/dc/de/l2c to d0/dc/d3d/l3e 0 2024-04-12T05:56:01.462 INFO:tasks.workunit.client.1.smithi194.stdout:1/263: symlink d0/d11/d18/d19/d1c/l49 0 2024-04-12T05:56:01.465 INFO:tasks.workunit.client.1.smithi194.stdout:3/200: truncate d0/d1/fd 570407 0 2024-04-12T05:56:01.469 INFO:tasks.workunit.client.1.smithi194.stdout:1/264: dwrite d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [0,4194304] 0 2024-04-12T05:56:01.472 INFO:tasks.workunit.client.1.smithi194.stdout:3/201: link d0/d1/f1c d0/dc/de/f3f 0 2024-04-12T05:56:01.475 INFO:tasks.workunit.client.1.smithi194.stdout:3/202: creat d0/dc/d3d/f40 x:0 0 0 2024-04-12T05:56:01.476 INFO:tasks.workunit.client.1.smithi194.stdout:3/203: chown d0/dc/de/d31/d34/d2d/l30 92 1 2024-04-12T05:56:01.478 INFO:tasks.workunit.client.1.smithi194.stdout:3/204: truncate d0/d1/d10/f2b 883218 0 2024-04-12T05:56:01.484 INFO:tasks.workunit.client.1.smithi194.stdout:3/205: dwrite d0/dc/de/d39/f3b [0,4194304] 0 2024-04-12T05:56:01.485 INFO:tasks.workunit.client.1.smithi194.stdout:1/265: truncate d0/f4 3120079 0 2024-04-12T05:56:01.487 INFO:tasks.workunit.client.1.smithi194.stdout:1/266: read d0/d11/d18/d19/f36 [3732758,21083] 0 2024-04-12T05:56:01.490 INFO:tasks.workunit.client.1.smithi194.stdout:1/267: unlink d0/df/f25 0 2024-04-12T05:56:01.494 INFO:tasks.workunit.client.1.smithi194.stdout:3/206: dwrite d0/d1/d10/f2b [0,4194304] 0 2024-04-12T05:56:01.496 INFO:tasks.workunit.client.1.smithi194.stdout:1/268: symlink d0/d11/d18/d19/d1a/d23/d27/d48/l4a 0 2024-04-12T05:56:01.498 INFO:tasks.workunit.client.1.smithi194.stdout:3/207: fsync d0/f11 0 2024-04-12T05:56:01.502 INFO:tasks.workunit.client.1.smithi194.stdout:1/269: dread d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [0,4194304] 0 2024-04-12T05:56:01.504 INFO:tasks.workunit.client.1.smithi194.stdout:1/270: dread - d0/d11/d18/d35/f3a zero size 2024-04-12T05:56:01.505 INFO:tasks.workunit.client.1.smithi194.stdout:3/208: write d0/f11 [2325740,36115] 0 2024-04-12T05:56:01.507 INFO:tasks.workunit.client.1.smithi194.stdout:1/271: mknod d0/d3e/c4b 0 2024-04-12T05:56:01.508 INFO:tasks.workunit.client.1.smithi194.stdout:1/272: fsync d0/d11/d18/d19/d1a/d23/f30 0 2024-04-12T05:56:01.511 INFO:tasks.workunit.client.1.smithi194.stdout:1/273: write d0/d11/d18/d19/d1a/f43 [201314,66564] 0 2024-04-12T05:56:01.514 INFO:tasks.workunit.client.1.smithi194.stdout:3/209: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:01.516 INFO:tasks.workunit.client.1.smithi194.stdout:3/210: creat d0/dc/de/d31/d34/f41 x:0 0 0 2024-04-12T05:56:01.519 INFO:tasks.workunit.client.1.smithi194.stdout:1/274: fsync d0/d11/d18/f2d 0 2024-04-12T05:56:01.523 INFO:tasks.workunit.client.1.smithi194.stdout:3/211: dwrite d0/f27 [0,4194304] 0 2024-04-12T05:56:01.529 INFO:tasks.workunit.client.1.smithi194.stdout:3/212: mkdir d0/d42 0 2024-04-12T05:56:01.529 INFO:tasks.workunit.client.1.smithi194.stdout:3/213: readlink d0/l3 0 2024-04-12T05:56:01.529 INFO:tasks.workunit.client.1.smithi194.stdout:3/214: creat d0/dc/de/d39/f43 x:0 0 0 2024-04-12T05:56:01.535 INFO:tasks.workunit.client.1.smithi194.stdout:1/275: dwrite d0/d11/d18/d19/f36 [0,4194304] 0 2024-04-12T05:56:01.541 INFO:tasks.workunit.client.1.smithi194.stdout:3/215: dwrite d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:01.542 INFO:tasks.workunit.client.1.smithi194.stdout:1/276: dread d0/f12 [0,4194304] 0 2024-04-12T05:56:01.542 INFO:tasks.workunit.client.1.smithi194.stdout:3/216: readlink d0/dc/l15 0 2024-04-12T05:56:01.547 INFO:tasks.workunit.client.1.smithi194.stdout:1/277: fdatasync d0/f4 0 2024-04-12T05:56:01.551 INFO:tasks.workunit.client.1.smithi194.stdout:3/217: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:01.562 INFO:tasks.workunit.client.1.smithi194.stdout:1/278: truncate d0/f4 3197584 0 2024-04-12T05:56:01.570 INFO:tasks.workunit.client.1.smithi194.stdout:1/279: dwrite d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [4194304,4194304] 0 2024-04-12T05:56:01.570 INFO:tasks.workunit.client.1.smithi194.stdout:1/280: fdatasync d0/d11/d18/d35/f3a 0 2024-04-12T05:56:01.575 INFO:tasks.workunit.client.1.smithi194.stdout:3/218: dread d0/dc/de/f3f [0,4194304] 0 2024-04-12T05:56:01.576 INFO:tasks.workunit.client.1.smithi194.stdout:1/281: link d0/d11/c40 d0/d11/c4c 0 2024-04-12T05:56:01.579 INFO:tasks.workunit.client.1.smithi194.stdout:3/219: rename d0/dc/d3d/f40 to d0/d1/d10/f44 0 2024-04-12T05:56:01.580 INFO:tasks.workunit.client.1.smithi194.stdout:3/220: chown d0/f27 1232316 1 2024-04-12T05:56:01.580 INFO:tasks.workunit.client.1.smithi194.stdout:3/221: readlink d0/l7 0 2024-04-12T05:56:01.581 INFO:tasks.workunit.client.1.smithi194.stdout:1/282: link d0/df/l34 d0/d11/d18/d19/d1a/d23/d27/l4d 0 2024-04-12T05:56:01.583 INFO:tasks.workunit.client.1.smithi194.stdout:3/222: write d0/d1/d10/f44 [388322,46322] 0 2024-04-12T05:56:01.583 INFO:tasks.workunit.client.1.smithi194.stdout:1/283: read d0/d11/d18/d19/d1a/f43 [201349,24806] 0 2024-04-12T05:56:01.584 INFO:tasks.workunit.client.1.smithi194.stdout:1/284: stat d0/d11/d18/d19/l2f 0 2024-04-12T05:56:01.586 INFO:tasks.workunit.client.1.smithi194.stdout:3/223: creat d0/dc/de/f45 x:0 0 0 2024-04-12T05:56:01.588 INFO:tasks.workunit.client.1.smithi194.stdout:3/224: creat d0/dc/de/d31/d34/f46 x:0 0 0 2024-04-12T05:56:01.594 INFO:tasks.workunit.client.1.smithi194.stdout:3/225: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:01.595 INFO:tasks.workunit.client.1.smithi194.stdout:3/226: dread - d0/dc/de/d31/d34/f46 zero size 2024-04-12T05:56:01.596 INFO:tasks.workunit.client.1.smithi194.stdout:1/285: dread d0/f4 [0,4194304] 0 2024-04-12T05:56:01.598 INFO:tasks.workunit.client.1.smithi194.stdout:3/227: mknod d0/dc/d1e/c47 0 2024-04-12T05:56:01.601 INFO:tasks.workunit.client.1.smithi194.stdout:1/286: creat d0/d11/d18/d19/f4e x:0 0 0 2024-04-12T05:56:01.604 INFO:tasks.workunit.client.1.smithi194.stdout:1/287: unlink d0/d11/d18/d19/d1a/f43 0 2024-04-12T05:56:01.604 INFO:tasks.workunit.client.1.smithi194.stdout:3/228: dread d0/d1/d10/f44 [0,4194304] 0 2024-04-12T05:56:01.606 INFO:tasks.workunit.client.1.smithi194.stdout:3/229: creat d0/dc/de/d31/f48 x:0 0 0 2024-04-12T05:56:01.608 INFO:tasks.workunit.client.1.smithi194.stdout:1/288: rename d0/d11/d18/c41 to d0/d11/d18/d19/d1a/d23/d27/c4f 0 2024-04-12T05:56:01.608 INFO:tasks.workunit.client.1.smithi194.stdout:3/230: symlink d0/d1/d10/l49 0 2024-04-12T05:56:01.609 INFO:tasks.workunit.client.1.smithi194.stdout:3/231: stat d0/dc/de/c3c 0 2024-04-12T05:56:01.610 INFO:tasks.workunit.client.1.smithi194.stdout:3/232: write d0/d1/f9 [4900435,53352] 0 2024-04-12T05:56:01.612 INFO:tasks.workunit.client.1.smithi194.stdout:1/289: dread d0/d11/d18/d19/d1c/f24 [0,4194304] 0 2024-04-12T05:56:01.614 INFO:tasks.workunit.client.1.smithi194.stdout:3/233: rename d0/dc/de/d39/f43 to d0/dc/d1e/d33/f4a 0 2024-04-12T05:56:01.615 INFO:tasks.workunit.client.1.smithi194.stdout:1/290: mkdir d0/d3e/d50 0 2024-04-12T05:56:01.621 INFO:tasks.workunit.client.1.smithi194.stdout:1/291: dwrite d0/f32 [0,4194304] 0 2024-04-12T05:56:01.623 INFO:tasks.workunit.client.1.smithi194.stdout:1/292: fsync d0/d11/f26 0 2024-04-12T05:56:01.624 INFO:tasks.workunit.client.1.smithi194.stdout:1/293: truncate d0/d11/d18/d19/d1c/f46 964032 0 2024-04-12T05:56:01.627 INFO:tasks.workunit.client.1.smithi194.stdout:1/294: symlink d0/d11/d18/d19/d1a/l51 0 2024-04-12T05:56:01.629 INFO:tasks.workunit.client.1.smithi194.stdout:1/295: mknod d0/d11/d18/d19/c52 0 2024-04-12T05:56:01.630 INFO:tasks.workunit.client.1.smithi194.stdout:1/296: chown d0/d11/c14 216903131 1 2024-04-12T05:56:01.634 INFO:tasks.workunit.client.1.smithi194.stdout:1/297: unlink d0/d11/d18/d19/d1c/f2a 0 2024-04-12T05:56:01.722 INFO:tasks.workunit.client.1.smithi194.stdout:6/261: sync 2024-04-12T05:56:01.722 INFO:tasks.workunit.client.1.smithi194.stdout:8/123: sync 2024-04-12T05:56:01.723 INFO:tasks.workunit.client.1.smithi194.stdout:8/124: chown d2/d8/d9/d1a/f25 2098418516 1 2024-04-12T05:56:01.724 INFO:tasks.workunit.client.1.smithi194.stdout:6/262: creat d3/d2d/d33/f4d x:0 0 0 2024-04-12T05:56:01.724 INFO:tasks.workunit.client.1.smithi194.stdout:8/125: creat d2/d8/d15/d17/f29 x:0 0 0 2024-04-12T05:56:01.725 INFO:tasks.workunit.client.1.smithi194.stdout:8/126: read d2/d8/d15/f16 [818753,107270] 0 2024-04-12T05:56:01.725 INFO:tasks.workunit.client.1.smithi194.stdout:6/263: read d3/d15/f28 [3370365,47497] 0 2024-04-12T05:56:01.727 INFO:tasks.workunit.client.1.smithi194.stdout:8/127: truncate d2/d8/d9/de/d10/f24 1179376 0 2024-04-12T05:56:01.728 INFO:tasks.workunit.client.1.smithi194.stdout:6/264: mkdir d3/d2f/d4e 0 2024-04-12T05:56:01.728 INFO:tasks.workunit.client.1.smithi194.stdout:8/128: chown d2/d8 228232 1 2024-04-12T05:56:01.746 INFO:tasks.workunit.client.1.smithi194.stdout:6/265: creat d3/d15/d2b/d48/f4f x:0 0 0 2024-04-12T05:56:01.747 INFO:tasks.workunit.client.1.smithi194.stdout:6/266: stat d3/f20 0 2024-04-12T05:56:01.748 INFO:tasks.workunit.client.1.smithi194.stdout:8/129: rmdir d2 39 2024-04-12T05:56:01.750 INFO:tasks.workunit.client.1.smithi194.stdout:6/267: rename d3/c1b to d3/d15/c50 0 2024-04-12T05:56:01.751 INFO:tasks.workunit.client.1.smithi194.stdout:6/268: fdatasync d3/d2d/f3d 0 2024-04-12T05:56:01.752 INFO:tasks.workunit.client.1.smithi194.stdout:6/269: dread - d3/d15/f21 zero size 2024-04-12T05:56:01.753 INFO:tasks.workunit.client.1.smithi194.stdout:8/130: read d2/d8/d15/d17/f18 [622832,72577] 0 2024-04-12T05:56:01.754 INFO:tasks.workunit.client.1.smithi194.stdout:6/270: mknod d3/d2f/c51 0 2024-04-12T05:56:01.758 INFO:tasks.workunit.client.1.smithi194.stdout:6/271: rename d3/d15/d2b/d48/d4c to d3/d2d/d33/d52 0 2024-04-12T05:56:01.760 INFO:tasks.workunit.client.1.smithi194.stdout:8/131: dwrite d2/d8/d15/d17/f18 [0,4194304] 0 2024-04-12T05:56:01.764 INFO:tasks.workunit.client.1.smithi194.stdout:8/132: creat d2/d8/d9/f2a x:0 0 0 2024-04-12T05:56:01.765 INFO:tasks.workunit.client.1.smithi194.stdout:8/133: truncate d2/d8/f26 85060 0 2024-04-12T05:56:01.767 INFO:tasks.workunit.client.1.smithi194.stdout:6/272: getdents d3/d2d/d33 0 2024-04-12T05:56:01.770 INFO:tasks.workunit.client.1.smithi194.stdout:8/134: link d2/d8/d15/d17/f1b d2/d8/d9/d1a/f2b 0 2024-04-12T05:56:01.772 INFO:tasks.workunit.client.1.smithi194.stdout:8/135: symlink d2/l2c 0 2024-04-12T05:56:01.772 INFO:tasks.workunit.client.1.smithi194.stdout:8/136: chown d2 813 1 2024-04-12T05:56:01.773 INFO:tasks.workunit.client.1.smithi194.stdout:8/137: chown d2/d8/d9/f2a 508912318 1 2024-04-12T05:56:01.774 INFO:tasks.workunit.client.1.smithi194.stdout:8/138: dread - d2/d8/d9/de/f1d zero size 2024-04-12T05:56:01.775 INFO:tasks.workunit.client.1.smithi194.stdout:8/139: truncate d2/d8/d9/f2a 316124 0 2024-04-12T05:56:01.777 INFO:tasks.workunit.client.1.smithi194.stdout:8/140: creat d2/f2d x:0 0 0 2024-04-12T05:56:01.779 INFO:tasks.workunit.client.1.smithi194.stdout:8/141: write d2/d8/d15/d17/f18 [613181,72854] 0 2024-04-12T05:56:01.780 INFO:tasks.workunit.client.1.smithi194.stdout:8/142: truncate d2/d8/d15/d17/f29 259787 0 2024-04-12T05:56:01.783 INFO:tasks.workunit.client.1.smithi194.stdout:6/273: dread d3/d2d/f32 [0,4194304] 0 2024-04-12T05:56:01.783 INFO:tasks.workunit.client.1.smithi194.stdout:8/143: creat d2/d8/d9/de/f2e x:0 0 0 2024-04-12T05:56:01.784 INFO:tasks.workunit.client.1.smithi194.stdout:8/144: readlink d2/d8/d9/de/d10/l13 0 2024-04-12T05:56:01.786 INFO:tasks.workunit.client.1.smithi194.stdout:8/145: mknod d2/d8/d15/c2f 0 2024-04-12T05:56:01.787 INFO:tasks.workunit.client.1.smithi194.stdout:6/274: getdents d3/d15/d2b 0 2024-04-12T05:56:01.788 INFO:tasks.workunit.client.1.smithi194.stdout:6/275: chown d3/d15/c11 10660 1 2024-04-12T05:56:01.790 INFO:tasks.workunit.client.1.smithi194.stdout:6/276: mkdir d3/d2f/d4e/d53 0 2024-04-12T05:56:01.793 INFO:tasks.workunit.client.1.smithi194.stdout:6/277: creat d3/d15/d2b/d48/f54 x:0 0 0 2024-04-12T05:56:01.793 INFO:tasks.workunit.client.1.smithi194.stdout:8/146: dwrite d2/f12 [0,4194304] 0 2024-04-12T05:56:01.797 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-12T05:56:01.797 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-12T05:56:01.798 INFO:tasks.workunit.client.1.smithi194.stdout:8/147: creat d2/d8/d9/d1a/f30 x:0 0 0 2024-04-12T05:56:01.801 INFO:tasks.workunit.client.1.smithi194.stdout:8/148: truncate d2/d8/d9/de/d10/f24 1317768 0 2024-04-12T05:56:01.801 INFO:tasks.workunit.client.1.smithi194.stdout:6/278: dwrite d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:01.804 INFO:tasks.workunit.client.1.smithi194.stdout:8/149: creat d2/d8/d15/d17/f31 x:0 0 0 2024-04-12T05:56:01.806 INFO:tasks.workunit.client.1.smithi194.stdout:8/150: mkdir d2/d8/d15/d17/d32 0 2024-04-12T05:56:01.809 INFO:tasks.workunit.client.1.smithi194.stdout:6/279: rmdir d3/d15/d2b/d48 39 2024-04-12T05:56:01.811 INFO:tasks.workunit.client.1.smithi194.stdout:6/280: read d3/d2f/d37/f45 [1187772,27816] 0 2024-04-12T05:56:01.812 INFO:tasks.workunit.client.1.smithi194.stdout:6/281: truncate d3/d2f/f39 632293 0 2024-04-12T05:56:01.812 INFO:tasks.workunit.client.1.smithi194.stdout:6/282: dread - d3/f20 zero size 2024-04-12T05:56:01.813 INFO:tasks.workunit.client.1.smithi194.stdout:6/283: chown d3/f18 15756203 1 2024-04-12T05:56:01.815 INFO:tasks.workunit.client.1.smithi194.stdout:6/284: read d3/d15/f4b [1406858,83014] 0 2024-04-12T05:56:01.817 INFO:tasks.workunit.client.1.smithi194.stdout:8/151: rename d2/d8/d15/d17 to d2/d8/d15/d33 0 2024-04-12T05:56:01.818 INFO:tasks.workunit.client.1.smithi194.stdout:6/285: write d3/d15/d2b/d48/f49 [2958452,111120] 0 2024-04-12T05:56:01.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:01 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/362540189' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:56:01.820 INFO:tasks.workunit.client.1.smithi194.stdout:6/286: stat d3/d2f/f39 0 2024-04-12T05:56:01.822 INFO:tasks.workunit.client.1.smithi194.stdout:8/152: rename d2/d8/d9/de/d10/c21 to d2/d8/d9/c34 0 2024-04-12T05:56:01.826 INFO:tasks.workunit.client.1.smithi194.stdout:8/153: mkdir d2/d8/d15/d35 0 2024-04-12T05:56:01.828 INFO:tasks.workunit.client.1.smithi194.stdout:6/287: rename d3/fe to d3/d2d/d33/d52/f55 0 2024-04-12T05:56:01.828 INFO:tasks.workunit.client.1.smithi194.stdout:8/154: read - d2/d8/d15/d33/f31 zero size 2024-04-12T05:56:01.835 INFO:tasks.workunit.client.1.smithi194.stdout:8/155: rename d2/d8/d9/f28 to d2/d8/f36 0 2024-04-12T05:56:01.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/156: write d2/d8/d9/de/f1d [274554,24966] 0 2024-04-12T05:56:01.839 INFO:tasks.workunit.client.1.smithi194.stdout:8/157: read - d2/d8/d9/d1a/f25 zero size 2024-04-12T05:56:01.850 INFO:tasks.workunit.client.1.smithi194.stdout:6/288: dread d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:01.852 INFO:tasks.workunit.client.1.smithi194.stdout:6/289: mkdir d3/d2f/d4e/d56 0 2024-04-12T05:56:01.855 INFO:tasks.workunit.client.1.smithi194.stdout:6/290: getdents d3/d2d 0 2024-04-12T05:56:01.857 INFO:tasks.workunit.client.1.smithi194.stdout:6/291: symlink d3/d15/d2b/l57 0 2024-04-12T05:56:01.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:01 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/362540189' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:56:01.914 INFO:tasks.workunit.client.1.smithi194.stdout:0/166: sync 2024-04-12T05:56:01.914 INFO:tasks.workunit.client.1.smithi194.stdout:2/205: sync 2024-04-12T05:56:01.914 INFO:tasks.workunit.client.1.smithi194.stdout:9/220: sync 2024-04-12T05:56:01.917 INFO:tasks.workunit.client.1.smithi194.stdout:2/206: write d7/d12/d25/f31 [1155778,106932] 0 2024-04-12T05:56:01.918 INFO:tasks.workunit.client.1.smithi194.stdout:9/221: write d1/d19/d1f/f2c [1524958,44707] 0 2024-04-12T05:56:01.919 INFO:tasks.workunit.client.1.smithi194.stdout:2/207: chown d7/d12/d1a/f3e 0 1 2024-04-12T05:56:01.919 INFO:tasks.workunit.client.1.smithi194.stdout:0/167: mknod d0/da/d1e/d1a/c38 0 2024-04-12T05:56:01.922 INFO:tasks.workunit.client.1.smithi194.stdout:9/222: mknod d1/d19/d1f/d20/d37/d38/c4a 0 2024-04-12T05:56:01.924 INFO:tasks.workunit.client.1.smithi194.stdout:9/223: truncate d1/d19/d1f/d20/f31 1778144 0 2024-04-12T05:56:01.925 INFO:tasks.workunit.client.1.smithi194.stdout:0/168: creat d0/da/f39 x:0 0 0 2024-04-12T05:56:01.926 INFO:tasks.workunit.client.1.smithi194.stdout:2/208: creat d7/d22/f4d x:0 0 0 2024-04-12T05:56:01.934 INFO:tasks.workunit.client.1.smithi194.stdout:9/224: unlink d1/d19/d1f/d20/f25 0 2024-04-12T05:56:01.934 INFO:tasks.workunit.client.1.smithi194.stdout:0/169: dread d0/f10 [0,4194304] 0 2024-04-12T05:56:01.934 INFO:tasks.workunit.client.1.smithi194.stdout:9/225: stat d1/d41/f43 0 2024-04-12T05:56:01.934 INFO:tasks.workunit.client.1.smithi194.stdout:2/209: symlink d7/l4e 0 2024-04-12T05:56:01.934 INFO:tasks.workunit.client.1.smithi194.stdout:0/170: mknod d0/da/d1e/d1a/d24/c3a 0 2024-04-12T05:56:01.935 INFO:tasks.workunit.client.1.smithi194.stdout:0/171: dread - d0/da/d1e/d27/f2a zero size 2024-04-12T05:56:01.937 INFO:tasks.workunit.client.1.smithi194.stdout:2/210: creat d7/d12/d25/f4f x:0 0 0 2024-04-12T05:56:01.938 INFO:tasks.workunit.client.1.smithi194.stdout:9/226: getdents d1/d19 0 2024-04-12T05:56:01.940 INFO:tasks.workunit.client.1.smithi194.stdout:0/172: dread d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:01.942 INFO:tasks.workunit.client.1.smithi194.stdout:2/211: dread d7/d12/f3f [0,4194304] 0 2024-04-12T05:56:01.947 INFO:tasks.workunit.client.1.smithi194.stdout:0/173: dwrite d0/da/f2d [4194304,4194304] 0 2024-04-12T05:56:01.947 INFO:tasks.workunit.client.1.smithi194.stdout:2/212: rename d7/d12/d25/c35 to d7/d12/d1a/c50 0 2024-04-12T05:56:01.948 INFO:tasks.workunit.client.1.smithi194.stdout:2/213: write d7/d12/f3b [430798,71706] 0 2024-04-12T05:56:01.953 INFO:tasks.workunit.client.1.smithi194.stdout:0/174: mkdir d0/d3b 0 2024-04-12T05:56:01.954 INFO:tasks.workunit.client.1.smithi194.stdout:0/175: write d0/da/f39 [396521,129086] 0 2024-04-12T05:56:01.960 INFO:tasks.workunit.client.1.smithi194.stdout:2/214: write d7/f48 [271516,82188] 0 2024-04-12T05:56:01.961 INFO:tasks.workunit.client.1.smithi194.stdout:2/215: write d7/d12/d1a/f3e [2890548,112284] 0 2024-04-12T05:56:01.963 INFO:tasks.workunit.client.1.smithi194.stdout:2/216: dread d7/d12/f38 [0,4194304] 0 2024-04-12T05:56:01.964 INFO:tasks.workunit.client.1.smithi194.stdout:2/217: readlink d7/l30 0 2024-04-12T05:56:01.965 INFO:tasks.workunit.client.1.smithi194.stdout:2/218: truncate d7/d12/d25/f4f 107418 0 2024-04-12T05:56:01.967 INFO:tasks.workunit.client.1.smithi194.stdout:2/219: creat d7/d22/d2b/f51 x:0 0 0 2024-04-12T05:56:01.968 INFO:tasks.workunit.client.1.smithi194.stdout:2/220: truncate d7/d12/d1a/f44 337113 0 2024-04-12T05:56:01.969 INFO:tasks.workunit.client.1.smithi194.stdout:2/221: rmdir d7/d22/d42 39 2024-04-12T05:56:01.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/222: dwrite d7/d12/d1a/f43 [0,4194304] 0 2024-04-12T05:56:01.979 INFO:tasks.workunit.client.1.smithi194.stdout:2/223: mkdir d7/d22/d2b/d47/d52 0 2024-04-12T05:56:01.982 INFO:tasks.workunit.client.1.smithi194.stdout:2/224: unlink d7/d12/d1a/f43 0 2024-04-12T05:56:01.987 INFO:tasks.workunit.client.1.smithi194.stdout:2/225: rename d7/d22/d42/c28 to d7/d12/c53 0 2024-04-12T05:56:01.988 INFO:tasks.workunit.client.1.smithi194.stdout:2/226: write d7/d22/d42/f1e [1496164,51074] 0 2024-04-12T05:56:01.991 INFO:tasks.workunit.client.1.smithi194.stdout:2/227: creat d7/d22/d42/f54 x:0 0 0 2024-04-12T05:56:01.992 INFO:tasks.workunit.client.1.smithi194.stdout:2/228: chown d7/d12/l13 52173651 1 2024-04-12T05:56:01.998 INFO:tasks.workunit.client.1.smithi194.stdout:2/229: truncate d7/d22/d42/f2a 3152098 0 2024-04-12T05:56:02.013 INFO:tasks.workunit.client.1.smithi194.stdout:2/230: dread d7/d12/f33 [0,4194304] 0 2024-04-12T05:56:02.014 INFO:tasks.workunit.client.1.smithi194.stdout:2/231: write d7/d22/d42/f1e [4503472,75198] 0 2024-04-12T05:56:02.021 INFO:tasks.workunit.client.1.smithi194.stdout:2/232: dwrite d7/d12/f33 [4194304,4194304] 0 2024-04-12T05:56:02.024 INFO:tasks.workunit.client.1.smithi194.stdout:2/233: chown d7/d22/d42/f1d 7 1 2024-04-12T05:56:02.024 INFO:tasks.workunit.client.1.smithi194.stdout:2/234: stat d7/d12/f33 0 2024-04-12T05:56:02.026 INFO:tasks.workunit.client.1.smithi194.stdout:2/235: fdatasync d7/d12/f3f 0 2024-04-12T05:56:02.029 INFO:tasks.workunit.client.1.smithi194.stdout:2/236: rename d7/d12/d34/f3a to d7/d22/d2b/d47/d52/f55 0 2024-04-12T05:56:02.030 INFO:tasks.workunit.client.1.smithi194.stdout:2/237: mknod d7/d12/d34/c56 0 2024-04-12T05:56:02.031 INFO:tasks.workunit.client.1.smithi194.stdout:2/238: write d7/d12/f33 [5119192,75625] 0 2024-04-12T05:56:02.033 INFO:tasks.workunit.client.1.smithi194.stdout:2/239: creat d7/d22/d2b/d47/f57 x:0 0 0 2024-04-12T05:56:02.035 INFO:tasks.workunit.client.1.smithi194.stdout:2/240: symlink d7/d12/d34/d4a/l58 0 2024-04-12T05:56:02.037 INFO:tasks.workunit.client.1.smithi194.stdout:2/241: symlink d7/d12/d34/d4a/l59 0 2024-04-12T05:56:02.038 INFO:tasks.workunit.client.1.smithi194.stdout:2/242: symlink d7/d22/l5a 0 2024-04-12T05:56:02.042 INFO:tasks.workunit.client.1.smithi194.stdout:2/243: link d7/d22/d42/f2a d7/f5b 0 2024-04-12T05:56:02.117 INFO:tasks.workunit.client.1.smithi194.stdout:5/268: sync 2024-04-12T05:56:02.118 INFO:tasks.workunit.client.1.smithi194.stdout:5/269: truncate d1/d4/d6/f13 714782 0 2024-04-12T05:56:02.120 INFO:tasks.workunit.client.1.smithi194.stdout:5/270: creat d1/d4/df/d23/f56 x:0 0 0 2024-04-12T05:56:02.127 INFO:tasks.workunit.client.1.smithi194.stdout:5/271: truncate d1/fa 4507134 0 2024-04-12T05:56:02.128 INFO:tasks.workunit.client.1.smithi194.stdout:5/272: write d1/d4/d6/f13 [651526,34322] 0 2024-04-12T05:56:02.131 INFO:tasks.workunit.client.1.smithi194.stdout:5/273: symlink d1/d4/d6/l57 0 2024-04-12T05:56:02.133 INFO:tasks.workunit.client.1.smithi194.stdout:5/274: fdatasync d1/d4/d6/db/d15/d26/f50 0 2024-04-12T05:56:02.136 INFO:tasks.workunit.client.1.smithi194.stdout:5/275: rename d1/d4/df/f1f to d1/d4/d6/db/d15/f58 0 2024-04-12T05:56:02.137 INFO:tasks.workunit.client.1.smithi194.stdout:5/276: read d1/d4/d6/f2d [2017721,50240] 0 2024-04-12T05:56:02.138 INFO:tasks.workunit.client.1.smithi194.stdout:5/277: write d1/d4/df/d23/f56 [130752,79976] 0 2024-04-12T05:56:02.143 INFO:tasks.workunit.client.1.smithi194.stdout:5/278: unlink d1/d4/d6/db/d15/d26/f28 0 2024-04-12T05:56:02.144 INFO:tasks.workunit.client.1.smithi194.stdout:5/279: mknod d1/d4/df/d23/c59 0 2024-04-12T05:56:02.146 INFO:tasks.workunit.client.1.smithi194.stdout:5/280: truncate d1/d4/d6/d34/f54 606190 0 2024-04-12T05:56:02.148 INFO:tasks.workunit.client.1.smithi194.stdout:5/281: creat d1/d4/d6/db/d38/f5a x:0 0 0 2024-04-12T05:56:02.150 INFO:tasks.workunit.client.1.smithi194.stdout:5/282: mkdir d1/d4/d6/db/d5b 0 2024-04-12T05:56:02.155 INFO:tasks.workunit.client.1.smithi194.stdout:5/283: dread d1/d4/df/f2e [0,4194304] 0 2024-04-12T05:56:02.162 INFO:tasks.workunit.client.1.smithi194.stdout:5/284: dwrite d1/d4/d6/db/d2b/f44 [0,4194304] 0 2024-04-12T05:56:02.165 INFO:tasks.workunit.client.1.smithi194.stdout:5/285: mkdir d1/d5c 0 2024-04-12T05:56:02.168 INFO:tasks.workunit.client.1.smithi194.stdout:5/286: truncate d1/d4/df/d23/f3c 9521 0 2024-04-12T05:56:02.169 INFO:tasks.workunit.client.1.smithi194.stdout:5/287: mknod d1/d4/d6/db/d38/c5d 0 2024-04-12T05:56:02.171 INFO:tasks.workunit.client.1.smithi194.stdout:5/288: mknod d1/d4/d6/db/d2b/c5e 0 2024-04-12T05:56:02.172 INFO:tasks.workunit.client.1.smithi194.stdout:5/289: write d1/d4/f1a [1821041,125330] 0 2024-04-12T05:56:02.174 INFO:tasks.workunit.client.1.smithi194.stdout:5/290: read d1/d4/df/d23/f56 [115477,2239] 0 2024-04-12T05:56:02.180 INFO:tasks.workunit.client.1.smithi194.stdout:5/291: dwrite d1/d4/d6/db/d15/d26/f50 [0,4194304] 0 2024-04-12T05:56:02.184 INFO:tasks.workunit.client.1.smithi194.stdout:5/292: write d1/d4/df/f2a [8268016,53114] 0 2024-04-12T05:56:02.188 INFO:tasks.workunit.client.1.smithi194.stdout:5/293: creat d1/d4/d6/d34/d52/f5f x:0 0 0 2024-04-12T05:56:02.188 INFO:tasks.workunit.client.1.smithi194.stdout:5/294: read - d1/d4/d6/db/f53 zero size 2024-04-12T05:56:02.190 INFO:tasks.workunit.client.1.smithi194.stdout:5/295: getdents d1/d4/d4b 0 2024-04-12T05:56:02.195 INFO:tasks.workunit.client.1.smithi194.stdout:5/296: truncate d1/d4/d6/d34/f54 272518 0 2024-04-12T05:56:02.196 INFO:tasks.workunit.client.1.smithi194.stdout:5/297: truncate d1/d4/d6/f35 5071976 0 2024-04-12T05:56:02.198 INFO:tasks.workunit.client.1.smithi194.stdout:5/298: read d1/d4/d6/db/d15/f58 [2266798,84130] 0 2024-04-12T05:56:02.201 INFO:tasks.workunit.client.1.smithi194.stdout:5/299: rename d1/d4/d6/db/d38/f41 to d1/d4/d6/db/d38/f60 0 2024-04-12T05:56:02.202 INFO:tasks.workunit.client.1.smithi194.stdout:5/300: write d1/d4/d6/db/d38/f5a [105669,53734] 0 2024-04-12T05:56:02.204 INFO:tasks.workunit.client.1.smithi194.stdout:5/301: creat d1/d4/d6/db/d38/f61 x:0 0 0 2024-04-12T05:56:02.206 INFO:tasks.workunit.client.1.smithi194.stdout:5/302: dread d1/d4/df/d23/f3c [0,4194304] 0 2024-04-12T05:56:02.207 INFO:tasks.workunit.client.1.smithi194.stdout:5/303: mknod d1/d4/d6/db/d3b/c62 0 2024-04-12T05:56:02.208 INFO:tasks.workunit.client.1.smithi194.stdout:5/304: dread - d1/d4/d6/db/d15/d26/f4e zero size 2024-04-12T05:56:02.208 INFO:tasks.workunit.client.1.smithi194.stdout:5/305: chown d1/d4/f5 16165 1 2024-04-12T05:56:02.210 INFO:tasks.workunit.client.1.smithi194.stdout:5/306: unlink d1/d4/df/c49 0 2024-04-12T05:56:02.213 INFO:tasks.workunit.client.1.smithi194.stdout:5/307: rename d1/d4/df/d23/l51 to d1/d4/d6/l63 0 2024-04-12T05:56:02.214 INFO:tasks.workunit.client.1.smithi194.stdout:5/308: write d1/f3 [5823608,53132] 0 2024-04-12T05:56:02.216 INFO:tasks.workunit.client.1.smithi194.stdout:5/309: mknod d1/d4/d6/db/d15/d26/c64 0 2024-04-12T05:56:02.221 INFO:tasks.workunit.client.1.smithi194.stdout:5/310: rmdir d1/d4/d6/d34 39 2024-04-12T05:56:02.222 INFO:tasks.workunit.client.1.smithi194.stdout:5/311: fsync d1/d4/df/d23/f2c 0 2024-04-12T05:56:02.230 INFO:tasks.workunit.client.1.smithi194.stdout:5/312: dwrite d1/d4/d6/db/f4a [0,4194304] 0 2024-04-12T05:56:02.231 INFO:tasks.workunit.client.1.smithi194.stdout:4/174: sync 2024-04-12T05:56:02.231 INFO:tasks.workunit.client.1.smithi194.stdout:4/175: dread - d2/d3/f27 zero size 2024-04-12T05:56:02.235 INFO:tasks.workunit.client.1.smithi194.stdout:4/176: symlink d2/d22/l37 0 2024-04-12T05:56:02.237 INFO:tasks.workunit.client.1.smithi194.stdout:5/313: link d1/d4/d6/f3d d1/d4/d6/d34/d52/f65 0 2024-04-12T05:56:02.239 INFO:tasks.workunit.client.1.smithi194.stdout:5/314: truncate d1/d4/df/d23/f3c 118242 0 2024-04-12T05:56:02.242 INFO:tasks.workunit.client.1.smithi194.stdout:5/315: rename d1/d4/df/f2a to d1/d4/d6/db/d2b/f66 0 2024-04-12T05:56:02.244 INFO:tasks.workunit.client.1.smithi194.stdout:5/316: mkdir d1/d4/d4b/d67 0 2024-04-12T05:56:02.247 INFO:tasks.workunit.client.1.smithi194.stdout:5/317: creat d1/d4/d6/f68 x:0 0 0 2024-04-12T05:56:02.249 INFO:tasks.workunit.client.1.smithi194.stdout:4/177: write f0 [1357296,82066] 0 2024-04-12T05:56:02.254 INFO:tasks.workunit.client.1.smithi194.stdout:4/178: link d2/f21 d2/d3/d1b/d32/f38 0 2024-04-12T05:56:02.254 INFO:tasks.workunit.client.1.smithi194.stdout:5/318: dwrite d1/d4/d6/db/f19 [4194304,4194304] 0 2024-04-12T05:56:02.255 INFO:tasks.workunit.client.1.smithi194.stdout:5/319: chown d1/fa 273146505 1 2024-04-12T05:56:02.256 INFO:tasks.workunit.client.1.smithi194.stdout:5/320: chown d1/l7 31052 1 2024-04-12T05:56:02.256 INFO:tasks.workunit.client.1.smithi194.stdout:4/179: creat d2/d11/f39 x:0 0 0 2024-04-12T05:56:02.258 INFO:tasks.workunit.client.1.smithi194.stdout:4/180: write d2/d3/f18 [7799975,128069] 0 2024-04-12T05:56:02.262 INFO:tasks.workunit.client.1.smithi194.stdout:5/321: link d1/d4/f25 d1/d4/df/f69 0 2024-04-12T05:56:02.263 INFO:tasks.workunit.client.1.smithi194.stdout:5/322: readlink d1/d4/d6/l57 0 2024-04-12T05:56:02.264 INFO:tasks.workunit.client.1.smithi194.stdout:4/181: mkdir d2/d3/d1b/d32/d3a 0 2024-04-12T05:56:02.267 INFO:tasks.workunit.client.1.smithi194.stdout:5/323: link d1/d4/df/d23/f2c d1/d4/df/f6a 0 2024-04-12T05:56:02.268 INFO:tasks.workunit.client.1.smithi194.stdout:5/324: chown d1/d4/df/d23/f56 1246688 1 2024-04-12T05:56:02.270 INFO:tasks.workunit.client.1.smithi194.stdout:5/325: creat d1/d4/d4b/d67/f6b x:0 0 0 2024-04-12T05:56:02.271 INFO:tasks.workunit.client.1.smithi194.stdout:9/227: sync 2024-04-12T05:56:02.271 INFO:tasks.workunit.client.1.smithi194.stdout:3/234: sync 2024-04-12T05:56:02.271 INFO:tasks.workunit.client.1.smithi194.stdout:7/73: sync 2024-04-12T05:56:02.271 INFO:tasks.workunit.client.1.smithi194.stdout:0/176: sync 2024-04-12T05:56:02.271 INFO:tasks.workunit.client.1.smithi194.stdout:8/158: sync 2024-04-12T05:56:02.272 INFO:tasks.workunit.client.1.smithi194.stdout:2/244: sync 2024-04-12T05:56:02.272 INFO:tasks.workunit.client.1.smithi194.stdout:6/292: sync 2024-04-12T05:56:02.272 INFO:tasks.workunit.client.1.smithi194.stdout:1/298: sync 2024-04-12T05:56:02.272 INFO:tasks.workunit.client.1.smithi194.stdout:5/326: write d1/d4/d6/db/d2b/f44 [3879604,18412] 0 2024-04-12T05:56:02.272 INFO:tasks.workunit.client.1.smithi194.stdout:7/74: chown d3 5279922 1 2024-04-12T05:56:02.274 INFO:tasks.workunit.client.1.smithi194.stdout:0/177: fdatasync d0/f14 0 2024-04-12T05:56:02.275 INFO:tasks.workunit.client.1.smithi194.stdout:8/159: write d2/fb [2857905,40602] 0 2024-04-12T05:56:02.277 INFO:tasks.workunit.client.1.smithi194.stdout:8/160: readlink d2/l2c 0 2024-04-12T05:56:02.277 INFO:tasks.workunit.client.1.smithi194.stdout:8/161: chown d2 1278 1 2024-04-12T05:56:02.278 INFO:tasks.workunit.client.1.smithi194.stdout:7/75: write d3/d16/f1a [360706,73292] 0 2024-04-12T05:56:02.279 INFO:tasks.workunit.client.1.smithi194.stdout:6/293: mkdir d3/d2d/d33/d58 0 2024-04-12T05:56:02.280 INFO:tasks.workunit.client.1.smithi194.stdout:0/178: write d0/f14 [121056,45839] 0 2024-04-12T05:56:02.280 INFO:tasks.workunit.client.1.smithi194.stdout:1/299: truncate d0/d11/d18/f2d 4489832 0 2024-04-12T05:56:02.281 INFO:tasks.workunit.client.1.smithi194.stdout:5/327: mknod d1/d4/d6/db/d38/c6c 0 2024-04-12T05:56:02.282 INFO:tasks.workunit.client.1.smithi194.stdout:0/179: dread - d0/da/d1e/f32 zero size 2024-04-12T05:56:02.284 INFO:tasks.workunit.client.1.smithi194.stdout:9/228: rename d1/d19/d1f/d20/f31 to d1/d19/d1f/f4b 0 2024-04-12T05:56:02.284 INFO:tasks.workunit.client.1.smithi194.stdout:1/300: write d0/d11/d18/d19/f36 [54598,20735] 0 2024-04-12T05:56:02.286 INFO:tasks.workunit.client.1.smithi194.stdout:2/245: creat d7/d22/f5c x:0 0 0 2024-04-12T05:56:02.287 INFO:tasks.workunit.client.1.smithi194.stdout:9/229: dread - d1/f46 zero size 2024-04-12T05:56:02.288 INFO:tasks.workunit.client.1.smithi194.stdout:6/294: unlink d3/f20 0 2024-04-12T05:56:02.289 INFO:tasks.workunit.client.1.smithi194.stdout:7/76: rmdir d3/d4/d12 39 2024-04-12T05:56:02.291 INFO:tasks.workunit.client.1.smithi194.stdout:9/230: fdatasync d1/f46 0 2024-04-12T05:56:02.291 INFO:tasks.workunit.client.1.smithi194.stdout:4/182: write d2/d22/f2a [573511,83752] 0 2024-04-12T05:56:02.295 INFO:tasks.workunit.client.1.smithi194.stdout:8/162: dwrite d2/d8/d15/d33/f27 [0,4194304] 0 2024-04-12T05:56:02.298 INFO:tasks.workunit.client.1.smithi194.stdout:0/180: dwrite d0/f37 [0,4194304] 0 2024-04-12T05:56:02.298 INFO:tasks.workunit.client.1.smithi194.stdout:8/163: fdatasync d2/d8/f26 0 2024-04-12T05:56:02.301 INFO:tasks.workunit.client.1.smithi194.stdout:5/328: rename d1/d4/d4b/d67 to d1/d4/d6/d6d 0 2024-04-12T05:56:02.303 INFO:tasks.workunit.client.1.smithi194.stdout:0/181: write d0/f37 [1304469,90580] 0 2024-04-12T05:56:02.313 INFO:tasks.workunit.client.1.smithi194.stdout:2/246: rename d7/d22/d2b to d7/d12/d25/d5d 0 2024-04-12T05:56:02.314 INFO:tasks.workunit.client.1.smithi194.stdout:8/164: creat d2/d8/d9/de/f37 x:0 0 0 2024-04-12T05:56:02.315 INFO:tasks.workunit.client.1.smithi194.stdout:7/77: creat d3/d4/f1b x:0 0 0 2024-04-12T05:56:02.315 INFO:tasks.workunit.client.1.smithi194.stdout:2/247: chown d7/d12/d25/f31 1465332 1 2024-04-12T05:56:02.316 INFO:tasks.workunit.client.1.smithi194.stdout:0/182: fsync d0/da/d1e/d1a/f2e 0 2024-04-12T05:56:02.318 INFO:tasks.workunit.client.1.smithi194.stdout:4/183: mkdir d2/d3/d1b/d32/d3a/d3b 0 2024-04-12T05:56:02.319 INFO:tasks.workunit.client.1.smithi194.stdout:7/78: read d3/d4/fa [705860,53008] 0 2024-04-12T05:56:02.320 INFO:tasks.workunit.client.1.smithi194.stdout:9/231: rename d1/d19/d1f/d20/c3c to d1/d21/d49/c4c 0 2024-04-12T05:56:02.321 INFO:tasks.workunit.client.1.smithi194.stdout:6/295: dread d3/f3a [0,4194304] 0 2024-04-12T05:56:02.322 INFO:tasks.workunit.client.1.smithi194.stdout:1/301: getdents d0/d11/d18/d19/d1c 0 2024-04-12T05:56:02.328 INFO:tasks.workunit.client.1.smithi194.stdout:2/248: dwrite d7/d22/f5c [0,4194304] 0 2024-04-12T05:56:02.337 INFO:tasks.workunit.client.1.smithi194.stdout:4/184: creat d2/d3/d1b/d32/f3c x:0 0 0 2024-04-12T05:56:02.337 INFO:tasks.workunit.client.1.smithi194.stdout:5/329: dread d1/d4/df/f69 [0,4194304] 0 2024-04-12T05:56:02.338 INFO:tasks.workunit.client.1.smithi194.stdout:0/183: chown d0/da/d1e/d1a/c26 1257849 1 2024-04-12T05:56:02.338 INFO:tasks.workunit.client.1.smithi194.stdout:4/185: chown d2/d11/d1e 36932 1 2024-04-12T05:56:02.340 INFO:tasks.workunit.client.1.smithi194.stdout:9/232: dread d1/f3f [0,4194304] 0 2024-04-12T05:56:02.341 INFO:tasks.workunit.client.1.smithi194.stdout:7/79: link d3/d4/f11 d3/d16/f1c 0 2024-04-12T05:56:02.341 INFO:tasks.workunit.client.1.smithi194.stdout:5/330: dread d1/fa [4194304,4194304] 0 2024-04-12T05:56:02.343 INFO:tasks.workunit.client.1.smithi194.stdout:6/296: creat d3/d2d/d33/d58/f59 x:0 0 0 2024-04-12T05:56:02.343 INFO:tasks.workunit.client.1.smithi194.stdout:0/184: mkdir d0/da/d1e/d27/d3c 0 2024-04-12T05:56:02.345 INFO:tasks.workunit.client.1.smithi194.stdout:4/186: write d2/d22/f34 [1234884,42307] 0 2024-04-12T05:56:02.346 INFO:tasks.workunit.client.1.smithi194.stdout:9/233: stat d1/c15 0 2024-04-12T05:56:02.346 INFO:tasks.workunit.client.1.smithi194.stdout:4/187: read - d2/d11/f39 zero size 2024-04-12T05:56:02.351 INFO:tasks.workunit.client.1.smithi194.stdout:6/297: fdatasync d3/d1a/f29 0 2024-04-12T05:56:02.354 INFO:tasks.workunit.client.1.smithi194.stdout:4/188: unlink d2/d11/c19 0 2024-04-12T05:56:02.356 INFO:tasks.workunit.client.1.smithi194.stdout:9/234: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:56:02.357 INFO:tasks.workunit.client.1.smithi194.stdout:1/302: rename d0/d11/c40 to d0/c53 0 2024-04-12T05:56:02.364 INFO:tasks.workunit.client.1.smithi194.stdout:8/165: write d2/d8/d9/d1a/f2b [4089848,81887] 0 2024-04-12T05:56:02.364 INFO:tasks.workunit.client.1.smithi194.stdout:2/249: getdents d7/d12/d34/d4a 0 2024-04-12T05:56:02.364 INFO:tasks.workunit.client.1.smithi194.stdout:0/185: dread d0/f4 [4194304,4194304] 0 2024-04-12T05:56:02.366 INFO:tasks.workunit.client.1.smithi194.stdout:7/80: fsync d3/d4/f11 0 2024-04-12T05:56:02.366 INFO:tasks.workunit.client.1.smithi194.stdout:2/250: chown d7/d12/d34/d4a 79 1 2024-04-12T05:56:02.366 INFO:tasks.workunit.client.1.smithi194.stdout:4/189: dwrite d2/d3/f18 [0,4194304] 0 2024-04-12T05:56:02.368 INFO:tasks.workunit.client.1.smithi194.stdout:7/81: readlink l1 0 2024-04-12T05:56:02.368 INFO:tasks.workunit.client.1.smithi194.stdout:9/235: creat d1/d21/f4d x:0 0 0 2024-04-12T05:56:02.368 INFO:tasks.workunit.client.1.smithi194.stdout:1/303: mkdir d0/d3e/d54 0 2024-04-12T05:56:02.370 INFO:tasks.workunit.client.1.smithi194.stdout:8/166: rmdir d2/d8 39 2024-04-12T05:56:02.370 INFO:tasks.workunit.client.1.smithi194.stdout:7/82: write d3/d16/f19 [523088,22686] 0 2024-04-12T05:56:02.375 INFO:tasks.workunit.client.1.smithi194.stdout:5/331: truncate d1/d4/d6/db/d15/d26/f50 2665925 0 2024-04-12T05:56:02.377 INFO:tasks.workunit.client.1.smithi194.stdout:0/186: mkdir d0/da/d1e/d1a/d3d 0 2024-04-12T05:56:02.380 INFO:tasks.workunit.client.1.smithi194.stdout:0/187: write d0/f30 [3582805,50035] 0 2024-04-12T05:56:02.381 INFO:tasks.workunit.client.1.smithi194.stdout:5/332: fdatasync d1/d4/d6/db/d15/d26/f4e 0 2024-04-12T05:56:02.382 INFO:tasks.workunit.client.1.smithi194.stdout:1/304: creat d0/d11/f55 x:0 0 0 2024-04-12T05:56:02.383 INFO:tasks.workunit.client.1.smithi194.stdout:2/251: dwrite d7/d22/f5c [0,4194304] 0 2024-04-12T05:56:02.387 INFO:tasks.workunit.client.1.smithi194.stdout:9/236: dwrite d1/fe [0,4194304] 0 2024-04-12T05:56:02.387 INFO:tasks.workunit.client.1.smithi194.stdout:7/83: unlink d3/fd 0 2024-04-12T05:56:02.389 INFO:tasks.workunit.client.1.smithi194.stdout:8/167: write d2/d8/d9/d1a/f30 [593776,97245] 0 2024-04-12T05:56:02.389 INFO:tasks.workunit.client.1.smithi194.stdout:6/298: getdents d3/d15/d2b 0 2024-04-12T05:56:02.390 INFO:tasks.workunit.client.1.smithi194.stdout:1/305: stat d0/le 0 2024-04-12T05:56:02.392 INFO:tasks.workunit.client.1.smithi194.stdout:2/252: write d7/d22/f4d [304085,91149] 0 2024-04-12T05:56:02.393 INFO:tasks.workunit.client.1.smithi194.stdout:8/168: fsync d2/d8/d9/d1a/f2b 0 2024-04-12T05:56:02.393 INFO:tasks.workunit.client.1.smithi194.stdout:0/188: dread d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:02.397 INFO:tasks.workunit.client.1.smithi194.stdout:9/237: mkdir d1/d4e 0 2024-04-12T05:56:02.398 INFO:tasks.workunit.client.1.smithi194.stdout:8/169: write d2/d8/d9/d1a/f25 [844084,44587] 0 2024-04-12T05:56:02.398 INFO:tasks.workunit.client.1.smithi194.stdout:6/299: rmdir d3/d15/d2b/d48 39 2024-04-12T05:56:02.399 INFO:tasks.workunit.client.1.smithi194.stdout:0/189: creat d0/da/f3e x:0 0 0 2024-04-12T05:56:02.399 INFO:tasks.workunit.client.1.smithi194.stdout:4/190: getdents d2/d11/d1f 0 2024-04-12T05:56:02.400 INFO:tasks.workunit.client.1.smithi194.stdout:1/306: symlink d0/d3e/d54/l56 0 2024-04-12T05:56:02.402 INFO:tasks.workunit.client.1.smithi194.stdout:6/300: write d3/f3a [9418562,14986] 0 2024-04-12T05:56:02.403 INFO:tasks.workunit.client.1.smithi194.stdout:1/307: symlink d0/d11/d18/l57 0 2024-04-12T05:56:02.405 INFO:tasks.workunit.client.1.smithi194.stdout:2/253: dread d7/fa [0,4194304] 0 2024-04-12T05:56:02.405 INFO:tasks.workunit.client.1.smithi194.stdout:4/191: mknod d2/d3/c3d 0 2024-04-12T05:56:02.406 INFO:tasks.workunit.client.1.smithi194.stdout:0/190: creat d0/da/d1e/d27/f3f x:0 0 0 2024-04-12T05:56:02.408 INFO:tasks.workunit.client.1.smithi194.stdout:8/170: truncate d2/d8/d15/d33/f31 895523 0 2024-04-12T05:56:02.410 INFO:tasks.workunit.client.1.smithi194.stdout:9/238: dread d1/d41/f43 [0,4194304] 0 2024-04-12T05:56:02.410 INFO:tasks.workunit.client.1.smithi194.stdout:2/254: write d7/d12/d25/f31 [2249515,53840] 0 2024-04-12T05:56:02.410 INFO:tasks.workunit.client.1.smithi194.stdout:6/301: dread d3/d1a/f35 [0,4194304] 0 2024-04-12T05:56:02.413 INFO:tasks.workunit.client.1.smithi194.stdout:1/308: rename d0/d11/d18/d19/d1a/d23 to d0/d11/d18/d19/d1a/d23/d58 22 2024-04-12T05:56:02.413 INFO:tasks.workunit.client.1.smithi194.stdout:8/171: write d2/d8/d9/de/f37 [910232,5961] 0 2024-04-12T05:56:02.416 INFO:tasks.workunit.client.1.smithi194.stdout:9/239: chown d1/f13 100504 1 2024-04-12T05:56:02.416 INFO:tasks.workunit.client.1.smithi194.stdout:2/255: stat d7/d22/d42/c20 0 2024-04-12T05:56:02.420 INFO:tasks.workunit.client.1.smithi194.stdout:6/302: fsync d3/d2f/d37/f3f 0 2024-04-12T05:56:02.423 INFO:tasks.workunit.client.1.smithi194.stdout:0/191: symlink d0/da/d1e/l40 0 2024-04-12T05:56:02.423 INFO:tasks.workunit.client.1.smithi194.stdout:8/172: mkdir d2/d8/d9/de/d38 0 2024-04-12T05:56:02.426 INFO:tasks.workunit.client.1.smithi194.stdout:5/333: write d1/d4/d6/db/d15/d26/f50 [2924359,42526] 0 2024-04-12T05:56:02.427 INFO:tasks.workunit.client.1.smithi194.stdout:1/309: dread d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [4194304,4194304] 0 2024-04-12T05:56:02.427 INFO:tasks.workunit.client.1.smithi194.stdout:7/84: unlink d3/d4/l15 0 2024-04-12T05:56:02.430 INFO:tasks.workunit.client.1.smithi194.stdout:5/334: dread - d1/d4/d6/d34/d52/f5f zero size 2024-04-12T05:56:02.431 INFO:tasks.workunit.client.1.smithi194.stdout:9/240: dread d1/f18 [0,4194304] 0 2024-04-12T05:56:02.431 INFO:tasks.workunit.client.1.smithi194.stdout:5/335: readlink d1/d4/d6/db/d15/l36 0 2024-04-12T05:56:02.433 INFO:tasks.workunit.client.1.smithi194.stdout:6/303: rename d3/d1a/f47 to d3/d2d/d33/d52/f5a 0 2024-04-12T05:56:02.437 INFO:tasks.workunit.client.1.smithi194.stdout:7/85: write d3/d16/f1a [1185319,18599] 0 2024-04-12T05:56:02.438 INFO:tasks.workunit.client.1.smithi194.stdout:4/192: getdents d2/d22 0 2024-04-12T05:56:02.439 INFO:tasks.workunit.client.1.smithi194.stdout:4/193: stat d2 0 2024-04-12T05:56:02.443 INFO:tasks.workunit.client.1.smithi194.stdout:7/86: fdatasync d3/d4/f1b 0 2024-04-12T05:56:02.446 INFO:tasks.workunit.client.1.smithi194.stdout:0/192: link d0/da/d1e/f35 d0/da/d1e/f41 0 2024-04-12T05:56:02.447 INFO:tasks.workunit.client.1.smithi194.stdout:8/173: dread d2/fb [0,4194304] 0 2024-04-12T05:56:02.450 INFO:tasks.workunit.client.1.smithi194.stdout:6/304: dwrite d3/d2f/d37/f4a [0,4194304] 0 2024-04-12T05:56:02.450 INFO:tasks.workunit.client.1.smithi194.stdout:5/336: rename d1/d4/d6/db/c27 to d1/d4/c6e 0 2024-04-12T05:56:02.453 INFO:tasks.workunit.client.1.smithi194.stdout:5/337: chown d1/d4/d6/db/d15/d26/c47 1721955 1 2024-04-12T05:56:02.454 INFO:tasks.workunit.client.1.smithi194.stdout:2/256: truncate d7/d12/f33 40454 0 2024-04-12T05:56:02.459 INFO:tasks.workunit.client.1.smithi194.stdout:5/338: chown d1/d4/f42 23 1 2024-04-12T05:56:02.460 INFO:tasks.workunit.client.1.smithi194.stdout:4/194: truncate d2/fc 2575752 0 2024-04-12T05:56:02.461 INFO:tasks.workunit.client.1.smithi194.stdout:7/87: dwrite f2 [0,4194304] 0 2024-04-12T05:56:02.461 INFO:tasks.workunit.client.1.smithi194.stdout:2/257: unlink d7/c9 0 2024-04-12T05:56:02.468 INFO:tasks.workunit.client.1.smithi194.stdout:0/193: link d0/da/l36 d0/da/d1e/d1a/l42 0 2024-04-12T05:56:02.468 INFO:tasks.workunit.client.1.smithi194.stdout:8/174: link d2/c7 d2/c39 0 2024-04-12T05:56:02.470 INFO:tasks.workunit.client.1.smithi194.stdout:6/305: dwrite d3/f3b [0,4194304] 0 2024-04-12T05:56:02.470 INFO:tasks.workunit.client.1.smithi194.stdout:4/195: creat d2/d3/d1b/d32/d3a/f3e x:0 0 0 2024-04-12T05:56:02.470 INFO:tasks.workunit.client.1.smithi194.stdout:9/241: dwrite d1/fb [0,4194304] 0 2024-04-12T05:56:02.475 INFO:tasks.workunit.client.1.smithi194.stdout:5/339: mknod d1/d4/d6/db/d5b/c6f 0 2024-04-12T05:56:02.479 INFO:tasks.workunit.client.1.smithi194.stdout:2/258: creat d7/f5e x:0 0 0 2024-04-12T05:56:02.479 INFO:tasks.workunit.client.1.smithi194.stdout:7/88: creat d3/d4/f1d x:0 0 0 2024-04-12T05:56:02.480 INFO:tasks.workunit.client.1.smithi194.stdout:2/259: dread - d7/d12/d25/d5d/d47/f57 zero size 2024-04-12T05:56:02.481 INFO:tasks.workunit.client.1.smithi194.stdout:9/242: mknod d1/d19/d1f/d20/c4f 0 2024-04-12T05:56:02.483 INFO:tasks.workunit.client.1.smithi194.stdout:8/175: creat d2/d8/d9/de/d38/f3a x:0 0 0 2024-04-12T05:56:02.488 INFO:tasks.workunit.client.1.smithi194.stdout:7/89: read d3/d4/fa [748904,105843] 0 2024-04-12T05:56:02.489 INFO:tasks.workunit.client.1.smithi194.stdout:4/196: getdents d2/d3/d1b/d32/d3a/d3b 0 2024-04-12T05:56:02.492 INFO:tasks.workunit.client.1.smithi194.stdout:0/194: dwrite d0/da/d1e/d1a/f1c [0,4194304] 0 2024-04-12T05:56:02.492 INFO:tasks.workunit.client.1.smithi194.stdout:9/243: unlink d1/d19/d1f/f47 0 2024-04-12T05:56:02.493 INFO:tasks.workunit.client.1.smithi194.stdout:6/306: dread d3/d2f/d37/f3f [0,4194304] 0 2024-04-12T05:56:02.493 INFO:tasks.workunit.client.1.smithi194.stdout:7/90: chown d3/d4/f11 93826 1 2024-04-12T05:56:02.500 INFO:tasks.workunit.client.1.smithi194.stdout:0/195: stat d0/da/d1e/fe 0 2024-04-12T05:56:02.500 INFO:tasks.workunit.client.1.smithi194.stdout:6/307: write d3/d15/f21 [76072,50065] 0 2024-04-12T05:56:02.500 INFO:tasks.workunit.client.1.smithi194.stdout:2/260: rename d7/d22/l29 to d7/d22/d42/l5f 0 2024-04-12T05:56:02.500 INFO:tasks.workunit.client.1.smithi194.stdout:0/196: chown d0/f4 910 1 2024-04-12T05:56:02.500 INFO:tasks.workunit.client.1.smithi194.stdout:9/244: fdatasync d1/d41/f43 0 2024-04-12T05:56:02.502 INFO:tasks.workunit.client.1.smithi194.stdout:5/340: dwrite d1/d4/d6/f13 [0,4194304] 0 2024-04-12T05:56:02.503 INFO:tasks.workunit.client.1.smithi194.stdout:4/197: creat d2/d3/f3f x:0 0 0 2024-04-12T05:56:02.510 INFO:tasks.workunit.client.1.smithi194.stdout:2/261: read d7/d12/d25/d5d/f2f [839042,59237] 0 2024-04-12T05:56:02.510 INFO:tasks.workunit.client.1.smithi194.stdout:9/245: chown d1/fb 747718 1 2024-04-12T05:56:02.510 INFO:tasks.workunit.client.1.smithi194.stdout:9/246: readlink d1/d19/d1f/d20/l23 0 2024-04-12T05:56:02.514 INFO:tasks.workunit.client.1.smithi194.stdout:7/91: dwrite d3/d4/f1d [0,4194304] 0 2024-04-12T05:56:02.515 INFO:tasks.workunit.client.1.smithi194.stdout:8/176: rmdir d2/d8/d9/de 39 2024-04-12T05:56:02.515 INFO:tasks.workunit.client.1.smithi194.stdout:0/197: rename d0/da/d1e/d1a/c2f to d0/da/d1e/d1a/d24/c43 0 2024-04-12T05:56:02.518 INFO:tasks.workunit.client.1.smithi194.stdout:5/341: symlink d1/d4/d6/db/d2b/l70 0 2024-04-12T05:56:02.521 INFO:tasks.workunit.client.1.smithi194.stdout:4/198: rename d2/d3/d1b to d2/d3/d1b/d40 22 2024-04-12T05:56:02.521 INFO:tasks.workunit.client.1.smithi194.stdout:9/247: symlink d1/d19/d1f/d20/d37/d38/l50 0 2024-04-12T05:56:02.524 INFO:tasks.workunit.client.1.smithi194.stdout:9/248: write d1/fe [3664223,102940] 0 2024-04-12T05:56:02.526 INFO:tasks.workunit.client.1.smithi194.stdout:2/262: dwrite d7/d22/f4d [0,4194304] 0 2024-04-12T05:56:02.528 INFO:tasks.workunit.client.1.smithi194.stdout:6/308: dwrite d3/d2f/d37/f4a [4194304,4194304] 0 2024-04-12T05:56:02.528 INFO:tasks.workunit.client.1.smithi194.stdout:7/92: symlink d3/d4/l1e 0 2024-04-12T05:56:02.533 INFO:tasks.workunit.client.1.smithi194.stdout:9/249: write d1/d41/f43 [168974,38304] 0 2024-04-12T05:56:02.533 INFO:tasks.workunit.client.1.smithi194.stdout:0/198: dread d0/da/d1e/f17 [0,4194304] 0 2024-04-12T05:56:02.539 INFO:tasks.workunit.client.1.smithi194.stdout:4/199: write d2/d3/d1b/d32/f3c [633816,59626] 0 2024-04-12T05:56:02.545 INFO:tasks.workunit.client.1.smithi194.stdout:4/200: unlink d2/d3/f31 0 2024-04-12T05:56:02.546 INFO:tasks.workunit.client.1.smithi194.stdout:6/309: unlink d3/d2d/c41 0 2024-04-12T05:56:02.550 INFO:tasks.workunit.client.1.smithi194.stdout:0/199: creat d0/da/d1e/d1a/f44 x:0 0 0 2024-04-12T05:56:02.550 INFO:tasks.workunit.client.1.smithi194.stdout:5/342: rename d1/d4/c6e to d1/d4/d6/db/c71 0 2024-04-12T05:56:02.552 INFO:tasks.workunit.client.1.smithi194.stdout:7/93: rename d3/d4/f11 to d3/d4/f1f 0 2024-04-12T05:56:02.553 INFO:tasks.workunit.client.1.smithi194.stdout:5/343: write f0 [4576695,27577] 0 2024-04-12T05:56:02.553 INFO:tasks.workunit.client.1.smithi194.stdout:6/310: symlink d3/d2d/d33/d52/l5b 0 2024-04-12T05:56:02.554 INFO:tasks.workunit.client.1.smithi194.stdout:5/344: stat d1 0 2024-04-12T05:56:02.555 INFO:tasks.workunit.client.1.smithi194.stdout:4/201: link d2/d3/f18 d2/d3/d1b/d32/d3a/d3b/f41 0 2024-04-12T05:56:02.556 INFO:tasks.workunit.client.1.smithi194.stdout:6/311: read d3/d15/f21 [106209,37685] 0 2024-04-12T05:56:02.557 INFO:tasks.workunit.client.1.smithi194.stdout:4/202: readlink d2/d11/d1f/l36 0 2024-04-12T05:56:02.558 INFO:tasks.workunit.client.1.smithi194.stdout:9/250: rename d1/d41/f43 to d1/d21/d39/f51 0 2024-04-12T05:56:02.559 INFO:tasks.workunit.client.1.smithi194.stdout:0/200: rename d0/da/d1e/d27/d3c to d0/da/d1e/d27/d45 0 2024-04-12T05:56:02.560 INFO:tasks.workunit.client.1.smithi194.stdout:2/263: dwrite d7/d12/d1a/f44 [0,4194304] 0 2024-04-12T05:56:02.561 INFO:tasks.workunit.client.1.smithi194.stdout:4/203: read d2/d3/d1b/d32/f3c [637573,9142] 0 2024-04-12T05:56:02.561 INFO:tasks.workunit.client.1.smithi194.stdout:0/201: chown d0/da 756 1 2024-04-12T05:56:02.562 INFO:tasks.workunit.client.1.smithi194.stdout:9/251: stat d1/d19/d1f/d20/d37/d38/c4a 0 2024-04-12T05:56:02.563 INFO:tasks.workunit.client.1.smithi194.stdout:6/312: read d3/d15/f28 [2384530,24994] 0 2024-04-12T05:56:02.564 INFO:tasks.workunit.client.1.smithi194.stdout:4/204: truncate d2/d11/f28 201219 0 2024-04-12T05:56:02.566 INFO:tasks.workunit.client.1.smithi194.stdout:5/345: creat d1/d4/df/f72 x:0 0 0 2024-04-12T05:56:02.566 INFO:tasks.workunit.client.1.smithi194.stdout:6/313: chown d3/d2d/d33/c38 20 1 2024-04-12T05:56:02.570 INFO:tasks.workunit.client.1.smithi194.stdout:0/202: mknod d0/da/d1e/d27/d45/c46 0 2024-04-12T05:56:02.573 INFO:tasks.workunit.client.1.smithi194.stdout:2/264: creat d7/d12/d25/d5d/d47/d52/f60 x:0 0 0 2024-04-12T05:56:02.574 INFO:tasks.workunit.client.1.smithi194.stdout:4/205: rename d2/d11/f28 to d2/d11/d1e/f42 0 2024-04-12T05:56:02.576 INFO:tasks.workunit.client.1.smithi194.stdout:5/346: write d1/d4/d6/db/d2b/f66 [6955657,86055] 0 2024-04-12T05:56:02.577 INFO:tasks.workunit.client.1.smithi194.stdout:6/314: mkdir d3/d2d/d33/d52/d5c 0 2024-04-12T05:56:02.582 INFO:tasks.workunit.client.1.smithi194.stdout:5/347: creat d1/d4/d6/db/d2b/f73 x:0 0 0 2024-04-12T05:56:02.582 INFO:tasks.workunit.client.1.smithi194.stdout:5/348: readlink d1/l2 0 2024-04-12T05:56:02.583 INFO:tasks.workunit.client.1.smithi194.stdout:5/349: chown d1/d4/d6/f13 2737 1 2024-04-12T05:56:02.584 INFO:tasks.workunit.client.1.smithi194.stdout:4/206: truncate d2/fc 3522824 0 2024-04-12T05:56:02.585 INFO:tasks.workunit.client.1.smithi194.stdout:2/265: creat d7/d22/f61 x:0 0 0 2024-04-12T05:56:02.586 INFO:tasks.workunit.client.1.smithi194.stdout:4/207: mkdir d2/d11/d43 0 2024-04-12T05:56:02.589 INFO:tasks.workunit.client.1.smithi194.stdout:0/203: dread d0/da/d1e/f19 [0,4194304] 0 2024-04-12T05:56:02.589 INFO:tasks.workunit.client.1.smithi194.stdout:4/208: write d2/d3/d1b/d32/d3a/f3e [251846,99112] 0 2024-04-12T05:56:02.590 INFO:tasks.workunit.client.1.smithi194.stdout:2/266: mknod d7/d22/c62 0 2024-04-12T05:56:02.592 INFO:tasks.workunit.client.1.smithi194.stdout:2/267: write d7/d22/f49 [197930,89786] 0 2024-04-12T05:56:02.592 INFO:tasks.workunit.client.1.smithi194.stdout:4/209: creat d2/d3/d1b/d32/f44 x:0 0 0 2024-04-12T05:56:02.593 INFO:tasks.workunit.client.1.smithi194.stdout:9/252: truncate d1/fe 2887960 0 2024-04-12T05:56:02.593 INFO:tasks.workunit.client.1.smithi194.stdout:4/210: fdatasync d2/d11/f39 0 2024-04-12T05:56:02.594 INFO:tasks.workunit.client.1.smithi194.stdout:0/204: mknod d0/da/d1e/d1a/d3d/c47 0 2024-04-12T05:56:02.595 INFO:tasks.workunit.client.1.smithi194.stdout:2/268: dread - d7/d12/d25/d5d/d47/d52/f60 zero size 2024-04-12T05:56:02.596 INFO:tasks.workunit.client.1.smithi194.stdout:2/269: stat d7/d22/d42/f54 0 2024-04-12T05:56:02.596 INFO:tasks.workunit.client.1.smithi194.stdout:9/253: creat d1/d19/d1f/f52 x:0 0 0 2024-04-12T05:56:02.596 INFO:tasks.workunit.client.1.smithi194.stdout:9/254: rename d1 to d1/d21/d53 22 2024-04-12T05:56:02.597 INFO:tasks.workunit.client.1.smithi194.stdout:0/205: stat d0/da/d1e/d1a/l42 0 2024-04-12T05:56:02.599 INFO:tasks.workunit.client.1.smithi194.stdout:9/255: creat d1/d19/f54 x:0 0 0 2024-04-12T05:56:02.600 INFO:tasks.workunit.client.1.smithi194.stdout:0/206: truncate d0/da/d1e/f35 815645 0 2024-04-12T05:56:02.601 INFO:tasks.workunit.client.1.smithi194.stdout:2/270: link d7/d12/d25/d5d/f2f d7/d22/f63 0 2024-04-12T05:56:02.602 INFO:tasks.workunit.client.1.smithi194.stdout:2/271: readlink d7/d12/d34/d4a/l59 0 2024-04-12T05:56:02.603 INFO:tasks.workunit.client.1.smithi194.stdout:0/207: creat d0/f48 x:0 0 0 2024-04-12T05:56:02.604 INFO:tasks.workunit.client.1.smithi194.stdout:0/208: stat d0/da/d1e/d27/f34 0 2024-04-12T05:56:02.605 INFO:tasks.workunit.client.1.smithi194.stdout:2/272: creat d7/d12/d25/f64 x:0 0 0 2024-04-12T05:56:02.605 INFO:tasks.workunit.client.1.smithi194.stdout:0/209: chown d0/da/d1e/fe 810816 1 2024-04-12T05:56:02.606 INFO:tasks.workunit.client.1.smithi194.stdout:0/210: write d0/da/d1e/d1a/f44 [734176,90912] 0 2024-04-12T05:56:02.610 INFO:tasks.workunit.client.1.smithi194.stdout:4/211: write d2/d3/f13 [4382656,79478] 0 2024-04-12T05:56:02.611 INFO:tasks.workunit.client.1.smithi194.stdout:4/212: write d2/d11/f39 [715478,73966] 0 2024-04-12T05:56:02.614 INFO:tasks.workunit.client.1.smithi194.stdout:3/235: sync 2024-04-12T05:56:02.615 INFO:tasks.workunit.client.1.smithi194.stdout:4/213: creat d2/d11/f45 x:0 0 0 2024-04-12T05:56:02.616 INFO:tasks.workunit.client.1.smithi194.stdout:2/273: link d7/d22/d42/c20 d7/d12/d34/c65 0 2024-04-12T05:56:02.625 INFO:tasks.workunit.client.1.smithi194.stdout:4/214: dwrite d2/d3/f13 [4194304,4194304] 0 2024-04-12T05:56:02.626 INFO:tasks.workunit.client.1.smithi194.stdout:3/236: rename d0/dc/de/c22 to d0/c4b 0 2024-04-12T05:56:02.627 INFO:tasks.workunit.client.1.smithi194.stdout:0/211: dread d0/da/d1e/f25 [0,4194304] 0 2024-04-12T05:56:02.627 INFO:tasks.workunit.client.1.smithi194.stdout:9/256: truncate d1/f1d 3965555 0 2024-04-12T05:56:02.629 INFO:tasks.workunit.client.1.smithi194.stdout:9/257: write d1/f13 [4466951,25191] 0 2024-04-12T05:56:02.633 INFO:tasks.workunit.client.1.smithi194.stdout:2/274: getdents d7/d12 0 2024-04-12T05:56:02.634 INFO:tasks.workunit.client.1.smithi194.stdout:4/215: mkdir d2/d11/d46 0 2024-04-12T05:56:02.636 INFO:tasks.workunit.client.1.smithi194.stdout:0/212: write d0/da/d1e/f32 [86065,117511] 0 2024-04-12T05:56:02.641 INFO:tasks.workunit.client.1.smithi194.stdout:2/275: fsync d7/d12/d1a/f3e 0 2024-04-12T05:56:02.643 INFO:tasks.workunit.client.1.smithi194.stdout:0/213: rename d0/da/d1e/d1a/c38 to d0/da/c49 0 2024-04-12T05:56:02.647 INFO:tasks.workunit.client.1.smithi194.stdout:9/258: dwrite d1/d21/d39/f51 [0,4194304] 0 2024-04-12T05:56:02.647 INFO:tasks.workunit.client.1.smithi194.stdout:2/276: mknod d7/d12/d34/d4a/c66 0 2024-04-12T05:56:02.649 INFO:tasks.workunit.client.1.smithi194.stdout:2/277: truncate d7/fa 833588 0 2024-04-12T05:56:02.650 INFO:tasks.workunit.client.1.smithi194.stdout:3/237: truncate d0/d1/f9 2952273 0 2024-04-12T05:56:02.651 INFO:tasks.workunit.client.1.smithi194.stdout:3/238: chown d0/c2f 1476 1 2024-04-12T05:56:02.652 INFO:tasks.workunit.client.1.smithi194.stdout:0/214: rename d0/da/d1e/d27/d45 to d0/da/d4a 0 2024-04-12T05:56:02.653 INFO:tasks.workunit.client.1.smithi194.stdout:3/239: stat d0/dc/de/d31 0 2024-04-12T05:56:02.655 INFO:tasks.workunit.client.1.smithi194.stdout:4/216: dread d2/d22/f34 [0,4194304] 0 2024-04-12T05:56:02.655 INFO:tasks.workunit.client.1.smithi194.stdout:0/215: fdatasync d0/f2 0 2024-04-12T05:56:02.656 INFO:tasks.workunit.client.1.smithi194.stdout:3/240: chown d0/d1/f1c 0 1 2024-04-12T05:56:02.659 INFO:tasks.workunit.client.1.smithi194.stdout:2/278: dwrite d7/d12/d25/d5d/d47/d52/f60 [0,4194304] 0 2024-04-12T05:56:02.661 INFO:tasks.workunit.client.1.smithi194.stdout:2/279: readlink d7/l30 0 2024-04-12T05:56:02.661 INFO:tasks.workunit.client.1.smithi194.stdout:2/280: chown d7/d12/l13 10 1 2024-04-12T05:56:02.662 INFO:tasks.workunit.client.1.smithi194.stdout:0/216: mkdir d0/da/d1e/d1a/d24/d4b 0 2024-04-12T05:56:02.662 INFO:tasks.workunit.client.1.smithi194.stdout:2/281: readlink d7/d12/d34/d4a/l59 0 2024-04-12T05:56:02.663 INFO:tasks.workunit.client.1.smithi194.stdout:0/217: truncate d0/da/d1e/d27/f2a 431845 0 2024-04-12T05:56:02.664 INFO:tasks.workunit.client.1.smithi194.stdout:3/241: rename d0/dc/c1a to d0/dc/de/d31/d34/c4c 0 2024-04-12T05:56:02.666 INFO:tasks.workunit.client.1.smithi194.stdout:2/282: symlink d7/d12/d34/l67 0 2024-04-12T05:56:02.670 INFO:tasks.workunit.client.1.smithi194.stdout:4/217: write d2/f21 [1454337,31995] 0 2024-04-12T05:56:02.670 INFO:tasks.workunit.client.1.smithi194.stdout:3/242: write d0/dc/de/f3f [2788705,62591] 0 2024-04-12T05:56:02.670 INFO:tasks.workunit.client.1.smithi194.stdout:2/283: unlink d7/d22/f49 0 2024-04-12T05:56:02.672 INFO:tasks.workunit.client.1.smithi194.stdout:4/218: fsync d2/d11/f39 0 2024-04-12T05:56:02.673 INFO:tasks.workunit.client.1.smithi194.stdout:3/243: stat d0/dc/c35 0 2024-04-12T05:56:02.674 INFO:tasks.workunit.client.1.smithi194.stdout:2/284: write d7/d12/d1a/f24 [1839010,115085] 0 2024-04-12T05:56:02.687 INFO:tasks.workunit.client.1.smithi194.stdout:4/219: dwrite d2/d11/f45 [0,4194304] 0 2024-04-12T05:56:02.688 INFO:tasks.workunit.client.1.smithi194.stdout:2/285: dwrite d7/d22/f5c [4194304,4194304] 0 2024-04-12T05:56:02.690 INFO:tasks.workunit.client.1.smithi194.stdout:0/218: dread d0/f14 [0,4194304] 0 2024-04-12T05:56:02.695 INFO:tasks.workunit.client.1.smithi194.stdout:4/220: mkdir d2/d11/d1e/d47 0 2024-04-12T05:56:02.696 INFO:tasks.workunit.client.1.smithi194.stdout:2/286: mkdir d7/d12/d1a/d68 0 2024-04-12T05:56:02.697 INFO:tasks.workunit.client.1.smithi194.stdout:4/221: chown d2/d3/c3d 1193 1 2024-04-12T05:56:02.698 INFO:tasks.workunit.client.1.smithi194.stdout:0/219: getdents d0 0 2024-04-12T05:56:02.700 INFO:tasks.workunit.client.1.smithi194.stdout:3/244: dread d0/d1/d10/f2b [0,4194304] 0 2024-04-12T05:56:02.700 INFO:tasks.workunit.client.1.smithi194.stdout:2/287: mkdir d7/d12/d1a/d69 0 2024-04-12T05:56:02.701 INFO:tasks.workunit.client.1.smithi194.stdout:4/222: rename d2/c4 to d2/d3/d1b/d32/d3a/c48 0 2024-04-12T05:56:02.703 INFO:tasks.workunit.client.1.smithi194.stdout:0/220: rmdir d0/da/d1e/d1a/d3d 39 2024-04-12T05:56:02.704 INFO:tasks.workunit.client.1.smithi194.stdout:3/245: mkdir d0/dc/de/d39/d4d 0 2024-04-12T05:56:02.705 INFO:tasks.workunit.client.1.smithi194.stdout:4/223: write d2/d11/d1e/f42 [482727,81129] 0 2024-04-12T05:56:02.706 INFO:tasks.workunit.client.1.smithi194.stdout:2/288: symlink d7/d12/d1a/d68/l6a 0 2024-04-12T05:56:02.707 INFO:tasks.workunit.client.1.smithi194.stdout:0/221: creat d0/d3b/f4c x:0 0 0 2024-04-12T05:56:02.709 INFO:tasks.workunit.client.1.smithi194.stdout:4/224: symlink d2/d3/d1b/d32/d3a/d3b/l49 0 2024-04-12T05:56:02.709 INFO:tasks.workunit.client.1.smithi194.stdout:3/246: write d0/d1/f1c [612513,22857] 0 2024-04-12T05:56:02.709 INFO:tasks.workunit.client.1.smithi194.stdout:2/289: mknod d7/d12/d1a/d68/c6b 0 2024-04-12T05:56:02.710 INFO:tasks.workunit.client.1.smithi194.stdout:4/225: stat d2/d22/c35 0 2024-04-12T05:56:02.711 INFO:tasks.workunit.client.1.smithi194.stdout:3/247: write d0/dc/de/f45 [383707,78642] 0 2024-04-12T05:56:02.713 INFO:tasks.workunit.client.1.smithi194.stdout:4/226: mkdir d2/d11/d1e/d2f/d4a 0 2024-04-12T05:56:02.714 INFO:tasks.workunit.client.1.smithi194.stdout:3/248: mknod d0/dc/de/d31/d34/c4e 0 2024-04-12T05:56:02.718 INFO:tasks.workunit.client.1.smithi194.stdout:4/227: mknod d2/d11/d43/c4b 0 2024-04-12T05:56:02.719 INFO:tasks.workunit.client.1.smithi194.stdout:4/228: chown d2/d11/d1f/l36 7681 1 2024-04-12T05:56:02.721 INFO:tasks.workunit.client.1.smithi194.stdout:2/290: write d7/d12/f33 [801215,11604] 0 2024-04-12T05:56:02.726 INFO:tasks.workunit.client.1.smithi194.stdout:2/291: rmdir d7/d12/d34/d4a 39 2024-04-12T05:56:02.727 INFO:tasks.workunit.client.1.smithi194.stdout:2/292: fsync d7/d12/d25/d5d/f51 0 2024-04-12T05:56:02.731 INFO:tasks.workunit.client.1.smithi194.stdout:2/293: rmdir d7/d12/d1a/d68 39 2024-04-12T05:56:02.737 INFO:tasks.workunit.client.1.smithi194.stdout:3/249: dread d0/dc/f1f [4194304,4194304] 0 2024-04-12T05:56:02.737 INFO:tasks.workunit.client.1.smithi194.stdout:3/250: readlink d0/d1/la 0 2024-04-12T05:56:02.739 INFO:tasks.workunit.client.1.smithi194.stdout:3/251: mkdir d0/d1/d4f 0 2024-04-12T05:56:02.740 INFO:tasks.workunit.client.1.smithi194.stdout:2/294: truncate d7/d12/d1a/f24 1727651 0 2024-04-12T05:56:02.742 INFO:tasks.workunit.client.1.smithi194.stdout:4/229: dread d2/f6 [4194304,4194304] 0 2024-04-12T05:56:02.743 INFO:tasks.workunit.client.1.smithi194.stdout:4/230: readlink d2/d3/d1b/l29 0 2024-04-12T05:56:02.743 INFO:tasks.workunit.client.1.smithi194.stdout:3/252: rename d0/dc/de/d31/d34/l21 to d0/d42/l50 0 2024-04-12T05:56:02.745 INFO:tasks.workunit.client.1.smithi194.stdout:3/253: write d0/dc/de/d31/d34/f41 [418374,30059] 0 2024-04-12T05:56:02.747 INFO:tasks.workunit.client.1.smithi194.stdout:3/254: fsync d0/dc/f1f 0 2024-04-12T05:56:02.751 INFO:tasks.workunit.client.1.smithi194.stdout:4/231: dwrite d2/d11/f45 [0,4194304] 0 2024-04-12T05:56:02.752 INFO:tasks.workunit.client.1.smithi194.stdout:3/255: creat d0/d42/f51 x:0 0 0 2024-04-12T05:56:02.753 INFO:tasks.workunit.client.1.smithi194.stdout:3/256: write d0/d1/f1c [2397444,76771] 0 2024-04-12T05:56:02.754 INFO:tasks.workunit.client.1.smithi194.stdout:3/257: write d0/dc/de/f32 [1537424,66064] 0 2024-04-12T05:56:02.758 INFO:tasks.workunit.client.1.smithi194.stdout:3/258: creat d0/d1/d10/f52 x:0 0 0 2024-04-12T05:56:02.760 INFO:tasks.workunit.client.1.smithi194.stdout:3/259: mkdir d0/dc/d53 0 2024-04-12T05:56:02.760 INFO:tasks.workunit.client.1.smithi194.stdout:3/260: fsync d0/dc/f12 0 2024-04-12T05:56:02.764 INFO:tasks.workunit.client.1.smithi194.stdout:4/232: truncate f0 179416 0 2024-04-12T05:56:02.764 INFO:tasks.workunit.client.1.smithi194.stdout:3/261: read d0/dc/de/f32 [3111832,110626] 0 2024-04-12T05:56:02.768 INFO:tasks.workunit.client.1.smithi194.stdout:4/233: dread d2/d11/f39 [0,4194304] 0 2024-04-12T05:56:02.770 INFO:tasks.workunit.client.1.smithi194.stdout:3/262: rename d0/d1/fd to d0/f54 0 2024-04-12T05:56:02.772 INFO:tasks.workunit.client.1.smithi194.stdout:3/263: mknod d0/dc/de/d31/c55 0 2024-04-12T05:56:02.772 INFO:tasks.workunit.client.1.smithi194.stdout:3/264: chown d0/dc/de/d31/d34/d2d/f3a 1541301 1 2024-04-12T05:56:02.773 INFO:tasks.workunit.client.1.smithi194.stdout:3/265: fdatasync d0/d42/f51 0 2024-04-12T05:56:02.779 INFO:tasks.workunit.client.1.smithi194.stdout:3/266: getdents d0/dc/de/d31 0 2024-04-12T05:56:02.785 INFO:tasks.workunit.client.1.smithi194.stdout:3/267: dwrite d0/f27 [0,4194304] 0 2024-04-12T05:56:02.787 INFO:tasks.workunit.client.1.smithi194.stdout:3/268: mknod d0/dc/de/d31/c56 0 2024-04-12T05:56:02.788 INFO:tasks.workunit.client.1.smithi194.stdout:3/269: write d0/d1/d10/f2e [713188,25729] 0 2024-04-12T05:56:02.790 INFO:tasks.workunit.client.1.smithi194.stdout:3/270: creat d0/d1/f57 x:0 0 0 2024-04-12T05:56:02.792 INFO:tasks.workunit.client.1.smithi194.stdout:3/271: mknod d0/dc/de/d31/d34/d2d/c58 0 2024-04-12T05:56:02.793 INFO:tasks.workunit.client.1.smithi194.stdout:3/272: write d0/dc/de/d31/d34/d2d/f3a [241611,17775] 0 2024-04-12T05:56:02.793 INFO:tasks.workunit.client.1.smithi194.stdout:3/273: read d0/dc/f1f [6193364,65209] 0 2024-04-12T05:56:02.796 INFO:tasks.workunit.client.1.smithi194.stdout:3/274: link d0/dc/de/d31/d34/f37 d0/d1/d4f/f59 0 2024-04-12T05:56:02.797 INFO:tasks.workunit.client.1.smithi194.stdout:3/275: write d0/d42/f51 [299981,52290] 0 2024-04-12T05:56:02.798 INFO:tasks.workunit.client.1.smithi194.stdout:3/276: read d0/dc/f12 [1215114,26838] 0 2024-04-12T05:56:02.800 INFO:tasks.workunit.client.1.smithi194.stdout:3/277: creat d0/dc/de/d31/d34/d2d/f5a x:0 0 0 2024-04-12T05:56:02.802 INFO:tasks.workunit.client.1.smithi194.stdout:3/278: mkdir d0/dc/de/d31/d5b 0 2024-04-12T05:56:02.805 INFO:tasks.workunit.client.1.smithi194.stdout:3/279: rename d0/dc/de/d31/c56 to d0/c5c 0 2024-04-12T05:56:02.811 INFO:tasks.workunit.client.1.smithi194.stdout:3/280: dwrite d0/d1/d10/f2b [0,4194304] 0 2024-04-12T05:56:02.823 INFO:tasks.workunit.client.1.smithi194.stdout:3/281: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:02.824 INFO:tasks.workunit.client.1.smithi194.stdout:3/282: truncate d0/dc/d1e/f2a 1742965 0 2024-04-12T05:56:02.827 INFO:tasks.workunit.client.1.smithi194.stdout:3/283: dread d0/f27 [0,4194304] 0 2024-04-12T05:56:02.828 INFO:tasks.workunit.client.1.smithi194.stdout:3/284: chown d0/dc/de/d39 37721862 1 2024-04-12T05:56:02.829 INFO:tasks.workunit.client.1.smithi194.stdout:3/285: truncate d0/d1/d10/f52 758404 0 2024-04-12T05:56:02.831 INFO:tasks.workunit.client.1.smithi194.stdout:3/286: creat d0/d1/f5d x:0 0 0 2024-04-12T05:56:02.832 INFO:tasks.workunit.client.1.smithi194.stdout:3/287: mknod d0/dc/de/d31/d34/c5e 0 2024-04-12T05:56:02.833 INFO:tasks.workunit.client.1.smithi194.stdout:3/288: chown d0/dc/de/c3c 85295501 1 2024-04-12T05:56:02.834 INFO:tasks.workunit.client.1.smithi194.stdout:3/289: creat d0/dc/d1e/f5f x:0 0 0 2024-04-12T05:56:02.835 INFO:tasks.workunit.client.1.smithi194.stdout:3/290: stat d0/dc/de/d31 0 2024-04-12T05:56:02.857 INFO:tasks.workunit.client.1.smithi194.stdout:3/291: dread d0/dc/de/d39/f3b [0,4194304] 0 2024-04-12T05:56:02.858 INFO:tasks.workunit.client.1.smithi194.stdout:3/292: readlink d0/dc/l15 0 2024-04-12T05:56:02.858 INFO:tasks.workunit.client.1.smithi194.stdout:3/293: stat d0/dc/de 0 2024-04-12T05:56:02.858 INFO:tasks.workunit.client.1.smithi194.stdout:3/294: readlink d0/d1/la 0 2024-04-12T05:56:02.860 INFO:tasks.workunit.client.1.smithi194.stdout:3/295: creat d0/d1/f60 x:0 0 0 2024-04-12T05:56:02.862 INFO:tasks.workunit.client.1.smithi194.stdout:3/296: readlink d0/d42/l50 0 2024-04-12T05:56:02.864 INFO:tasks.workunit.client.1.smithi194.stdout:3/297: creat d0/dc/d53/f61 x:0 0 0 2024-04-12T05:56:02.866 INFO:tasks.workunit.client.1.smithi194.stdout:3/298: fsync d0/dc/de/d39/f3b 0 2024-04-12T05:56:02.871 INFO:tasks.workunit.client.1.smithi194.stdout:3/299: dwrite d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:02.875 INFO:tasks.workunit.client.1.smithi194.stdout:3/300: creat d0/dc/de/d39/d4d/f62 x:0 0 0 2024-04-12T05:56:02.880 INFO:tasks.workunit.client.1.smithi194.stdout:3/301: dwrite d0/fb [4194304,4194304] 0 2024-04-12T05:56:02.882 INFO:tasks.workunit.client.1.smithi194.stdout:3/302: mkdir d0/dc/de/d39/d4d/d63 0 2024-04-12T05:56:02.894 INFO:tasks.workunit.client.1.smithi194.stdout:1/310: sync 2024-04-12T05:56:02.898 INFO:tasks.workunit.client.1.smithi194.stdout:1/311: creat d0/d11/f59 x:0 0 0 2024-04-12T05:56:02.901 INFO:tasks.workunit.client.1.smithi194.stdout:1/312: mkdir d0/d11/d18/d19/d1a/d23/d27/d5a 0 2024-04-12T05:56:02.902 INFO:tasks.workunit.client.1.smithi194.stdout:1/313: truncate d0/d11/f55 392077 0 2024-04-12T05:56:02.903 INFO:tasks.workunit.client.1.smithi194.stdout:1/314: mkdir d0/d3e/d5b 0 2024-04-12T05:56:02.913 INFO:tasks.workunit.client.1.smithi194.stdout:3/303: write d0/d1/f9 [1008096,16567] 0 2024-04-12T05:56:02.914 INFO:tasks.workunit.client.1.smithi194.stdout:1/315: dwrite d0/d11/d18/d19/d1c/f20 [0,4194304] 0 2024-04-12T05:56:02.914 INFO:tasks.workunit.client.1.smithi194.stdout:3/304: chown d0/dc/de/d39/d4d/d63 1 1 2024-04-12T05:56:02.924 INFO:tasks.workunit.client.1.smithi194.stdout:3/305: mkdir d0/dc/de/d31/d5b/d64 0 2024-04-12T05:56:02.926 INFO:tasks.workunit.client.1.smithi194.stdout:1/316: dwrite d0/f32 [0,4194304] 0 2024-04-12T05:56:02.933 INFO:tasks.workunit.client.1.smithi194.stdout:3/306: dwrite d0/dc/d1e/d33/f4a [0,4194304] 0 2024-04-12T05:56:02.944 INFO:tasks.workunit.client.1.smithi194.stdout:1/317: getdents d0/d11/d18/d19/d1a/d23/d27 0 2024-04-12T05:56:02.967 INFO:tasks.workunit.client.1.smithi194.stdout:1/318: dwrite d0/d11/d18/d19/d1c/f24 [0,4194304] 0 2024-04-12T05:56:02.978 INFO:tasks.workunit.client.1.smithi194.stdout:1/319: dwrite d0/d11/d18/d19/d1c/f20 [0,4194304] 0 2024-04-12T05:56:02.981 INFO:tasks.workunit.client.1.smithi194.stdout:1/320: write d0/d11/d18/d19/f1f [502417,73914] 0 2024-04-12T05:56:02.984 INFO:tasks.workunit.client.1.smithi194.stdout:1/321: fsync d0/d11/d18/d19/d1c/f22 0 2024-04-12T05:56:02.986 INFO:tasks.workunit.client.1.smithi194.stdout:1/322: read d0/d11/d18/d19/d1a/d23/d27/d2e/f33 [7833527,45237] 0 2024-04-12T05:56:02.988 INFO:tasks.workunit.client.1.smithi194.stdout:1/323: truncate d0/d11/d18/d19/d1c/f24 7051223 0 2024-04-12T05:56:02.988 INFO:tasks.workunit.client.1.smithi194.stdout:1/324: fdatasync d0/d11/f26 0 2024-04-12T05:56:02.993 INFO:tasks.workunit.client.1.smithi194.stdout:1/325: rename d0/d3e/d54/l56 to d0/d11/d18/d19/d1a/l5c 0 2024-04-12T05:56:02.994 INFO:tasks.workunit.client.1.smithi194.stdout:1/326: chown d0/f4 3 1 2024-04-12T05:56:02.996 INFO:tasks.workunit.client.1.smithi194.stdout:1/327: write d0/d11/d18/d35/f3a [614733,116836] 0 2024-04-12T05:56:02.997 INFO:tasks.workunit.client.1.smithi194.stdout:1/328: write d0/d11/f59 [526288,2540] 0 2024-04-12T05:56:03.003 INFO:tasks.workunit.client.1.smithi194.stdout:1/329: unlink d0/d11/d18/d19/d1a/d23/d27/l4d 0 2024-04-12T05:56:03.007 INFO:tasks.workunit.client.1.smithi194.stdout:1/330: mknod d0/d3e/d5b/c5d 0 2024-04-12T05:56:03.014 INFO:tasks.workunit.client.1.smithi194.stdout:1/331: dwrite d0/d11/d18/d19/f36 [0,4194304] 0 2024-04-12T05:56:03.017 INFO:tasks.workunit.client.1.smithi194.stdout:1/332: symlink d0/d3e/d54/l5e 0 2024-04-12T05:56:03.018 INFO:tasks.workunit.client.1.smithi194.stdout:1/333: write d0/d11/d18/d19/d1a/d23/f30 [5242281,3924] 0 2024-04-12T05:56:03.023 INFO:tasks.workunit.client.1.smithi194.stdout:1/334: link d0/d11/d18/l3f d0/d11/d18/d19/d1c/l5f 0 2024-04-12T05:56:03.025 INFO:tasks.workunit.client.1.smithi194.stdout:1/335: write d0/d11/f59 [589663,33776] 0 2024-04-12T05:56:03.025 INFO:tasks.workunit.client.1.smithi194.stdout:1/336: fsync d0/d11/d18/f45 0 2024-04-12T05:56:03.030 INFO:tasks.workunit.client.1.smithi194.stdout:1/337: link d0/d11/d18/d19/d1a/l47 d0/d11/d18/d35/l60 0 2024-04-12T05:56:03.032 INFO:tasks.workunit.client.1.smithi194.stdout:1/338: creat d0/d11/d18/d19/d1a/d23/d27/d2e/d42/f61 x:0 0 0 2024-04-12T05:56:03.034 INFO:tasks.workunit.client.1.smithi194.stdout:1/339: creat d0/d11/d18/d19/d1a/d23/d27/d48/f62 x:0 0 0 2024-04-12T05:56:03.040 INFO:tasks.workunit.client.1.smithi194.stdout:1/340: dwrite d0/f32 [4194304,4194304] 0 2024-04-12T05:56:03.044 INFO:tasks.workunit.client.1.smithi194.stdout:1/341: mkdir d0/d11/d18/d19/d1a/d23/d63 0 2024-04-12T05:56:03.045 INFO:tasks.workunit.client.1.smithi194.stdout:1/342: fdatasync d0/d11/d18/d19/f36 0 2024-04-12T05:56:03.061 INFO:tasks.workunit.client.1.smithi194.stdout:1/343: dread d0/d11/d18/d19/d1c/f24 [4194304,4194304] 0 2024-04-12T05:56:03.063 INFO:tasks.workunit.client.1.smithi194.stdout:1/344: creat d0/d3e/d54/f64 x:0 0 0 2024-04-12T05:56:03.096 INFO:tasks.workunit.client.1.smithi194.stdout:8/177: sync 2024-04-12T05:56:03.114 INFO:tasks.workunit.client.1.smithi194.stdout:1/345: write d0/d11/d18/d19/d1c/f46 [64674,17013] 0 2024-04-12T05:56:03.116 INFO:tasks.workunit.client.1.smithi194.stdout:8/178: write d2/d8/d15/d33/f18 [1654022,115613] 0 2024-04-12T05:56:03.117 INFO:tasks.workunit.client.1.smithi194.stdout:1/346: mknod d0/d11/d18/c65 0 2024-04-12T05:56:03.118 INFO:tasks.workunit.client.1.smithi194.stdout:8/179: write d2/d8/d9/de/d10/f24 [1440674,97835] 0 2024-04-12T05:56:03.120 INFO:tasks.workunit.client.1.smithi194.stdout:1/347: mknod d0/d11/d18/c66 0 2024-04-12T05:56:03.120 INFO:tasks.workunit.client.1.smithi194.stdout:8/180: write d2/d8/d15/d33/f18 [190010,30902] 0 2024-04-12T05:56:03.121 INFO:tasks.workunit.client.1.smithi194.stdout:8/181: chown d2/fb 3799234 1 2024-04-12T05:56:03.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:03.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: pgmap v27: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 45 MiB/s wr, 211 op/s 2024-04-12T05:56:03.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:03 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.171 INFO:tasks.workunit.client.1.smithi194.stdout:1/348: write d0/d11/d18/d19/d1c/f22 [724577,42305] 0 2024-04-12T05:56:03.173 INFO:tasks.workunit.client.1.smithi194.stdout:1/349: unlink d0/d11/c28 0 2024-04-12T05:56:03.177 INFO:tasks.workunit.client.1.smithi194.stdout:8/182: dwrite d2/fb [0,4194304] 0 2024-04-12T05:56:03.179 INFO:tasks.workunit.client.1.smithi194.stdout:8/183: write d2/d8/d9/de/f2e [1008436,3990] 0 2024-04-12T05:56:03.180 INFO:tasks.workunit.client.1.smithi194.stdout:8/184: fdatasync d2/d8/f26 0 2024-04-12T05:56:03.182 INFO:tasks.workunit.client.1.smithi194.stdout:1/350: dwrite d0/d11/f55 [0,4194304] 0 2024-04-12T05:56:03.185 INFO:tasks.workunit.client.1.smithi194.stdout:1/351: dread - d0/d11/d18/d19/d1a/d23/d27/d48/f62 zero size 2024-04-12T05:56:03.188 INFO:tasks.workunit.client.1.smithi194.stdout:8/185: dwrite d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:03.189 INFO:tasks.workunit.client.1.smithi194.stdout:1/352: write d0/d11/d18/d19/d1a/d23/d27/d48/f62 [110648,94574] 0 2024-04-12T05:56:03.192 INFO:tasks.workunit.client.1.smithi194.stdout:8/186: mkdir d2/d8/d15/d3b 0 2024-04-12T05:56:03.193 INFO:tasks.workunit.client.1.smithi194.stdout:1/353: mkdir d0/d11/d18/d19/d1c/d67 0 2024-04-12T05:56:03.195 INFO:tasks.workunit.client.1.smithi194.stdout:8/187: creat d2/d8/d9/de/d10/f3c x:0 0 0 2024-04-12T05:56:03.196 INFO:tasks.workunit.client.1.smithi194.stdout:1/354: mkdir d0/df/d68 0 2024-04-12T05:56:03.198 INFO:tasks.workunit.client.1.smithi194.stdout:1/355: readlink d0/d11/d18/d19/d1a/d23/l3c 0 2024-04-12T05:56:03.199 INFO:tasks.workunit.client.1.smithi194.stdout:1/356: stat d0/d11/d18/d19/d1a/d23/d27/d48 0 2024-04-12T05:56:03.201 INFO:tasks.workunit.client.1.smithi194.stdout:1/357: chown d0/d11/d18/d19/d1c/l2b 823609 1 2024-04-12T05:56:03.206 INFO:tasks.workunit.client.1.smithi194.stdout:1/358: dwrite d0/f12 [4194304,4194304] 0 2024-04-12T05:56:03.215 INFO:tasks.workunit.client.1.smithi194.stdout:1/359: truncate d0/d11/d18/d19/f39 3053437 0 2024-04-12T05:56:03.216 INFO:tasks.workunit.client.1.smithi194.stdout:1/360: write d0/d11/f59 [1391744,7121] 0 2024-04-12T05:56:03.217 INFO:tasks.workunit.client.1.smithi194.stdout:1/361: dread - d0/d11/d18/d19/f4e zero size 2024-04-12T05:56:03.219 INFO:tasks.workunit.client.1.smithi194.stdout:1/362: symlink d0/d11/d18/d19/d1a/l69 0 2024-04-12T05:56:03.220 INFO:tasks.workunit.client.1.smithi194.stdout:1/363: chown d0/d3e/d5b 60179259 1 2024-04-12T05:56:03.226 INFO:tasks.workunit.client.1.smithi194.stdout:1/364: rename d0/d11/d18/d19/d1a/d23 to d0/d11/d18/d19/d1a/d6a 0 2024-04-12T05:56:03.226 INFO:tasks.workunit.client.1.smithi194.stdout:1/365: fsync d0/f12 0 2024-04-12T05:56:03.233 INFO:tasks.workunit.client.1.smithi194.stdout:1/366: dwrite d0/d11/d18/f2d [4194304,4194304] 0 2024-04-12T05:56:03.234 INFO:tasks.workunit.client.1.smithi194.stdout:1/367: write d0/d11/d18/d19/d1a/d6a/f30 [1701007,97457] 0 2024-04-12T05:56:03.238 INFO:tasks.workunit.client.1.smithi194.stdout:1/368: creat d0/d11/d18/d19/d1c/f6b x:0 0 0 2024-04-12T05:56:03.242 INFO:tasks.workunit.client.1.smithi194.stdout:1/369: mkdir d0/d11/d18/d19/d1a/d6a/d27/d6c 0 2024-04-12T05:56:03.248 INFO:tasks.workunit.client.1.smithi194.stdout:1/370: truncate d0/d11/f55 4082509 0 2024-04-12T05:56:03.251 INFO:tasks.workunit.client.1.smithi194.stdout:1/371: link d0/d3e/d5b/c5d d0/df/c6d 0 2024-04-12T05:56:03.251 INFO:tasks.workunit.client.1.smithi194.stdout:1/372: stat d0/d11/d18/c66 0 2024-04-12T05:56:03.261 INFO:tasks.workunit.client.1.smithi194.stdout:1/373: dwrite d0/d11/f55 [0,4194304] 0 2024-04-12T05:56:03.280 INFO:tasks.workunit.client.1.smithi194.stdout:7/94: sync 2024-04-12T05:56:03.281 INFO:tasks.workunit.client.1.smithi194.stdout:7/95: write d3/d4/f7 [992983,64921] 0 2024-04-12T05:56:03.283 INFO:tasks.workunit.client.1.smithi194.stdout:7/96: symlink d3/d4/l20 0 2024-04-12T05:56:03.290 INFO:tasks.workunit.client.1.smithi194.stdout:7/97: dwrite d3/d4/d12/f18 [0,4194304] 0 2024-04-12T05:56:03.293 INFO:tasks.workunit.client.1.smithi194.stdout:7/98: mkdir d3/d16/d21 0 2024-04-12T05:56:03.294 INFO:tasks.workunit.client.1.smithi194.stdout:7/99: truncate d3/d16/f1a 1483890 0 2024-04-12T05:56:03.297 INFO:tasks.workunit.client.1.smithi194.stdout:7/100: rename d3/d4/d12 to d3/d4/d22 0 2024-04-12T05:56:03.297 INFO:tasks.workunit.client.1.smithi194.stdout:7/101: fdatasync d3/d16/f1a 0 2024-04-12T05:56:03.303 INFO:tasks.workunit.client.1.smithi194.stdout:7/102: dwrite d3/d4/d22/f18 [0,4194304] 0 2024-04-12T05:56:03.307 INFO:tasks.workunit.client.1.smithi194.stdout:7/103: truncate d3/d4/f1f 829585 0 2024-04-12T05:56:03.308 INFO:tasks.workunit.client.1.smithi194.stdout:7/104: read d3/d16/f1a [422300,107880] 0 2024-04-12T05:56:03.310 INFO:tasks.workunit.client.1.smithi194.stdout:7/105: mkdir d3/d16/d23 0 2024-04-12T05:56:03.310 INFO:tasks.workunit.client.1.smithi194.stdout:7/106: readlink l1 0 2024-04-12T05:56:03.313 INFO:tasks.workunit.client.1.smithi194.stdout:7/107: link d3/d4/l20 d3/d16/d23/l24 0 2024-04-12T05:56:03.313 INFO:tasks.workunit.client.1.smithi194.stdout:7/108: read - d3/d4/f1b zero size 2024-04-12T05:56:03.319 INFO:tasks.workunit.client.1.smithi194.stdout:7/109: dwrite d3/d4/d22/f18 [0,4194304] 0 2024-04-12T05:56:03.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:03.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: pgmap v27: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 45 MiB/s wr, 211 op/s 2024-04-12T05:56:03.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:03 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:03.327 INFO:tasks.workunit.client.1.smithi194.stdout:7/110: dread d3/d4/f1f [0,4194304] 0 2024-04-12T05:56:03.333 INFO:tasks.workunit.client.1.smithi194.stdout:7/111: dwrite d3/d4/f7 [0,4194304] 0 2024-04-12T05:56:03.334 INFO:tasks.workunit.client.1.smithi194.stdout:7/112: rmdir d3/d16/d23 39 2024-04-12T05:56:03.338 INFO:tasks.workunit.client.1.smithi194.stdout:7/113: mknod d3/d16/c25 0 2024-04-12T05:56:03.339 INFO:tasks.workunit.client.1.smithi194.stdout:7/114: write d3/d4/d22/f18 [1772548,118039] 0 2024-04-12T05:56:03.342 INFO:tasks.workunit.client.1.smithi194.stdout:7/115: link d3/d4/d22/c10 d3/d4/c26 0 2024-04-12T05:56:03.350 INFO:tasks.workunit.client.1.smithi194.stdout:7/116: rename d3/d4 to d3/d27 0 2024-04-12T05:56:03.356 INFO:tasks.workunit.client.1.smithi194.stdout:7/117: rename f2 to d3/d27/f28 0 2024-04-12T05:56:03.358 INFO:tasks.workunit.client.1.smithi194.stdout:7/118: write d3/d16/f1a [2359913,104009] 0 2024-04-12T05:56:03.361 INFO:tasks.workunit.client.1.smithi194.stdout:7/119: rename d3/d27/d22/f18 to d3/d27/d22/f29 0 2024-04-12T05:56:03.362 INFO:tasks.workunit.client.1.smithi194.stdout:7/120: stat d3/d27/c26 0 2024-04-12T05:56:03.364 INFO:tasks.workunit.client.1.smithi194.stdout:7/121: mknod d3/d27/c2a 0 2024-04-12T05:56:03.370 INFO:tasks.workunit.client.1.smithi194.stdout:7/122: dwrite d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:03.374 INFO:tasks.workunit.client.1.smithi194.stdout:7/123: link d3/d27/c2a d3/d16/d21/c2b 0 2024-04-12T05:56:03.375 INFO:tasks.workunit.client.1.smithi194.stdout:7/124: readlink l1 0 2024-04-12T05:56:03.378 INFO:tasks.workunit.client.1.smithi194.stdout:7/125: rename d3/d27/f28 to d3/d27/f2c 0 2024-04-12T05:56:03.383 INFO:tasks.workunit.client.1.smithi194.stdout:7/126: dwrite d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:03.390 INFO:tasks.workunit.client.1.smithi194.stdout:7/127: write d3/d27/f1f [849567,103996] 0 2024-04-12T05:56:03.397 INFO:tasks.workunit.client.1.smithi194.stdout:7/128: dwrite d3/d27/d22/f29 [0,4194304] 0 2024-04-12T05:56:03.398 INFO:tasks.workunit.client.1.smithi194.stdout:7/129: chown d3/d27/c26 3038370 1 2024-04-12T05:56:03.401 INFO:tasks.workunit.client.1.smithi194.stdout:7/130: link d3/d27/d22/c10 d3/d16/d21/c2d 0 2024-04-12T05:56:03.403 INFO:tasks.workunit.client.1.smithi194.stdout:7/131: creat d3/d27/d22/f2e x:0 0 0 2024-04-12T05:56:03.403 INFO:tasks.workunit.client.1.smithi194.stdout:7/132: chown d3/d27/d22/f29 2641 1 2024-04-12T05:56:03.404 INFO:tasks.workunit.client.1.smithi194.stdout:7/133: rmdir d3/d16/d21 39 2024-04-12T05:56:03.405 INFO:tasks.workunit.client.1.smithi194.stdout:7/134: write d3/d16/f19 [423800,66470] 0 2024-04-12T05:56:03.410 INFO:tasks.workunit.client.1.smithi194.stdout:7/135: rename d3/d16/f1c to d3/d27/f2f 0 2024-04-12T05:56:03.414 INFO:tasks.workunit.client.1.smithi194.stdout:7/136: link d3/d27/c2a d3/d16/c30 0 2024-04-12T05:56:03.447 INFO:tasks.workunit.client.1.smithi194.stdout:6/315: sync 2024-04-12T05:56:03.447 INFO:tasks.workunit.client.1.smithi194.stdout:0/222: sync 2024-04-12T05:56:03.447 INFO:tasks.workunit.client.1.smithi194.stdout:5/350: sync 2024-04-12T05:56:03.447 INFO:tasks.workunit.client.1.smithi194.stdout:9/259: sync 2024-04-12T05:56:03.450 INFO:tasks.workunit.client.1.smithi194.stdout:0/223: mknod d0/d3b/c4d 0 2024-04-12T05:56:03.451 INFO:tasks.workunit.client.1.smithi194.stdout:5/351: fdatasync d1/d4/df/f2e 0 2024-04-12T05:56:03.453 INFO:tasks.workunit.client.1.smithi194.stdout:6/316: creat d3/d34/f5d x:0 0 0 2024-04-12T05:56:03.455 INFO:tasks.workunit.client.1.smithi194.stdout:5/352: symlink d1/d4/d4b/l74 0 2024-04-12T05:56:03.458 INFO:tasks.workunit.client.1.smithi194.stdout:0/224: dwrite d0/f48 [0,4194304] 0 2024-04-12T05:56:03.458 INFO:tasks.workunit.client.1.smithi194.stdout:9/260: getdents d1/d19 0 2024-04-12T05:56:03.460 INFO:tasks.workunit.client.1.smithi194.stdout:6/317: truncate d3/d2f/d37/f45 1509538 0 2024-04-12T05:56:03.460 INFO:tasks.workunit.client.1.smithi194.stdout:5/353: fsync d1/d4/d6/d34/f54 0 2024-04-12T05:56:03.465 INFO:tasks.workunit.client.1.smithi194.stdout:9/261: creat d1/d4e/f55 x:0 0 0 2024-04-12T05:56:03.466 INFO:tasks.workunit.client.1.smithi194.stdout:9/262: chown d1/c15 219066746 1 2024-04-12T05:56:03.466 INFO:tasks.workunit.client.1.smithi194.stdout:5/354: write d1/d4/df/d23/f56 [943107,129192] 0 2024-04-12T05:56:03.469 INFO:tasks.workunit.client.1.smithi194.stdout:9/263: write d1/f46 [952856,83515] 0 2024-04-12T05:56:03.470 INFO:tasks.workunit.client.1.smithi194.stdout:0/225: dwrite d0/da/d1e/f25 [0,4194304] 0 2024-04-12T05:56:03.474 INFO:tasks.workunit.client.1.smithi194.stdout:0/226: write d0/da/d1e/f32 [236899,3001] 0 2024-04-12T05:56:03.476 INFO:tasks.workunit.client.1.smithi194.stdout:5/355: mkdir d1/d4/d75 0 2024-04-12T05:56:03.477 INFO:tasks.workunit.client.1.smithi194.stdout:9/264: write d1/d21/d39/f51 [3710916,106021] 0 2024-04-12T05:56:03.479 INFO:tasks.workunit.client.1.smithi194.stdout:0/227: rename d0/f2 to d0/da/d4a/f4e 0 2024-04-12T05:56:03.480 INFO:tasks.workunit.client.1.smithi194.stdout:5/356: fdatasync d1/d4/f1c 0 2024-04-12T05:56:03.484 INFO:tasks.workunit.client.1.smithi194.stdout:0/228: creat d0/da/d4a/f4f x:0 0 0 2024-04-12T05:56:03.488 INFO:tasks.workunit.client.1.smithi194.stdout:6/318: link d3/d2d/d33/d58/f59 d3/d15/d2b/f5e 0 2024-04-12T05:56:03.494 INFO:tasks.workunit.client.1.smithi194.stdout:6/319: rename d3/d1a/f29 to d3/d2d/d33/d52/d5c/f5f 0 2024-04-12T05:56:03.495 INFO:tasks.workunit.client.1.smithi194.stdout:9/265: link d1/lc d1/d19/d1f/d20/l56 0 2024-04-12T05:56:03.498 INFO:tasks.workunit.client.1.smithi194.stdout:6/320: mknod d3/d2d/d33/d52/d5c/c60 0 2024-04-12T05:56:03.500 INFO:tasks.workunit.client.1.smithi194.stdout:6/321: read d3/d15/f28 [4050377,75282] 0 2024-04-12T05:56:03.501 INFO:tasks.workunit.client.1.smithi194.stdout:9/266: creat d1/d19/d1f/d20/f57 x:0 0 0 2024-04-12T05:56:03.501 INFO:tasks.workunit.client.1.smithi194.stdout:9/267: stat f0 0 2024-04-12T05:56:03.502 INFO:tasks.workunit.client.1.smithi194.stdout:0/229: truncate d0/da/f39 417718 0 2024-04-12T05:56:03.502 INFO:tasks.workunit.client.1.smithi194.stdout:9/268: chown d1/d19 46453 1 2024-04-12T05:56:03.504 INFO:tasks.workunit.client.1.smithi194.stdout:0/230: write d0/da/d1e/f19 [705184,15421] 0 2024-04-12T05:56:03.506 INFO:tasks.workunit.client.1.smithi194.stdout:6/322: truncate d3/d15/d2b/d48/f49 4523620 0 2024-04-12T05:56:03.507 INFO:tasks.workunit.client.1.smithi194.stdout:9/269: link d1/l9 d1/d21/d39/l58 0 2024-04-12T05:56:03.508 INFO:tasks.workunit.client.1.smithi194.stdout:6/323: dread - d3/d15/d2b/f5e zero size 2024-04-12T05:56:03.509 INFO:tasks.workunit.client.1.smithi194.stdout:0/231: rename d0/da/ld to d0/da/d1e/d1a/d24/l50 0 2024-04-12T05:56:03.509 INFO:tasks.workunit.client.1.smithi194.stdout:6/324: write d3/f3b [829818,46468] 0 2024-04-12T05:56:03.515 INFO:tasks.workunit.client.1.smithi194.stdout:0/232: unlink d0/da/d1e/f25 0 2024-04-12T05:56:03.515 INFO:tasks.workunit.client.1.smithi194.stdout:6/325: rmdir d3/d1a 39 2024-04-12T05:56:03.517 INFO:tasks.workunit.client.1.smithi194.stdout:9/270: rename d1/la to d1/d19/d1f/d20/d37/d38/l59 0 2024-04-12T05:56:03.518 INFO:tasks.workunit.client.1.smithi194.stdout:9/271: fsync d1/d21/f3e 0 2024-04-12T05:56:03.519 INFO:tasks.workunit.client.1.smithi194.stdout:9/272: stat d1/c1e 0 2024-04-12T05:56:03.524 INFO:tasks.workunit.client.1.smithi194.stdout:9/273: link d1/c2a d1/d19/d1f/c5a 0 2024-04-12T05:56:03.525 INFO:tasks.workunit.client.1.smithi194.stdout:9/274: chown d1/f1d 125104286 1 2024-04-12T05:56:03.531 INFO:tasks.workunit.client.1.smithi194.stdout:9/275: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:56:03.535 INFO:tasks.workunit.client.1.smithi194.stdout:9/276: creat d1/f5b x:0 0 0 2024-04-12T05:56:03.537 INFO:tasks.workunit.client.1.smithi194.stdout:9/277: creat d1/d19/d1f/d20/d37/d38/f5c x:0 0 0 2024-04-12T05:56:03.546 INFO:tasks.workunit.client.1.smithi194.stdout:9/278: write d1/fe [1750268,97773] 0 2024-04-12T05:56:03.547 INFO:tasks.workunit.client.1.smithi194.stdout:9/279: fdatasync d1/f46 0 2024-04-12T05:56:03.553 INFO:tasks.workunit.client.1.smithi194.stdout:9/280: dwrite d1/f3f [0,4194304] 0 2024-04-12T05:56:03.555 INFO:tasks.workunit.client.1.smithi194.stdout:9/281: symlink d1/d21/d39/l5d 0 2024-04-12T05:56:03.557 INFO:tasks.workunit.client.1.smithi194.stdout:9/282: write d1/d19/f2e [112468,81407] 0 2024-04-12T05:56:03.559 INFO:tasks.workunit.client.1.smithi194.stdout:9/283: unlink d1/l9 0 2024-04-12T05:56:03.560 INFO:tasks.workunit.client.1.smithi194.stdout:9/284: chown d1/d19/d1f/d20/f57 1331 1 2024-04-12T05:56:03.566 INFO:tasks.workunit.client.1.smithi194.stdout:9/285: dwrite f0 [0,4194304] 0 2024-04-12T05:56:03.574 INFO:tasks.workunit.client.1.smithi194.stdout:9/286: dwrite f0 [4194304,4194304] 0 2024-04-12T05:56:03.575 INFO:tasks.workunit.client.1.smithi194.stdout:9/287: write d1/fe [1847438,7789] 0 2024-04-12T05:56:03.579 INFO:tasks.workunit.client.1.smithi194.stdout:9/288: creat d1/d41/f5e x:0 0 0 2024-04-12T05:56:03.580 INFO:tasks.workunit.client.1.smithi194.stdout:9/289: chown d1/fe 45530 1 2024-04-12T05:56:03.588 INFO:tasks.workunit.client.1.smithi194.stdout:9/290: dwrite d1/d19/f54 [0,4194304] 0 2024-04-12T05:56:03.599 INFO:tasks.workunit.client.1.smithi194.stdout:9/291: truncate d1/d19/d1f/d20/f35 283433 0 2024-04-12T05:56:03.601 INFO:tasks.workunit.client.1.smithi194.stdout:9/292: rmdir d1/d21 39 2024-04-12T05:56:03.615 INFO:tasks.workunit.client.1.smithi194.stdout:9/293: dwrite d1/d19/d1f/d20/f35 [0,4194304] 0 2024-04-12T05:56:03.618 INFO:tasks.workunit.client.1.smithi194.stdout:9/294: mkdir d1/d19/d1f/d5f 0 2024-04-12T05:56:03.621 INFO:tasks.workunit.client.1.smithi194.stdout:9/295: mkdir d1/d19/d1f/d20/d37/d60 0 2024-04-12T05:56:03.622 INFO:tasks.workunit.client.1.smithi194.stdout:9/296: fdatasync d1/f5b 0 2024-04-12T05:56:03.623 INFO:tasks.workunit.client.1.smithi194.stdout:9/297: truncate d1/d41/f44 287044 0 2024-04-12T05:56:03.626 INFO:tasks.workunit.client.1.smithi194.stdout:9/298: fdatasync d1/d21/d39/f51 0 2024-04-12T05:56:03.632 INFO:tasks.workunit.client.1.smithi194.stdout:9/299: dwrite d1/d19/d1f/d20/f35 [0,4194304] 0 2024-04-12T05:56:03.635 INFO:tasks.workunit.client.1.smithi194.stdout:9/300: creat d1/d21/f61 x:0 0 0 2024-04-12T05:56:03.638 INFO:tasks.workunit.client.1.smithi194.stdout:9/301: symlink d1/d19/l62 0 2024-04-12T05:56:03.642 INFO:tasks.workunit.client.1.smithi194.stdout:9/302: dread d1/f18 [0,4194304] 0 2024-04-12T05:56:03.642 INFO:tasks.workunit.client.1.smithi194.stdout:9/303: chown d1/d21/d39 11590925 1 2024-04-12T05:56:03.648 INFO:tasks.workunit.client.1.smithi194.stdout:9/304: dwrite d1/d21/f61 [0,4194304] 0 2024-04-12T05:56:03.655 INFO:tasks.workunit.client.1.smithi194.stdout:9/305: mkdir d1/d19/d1f/d63 0 2024-04-12T05:56:03.657 INFO:tasks.workunit.client.1.smithi194.stdout:9/306: unlink d1/d21/d39/l58 0 2024-04-12T05:56:03.658 INFO:tasks.workunit.client.1.smithi194.stdout:9/307: chown d1/d19/d1f/d20/c2d 128737 1 2024-04-12T05:56:03.658 INFO:tasks.workunit.client.1.smithi194.stdout:9/308: stat d1/f3f 0 2024-04-12T05:56:03.660 INFO:tasks.workunit.client.1.smithi194.stdout:9/309: link d1/c3a d1/d19/d1f/d20/d37/d38/c64 0 2024-04-12T05:56:03.661 INFO:tasks.workunit.client.1.smithi194.stdout:9/310: read - d1/d4e/f55 zero size 2024-04-12T05:56:03.663 INFO:tasks.workunit.client.1.smithi194.stdout:9/311: creat d1/d21/d39/f65 x:0 0 0 2024-04-12T05:56:03.666 INFO:tasks.workunit.client.1.smithi194.stdout:9/312: getdents d1/d19/d1f/d20/d37 0 2024-04-12T05:56:03.671 INFO:tasks.workunit.client.1.smithi194.stdout:9/313: dwrite d1/f13 [4194304,4194304] 0 2024-04-12T05:56:03.671 INFO:tasks.workunit.client.1.smithi194.stdout:9/314: fsync d1/d21/f4d 0 2024-04-12T05:56:03.673 INFO:tasks.workunit.client.1.smithi194.stdout:9/315: mknod d1/d19/d1f/c66 0 2024-04-12T05:56:03.681 INFO:tasks.workunit.client.1.smithi194.stdout:9/316: dwrite d1/d19/d1f/d20/d37/d38/f5c [0,4194304] 0 2024-04-12T05:56:03.699 INFO:tasks.workunit.client.1.smithi194.stdout:4/234: sync 2024-04-12T05:56:03.699 INFO:tasks.workunit.client.1.smithi194.stdout:2/295: sync 2024-04-12T05:56:03.699 INFO:tasks.workunit.client.1.smithi194.stdout:3/307: sync 2024-04-12T05:56:03.699 INFO:tasks.workunit.client.1.smithi194.stdout:2/296: truncate d7/d12/f3d 364017 0 2024-04-12T05:56:03.700 INFO:tasks.workunit.client.1.smithi194.stdout:4/235: creat d2/f4c x:0 0 0 2024-04-12T05:56:03.702 INFO:tasks.workunit.client.1.smithi194.stdout:4/236: write d2/d3/d1b/d32/d3a/f3e [81636,95824] 0 2024-04-12T05:56:03.707 INFO:tasks.workunit.client.1.smithi194.stdout:3/308: creat d0/dc/f65 x:0 0 0 2024-04-12T05:56:03.708 INFO:tasks.workunit.client.1.smithi194.stdout:2/297: unlink d7/c18 0 2024-04-12T05:56:03.708 INFO:tasks.workunit.client.1.smithi194.stdout:3/309: read d0/dc/de/f45 [406621,87342] 0 2024-04-12T05:56:03.711 INFO:tasks.workunit.client.1.smithi194.stdout:4/237: rename d2/d3/d1b/d32/f44 to d2/d11/d1e/f4d 0 2024-04-12T05:56:03.712 INFO:tasks.workunit.client.1.smithi194.stdout:2/298: symlink d7/d12/l6c 0 2024-04-12T05:56:03.717 INFO:tasks.workunit.client.1.smithi194.stdout:4/238: fdatasync d2/d3/d1b/d32/d3a/f3e 0 2024-04-12T05:56:03.718 INFO:tasks.workunit.client.1.smithi194.stdout:2/299: dread d7/fa [0,4194304] 0 2024-04-12T05:56:03.719 INFO:tasks.workunit.client.1.smithi194.stdout:3/310: dwrite d0/dc/de/d31/d34/f46 [0,4194304] 0 2024-04-12T05:56:03.720 INFO:tasks.workunit.client.1.smithi194.stdout:9/317: truncate d1/d19/d1f/d20/f3d 321047 0 2024-04-12T05:56:03.725 INFO:tasks.workunit.client.1.smithi194.stdout:9/318: mknod d1/d21/d49/c67 0 2024-04-12T05:56:03.730 INFO:tasks.workunit.client.1.smithi194.stdout:9/319: dread d1/d19/f54 [0,4194304] 0 2024-04-12T05:56:03.733 INFO:tasks.workunit.client.1.smithi194.stdout:2/300: fdatasync d7/d12/f3d 0 2024-04-12T05:56:03.736 INFO:tasks.workunit.client.1.smithi194.stdout:4/239: getdents d2/d3/d1b/d32/d3a/d3b 0 2024-04-12T05:56:03.739 INFO:tasks.workunit.client.1.smithi194.stdout:2/301: mkdir d7/d12/d6d 0 2024-04-12T05:56:03.739 INFO:tasks.workunit.client.1.smithi194.stdout:9/320: dwrite d1/f3f [4194304,4194304] 0 2024-04-12T05:56:03.740 INFO:tasks.workunit.client.1.smithi194.stdout:4/240: dread d2/d11/f39 [0,4194304] 0 2024-04-12T05:56:03.741 INFO:tasks.workunit.client.1.smithi194.stdout:3/311: truncate d0/dc/de/d31/d34/f46 1494571 0 2024-04-12T05:56:03.745 INFO:tasks.workunit.client.1.smithi194.stdout:3/312: fsync d0/d1/d10/f2e 0 2024-04-12T05:56:03.748 INFO:tasks.workunit.client.1.smithi194.stdout:2/302: symlink d7/d12/d25/d5d/d47/l6e 0 2024-04-12T05:56:03.751 INFO:tasks.workunit.client.1.smithi194.stdout:4/241: dread d2/d3/f13 [4194304,4194304] 0 2024-04-12T05:56:03.752 INFO:tasks.workunit.client.1.smithi194.stdout:3/313: creat d0/dc/d3d/f66 x:0 0 0 2024-04-12T05:56:03.753 INFO:tasks.workunit.client.1.smithi194.stdout:3/314: write d0/d1/f5d [226055,36183] 0 2024-04-12T05:56:03.753 INFO:tasks.workunit.client.1.smithi194.stdout:4/242: creat d2/f4e x:0 0 0 2024-04-12T05:56:03.754 INFO:tasks.workunit.client.1.smithi194.stdout:2/303: dread d7/d12/f3d [0,4194304] 0 2024-04-12T05:56:03.756 INFO:tasks.workunit.client.1.smithi194.stdout:3/315: dread - d0/dc/de/d31/f48 zero size 2024-04-12T05:56:03.758 INFO:tasks.workunit.client.1.smithi194.stdout:4/243: mknod d2/d11/d1e/d47/c4f 0 2024-04-12T05:56:03.758 INFO:tasks.workunit.client.1.smithi194.stdout:2/304: stat d7/d12/d34/d4a/l4c 0 2024-04-12T05:56:03.763 INFO:tasks.workunit.client.1.smithi194.stdout:9/321: dread d1/d19/f2e [0,4194304] 0 2024-04-12T05:56:03.764 INFO:tasks.workunit.client.1.smithi194.stdout:9/322: chown d1/d41/f5e 1 1 2024-04-12T05:56:03.765 INFO:tasks.workunit.client.1.smithi194.stdout:3/316: dwrite d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:03.765 INFO:tasks.workunit.client.1.smithi194.stdout:2/305: rename d7/c8 to d7/d12/d1a/d69/c6f 0 2024-04-12T05:56:03.768 INFO:tasks.workunit.client.1.smithi194.stdout:2/306: mkdir d7/d12/d25/d70 0 2024-04-12T05:56:03.774 INFO:tasks.workunit.client.1.smithi194.stdout:2/307: symlink d7/d22/l71 0 2024-04-12T05:56:03.775 INFO:tasks.workunit.client.1.smithi194.stdout:3/317: link d0/dc/l20 d0/dc/de/d31/d34/d2d/l67 0 2024-04-12T05:56:03.776 INFO:tasks.workunit.client.1.smithi194.stdout:2/308: write d7/d12/d25/d5d/f51 [485461,32081] 0 2024-04-12T05:56:03.776 INFO:tasks.workunit.client.1.smithi194.stdout:4/244: write d2/d3/ff [247001,92871] 0 2024-04-12T05:56:03.779 INFO:tasks.workunit.client.1.smithi194.stdout:4/245: chown d2/d3/d1b/d32/d3a/c48 133152365 1 2024-04-12T05:56:03.780 INFO:tasks.workunit.client.1.smithi194.stdout:3/318: truncate d0/dc/de/f45 1301079 0 2024-04-12T05:56:03.782 INFO:tasks.workunit.client.1.smithi194.stdout:9/323: truncate d1/d19/d1f/f4b 412786 0 2024-04-12T05:56:03.783 INFO:tasks.workunit.client.1.smithi194.stdout:9/324: fdatasync d1/d19/d1f/f2c 0 2024-04-12T05:56:03.784 INFO:tasks.workunit.client.1.smithi194.stdout:4/246: unlink d2/d3/d1b/d32/d3a/d3b/l49 0 2024-04-12T05:56:03.785 INFO:tasks.workunit.client.1.smithi194.stdout:3/319: dread d0/fb [4194304,4194304] 0 2024-04-12T05:56:03.786 INFO:tasks.workunit.client.1.smithi194.stdout:9/325: mknod d1/d21/d49/c68 0 2024-04-12T05:56:03.786 INFO:tasks.workunit.client.1.smithi194.stdout:4/247: fdatasync d2/d3/d1b/d32/d3a/f3e 0 2024-04-12T05:56:03.787 INFO:tasks.workunit.client.1.smithi194.stdout:3/320: write d0/dc/f65 [856426,98562] 0 2024-04-12T05:56:03.789 INFO:tasks.workunit.client.1.smithi194.stdout:4/248: mkdir d2/d3/d50 0 2024-04-12T05:56:03.790 INFO:tasks.workunit.client.1.smithi194.stdout:9/326: creat d1/f69 x:0 0 0 2024-04-12T05:56:03.791 INFO:tasks.workunit.client.1.smithi194.stdout:3/321: creat d0/dc/d1e/d33/f68 x:0 0 0 2024-04-12T05:56:03.793 INFO:tasks.workunit.client.1.smithi194.stdout:4/249: rmdir d2/d3/d1b 39 2024-04-12T05:56:03.793 INFO:tasks.workunit.client.1.smithi194.stdout:4/250: chown d2 873566 1 2024-04-12T05:56:03.793 INFO:tasks.workunit.client.1.smithi194.stdout:3/322: dread - d0/dc/d3d/f66 zero size 2024-04-12T05:56:03.794 INFO:tasks.workunit.client.1.smithi194.stdout:2/309: dread d7/d22/d42/f2a [0,4194304] 0 2024-04-12T05:56:03.795 INFO:tasks.workunit.client.1.smithi194.stdout:9/327: chown d1/d19/c34 84937 1 2024-04-12T05:56:03.795 INFO:tasks.workunit.client.1.smithi194.stdout:4/251: stat d2/d3/l8 0 2024-04-12T05:56:03.797 INFO:tasks.workunit.client.1.smithi194.stdout:3/323: creat d0/dc/d3d/f69 x:0 0 0 2024-04-12T05:56:03.798 INFO:tasks.workunit.client.1.smithi194.stdout:4/252: truncate d2/d3/f18 8626467 0 2024-04-12T05:56:03.802 INFO:tasks.workunit.client.1.smithi194.stdout:9/328: dread f0 [0,4194304] 0 2024-04-12T05:56:03.804 INFO:tasks.workunit.client.1.smithi194.stdout:3/324: symlink d0/d1/l6a 0 2024-04-12T05:56:03.807 INFO:tasks.workunit.client.1.smithi194.stdout:4/253: mkdir d2/d3/d51 0 2024-04-12T05:56:03.809 INFO:tasks.workunit.client.1.smithi194.stdout:4/254: write d2/d22/f2a [1339484,109425] 0 2024-04-12T05:56:03.810 INFO:tasks.workunit.client.1.smithi194.stdout:9/329: symlink d1/d19/d1f/l6a 0 2024-04-12T05:56:03.811 INFO:tasks.workunit.client.1.smithi194.stdout:9/330: write d1/d21/f3e [946398,22124] 0 2024-04-12T05:56:03.813 INFO:tasks.workunit.client.1.smithi194.stdout:3/325: creat d0/dc/de/d31/f6b x:0 0 0 2024-04-12T05:56:03.814 INFO:tasks.workunit.client.1.smithi194.stdout:2/310: write d7/d12/f38 [1163491,84963] 0 2024-04-12T05:56:03.815 INFO:tasks.workunit.client.1.smithi194.stdout:4/255: creat d2/d11/f52 x:0 0 0 2024-04-12T05:56:03.819 INFO:tasks.workunit.client.1.smithi194.stdout:2/311: unlink d7/f5b 0 2024-04-12T05:56:03.820 INFO:tasks.workunit.client.1.smithi194.stdout:2/312: readlink d7/l30 0 2024-04-12T05:56:03.823 INFO:tasks.workunit.client.1.smithi194.stdout:3/326: dwrite d0/dc/d1e/f2a [0,4194304] 0 2024-04-12T05:56:03.826 INFO:tasks.workunit.client.1.smithi194.stdout:2/313: read d7/d12/f3b [150736,122486] 0 2024-04-12T05:56:03.826 INFO:tasks.workunit.client.1.smithi194.stdout:4/256: dwrite d2/d11/f45 [0,4194304] 0 2024-04-12T05:56:03.829 INFO:tasks.workunit.client.1.smithi194.stdout:2/314: write d7/d12/f3d [988899,22510] 0 2024-04-12T05:56:03.832 INFO:tasks.workunit.client.1.smithi194.stdout:4/257: rmdir d2 39 2024-04-12T05:56:03.833 INFO:tasks.workunit.client.1.smithi194.stdout:3/327: creat d0/dc/f6c x:0 0 0 2024-04-12T05:56:03.834 INFO:tasks.workunit.client.1.smithi194.stdout:2/315: readlink d7/d12/d1a/d68/l6a 0 2024-04-12T05:56:03.836 INFO:tasks.workunit.client.1.smithi194.stdout:3/328: creat d0/dc/d1e/f6d x:0 0 0 2024-04-12T05:56:03.837 INFO:tasks.workunit.client.1.smithi194.stdout:4/258: symlink d2/d11/d1e/d47/l53 0 2024-04-12T05:56:03.839 INFO:tasks.workunit.client.1.smithi194.stdout:2/316: unlink d7/d12/d25/d5d/d47/l6e 0 2024-04-12T05:56:03.841 INFO:tasks.workunit.client.1.smithi194.stdout:3/329: mkdir d0/dc/d3d/d6e 0 2024-04-12T05:56:03.844 INFO:tasks.workunit.client.1.smithi194.stdout:2/317: rename d7/d22/f2e to d7/d12/d6d/f72 0 2024-04-12T05:56:03.845 INFO:tasks.workunit.client.1.smithi194.stdout:3/330: mknod d0/dc/de/c6f 0 2024-04-12T05:56:03.846 INFO:tasks.workunit.client.1.smithi194.stdout:2/318: mknod d7/d12/d6d/c73 0 2024-04-12T05:56:03.849 INFO:tasks.workunit.client.1.smithi194.stdout:3/331: creat d0/dc/de/d39/f70 x:0 0 0 2024-04-12T05:56:03.850 INFO:tasks.workunit.client.1.smithi194.stdout:3/332: dread - d0/dc/d1e/f5f zero size 2024-04-12T05:56:03.850 INFO:tasks.workunit.client.1.smithi194.stdout:3/333: fdatasync d0/dc/de/d31/d34/f41 0 2024-04-12T05:56:03.860 INFO:tasks.workunit.client.1.smithi194.stdout:2/319: write d7/d22/d42/f2a [1847042,45328] 0 2024-04-12T05:56:03.860 INFO:tasks.workunit.client.1.smithi194.stdout:3/334: dwrite d0/dc/de/d31/f48 [0,4194304] 0 2024-04-12T05:56:03.863 INFO:tasks.workunit.client.1.smithi194.stdout:4/259: dread d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:03.955 INFO:tasks.workunit.client.1.smithi194.stdout:1/374: sync 2024-04-12T05:56:03.955 INFO:tasks.workunit.client.1.smithi194.stdout:8/188: sync 2024-04-12T05:56:03.956 INFO:tasks.workunit.client.1.smithi194.stdout:7/137: sync 2024-04-12T05:56:03.956 INFO:tasks.workunit.client.1.smithi194.stdout:5/357: sync 2024-04-12T05:56:03.956 INFO:tasks.workunit.client.1.smithi194.stdout:7/138: read d3/d27/f1f [588178,26810] 0 2024-04-12T05:56:03.957 INFO:tasks.workunit.client.1.smithi194.stdout:5/358: write d1/d4/d6/db/d15/d26/f4e [239390,20731] 0 2024-04-12T05:56:03.958 INFO:tasks.workunit.client.1.smithi194.stdout:8/189: read d2/d8/d9/de/f37 [711636,57512] 0 2024-04-12T05:56:03.958 INFO:tasks.workunit.client.1.smithi194.stdout:5/359: fsync d1/d4/d6/d6d/f6b 0 2024-04-12T05:56:03.977 INFO:tasks.workunit.client.1.smithi194.stdout:8/190: dwrite d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:03.986 INFO:tasks.workunit.client.1.smithi194.stdout:2/320: link d7/d12/c17 d7/d22/d42/c74 0 2024-04-12T05:56:03.986 INFO:tasks.workunit.client.1.smithi194.stdout:4/260: link d2/ce d2/d11/d46/c54 0 2024-04-12T05:56:03.987 INFO:tasks.workunit.client.1.smithi194.stdout:2/321: chown d7/d12/d25/d5d/d47 25 1 2024-04-12T05:56:03.990 INFO:tasks.workunit.client.1.smithi194.stdout:3/335: getdents d0/dc/de/d31/d34 0 2024-04-12T05:56:03.990 INFO:tasks.workunit.client.1.smithi194.stdout:1/375: getdents d0/d11 0 2024-04-12T05:56:03.991 INFO:tasks.workunit.client.1.smithi194.stdout:7/139: getdents d3/d16 0 2024-04-12T05:56:03.992 INFO:tasks.workunit.client.1.smithi194.stdout:5/360: link d1/d4/d6/d34/f54 d1/d4/d6/f76 0 2024-04-12T05:56:03.995 INFO:tasks.workunit.client.1.smithi194.stdout:2/322: rename d7/d12/d34/c39 to d7/d12/d25/d5d/c75 0 2024-04-12T05:56:03.998 INFO:tasks.workunit.client.1.smithi194.stdout:7/140: mkdir d3/d27/d22/d31 0 2024-04-12T05:56:03.999 INFO:tasks.workunit.client.1.smithi194.stdout:4/261: dwrite d2/d3/f27 [0,4194304] 0 2024-04-12T05:56:04.002 INFO:tasks.workunit.client.1.smithi194.stdout:1/376: dread d0/d11/d18/d19/d1c/f24 [4194304,4194304] 0 2024-04-12T05:56:04.002 INFO:tasks.workunit.client.1.smithi194.stdout:3/336: creat d0/dc/de/d31/d34/f71 x:0 0 0 2024-04-12T05:56:04.004 INFO:tasks.workunit.client.1.smithi194.stdout:7/141: mknod d3/d16/d23/c32 0 2024-04-12T05:56:04.007 INFO:tasks.workunit.client.1.smithi194.stdout:4/262: creat d2/d3/d51/f55 x:0 0 0 2024-04-12T05:56:04.007 INFO:tasks.workunit.client.1.smithi194.stdout:5/361: dread d1/d4/f1a [0,4194304] 0 2024-04-12T05:56:04.010 INFO:tasks.workunit.client.1.smithi194.stdout:7/142: creat d3/d27/d22/d31/f33 x:0 0 0 2024-04-12T05:56:04.016 INFO:tasks.workunit.client.1.smithi194.stdout:5/362: creat d1/d4/d6/d34/d52/f77 x:0 0 0 2024-04-12T05:56:04.020 INFO:tasks.workunit.client.1.smithi194.stdout:5/363: chown d1/d4/d6/db/d2b/f73 7682 1 2024-04-12T05:56:04.023 INFO:tasks.workunit.client.1.smithi194.stdout:3/337: dwrite d0/dc/de/d39/f3b [4194304,4194304] 0 2024-04-12T05:56:04.023 INFO:tasks.workunit.client.1.smithi194.stdout:7/143: dwrite d3/d27/f1b [0,4194304] 0 2024-04-12T05:56:04.025 INFO:tasks.workunit.client.1.smithi194.stdout:1/377: dread d0/f12 [0,4194304] 0 2024-04-12T05:56:04.028 INFO:tasks.workunit.client.1.smithi194.stdout:5/364: fdatasync d1/d4/d6/db/fc 0 2024-04-12T05:56:04.032 INFO:tasks.workunit.client.1.smithi194.stdout:3/338: mknod d0/d1/d10/c72 0 2024-04-12T05:56:04.033 INFO:tasks.workunit.client.1.smithi194.stdout:7/144: truncate d3/d16/f19 1362513 0 2024-04-12T05:56:04.034 INFO:tasks.workunit.client.1.smithi194.stdout:2/323: dread d7/d12/d6d/f72 [0,4194304] 0 2024-04-12T05:56:04.035 INFO:tasks.workunit.client.1.smithi194.stdout:1/378: readlink d0/df/l34 0 2024-04-12T05:56:04.036 INFO:tasks.workunit.client.1.smithi194.stdout:5/365: mknod d1/d4/d6/db/d15/c78 0 2024-04-12T05:56:04.039 INFO:tasks.workunit.client.1.smithi194.stdout:4/263: dwrite d2/d3/f18 [4194304,4194304] 0 2024-04-12T05:56:04.039 INFO:tasks.workunit.client.1.smithi194.stdout:7/145: rename d3/d27/d22/d31/f33 to d3/d16/d23/f34 0 2024-04-12T05:56:04.040 INFO:tasks.workunit.client.1.smithi194.stdout:3/339: truncate d0/f54 711611 0 2024-04-12T05:56:04.043 INFO:tasks.workunit.client.1.smithi194.stdout:3/340: fsync d0/f11 0 2024-04-12T05:56:04.044 INFO:tasks.workunit.client.1.smithi194.stdout:5/366: fsync d1/d4/df/f6a 0 2024-04-12T05:56:04.047 INFO:tasks.workunit.client.1.smithi194.stdout:4/264: chown d2/d3/d1b/d32/d3a/c48 0 1 2024-04-12T05:56:04.047 INFO:tasks.workunit.client.1.smithi194.stdout:1/379: dwrite d0/d11/d18/d35/f3a [0,4194304] 0 2024-04-12T05:56:04.048 INFO:tasks.workunit.client.1.smithi194.stdout:7/146: getdents d3/d16/d21 0 2024-04-12T05:56:04.049 INFO:tasks.workunit.client.1.smithi194.stdout:3/341: truncate d0/dc/de/d39/d4d/f62 132956 0 2024-04-12T05:56:04.057 INFO:tasks.workunit.client.1.smithi194.stdout:1/380: mknod d0/d3e/d54/c6e 0 2024-04-12T05:56:04.057 INFO:tasks.workunit.client.1.smithi194.stdout:3/342: creat d0/d42/f73 x:0 0 0 2024-04-12T05:56:04.058 INFO:tasks.workunit.client.1.smithi194.stdout:7/147: dwrite d3/d27/f17 [0,4194304] 0 2024-04-12T05:56:04.059 INFO:tasks.workunit.client.1.smithi194.stdout:4/265: link d2/d11/d1f/l36 d2/d3/d1b/l56 0 2024-04-12T05:56:04.060 INFO:tasks.workunit.client.1.smithi194.stdout:1/381: dread - d0/d11/d18/f45 zero size 2024-04-12T05:56:04.061 INFO:tasks.workunit.client.1.smithi194.stdout:7/148: write d3/d27/f1d [5051326,99078] 0 2024-04-12T05:56:04.063 INFO:tasks.workunit.client.1.smithi194.stdout:7/149: truncate d3/d16/f19 2180285 0 2024-04-12T05:56:04.064 INFO:tasks.workunit.client.1.smithi194.stdout:3/343: rename d0/dc/de to d0/d1/d74 0 2024-04-12T05:56:04.066 INFO:tasks.workunit.client.1.smithi194.stdout:3/344: chown d0/d1/d74/d31/d34/c29 0 1 2024-04-12T05:56:04.068 INFO:tasks.workunit.client.1.smithi194.stdout:4/266: fdatasync d2/d3/d1b/d32/f3c 0 2024-04-12T05:56:04.069 INFO:tasks.workunit.client.1.smithi194.stdout:1/382: mkdir d0/d11/d18/d35/d6f 0 2024-04-12T05:56:04.071 INFO:tasks.workunit.client.1.smithi194.stdout:3/345: creat d0/dc/d53/f75 x:0 0 0 2024-04-12T05:56:04.075 INFO:tasks.workunit.client.1.smithi194.stdout:1/383: dread d0/f32 [0,4194304] 0 2024-04-12T05:56:04.076 INFO:tasks.workunit.client.1.smithi194.stdout:3/346: mkdir d0/dc/d1e/d33/d76 0 2024-04-12T05:56:04.077 INFO:tasks.workunit.client.1.smithi194.stdout:1/384: fdatasync d0/d11/d18/d19/d1c/f20 0 2024-04-12T05:56:04.078 INFO:tasks.workunit.client.1.smithi194.stdout:1/385: fdatasync d0/d11/d18/d19/d1c/f20 0 2024-04-12T05:56:04.082 INFO:tasks.workunit.client.1.smithi194.stdout:4/267: dread f0 [0,4194304] 0 2024-04-12T05:56:04.082 INFO:tasks.workunit.client.1.smithi194.stdout:3/347: creat d0/dc/d3d/d6e/f77 x:0 0 0 2024-04-12T05:56:04.083 INFO:tasks.workunit.client.1.smithi194.stdout:4/268: dread - d2/f4c zero size 2024-04-12T05:56:04.084 INFO:tasks.workunit.client.1.smithi194.stdout:3/348: write d0/dc/f1f [3803674,52286] 0 2024-04-12T05:56:04.085 INFO:tasks.workunit.client.1.smithi194.stdout:7/150: dread d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:04.088 INFO:tasks.workunit.client.1.smithi194.stdout:1/386: dwrite d0/d11/d18/d19/d1a/d6a/d27/d48/f62 [0,4194304] 0 2024-04-12T05:56:04.089 INFO:tasks.workunit.client.1.smithi194.stdout:4/269: rename d2/d11/d1e/d47 to d2/d22/d57 0 2024-04-12T05:56:04.090 INFO:tasks.workunit.client.1.smithi194.stdout:4/270: write d2/d3/f18 [4075024,6361] 0 2024-04-12T05:56:04.093 INFO:tasks.workunit.client.1.smithi194.stdout:7/151: creat d3/d16/f35 x:0 0 0 2024-04-12T05:56:04.095 INFO:tasks.workunit.client.1.smithi194.stdout:3/349: creat d0/d1/d4f/f78 x:0 0 0 2024-04-12T05:56:04.095 INFO:tasks.workunit.client.1.smithi194.stdout:4/271: write d2/d11/d1e/f4d [245355,33133] 0 2024-04-12T05:56:04.095 INFO:tasks.workunit.client.1.smithi194.stdout:7/152: stat d3/d27/f1d 0 2024-04-12T05:56:04.098 INFO:tasks.workunit.client.1.smithi194.stdout:1/387: dwrite d0/d11/f59 [0,4194304] 0 2024-04-12T05:56:04.100 INFO:tasks.workunit.client.1.smithi194.stdout:4/272: mkdir d2/d3/d58 0 2024-04-12T05:56:04.108 INFO:tasks.workunit.client.1.smithi194.stdout:7/153: dwrite d3/d27/f2f [0,4194304] 0 2024-04-12T05:56:04.110 INFO:tasks.workunit.client.1.smithi194.stdout:7/154: write d3/d16/f1a [3056974,54606] 0 2024-04-12T05:56:04.190 INFO:tasks.workunit.client.1.smithi194.stdout:0/233: sync 2024-04-12T05:56:04.190 INFO:tasks.workunit.client.1.smithi194.stdout:4/273: fdatasync d2/d3/d1b/d32/f3c 0 2024-04-12T05:56:04.191 INFO:tasks.workunit.client.1.smithi194.stdout:0/234: chown d0/da/d1e/d27/f34 488025601 1 2024-04-12T05:56:04.196 INFO:tasks.workunit.client.1.smithi194.stdout:4/274: dread d2/d3/d1b/d32/f38 [0,4194304] 0 2024-04-12T05:56:04.197 INFO:tasks.workunit.client.1.smithi194.stdout:0/235: unlink d0/da/c16 0 2024-04-12T05:56:04.198 INFO:tasks.workunit.client.1.smithi194.stdout:0/236: write d0/f30 [5028535,114883] 0 2024-04-12T05:56:04.199 INFO:tasks.workunit.client.1.smithi194.stdout:0/237: write d0/f48 [857219,11580] 0 2024-04-12T05:56:04.202 INFO:tasks.workunit.client.1.smithi194.stdout:0/238: read d0/da/d1e/d27/f2a [258033,39858] 0 2024-04-12T05:56:04.203 INFO:tasks.workunit.client.1.smithi194.stdout:1/388: write d0/f4 [97157,128140] 0 2024-04-12T05:56:04.204 INFO:tasks.workunit.client.1.smithi194.stdout:4/275: getdents d2/d22 0 2024-04-12T05:56:04.208 INFO:tasks.workunit.client.1.smithi194.stdout:0/239: write d0/da/d4a/f4f [633491,50269] 0 2024-04-12T05:56:04.210 INFO:tasks.workunit.client.1.smithi194.stdout:3/350: dwrite d0/dc/d1e/f2a [4194304,4194304] 0 2024-04-12T05:56:04.212 INFO:tasks.workunit.client.1.smithi194.stdout:1/389: truncate d0/d11/d18/d19/d1c/f22 1417340 0 2024-04-12T05:56:04.213 INFO:tasks.workunit.client.1.smithi194.stdout:3/351: fsync d0/d1/f57 0 2024-04-12T05:56:04.218 INFO:tasks.workunit.client.1.smithi194.stdout:4/276: dwrite d2/d11/f52 [0,4194304] 0 2024-04-12T05:56:04.226 INFO:tasks.workunit.client.1.smithi194.stdout:1/390: dread d0/d11/d18/d19/d1c/f22 [0,4194304] 0 2024-04-12T05:56:04.231 INFO:tasks.workunit.client.1.smithi194.stdout:1/391: symlink d0/d3e/d5b/l70 0 2024-04-12T05:56:04.231 INFO:tasks.workunit.client.1.smithi194.stdout:4/277: dwrite d2/d11/d1e/f4d [0,4194304] 0 2024-04-12T05:56:04.231 INFO:tasks.workunit.client.1.smithi194.stdout:3/352: dwrite d0/fb [4194304,4194304] 0 2024-04-12T05:56:04.233 INFO:tasks.workunit.client.1.smithi194.stdout:3/353: fsync d0/d1/f5d 0 2024-04-12T05:56:04.236 INFO:tasks.workunit.client.1.smithi194.stdout:4/278: symlink d2/l59 0 2024-04-12T05:56:04.239 INFO:tasks.workunit.client.1.smithi194.stdout:1/392: unlink d0/d11/d18/d19/d1a/l13 0 2024-04-12T05:56:04.239 INFO:tasks.workunit.client.1.smithi194.stdout:0/240: dread d0/f14 [0,4194304] 0 2024-04-12T05:56:04.240 INFO:tasks.workunit.client.1.smithi194.stdout:4/279: creat d2/d11/d1e/f5a x:0 0 0 2024-04-12T05:56:04.241 INFO:tasks.workunit.client.1.smithi194.stdout:3/354: mkdir d0/d1/d74/d31/d5b/d64/d79 0 2024-04-12T05:56:04.243 INFO:tasks.workunit.client.1.smithi194.stdout:4/280: symlink d2/d11/d1e/d25/l5b 0 2024-04-12T05:56:04.244 INFO:tasks.workunit.client.1.smithi194.stdout:1/393: symlink d0/d3e/d54/l71 0 2024-04-12T05:56:04.245 INFO:tasks.workunit.client.1.smithi194.stdout:1/394: readlink d0/d11/d18/d19/d1a/l51 0 2024-04-12T05:56:04.247 INFO:tasks.workunit.client.1.smithi194.stdout:0/241: dread d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:04.248 INFO:tasks.workunit.client.1.smithi194.stdout:4/281: rename d2/d3/d1b/d32/f3c to d2/d3/d58/f5c 0 2024-04-12T05:56:04.248 INFO:tasks.workunit.client.1.smithi194.stdout:4/282: readlink d2/l5 0 2024-04-12T05:56:04.248 INFO:tasks.workunit.client.1.smithi194.stdout:3/355: link d0/d1/d74/d31/d34/c4e d0/dc/c7a 0 2024-04-12T05:56:04.249 INFO:tasks.workunit.client.1.smithi194.stdout:1/395: write d0/d11/d18/f2d [3709336,119786] 0 2024-04-12T05:56:04.249 INFO:tasks.workunit.client.1.smithi194.stdout:0/242: rmdir d0/da/d1e/d1a 39 2024-04-12T05:56:04.251 INFO:tasks.workunit.client.1.smithi194.stdout:3/356: creat d0/dc/d1e/f7b x:0 0 0 2024-04-12T05:56:04.252 INFO:tasks.workunit.client.1.smithi194.stdout:4/283: mkdir d2/d11/d1e/d25/d2b/d5d 0 2024-04-12T05:56:04.254 INFO:tasks.workunit.client.1.smithi194.stdout:1/396: read d0/d11/d18/d19/d1a/d6a/d27/d48/f62 [2353139,109896] 0 2024-04-12T05:56:04.256 INFO:tasks.workunit.client.1.smithi194.stdout:3/357: unlink d0/dc/l19 0 2024-04-12T05:56:04.258 INFO:tasks.workunit.client.1.smithi194.stdout:4/284: creat d2/d3/d1b/f5e x:0 0 0 2024-04-12T05:56:04.259 INFO:tasks.workunit.client.1.smithi194.stdout:4/285: stat d2/l7 0 2024-04-12T05:56:04.261 INFO:tasks.workunit.client.1.smithi194.stdout:4/286: chown d2/ce 11 1 2024-04-12T05:56:04.262 INFO:tasks.workunit.client.1.smithi194.stdout:1/397: creat d0/d11/d18/d19/d1a/d6a/d63/f72 x:0 0 0 2024-04-12T05:56:04.266 INFO:tasks.workunit.client.1.smithi194.stdout:1/398: fsync d0/d11/d18/d19/d1a/d6a/f30 0 2024-04-12T05:56:04.266 INFO:tasks.workunit.client.1.smithi194.stdout:3/358: dwrite d0/d1/f9 [0,4194304] 0 2024-04-12T05:56:04.269 INFO:tasks.workunit.client.1.smithi194.stdout:0/243: dread d0/da/d1e/f19 [0,4194304] 0 2024-04-12T05:56:04.274 INFO:tasks.workunit.client.1.smithi194.stdout:4/287: dread d2/d22/f2a [0,4194304] 0 2024-04-12T05:56:04.274 INFO:tasks.workunit.client.1.smithi194.stdout:1/399: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 x:0 0 0 2024-04-12T05:56:04.276 INFO:tasks.workunit.client.1.smithi194.stdout:3/359: dread d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:04.277 INFO:tasks.workunit.client.1.smithi194.stdout:4/288: creat d2/d11/d1e/d25/f5f x:0 0 0 2024-04-12T05:56:04.280 INFO:tasks.workunit.client.1.smithi194.stdout:3/360: mkdir d0/d42/d7c 0 2024-04-12T05:56:04.282 INFO:tasks.workunit.client.1.smithi194.stdout:4/289: rename d2/l15 to d2/d3/d58/l60 0 2024-04-12T05:56:04.283 INFO:tasks.workunit.client.1.smithi194.stdout:1/400: dwrite d0/d3e/d54/f64 [0,4194304] 0 2024-04-12T05:56:04.285 INFO:tasks.workunit.client.1.smithi194.stdout:3/361: rmdir d0/d1/d74/d31/d5b 39 2024-04-12T05:56:04.287 INFO:tasks.workunit.client.1.smithi194.stdout:4/290: mkdir d2/d11/d46/d61 0 2024-04-12T05:56:04.288 INFO:tasks.workunit.client.1.smithi194.stdout:3/362: unlink d0/d1/f9 0 2024-04-12T05:56:04.290 INFO:tasks.workunit.client.1.smithi194.stdout:4/291: truncate d2/d3/d1b/d32/d3a/f3e 1379604 0 2024-04-12T05:56:04.292 INFO:tasks.workunit.client.1.smithi194.stdout:0/244: truncate d0/f1d 2499391 0 2024-04-12T05:56:04.293 INFO:tasks.workunit.client.1.smithi194.stdout:3/363: symlink d0/d1/d74/d31/d5b/l7d 0 2024-04-12T05:56:04.297 INFO:tasks.workunit.client.1.smithi194.stdout:4/292: rename d2/d11/d1e/f4d to d2/d11/d43/f62 0 2024-04-12T05:56:04.297 INFO:tasks.workunit.client.1.smithi194.stdout:1/401: dread d0/d11/d18/d19/f36 [0,4194304] 0 2024-04-12T05:56:04.301 INFO:tasks.workunit.client.1.smithi194.stdout:3/364: dread d0/d1/d74/f3f [4194304,4194304] 0 2024-04-12T05:56:04.301 INFO:tasks.workunit.client.1.smithi194.stdout:1/402: write d0/d11/d18/d19/d1c/f46 [1203417,4540] 0 2024-04-12T05:56:04.303 INFO:tasks.workunit.client.1.smithi194.stdout:3/365: dread - d0/d1/f57 zero size 2024-04-12T05:56:04.304 INFO:tasks.workunit.client.1.smithi194.stdout:0/245: unlink d0/da/d1e/d1a/d24/c3a 0 2024-04-12T05:56:04.305 INFO:tasks.workunit.client.1.smithi194.stdout:4/293: dread d2/d3/d1b/d32/d3a/f3e [0,4194304] 0 2024-04-12T05:56:04.307 INFO:tasks.workunit.client.1.smithi194.stdout:3/366: creat d0/d1/d74/d39/f7e x:0 0 0 2024-04-12T05:56:04.309 INFO:tasks.workunit.client.1.smithi194.stdout:1/403: creat d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 x:0 0 0 2024-04-12T05:56:04.312 INFO:tasks.workunit.client.1.smithi194.stdout:4/294: mkdir d2/d11/d46/d61/d63 0 2024-04-12T05:56:04.317 INFO:tasks.workunit.client.1.smithi194.stdout:4/295: truncate d2/d11/f39 1392282 0 2024-04-12T05:56:04.318 INFO:tasks.workunit.client.1.smithi194.stdout:0/246: rename d0/da/d1e/d27/f34 to d0/da/d1e/d1a/d3d/f51 0 2024-04-12T05:56:04.318 INFO:tasks.workunit.client.1.smithi194.stdout:3/367: dwrite d0/dc/d53/f61 [0,4194304] 0 2024-04-12T05:56:04.320 INFO:tasks.workunit.client.1.smithi194.stdout:0/247: readlink d0/da/l36 0 2024-04-12T05:56:04.324 INFO:tasks.workunit.client.1.smithi194.stdout:0/248: write d0/da/f3e [256621,44596] 0 2024-04-12T05:56:04.324 INFO:tasks.workunit.client.1.smithi194.stdout:1/404: dwrite d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 [0,4194304] 0 2024-04-12T05:56:04.329 INFO:tasks.workunit.client.1.smithi194.stdout:3/368: write d0/f27 [3536821,53920] 0 2024-04-12T05:56:04.332 INFO:tasks.workunit.client.1.smithi194.stdout:0/249: truncate d0/da/d1e/d1a/f31 793192 0 2024-04-12T05:56:04.333 INFO:tasks.workunit.client.1.smithi194.stdout:1/405: mknod d0/d11/d18/d19/d1c/d67/c75 0 2024-04-12T05:56:04.334 INFO:tasks.workunit.client.1.smithi194.stdout:0/250: mkdir d0/d3b/d52 0 2024-04-12T05:56:04.337 INFO:tasks.workunit.client.1.smithi194.stdout:3/369: dwrite d0/d1/d10/f2e [0,4194304] 0 2024-04-12T05:56:04.342 INFO:tasks.workunit.client.1.smithi194.stdout:0/251: dread d0/da/d1e/f19 [0,4194304] 0 2024-04-12T05:56:04.345 INFO:tasks.workunit.client.1.smithi194.stdout:0/252: unlink d0/f10 0 2024-04-12T05:56:04.346 INFO:tasks.workunit.client.1.smithi194.stdout:3/370: dread d0/dc/d1e/f2a [0,4194304] 0 2024-04-12T05:56:04.348 INFO:tasks.workunit.client.1.smithi194.stdout:0/253: creat d0/f53 x:0 0 0 2024-04-12T05:56:04.348 INFO:tasks.workunit.client.1.smithi194.stdout:3/371: dread d0/d1/d74/d39/d4d/f62 [0,4194304] 0 2024-04-12T05:56:04.349 INFO:tasks.workunit.client.1.smithi194.stdout:1/406: truncate d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 2012436 0 2024-04-12T05:56:04.352 INFO:tasks.workunit.client.1.smithi194.stdout:1/407: rmdir d0/d11/d18/d19/d1a/d6a/d27/d6c 39 2024-04-12T05:56:04.354 INFO:tasks.workunit.client.1.smithi194.stdout:1/408: creat d0/d11/d18/f76 x:0 0 0 2024-04-12T05:56:04.357 INFO:tasks.workunit.client.1.smithi194.stdout:3/372: getdents d0/d1/d74 0 2024-04-12T05:56:04.357 INFO:tasks.workunit.client.1.smithi194.stdout:0/254: dwrite d0/da/f3e [0,4194304] 0 2024-04-12T05:56:04.358 INFO:tasks.workunit.client.1.smithi194.stdout:1/409: mknod d0/d11/c77 0 2024-04-12T05:56:04.359 INFO:tasks.workunit.client.1.smithi194.stdout:1/410: read - d0/d11/d18/f76 zero size 2024-04-12T05:56:04.361 INFO:tasks.workunit.client.1.smithi194.stdout:3/373: rmdir d0/dc/d3d/d6e 39 2024-04-12T05:56:04.363 INFO:tasks.workunit.client.1.smithi194.stdout:0/255: mkdir d0/da/d1e/d27/d54 0 2024-04-12T05:56:04.365 INFO:tasks.workunit.client.1.smithi194.stdout:1/411: symlink d0/d3e/d54/l78 0 2024-04-12T05:56:04.366 INFO:tasks.workunit.client.1.smithi194.stdout:1/412: truncate d0/d11/d18/d19/f1f 2425570 0 2024-04-12T05:56:04.368 INFO:tasks.workunit.client.1.smithi194.stdout:3/374: creat d0/f7f x:0 0 0 2024-04-12T05:56:04.368 INFO:tasks.workunit.client.1.smithi194.stdout:0/256: getdents d0/d3b/d52 0 2024-04-12T05:56:04.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/413: unlink d0/d11/d18/f45 0 2024-04-12T05:56:04.371 INFO:tasks.workunit.client.1.smithi194.stdout:1/414: write d0/d11/d18/d35/f3a [3205855,13290] 0 2024-04-12T05:56:04.371 INFO:tasks.workunit.client.1.smithi194.stdout:3/375: mknod d0/d42/c80 0 2024-04-12T05:56:04.372 INFO:tasks.workunit.client.1.smithi194.stdout:0/257: mkdir d0/da/d4a/d55 0 2024-04-12T05:56:04.373 INFO:tasks.workunit.client.1.smithi194.stdout:3/376: chown d0/d1/d74/d31/d34/d2d/l30 14 1 2024-04-12T05:56:04.374 INFO:tasks.workunit.client.1.smithi194.stdout:0/258: mkdir d0/da/d1e/d1a/d24/d56 0 2024-04-12T05:56:04.376 INFO:tasks.workunit.client.1.smithi194.stdout:1/415: creat d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 x:0 0 0 2024-04-12T05:56:04.378 INFO:tasks.workunit.client.1.smithi194.stdout:3/377: rename d0/d1/d74/d31 to d0/dc/d1e/d81 0 2024-04-12T05:56:04.379 INFO:tasks.workunit.client.1.smithi194.stdout:0/259: write d0/da/d1e/d1a/f1c [4705582,96141] 0 2024-04-12T05:56:04.380 INFO:tasks.workunit.client.1.smithi194.stdout:1/416: rmdir d0/d11/d18/d19/d1c 39 2024-04-12T05:56:04.382 INFO:tasks.workunit.client.1.smithi194.stdout:3/378: symlink d0/dc/d1e/d81/d34/l82 0 2024-04-12T05:56:04.384 INFO:tasks.workunit.client.1.smithi194.stdout:1/417: truncate d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 345875 0 2024-04-12T05:56:04.385 INFO:tasks.workunit.client.1.smithi194.stdout:1/418: write d0/d11/d18/d19/f1f [1268560,59714] 0 2024-04-12T05:56:04.387 INFO:tasks.workunit.client.1.smithi194.stdout:3/379: mknod d0/dc/d1e/d81/d5b/d64/c83 0 2024-04-12T05:56:04.389 INFO:tasks.workunit.client.1.smithi194.stdout:1/419: rmdir d0/d11/d18/d19/d1a/d6a/d27 39 2024-04-12T05:56:04.390 INFO:tasks.workunit.client.1.smithi194.stdout:1/420: chown d0/d3e/c4b 14031 1 2024-04-12T05:56:04.395 INFO:tasks.workunit.client.1.smithi194.stdout:0/260: truncate d0/da/d1e/d1a/f44 239893 0 2024-04-12T05:56:04.395 INFO:tasks.workunit.client.1.smithi194.stdout:1/421: fdatasync d0/d11/d18/d19/d1c/f22 0 2024-04-12T05:56:04.397 INFO:tasks.workunit.client.1.smithi194.stdout:3/380: getdents d0/d42 0 2024-04-12T05:56:04.397 INFO:tasks.workunit.client.1.smithi194.stdout:0/261: mknod d0/da/d1e/d27/c57 0 2024-04-12T05:56:04.398 INFO:tasks.workunit.client.1.smithi194.stdout:3/381: read - d0/dc/d1e/d81/d34/f71 zero size 2024-04-12T05:56:04.400 INFO:tasks.workunit.client.1.smithi194.stdout:1/422: mknod d0/d11/c7a 0 2024-04-12T05:56:04.403 INFO:tasks.workunit.client.1.smithi194.stdout:0/262: mkdir d0/da/d1e/d1a/d58 0 2024-04-12T05:56:04.403 INFO:tasks.workunit.client.1.smithi194.stdout:1/423: mkdir d0/d3e/d54/d7b 0 2024-04-12T05:56:04.405 INFO:tasks.workunit.client.1.smithi194.stdout:0/263: write d0/da/d1e/f8 [7998681,15294] 0 2024-04-12T05:56:04.407 INFO:tasks.workunit.client.1.smithi194.stdout:1/424: creat d0/d11/f7c x:0 0 0 2024-04-12T05:56:04.408 INFO:tasks.workunit.client.1.smithi194.stdout:3/382: link d0/d1/d74/d39/f70 d0/dc/d1e/d81/d5b/d64/d79/f84 0 2024-04-12T05:56:04.411 INFO:tasks.workunit.client.1.smithi194.stdout:0/264: rmdir d0/da/d4a/d55 0 2024-04-12T05:56:04.413 INFO:tasks.workunit.client.1.smithi194.stdout:1/425: creat d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f7d x:0 0 0 2024-04-12T05:56:04.413 INFO:tasks.workunit.client.1.smithi194.stdout:3/383: dread d0/d1/d10/f2e [0,4194304] 0 2024-04-12T05:56:04.413 INFO:tasks.workunit.client.1.smithi194.stdout:0/265: symlink d0/da/d1e/d1a/d24/l59 0 2024-04-12T05:56:04.416 INFO:tasks.workunit.client.1.smithi194.stdout:3/384: truncate d0/dc/d1e/d81/d34/d2d/f5a 536995 0 2024-04-12T05:56:04.416 INFO:tasks.workunit.client.1.smithi194.stdout:0/266: write d0/da/d1e/f19 [3047558,84784] 0 2024-04-12T05:56:04.418 INFO:tasks.workunit.client.1.smithi194.stdout:1/426: symlink d0/d11/d18/d19/l7e 0 2024-04-12T05:56:04.419 INFO:tasks.workunit.client.1.smithi194.stdout:1/427: chown d0/d11/d18/d19/d1c/f6b 664 1 2024-04-12T05:56:04.425 INFO:tasks.workunit.client.1.smithi194.stdout:0/267: dwrite d0/da/d1e/f8 [8388608,4194304] 0 2024-04-12T05:56:04.429 INFO:tasks.workunit.client.1.smithi194.stdout:6/326: sync 2024-04-12T05:56:04.430 INFO:tasks.workunit.client.1.smithi194.stdout:1/428: dread d0/d11/d18/d19/d1a/d6a/d27/d48/f62 [0,4194304] 0 2024-04-12T05:56:04.436 INFO:tasks.workunit.client.1.smithi194.stdout:6/327: write d3/f1f [2274899,72023] 0 2024-04-12T05:56:04.437 INFO:tasks.workunit.client.1.smithi194.stdout:3/385: dread d0/d1/d74/f32 [0,4194304] 0 2024-04-12T05:56:04.441 INFO:tasks.workunit.client.1.smithi194.stdout:1/429: dwrite d0/d11/d18/d35/f3a [0,4194304] 0 2024-04-12T05:56:04.446 INFO:tasks.workunit.client.1.smithi194.stdout:0/268: dwrite d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:04.447 INFO:tasks.workunit.client.1.smithi194.stdout:6/328: dwrite d3/d34/f5d [0,4194304] 0 2024-04-12T05:56:04.451 INFO:tasks.workunit.client.1.smithi194.stdout:0/269: symlink d0/d3b/l5a 0 2024-04-12T05:56:04.451 INFO:tasks.workunit.client.1.smithi194.stdout:6/329: mknod d3/d2d/d33/d58/c61 0 2024-04-12T05:56:04.454 INFO:tasks.workunit.client.1.smithi194.stdout:3/386: link d0/d1/d10/f2b d0/d1/d4f/f85 0 2024-04-12T05:56:04.455 INFO:tasks.workunit.client.1.smithi194.stdout:1/430: link d0/d11/f59 d0/df/f7f 0 2024-04-12T05:56:04.455 INFO:tasks.workunit.client.1.smithi194.stdout:3/387: write d0/f7f [853290,3837] 0 2024-04-12T05:56:04.458 INFO:tasks.workunit.client.1.smithi194.stdout:3/388: dread - d0/dc/d53/f75 zero size 2024-04-12T05:56:04.458 INFO:tasks.workunit.client.1.smithi194.stdout:0/270: mkdir d0/da/d1e/d1a/d58/d5b 0 2024-04-12T05:56:04.460 INFO:tasks.workunit.client.1.smithi194.stdout:6/330: chown d3/d1a 0 1 2024-04-12T05:56:04.461 INFO:tasks.workunit.client.1.smithi194.stdout:0/271: mkdir d0/da/d1e/d27/d5c 0 2024-04-12T05:56:04.465 INFO:tasks.workunit.client.1.smithi194.stdout:6/331: mkdir d3/d2d/d33/d52/d62 0 2024-04-12T05:56:04.465 INFO:tasks.workunit.client.1.smithi194.stdout:0/272: truncate d0/da/d1e/d1a/d3d/f51 934135 0 2024-04-12T05:56:04.467 INFO:tasks.workunit.client.1.smithi194.stdout:1/431: dwrite d0/d11/f55 [0,4194304] 0 2024-04-12T05:56:04.473 INFO:tasks.workunit.client.1.smithi194.stdout:6/332: write d3/d2d/f32 [1679684,47439] 0 2024-04-12T05:56:04.473 INFO:tasks.workunit.client.1.smithi194.stdout:1/432: unlink d0/d11/d18/d35/f3a 0 2024-04-12T05:56:04.475 INFO:tasks.workunit.client.1.smithi194.stdout:0/273: dwrite d0/da/d1e/f15 [0,4194304] 0 2024-04-12T05:56:04.476 INFO:tasks.workunit.client.1.smithi194.stdout:0/274: fsync d0/f37 0 2024-04-12T05:56:04.480 INFO:tasks.workunit.client.1.smithi194.stdout:6/333: write d3/d2d/d33/d52/d5c/f5f [4066802,38693] 0 2024-04-12T05:56:04.489 INFO:tasks.workunit.client.1.smithi194.stdout:3/389: dread d0/dc/f1f [0,4194304] 0 2024-04-12T05:56:04.491 INFO:tasks.workunit.client.1.smithi194.stdout:3/390: fsync d0/dc/d1e/d81/d5b/d64/d79/f84 0 2024-04-12T05:56:04.495 INFO:tasks.workunit.client.1.smithi194.stdout:3/391: creat d0/dc/d1e/d81/d5b/f86 x:0 0 0 2024-04-12T05:56:04.498 INFO:tasks.workunit.client.1.smithi194.stdout:0/275: truncate d0/da/d1e/d1a/f1c 4099871 0 2024-04-12T05:56:04.503 INFO:tasks.workunit.client.1.smithi194.stdout:3/392: dwrite d0/dc/d1e/d81/d34/f41 [0,4194304] 0 2024-04-12T05:56:04.504 INFO:tasks.workunit.client.1.smithi194.stdout:0/276: getdents d0/da/d1e/d27/d5c 0 2024-04-12T05:56:04.507 INFO:tasks.workunit.client.1.smithi194.stdout:0/277: symlink d0/da/l5d 0 2024-04-12T05:56:04.507 INFO:tasks.workunit.client.1.smithi194.stdout:3/393: dread d0/d42/f51 [0,4194304] 0 2024-04-12T05:56:04.509 INFO:tasks.workunit.client.1.smithi194.stdout:6/334: dread d3/f1f [0,4194304] 0 2024-04-12T05:56:04.512 INFO:tasks.workunit.client.1.smithi194.stdout:0/278: dread d0/da/d1e/d1a/d3d/f51 [0,4194304] 0 2024-04-12T05:56:04.512 INFO:tasks.workunit.client.1.smithi194.stdout:3/394: unlink d0/d1/d10/c16 0 2024-04-12T05:56:04.513 INFO:tasks.workunit.client.1.smithi194.stdout:3/395: chown d0/d1/d10/f44 247136644 1 2024-04-12T05:56:04.513 INFO:tasks.workunit.client.1.smithi194.stdout:6/335: mkdir d3/d2f/d4e/d63 0 2024-04-12T05:56:04.514 INFO:tasks.workunit.client.1.smithi194.stdout:3/396: chown d0/d42/f73 38 1 2024-04-12T05:56:04.515 INFO:tasks.workunit.client.1.smithi194.stdout:0/279: creat d0/da/d4a/f5e x:0 0 0 2024-04-12T05:56:04.515 INFO:tasks.workunit.client.1.smithi194.stdout:3/397: readlink d0/dc/d1e/d81/d5b/l7d 0 2024-04-12T05:56:04.515 INFO:tasks.workunit.client.1.smithi194.stdout:6/336: fdatasync d3/d2d/d33/d52/f5a 0 2024-04-12T05:56:04.516 INFO:tasks.workunit.client.1.smithi194.stdout:3/398: chown d0/dc/d3d/f69 56600003 1 2024-04-12T05:56:04.518 INFO:tasks.workunit.client.1.smithi194.stdout:3/399: write d0/dc/d1e/d81/d5b/f86 [90714,44231] 0 2024-04-12T05:56:04.520 INFO:tasks.workunit.client.1.smithi194.stdout:3/400: write d0/d1/d74/d39/f7e [629574,74687] 0 2024-04-12T05:56:04.526 INFO:tasks.workunit.client.1.smithi194.stdout:6/337: dwrite d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:04.531 INFO:tasks.workunit.client.1.smithi194.stdout:6/338: mkdir d3/d2f/d4e/d63/d64 0 2024-04-12T05:56:04.531 INFO:tasks.workunit.client.1.smithi194.stdout:3/401: dwrite d0/dc/d53/f75 [0,4194304] 0 2024-04-12T05:56:04.533 INFO:tasks.workunit.client.1.smithi194.stdout:6/339: write d3/f3b [995462,65839] 0 2024-04-12T05:56:04.534 INFO:tasks.workunit.client.1.smithi194.stdout:6/340: readlink d3/l40 0 2024-04-12T05:56:04.534 INFO:tasks.workunit.client.1.smithi194.stdout:3/402: write d0/dc/d1e/d81/d34/f71 [271939,61627] 0 2024-04-12T05:56:04.546 INFO:tasks.workunit.client.1.smithi194.stdout:3/403: truncate d0/d1/f1c 5264020 0 2024-04-12T05:56:04.548 INFO:tasks.workunit.client.1.smithi194.stdout:3/404: write d0/dc/d1e/d81/d5b/f86 [710053,4662] 0 2024-04-12T05:56:04.551 INFO:tasks.workunit.client.1.smithi194.stdout:3/405: mkdir d0/dc/d1e/d81/d5b/d64/d79/d87 0 2024-04-12T05:56:04.556 INFO:tasks.workunit.client.1.smithi194.stdout:3/406: dread d0/dc/d53/f61 [0,4194304] 0 2024-04-12T05:56:04.563 INFO:tasks.workunit.client.1.smithi194.stdout:3/407: write d0/d1/f1c [3166060,58802] 0 2024-04-12T05:56:04.564 INFO:tasks.workunit.client.1.smithi194.stdout:3/408: write d0/dc/d1e/d81/d34/d2d/f3a [1140020,2476] 0 2024-04-12T05:56:04.568 INFO:tasks.workunit.client.1.smithi194.stdout:3/409: dread d0/dc/f1f [4194304,4194304] 0 2024-04-12T05:56:04.568 INFO:tasks.workunit.client.1.smithi194.stdout:3/410: chown d0/d42 11642853 1 2024-04-12T05:56:04.570 INFO:tasks.workunit.client.1.smithi194.stdout:3/411: getdents d0/d1/d74/d39/d4d/d63 0 2024-04-12T05:56:04.571 INFO:tasks.workunit.client.1.smithi194.stdout:3/412: truncate d0/dc/d1e/d81/d34/f37 992433 0 2024-04-12T05:56:04.572 INFO:tasks.workunit.client.1.smithi194.stdout:3/413: write d0/dc/d1e/f6d [107815,96981] 0 2024-04-12T05:56:04.574 INFO:tasks.workunit.client.1.smithi194.stdout:3/414: truncate d0/d1/d74/d39/f70 206711 0 2024-04-12T05:56:04.577 INFO:tasks.workunit.client.1.smithi194.stdout:3/415: rmdir d0/d42/d7c 0 2024-04-12T05:56:04.580 INFO:tasks.workunit.client.1.smithi194.stdout:3/416: link d0/dc/d1e/f2a d0/dc/f88 0 2024-04-12T05:56:04.581 INFO:tasks.workunit.client.1.smithi194.stdout:3/417: mkdir d0/d1/d10/d89 0 2024-04-12T05:56:04.583 INFO:tasks.workunit.client.1.smithi194.stdout:3/418: symlink d0/d1/d74/d39/d4d/l8a 0 2024-04-12T05:56:04.584 INFO:tasks.workunit.client.1.smithi194.stdout:3/419: creat d0/dc/d1e/d81/d5b/f8b x:0 0 0 2024-04-12T05:56:04.593 INFO:tasks.workunit.client.1.smithi194.stdout:3/420: getdents d0/dc/d1e/d81/d5b 0 2024-04-12T05:56:04.594 INFO:tasks.workunit.client.1.smithi194.stdout:3/421: mkdir d0/d1/d10/d8c 0 2024-04-12T05:56:04.596 INFO:tasks.workunit.client.1.smithi194.stdout:3/422: symlink d0/dc/l8d 0 2024-04-12T05:56:04.596 INFO:tasks.workunit.client.1.smithi194.stdout:3/423: readlink d0/d1/d10/l49 0 2024-04-12T05:56:04.599 INFO:tasks.workunit.client.1.smithi194.stdout:3/424: getdents d0/dc/d1e/d81/d34 0 2024-04-12T05:56:04.600 INFO:tasks.workunit.client.1.smithi194.stdout:3/425: fdatasync d0/dc/f12 0 2024-04-12T05:56:04.605 INFO:tasks.workunit.client.1.smithi194.stdout:3/426: dwrite d0/f7f [0,4194304] 0 2024-04-12T05:56:04.609 INFO:tasks.workunit.client.1.smithi194.stdout:3/427: mkdir d0/dc/d8e 0 2024-04-12T05:56:04.612 INFO:tasks.workunit.client.1.smithi194.stdout:3/428: mkdir d0/d1/d10/d8c/d8f 0 2024-04-12T05:56:04.612 INFO:tasks.workunit.client.1.smithi194.stdout:3/429: chown d0/d1/d74/d39/d4d 7 1 2024-04-12T05:56:04.618 INFO:tasks.workunit.client.1.smithi194.stdout:3/430: dwrite d0/dc/d1e/d81/f48 [0,4194304] 0 2024-04-12T05:56:04.623 INFO:tasks.workunit.client.1.smithi194.stdout:3/431: rename d0/d1/f5d to d0/dc/d3d/f90 0 2024-04-12T05:56:04.626 INFO:tasks.workunit.client.1.smithi194.stdout:3/432: creat d0/dc/f91 x:0 0 0 2024-04-12T05:56:04.628 INFO:tasks.workunit.client.1.smithi194.stdout:3/433: unlink d0/d1/d74/c38 0 2024-04-12T05:56:04.639 INFO:tasks.workunit.client.1.smithi194.stdout:3/434: dwrite d0/f54 [0,4194304] 0 2024-04-12T05:56:04.641 INFO:tasks.workunit.client.1.smithi194.stdout:3/435: symlink d0/d1/d10/d89/l92 0 2024-04-12T05:56:04.643 INFO:tasks.workunit.client.1.smithi194.stdout:3/436: symlink d0/d1/d74/d39/d4d/l93 0 2024-04-12T05:56:04.647 INFO:tasks.workunit.client.1.smithi194.stdout:3/437: dread d0/dc/d1e/f2a [0,4194304] 0 2024-04-12T05:56:04.653 INFO:tasks.workunit.client.1.smithi194.stdout:3/438: rename d0/d1/d74/f45 to d0/dc/d3d/f94 0 2024-04-12T05:56:04.656 INFO:tasks.workunit.client.1.smithi194.stdout:3/439: rename d0/c4b to d0/d1/d10/d8c/c95 0 2024-04-12T05:56:04.659 INFO:tasks.workunit.client.1.smithi194.stdout:3/440: truncate d0/dc/f88 5440952 0 2024-04-12T05:56:04.674 INFO:tasks.workunit.client.1.smithi194.stdout:3/441: dread d0/d1/d4f/f85 [0,4194304] 0 2024-04-12T05:56:04.679 INFO:tasks.workunit.client.1.smithi194.stdout:3/442: dwrite d0/dc/d1e/d81/d34/f71 [0,4194304] 0 2024-04-12T05:56:04.685 INFO:tasks.workunit.client.1.smithi194.stdout:3/443: dread d0/d1/d10/f2e [0,4194304] 0 2024-04-12T05:56:04.686 INFO:tasks.workunit.client.1.smithi194.stdout:3/444: fdatasync d0/dc/d53/f61 0 2024-04-12T05:56:04.687 INFO:tasks.workunit.client.1.smithi194.stdout:3/445: write d0/f54 [4354643,85272] 0 2024-04-12T05:56:04.688 INFO:tasks.workunit.client.1.smithi194.stdout:3/446: readlink d0/dc/l8d 0 2024-04-12T05:56:04.695 INFO:tasks.workunit.client.1.smithi194.stdout:3/447: dwrite d0/d1/d4f/f85 [0,4194304] 0 2024-04-12T05:56:04.697 INFO:tasks.workunit.client.1.smithi194.stdout:3/448: unlink d0/d1/f1c 0 2024-04-12T05:56:04.699 INFO:tasks.workunit.client.1.smithi194.stdout:3/449: mknod d0/d1/d10/c96 0 2024-04-12T05:56:04.700 INFO:tasks.workunit.client.1.smithi194.stdout:3/450: truncate d0/d1/d74/f32 4508740 0 2024-04-12T05:56:04.702 INFO:tasks.workunit.client.1.smithi194.stdout:3/451: rename d0/dc/d1e/f7b to d0/d1/f97 0 2024-04-12T05:56:04.706 INFO:tasks.workunit.client.1.smithi194.stdout:3/452: truncate d0/d1/d74/d39/f70 985805 0 2024-04-12T05:56:04.707 INFO:tasks.workunit.client.1.smithi194.stdout:3/453: fsync d0/d42/f51 0 2024-04-12T05:56:04.708 INFO:tasks.workunit.client.1.smithi194.stdout:3/454: fsync d0/d1/d74/d39/f3b 0 2024-04-12T05:56:04.709 INFO:tasks.workunit.client.1.smithi194.stdout:3/455: mknod d0/d1/d74/d39/d4d/c98 0 2024-04-12T05:56:04.710 INFO:tasks.workunit.client.1.smithi194.stdout:3/456: write d0/d1/d74/d39/f7e [1126990,35855] 0 2024-04-12T05:56:04.712 INFO:tasks.workunit.client.1.smithi194.stdout:3/457: mknod d0/d1/d10/d89/c99 0 2024-04-12T05:56:04.714 INFO:tasks.workunit.client.1.smithi194.stdout:3/458: mkdir d0/dc/d1e/d33/d76/d9a 0 2024-04-12T05:56:04.716 INFO:tasks.workunit.client.1.smithi194.stdout:3/459: symlink d0/d1/d4f/l9b 0 2024-04-12T05:56:04.753 INFO:tasks.workunit.client.1.smithi194.stdout:9/331: sync 2024-04-12T05:56:04.753 INFO:tasks.workunit.client.1.smithi194.stdout:2/324: sync 2024-04-12T05:56:04.753 INFO:tasks.workunit.client.1.smithi194.stdout:7/155: sync 2024-04-12T05:56:04.753 INFO:tasks.workunit.client.1.smithi194.stdout:8/191: sync 2024-04-12T05:56:04.754 INFO:tasks.workunit.client.1.smithi194.stdout:5/367: sync 2024-04-12T05:56:04.758 INFO:tasks.workunit.client.1.smithi194.stdout:8/192: chown d2/d8/d9/d1a 159973977 1 2024-04-12T05:56:04.759 INFO:tasks.workunit.client.1.smithi194.stdout:5/368: write d1/d4/d6/db/f19 [4954340,81537] 0 2024-04-12T05:56:04.761 INFO:tasks.workunit.client.1.smithi194.stdout:2/325: symlink d7/d12/d34/d4a/l76 0 2024-04-12T05:56:04.761 INFO:tasks.workunit.client.1.smithi194.stdout:9/332: chown d1/f5 187841980 1 2024-04-12T05:56:04.762 INFO:tasks.workunit.client.1.smithi194.stdout:8/193: truncate d2/d8/d9/de/d38/f3a 782292 0 2024-04-12T05:56:04.763 INFO:tasks.workunit.client.1.smithi194.stdout:5/369: write d1/d4/d6/db/d38/f5a [132628,103256] 0 2024-04-12T05:56:04.769 INFO:tasks.workunit.client.1.smithi194.stdout:7/156: dwrite d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:04.772 INFO:tasks.workunit.client.1.smithi194.stdout:8/194: creat d2/d8/d9/d1a/f3d x:0 0 0 2024-04-12T05:56:04.772 INFO:tasks.workunit.client.1.smithi194.stdout:5/370: dwrite d1/f3 [4194304,4194304] 0 2024-04-12T05:56:04.773 INFO:tasks.workunit.client.1.smithi194.stdout:2/326: link d7/d12/d1a/f3e d7/d22/d42/f77 0 2024-04-12T05:56:04.779 INFO:tasks.workunit.client.1.smithi194.stdout:7/157: mknod d3/d16/d21/c36 0 2024-04-12T05:56:04.779 INFO:tasks.workunit.client.1.smithi194.stdout:2/327: dread d7/d12/f3d [0,4194304] 0 2024-04-12T05:56:04.780 INFO:tasks.workunit.client.1.smithi194.stdout:2/328: chown d7/c21 8000309 1 2024-04-12T05:56:04.782 INFO:tasks.workunit.client.1.smithi194.stdout:5/371: mknod d1/d4/d75/c79 0 2024-04-12T05:56:04.784 INFO:tasks.workunit.client.1.smithi194.stdout:5/372: dread - d1/d4/d6/f68 zero size 2024-04-12T05:56:04.787 INFO:tasks.workunit.client.1.smithi194.stdout:5/373: write d1/d4/d6/f68 [341548,117550] 0 2024-04-12T05:56:04.789 INFO:tasks.workunit.client.1.smithi194.stdout:2/329: rename d7/d12/d34/l67 to d7/d12/d25/d70/l78 0 2024-04-12T05:56:04.790 INFO:tasks.workunit.client.1.smithi194.stdout:7/158: dwrite d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:04.790 INFO:tasks.workunit.client.1.smithi194.stdout:5/374: chown d1/d4/d6/f35 16498764 1 2024-04-12T05:56:04.792 INFO:tasks.workunit.client.1.smithi194.stdout:2/330: rmdir d7/d12/d1a/d68 39 2024-04-12T05:56:04.793 INFO:tasks.workunit.client.1.smithi194.stdout:7/159: write d3/d16/f19 [238555,107536] 0 2024-04-12T05:56:04.797 INFO:tasks.workunit.client.1.smithi194.stdout:3/460: dwrite d0/dc/d1e/f2a [4194304,4194304] 0 2024-04-12T05:56:04.799 INFO:tasks.workunit.client.1.smithi194.stdout:2/331: chown d7/d12/c17 1 1 2024-04-12T05:56:04.808 INFO:tasks.workunit.client.1.smithi194.stdout:5/375: dwrite d1/d4/d6/db/d38/f60 [4194304,4194304] 0 2024-04-12T05:56:04.811 INFO:tasks.workunit.client.1.smithi194.stdout:7/160: dread d3/d16/f19 [0,4194304] 0 2024-04-12T05:56:04.813 INFO:tasks.workunit.client.1.smithi194.stdout:2/332: dwrite f5 [0,4194304] 0 2024-04-12T05:56:04.813 INFO:tasks.workunit.client.1.smithi194.stdout:3/461: mknod d0/d1/d74/d39/d4d/d63/c9c 0 2024-04-12T05:56:04.815 INFO:tasks.workunit.client.1.smithi194.stdout:7/161: write d3/d27/f1f [1214284,91208] 0 2024-04-12T05:56:04.818 INFO:tasks.workunit.client.1.smithi194.stdout:7/162: chown d3/d27/f1f 1600 1 2024-04-12T05:56:04.819 INFO:tasks.workunit.client.1.smithi194.stdout:7/163: chown d3/d27 1 1 2024-04-12T05:56:04.820 INFO:tasks.workunit.client.1.smithi194.stdout:3/462: rename d0/dc/d1e/c36 to d0/dc/d1e/d81/d5b/d64/d79/d87/c9d 0 2024-04-12T05:56:04.821 INFO:tasks.workunit.client.1.smithi194.stdout:3/463: chown d0/f11 250080 1 2024-04-12T05:56:04.821 INFO:tasks.workunit.client.1.smithi194.stdout:5/376: creat d1/d4/d6/db/f7a x:0 0 0 2024-04-12T05:56:04.823 INFO:tasks.workunit.client.1.smithi194.stdout:7/164: mkdir d3/d16/d23/d37 0 2024-04-12T05:56:04.825 INFO:tasks.workunit.client.1.smithi194.stdout:5/377: creat d1/d4/d4b/f7b x:0 0 0 2024-04-12T05:56:04.830 INFO:tasks.workunit.client.1.smithi194.stdout:3/464: getdents d0/d1/d74/d39/d4d 0 2024-04-12T05:56:04.833 INFO:tasks.workunit.client.1.smithi194.stdout:7/165: dwrite d3/d16/d23/f34 [0,4194304] 0 2024-04-12T05:56:04.834 INFO:tasks.workunit.client.1.smithi194.stdout:3/465: unlink d0/d1/l6a 0 2024-04-12T05:56:04.835 INFO:tasks.workunit.client.1.smithi194.stdout:5/378: getdents d1/d4/d6/db/d5b 0 2024-04-12T05:56:04.838 INFO:tasks.workunit.client.1.smithi194.stdout:3/466: write d0/d1/d74/d39/f7e [1585279,68698] 0 2024-04-12T05:56:04.839 INFO:tasks.workunit.client.1.smithi194.stdout:7/166: creat d3/d27/d22/f38 x:0 0 0 2024-04-12T05:56:04.842 INFO:tasks.workunit.client.1.smithi194.stdout:3/467: mkdir d0/d1/d74/d39/d9e 0 2024-04-12T05:56:04.844 INFO:tasks.workunit.client.1.smithi194.stdout:2/333: dwrite d7/d12/d1a/f24 [0,4194304] 0 2024-04-12T05:56:04.845 INFO:tasks.workunit.client.1.smithi194.stdout:3/468: write d0/d1/f97 [692844,121914] 0 2024-04-12T05:56:04.846 INFO:tasks.workunit.client.1.smithi194.stdout:5/379: link d1/d4/df/f69 d1/d4/d6/db/d15/d26/f7c 0 2024-04-12T05:56:04.856 INFO:tasks.workunit.client.1.smithi194.stdout:5/380: link d1/d4/d6/db/d3b/c4c d1/d4/d6/d34/c7d 0 2024-04-12T05:56:04.858 INFO:tasks.workunit.client.1.smithi194.stdout:5/381: truncate d1/d4/d6/f2d 3073330 0 2024-04-12T05:56:04.860 INFO:tasks.workunit.client.1.smithi194.stdout:5/382: mknod d1/d4/d6/db/d38/c7e 0 2024-04-12T05:56:04.862 INFO:tasks.workunit.client.1.smithi194.stdout:5/383: mknod d1/d4/d6/d34/d52/c7f 0 2024-04-12T05:56:04.866 INFO:tasks.workunit.client.1.smithi194.stdout:3/469: dread d0/d1/d74/d39/f3b [0,4194304] 0 2024-04-12T05:56:04.868 INFO:tasks.workunit.client.1.smithi194.stdout:5/384: write d1/d4/d6/d34/f54 [717518,72891] 0 2024-04-12T05:56:04.870 INFO:tasks.workunit.client.1.smithi194.stdout:5/385: write d1/d4/d6/d6d/f6b [725871,45871] 0 2024-04-12T05:56:04.871 INFO:tasks.workunit.client.1.smithi194.stdout:3/470: dread d0/dc/d1e/d81/d5b/d64/d79/f84 [0,4194304] 0 2024-04-12T05:56:04.874 INFO:tasks.workunit.client.1.smithi194.stdout:5/386: write d1/d4/d6/db/d15/f58 [197741,96595] 0 2024-04-12T05:56:04.877 INFO:tasks.workunit.client.1.smithi194.stdout:3/471: dread d0/dc/f88 [4194304,4194304] 0 2024-04-12T05:56:04.881 INFO:tasks.workunit.client.1.smithi194.stdout:3/472: dread d0/dc/d53/f75 [0,4194304] 0 2024-04-12T05:56:04.882 INFO:tasks.workunit.client.1.smithi194.stdout:3/473: write d0/d1/d4f/f78 [470051,13278] 0 2024-04-12T05:56:04.883 INFO:tasks.workunit.client.1.smithi194.stdout:3/474: write d0/dc/d1e/d81/f6b [820262,4965] 0 2024-04-12T05:56:04.887 INFO:tasks.workunit.client.1.smithi194.stdout:3/475: creat d0/d1/d74/f9f x:0 0 0 2024-04-12T05:56:04.887 INFO:tasks.workunit.client.1.smithi194.stdout:5/387: dread d1/d4/f1c [0,4194304] 0 2024-04-12T05:56:04.887 INFO:tasks.workunit.client.1.smithi194.stdout:3/476: readlink d0/d1/d4f/l9b 0 2024-04-12T05:56:04.890 INFO:tasks.workunit.client.1.smithi194.stdout:5/388: rename d1/d4/df/d23/c59 to d1/c80 0 2024-04-12T05:56:04.895 INFO:tasks.workunit.client.1.smithi194.stdout:5/389: dread d1/d4/f5 [4194304,4194304] 0 2024-04-12T05:56:04.899 INFO:tasks.workunit.client.1.smithi194.stdout:5/390: creat d1/d4/d6/db/d5b/f81 x:0 0 0 2024-04-12T05:56:04.903 INFO:tasks.workunit.client.1.smithi194.stdout:5/391: creat d1/f82 x:0 0 0 2024-04-12T05:56:04.907 INFO:tasks.workunit.client.1.smithi194.stdout:5/392: symlink d1/d4/d6/d34/l83 0 2024-04-12T05:56:04.909 INFO:tasks.workunit.client.1.smithi194.stdout:5/393: chown d1/d4/l1b 515 1 2024-04-12T05:56:04.924 INFO:tasks.workunit.client.1.smithi194.stdout:5/394: dwrite d1/d4/d6/f35 [0,4194304] 0 2024-04-12T05:56:04.934 INFO:tasks.workunit.client.1.smithi194.stdout:5/395: getdents d1/d4/d6/db/d5b 0 2024-04-12T05:56:04.936 INFO:tasks.workunit.client.1.smithi194.stdout:5/396: mkdir d1/d4/d84 0 2024-04-12T05:56:04.936 INFO:tasks.workunit.client.1.smithi194.stdout:5/397: fdatasync d1/f82 0 2024-04-12T05:56:04.940 INFO:tasks.workunit.client.1.smithi194.stdout:5/398: read d1/d4/d6/db/f19 [240041,71414] 0 2024-04-12T05:56:04.942 INFO:tasks.workunit.client.1.smithi194.stdout:5/399: creat d1/d4/d75/f85 x:0 0 0 2024-04-12T05:56:04.945 INFO:tasks.workunit.client.1.smithi194.stdout:5/400: unlink d1/c4d 0 2024-04-12T05:56:04.947 INFO:tasks.workunit.client.1.smithi194.stdout:5/401: symlink d1/d4/d4b/l86 0 2024-04-12T05:56:04.981 INFO:tasks.workunit.client.1.smithi194.stdout:4/296: sync 2024-04-12T05:56:04.981 INFO:tasks.workunit.client.1.smithi194.stdout:1/433: sync 2024-04-12T05:56:04.985 INFO:tasks.workunit.client.1.smithi194.stdout:1/434: creat d0/d11/d18/d19/f80 x:0 0 0 2024-04-12T05:56:04.988 INFO:tasks.workunit.client.1.smithi194.stdout:1/435: mknod d0/d11/d18/d19/d1a/d6a/d27/c81 0 2024-04-12T05:56:04.991 INFO:tasks.workunit.client.1.smithi194.stdout:1/436: getdents d0/d11/d18/d19/d1c 0 2024-04-12T05:56:04.993 INFO:tasks.workunit.client.1.smithi194.stdout:4/297: dwrite d2/fc [0,4194304] 0 2024-04-12T05:56:04.994 INFO:tasks.workunit.client.1.smithi194.stdout:1/437: creat d0/df/f82 x:0 0 0 2024-04-12T05:56:04.996 INFO:tasks.workunit.client.1.smithi194.stdout:1/438: creat d0/d11/f83 x:0 0 0 2024-04-12T05:56:04.997 INFO:tasks.workunit.client.1.smithi194.stdout:1/439: read - d0/df/f82 zero size 2024-04-12T05:56:05.001 INFO:tasks.workunit.client.1.smithi194.stdout:4/298: rename d2/d11/f45 to d2/d3/d1b/d32/f64 0 2024-04-12T05:56:05.001 INFO:tasks.workunit.client.1.smithi194.stdout:1/440: write d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 [247615,2873] 0 2024-04-12T05:56:05.002 INFO:tasks.workunit.client.1.smithi194.stdout:4/299: mknod d2/d11/d46/d61/d63/c65 0 2024-04-12T05:56:05.003 INFO:tasks.workunit.client.1.smithi194.stdout:4/300: mknod d2/d11/d46/d61/d63/c66 0 2024-04-12T05:56:05.005 INFO:tasks.workunit.client.1.smithi194.stdout:4/301: creat d2/f67 x:0 0 0 2024-04-12T05:56:05.006 INFO:tasks.workunit.client.1.smithi194.stdout:4/302: mknod d2/d11/c68 0 2024-04-12T05:56:05.007 INFO:tasks.workunit.client.1.smithi194.stdout:4/303: chown d2/l59 68 1 2024-04-12T05:56:05.007 INFO:tasks.workunit.client.1.smithi194.stdout:4/304: dread - d2/f67 zero size 2024-04-12T05:56:05.009 INFO:tasks.workunit.client.1.smithi194.stdout:4/305: write d2/d3/f1a [2012913,104272] 0 2024-04-12T05:56:05.012 INFO:tasks.workunit.client.1.smithi194.stdout:1/441: write d0/d11/d18/d19/d1c/f22 [917042,30421] 0 2024-04-12T05:56:05.012 INFO:tasks.workunit.client.1.smithi194.stdout:1/442: fsync d0/d11/f83 0 2024-04-12T05:56:05.013 INFO:tasks.workunit.client.1.smithi194.stdout:4/306: link d2/d3/d58/f5c d2/d3/d58/f69 0 2024-04-12T05:56:05.015 INFO:tasks.workunit.client.1.smithi194.stdout:1/443: mknod d0/d3e/d50/c84 0 2024-04-12T05:56:05.016 INFO:tasks.workunit.client.1.smithi194.stdout:4/307: mknod d2/d11/d1e/d25/c6a 0 2024-04-12T05:56:05.017 INFO:tasks.workunit.client.1.smithi194.stdout:4/308: symlink d2/d3/d50/l6b 0 2024-04-12T05:56:05.020 INFO:tasks.workunit.client.1.smithi194.stdout:4/309: getdents d2/d3/d1b 0 2024-04-12T05:56:05.021 INFO:tasks.workunit.client.1.smithi194.stdout:4/310: rmdir d2/d3/d51 39 2024-04-12T05:56:05.023 INFO:tasks.workunit.client.1.smithi194.stdout:4/311: symlink d2/d11/l6c 0 2024-04-12T05:56:05.026 INFO:tasks.workunit.client.1.smithi194.stdout:4/312: write f0 [436914,122658] 0 2024-04-12T05:56:05.029 INFO:tasks.workunit.client.1.smithi194.stdout:4/313: link d2/d3/ld d2/d11/d1e/l6d 0 2024-04-12T05:56:05.031 INFO:tasks.workunit.client.1.smithi194.stdout:4/314: creat d2/d11/d1e/d25/f6e x:0 0 0 2024-04-12T05:56:05.032 INFO:tasks.workunit.client.1.smithi194.stdout:4/315: truncate d2/d3/d51/f55 985672 0 2024-04-12T05:56:05.038 INFO:tasks.workunit.client.1.smithi194.stdout:4/316: dwrite d2/d11/d1e/f42 [0,4194304] 0 2024-04-12T05:56:05.041 INFO:tasks.workunit.client.1.smithi194.stdout:4/317: creat d2/d11/d1e/f6f x:0 0 0 2024-04-12T05:56:05.042 INFO:tasks.workunit.client.1.smithi194.stdout:4/318: stat d2/d3/c3d 0 2024-04-12T05:56:05.045 INFO:tasks.workunit.client.1.smithi194.stdout:4/319: creat d2/d11/d46/d61/f70 x:0 0 0 2024-04-12T05:56:05.047 INFO:tasks.workunit.client.1.smithi194.stdout:4/320: dread d2/f21 [0,4194304] 0 2024-04-12T05:56:05.049 INFO:tasks.workunit.client.1.smithi194.stdout:4/321: creat d2/d11/d1e/d2f/f71 x:0 0 0 2024-04-12T05:56:05.052 INFO:tasks.workunit.client.1.smithi194.stdout:4/322: rename d2/d11/c33 to d2/d11/d46/d61/d63/c72 0 2024-04-12T05:56:05.055 INFO:tasks.workunit.client.1.smithi194.stdout:4/323: rename d2/d3/l8 to d2/d11/d1f/l73 0 2024-04-12T05:56:05.057 INFO:tasks.workunit.client.1.smithi194.stdout:4/324: creat d2/d3/f74 x:0 0 0 2024-04-12T05:56:05.058 INFO:tasks.workunit.client.1.smithi194.stdout:4/325: write d2/f4e [755815,44549] 0 2024-04-12T05:56:05.060 INFO:tasks.workunit.client.1.smithi194.stdout:4/326: fsync d2/f4e 0 2024-04-12T05:56:05.064 INFO:tasks.workunit.client.1.smithi194.stdout:4/327: link d2/d22/f34 d2/d3/d1b/d32/d3a/d3b/f75 0 2024-04-12T05:56:05.070 INFO:tasks.workunit.client.1.smithi194.stdout:4/328: dwrite d2/d22/f34 [0,4194304] 0 2024-04-12T05:56:05.071 INFO:tasks.workunit.client.1.smithi194.stdout:4/329: read d2/d22/f2a [1082427,33775] 0 2024-04-12T05:56:05.077 INFO:tasks.workunit.client.1.smithi194.stdout:4/330: link d2/f4e d2/d11/d1e/d2f/f76 0 2024-04-12T05:56:05.077 INFO:tasks.workunit.client.1.smithi194.stdout:4/331: truncate d2/d11/f39 1837554 0 2024-04-12T05:56:05.094 INFO:tasks.workunit.client.1.smithi194.stdout:4/332: dread d2/d3/f27 [0,4194304] 0 2024-04-12T05:56:05.102 INFO:tasks.workunit.client.1.smithi194.stdout:4/333: dwrite d2/d3/d58/f5c [0,4194304] 0 2024-04-12T05:56:05.104 INFO:tasks.workunit.client.1.smithi194.stdout:4/334: dread - d2/d11/d1e/f6f zero size 2024-04-12T05:56:05.107 INFO:tasks.workunit.client.1.smithi194.stdout:4/335: creat d2/d3/f77 x:0 0 0 2024-04-12T05:56:05.109 INFO:tasks.workunit.client.1.smithi194.stdout:4/336: stat d2/d11/d1e/l6d 0 2024-04-12T05:56:05.109 INFO:tasks.workunit.client.1.smithi194.stdout:4/337: readlink d2/d3/d1b/l29 0 2024-04-12T05:56:05.110 INFO:tasks.workunit.client.1.smithi194.stdout:4/338: chown d2/d3/d1b/d32/f38 213983 1 2024-04-12T05:56:05.112 INFO:tasks.workunit.client.1.smithi194.stdout:4/339: creat d2/d11/d46/d61/f78 x:0 0 0 2024-04-12T05:56:05.248 INFO:tasks.workunit.client.1.smithi194.stdout:0/280: sync 2024-04-12T05:56:05.250 INFO:tasks.workunit.client.1.smithi194.stdout:0/281: getdents d0/da/d1e/d1a/d58/d5b 0 2024-04-12T05:56:05.252 INFO:tasks.workunit.client.1.smithi194.stdout:0/282: dread d0/da/d1e/d1a/d3d/f51 [0,4194304] 0 2024-04-12T05:56:05.254 INFO:tasks.workunit.client.1.smithi194.stdout:0/283: truncate d0/da/d1e/d27/f2a 1152229 0 2024-04-12T05:56:05.255 INFO:tasks.workunit.client.1.smithi194.stdout:0/284: chown d0/da/d1e/d1a/f44 123 1 2024-04-12T05:56:05.259 INFO:tasks.workunit.client.1.smithi194.stdout:0/285: dread d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:05.263 INFO:tasks.workunit.client.1.smithi194.stdout:0/286: write d0/f1d [1211034,84387] 0 2024-04-12T05:56:05.264 INFO:tasks.workunit.client.1.smithi194.stdout:0/287: write d0/f30 [848921,34376] 0 2024-04-12T05:56:05.268 INFO:tasks.workunit.client.1.smithi194.stdout:0/288: write d0/da/f13 [1010370,105965] 0 2024-04-12T05:56:05.270 INFO:tasks.workunit.client.1.smithi194.stdout:0/289: mkdir d0/d5f 0 2024-04-12T05:56:05.299 INFO:tasks.workunit.client.1.smithi194.stdout:0/290: getdents d0/d3b 0 2024-04-12T05:56:05.315 INFO:tasks.workunit.client.1.smithi194.stdout:0/291: unlink d0/da/l20 0 2024-04-12T05:56:05.317 INFO:tasks.workunit.client.1.smithi194.stdout:0/292: symlink d0/da/d1e/d1a/d58/l60 0 2024-04-12T05:56:05.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:05 smithi194 ceph-mon[49771]: pgmap v28: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 45 MiB/s wr, 211 op/s 2024-04-12T05:56:05.322 INFO:tasks.workunit.client.1.smithi194.stdout:0/293: dwrite d0/da/d4a/f4f [0,4194304] 0 2024-04-12T05:56:05.323 INFO:tasks.workunit.client.1.smithi194.stdout:0/294: fsync d0/da/d1e/d27/f3f 0 2024-04-12T05:56:05.324 INFO:tasks.workunit.client.1.smithi194.stdout:0/295: chown d0/da/d1e/d1a/d58/d5b 1666871046 1 2024-04-12T05:56:05.329 INFO:tasks.workunit.client.1.smithi194.stdout:0/296: rmdir d0/da/d1e/d27/d5c 0 2024-04-12T05:56:05.330 INFO:tasks.workunit.client.1.smithi194.stdout:0/297: write d0/da/d1e/d1a/f31 [130633,88781] 0 2024-04-12T05:56:05.333 INFO:tasks.workunit.client.1.smithi194.stdout:0/298: unlink d0/da/d1e/c1b 0 2024-04-12T05:56:05.336 INFO:tasks.workunit.client.1.smithi194.stdout:0/299: rename d0/da/d1e/f8 to d0/da/d1e/d1a/d24/f61 0 2024-04-12T05:56:05.344 INFO:tasks.workunit.client.1.smithi194.stdout:0/300: write d0/da/d1e/d27/f2a [959850,5008] 0 2024-04-12T05:56:05.346 INFO:tasks.workunit.client.1.smithi194.stdout:0/301: creat d0/da/d1e/d1a/d24/f62 x:0 0 0 2024-04-12T05:56:05.348 INFO:tasks.workunit.client.1.smithi194.stdout:0/302: symlink d0/da/d1e/d1a/d24/d4b/l63 0 2024-04-12T05:56:05.349 INFO:tasks.workunit.client.1.smithi194.stdout:0/303: readlink d0/da/d1e/d1a/l42 0 2024-04-12T05:56:05.349 INFO:tasks.workunit.client.1.smithi194.stdout:0/304: chown d0/c3 1769931098 1 2024-04-12T05:56:05.351 INFO:tasks.workunit.client.1.smithi194.stdout:0/305: creat d0/da/d1e/d1a/f64 x:0 0 0 2024-04-12T05:56:05.358 INFO:tasks.workunit.client.1.smithi194.stdout:0/306: dwrite d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:05.359 INFO:tasks.workunit.client.1.smithi194.stdout:9/333: sync 2024-04-12T05:56:05.360 INFO:tasks.workunit.client.1.smithi194.stdout:8/195: sync 2024-04-12T05:56:05.360 INFO:tasks.workunit.client.1.smithi194.stdout:6/341: sync 2024-04-12T05:56:05.362 INFO:tasks.workunit.client.1.smithi194.stdout:9/334: chown d1/d21/d39/f65 971 1 2024-04-12T05:56:05.362 INFO:tasks.workunit.client.1.smithi194.stdout:0/307: creat d0/da/d1e/d1a/d24/d4b/f65 x:0 0 0 2024-04-12T05:56:05.362 INFO:tasks.workunit.client.1.smithi194.stdout:8/196: chown d2/d8/d9/d1a/f25 281 1 2024-04-12T05:56:05.366 INFO:tasks.workunit.client.1.smithi194.stdout:6/342: mknod d3/d2d/d33/c65 0 2024-04-12T05:56:05.367 INFO:tasks.workunit.client.1.smithi194.stdout:0/308: chown d0/da/f13 23 1 2024-04-12T05:56:05.367 INFO:tasks.workunit.client.1.smithi194.stdout:9/335: write d1/d19/d1f/f2c [853530,37005] 0 2024-04-12T05:56:05.367 INFO:tasks.workunit.client.1.smithi194.stdout:0/309: write d0/da/d4a/f4f [2192238,124135] 0 2024-04-12T05:56:05.367 INFO:tasks.workunit.client.1.smithi194.stdout:8/197: creat d2/f3e x:0 0 0 2024-04-12T05:56:05.368 INFO:tasks.workunit.client.1.smithi194.stdout:9/336: stat d1/d19/d1f/d20/d37/d38/l59 0 2024-04-12T05:56:05.369 INFO:tasks.workunit.client.1.smithi194.stdout:8/198: read d2/fb [2073300,95879] 0 2024-04-12T05:56:05.371 INFO:tasks.workunit.client.1.smithi194.stdout:0/310: mkdir d0/da/d1e/d1a/d24/d56/d66 0 2024-04-12T05:56:05.373 INFO:tasks.workunit.client.1.smithi194.stdout:9/337: symlink d1/d19/d1f/d20/l6b 0 2024-04-12T05:56:05.374 INFO:tasks.workunit.client.1.smithi194.stdout:6/343: dwrite d3/d15/d2b/d48/f54 [0,4194304] 0 2024-04-12T05:56:05.376 INFO:tasks.workunit.client.1.smithi194.stdout:0/311: symlink d0/l67 0 2024-04-12T05:56:05.376 INFO:tasks.workunit.client.1.smithi194.stdout:9/338: fdatasync d1/d19/f54 0 2024-04-12T05:56:05.378 INFO:tasks.workunit.client.1.smithi194.stdout:9/339: truncate d1/f5b 837953 0 2024-04-12T05:56:05.379 INFO:tasks.workunit.client.1.smithi194.stdout:6/344: rename d3/d2d/d33/d58/c61 to d3/d34/c66 0 2024-04-12T05:56:05.379 INFO:tasks.workunit.client.1.smithi194.stdout:9/340: dread - d1/d21/f4d zero size 2024-04-12T05:56:05.384 INFO:tasks.workunit.client.1.smithi194.stdout:0/312: unlink d0/da/c18 0 2024-04-12T05:56:05.385 INFO:tasks.workunit.client.1.smithi194.stdout:0/313: write d0/f1d [2352398,78944] 0 2024-04-12T05:56:05.388 INFO:tasks.workunit.client.1.smithi194.stdout:9/341: creat d1/d19/f6c x:0 0 0 2024-04-12T05:56:05.389 INFO:tasks.workunit.client.1.smithi194.stdout:8/199: truncate d2/d8/d15/f16 684545 0 2024-04-12T05:56:05.392 INFO:tasks.workunit.client.1.smithi194.stdout:8/200: rmdir d2/d8/d15 39 2024-04-12T05:56:05.393 INFO:tasks.workunit.client.1.smithi194.stdout:0/314: dwrite d0/d3b/f4c [0,4194304] 0 2024-04-12T05:56:05.395 INFO:tasks.workunit.client.1.smithi194.stdout:0/315: fdatasync d0/da/d1e/fe 0 2024-04-12T05:56:05.399 INFO:tasks.workunit.client.1.smithi194.stdout:8/201: creat d2/d8/d15/d35/f3f x:0 0 0 2024-04-12T05:56:05.402 INFO:tasks.workunit.client.1.smithi194.stdout:0/316: link d0/da/d1e/d1a/d24/f62 d0/da/d1e/d1a/d24/d56/d66/f68 0 2024-04-12T05:56:05.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:05 smithi008 ceph-mon[39722]: pgmap v28: 65 pgs: 65 active+clean; 1.0 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 2.3 MiB/s rd, 45 MiB/s wr, 211 op/s 2024-04-12T05:56:05.404 INFO:tasks.workunit.client.1.smithi194.stdout:8/202: unlink d2/d8/d15/d33/f31 0 2024-04-12T05:56:05.405 INFO:tasks.workunit.client.1.smithi194.stdout:8/203: chown d2/d8/f26 1486305 1 2024-04-12T05:56:05.408 INFO:tasks.workunit.client.1.smithi194.stdout:8/204: creat d2/d8/d15/d35/f40 x:0 0 0 2024-04-12T05:56:05.408 INFO:tasks.workunit.client.1.smithi194.stdout:0/317: link d0/da/d4a/c46 d0/d5f/c69 0 2024-04-12T05:56:05.409 INFO:tasks.workunit.client.1.smithi194.stdout:8/205: stat d2/d8/d15/d33/f27 0 2024-04-12T05:56:05.409 INFO:tasks.workunit.client.1.smithi194.stdout:9/342: dread d1/f18 [0,4194304] 0 2024-04-12T05:56:05.411 INFO:tasks.workunit.client.1.smithi194.stdout:8/206: write d2/d8/d15/d33/f27 [5126143,31178] 0 2024-04-12T05:56:05.411 INFO:tasks.workunit.client.1.smithi194.stdout:9/343: write d1/f46 [1745222,93837] 0 2024-04-12T05:56:05.415 INFO:tasks.workunit.client.1.smithi194.stdout:9/344: fdatasync d1/d19/f6c 0 2024-04-12T05:56:05.415 INFO:tasks.workunit.client.1.smithi194.stdout:8/207: chown d2/d8/d9/de/f2e 12237 1 2024-04-12T05:56:05.417 INFO:tasks.workunit.client.1.smithi194.stdout:9/345: creat d1/d4e/f6d x:0 0 0 2024-04-12T05:56:05.418 INFO:tasks.workunit.client.1.smithi194.stdout:8/208: creat d2/d8/d9/de/d10/f41 x:0 0 0 2024-04-12T05:56:05.419 INFO:tasks.workunit.client.1.smithi194.stdout:0/318: truncate d0/da/d1e/f35 292286 0 2024-04-12T05:56:05.420 INFO:tasks.workunit.client.1.smithi194.stdout:0/319: readlink d0/da/d1e/l40 0 2024-04-12T05:56:05.421 INFO:tasks.workunit.client.1.smithi194.stdout:0/320: readlink d0/da/d1e/l40 0 2024-04-12T05:56:05.421 INFO:tasks.workunit.client.1.smithi194.stdout:8/209: rmdir d2/d8/d15/d33 39 2024-04-12T05:56:05.423 INFO:tasks.workunit.client.1.smithi194.stdout:8/210: write d2/d8/d9/d1a/f30 [442440,92490] 0 2024-04-12T05:56:05.423 INFO:tasks.workunit.client.1.smithi194.stdout:9/346: rename d1/d19/d1f/d20/c2d to d1/d19/d1f/d20/d37/d60/c6e 0 2024-04-12T05:56:05.424 INFO:tasks.workunit.client.1.smithi194.stdout:8/211: chown d2/d8/d9/f2a 17334 1 2024-04-12T05:56:05.431 INFO:tasks.workunit.client.1.smithi194.stdout:9/347: rmdir d1/d19/d1f/d63 0 2024-04-12T05:56:05.433 INFO:tasks.workunit.client.1.smithi194.stdout:0/321: truncate d0/f30 4574497 0 2024-04-12T05:56:05.436 INFO:tasks.workunit.client.1.smithi194.stdout:0/322: creat d0/d5f/f6a x:0 0 0 2024-04-12T05:56:05.439 INFO:tasks.workunit.client.1.smithi194.stdout:8/212: dwrite d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:05.442 INFO:tasks.workunit.client.1.smithi194.stdout:8/213: write d2/d8/d9/de/d10/f24 [4810443,36409] 0 2024-04-12T05:56:05.447 INFO:tasks.workunit.client.1.smithi194.stdout:9/348: dwrite d1/d19/d1f/d20/f3d [0,4194304] 0 2024-04-12T05:56:05.453 INFO:tasks.workunit.client.1.smithi194.stdout:9/349: dread d1/d19/d1f/f2c [0,4194304] 0 2024-04-12T05:56:05.456 INFO:tasks.workunit.client.1.smithi194.stdout:9/350: creat d1/d41/f6f x:0 0 0 2024-04-12T05:56:05.458 INFO:tasks.workunit.client.1.smithi194.stdout:9/351: creat d1/d19/d1f/f70 x:0 0 0 2024-04-12T05:56:05.460 INFO:tasks.workunit.client.1.smithi194.stdout:9/352: rmdir d1/d21 39 2024-04-12T05:56:05.460 INFO:tasks.workunit.client.1.smithi194.stdout:9/353: write d1/d41/f6f [641891,2577] 0 2024-04-12T05:56:05.462 INFO:tasks.workunit.client.1.smithi194.stdout:9/354: dread - d1/d21/d39/f65 zero size 2024-04-12T05:56:05.465 INFO:tasks.workunit.client.1.smithi194.stdout:8/214: dread d2/fb [0,4194304] 0 2024-04-12T05:56:05.475 INFO:tasks.workunit.client.1.smithi194.stdout:9/355: dread d1/fe [0,4194304] 0 2024-04-12T05:56:05.476 INFO:tasks.workunit.client.1.smithi194.stdout:8/215: dwrite d2/d8/d9/de/f37 [0,4194304] 0 2024-04-12T05:56:05.477 INFO:tasks.workunit.client.1.smithi194.stdout:8/216: chown d2/d8/d9/d1a/f3d 1 1 2024-04-12T05:56:05.477 INFO:tasks.workunit.client.1.smithi194.stdout:9/356: read d1/d41/f44 [89379,10245] 0 2024-04-12T05:56:05.479 INFO:tasks.workunit.client.1.smithi194.stdout:8/217: mknod d2/d8/d15/d35/c42 0 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:8/218: chown d2/d8/d9/de/d10/l13 0 1 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:2/334: sync 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:5/402: sync 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:3/477: sync 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:4/340: sync 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:7/167: sync 2024-04-12T05:56:05.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/444: sync 2024-04-12T05:56:05.483 INFO:tasks.workunit.client.1.smithi194.stdout:5/403: readlink d1/le 0 2024-04-12T05:56:05.485 INFO:tasks.workunit.client.1.smithi194.stdout:4/341: creat d2/d22/f79 x:0 0 0 2024-04-12T05:56:05.489 INFO:tasks.workunit.client.1.smithi194.stdout:9/357: dwrite d1/d21/d39/f65 [0,4194304] 0 2024-04-12T05:56:05.489 INFO:tasks.workunit.client.1.smithi194.stdout:8/219: truncate d2/d8/d9/f2a 1283793 0 2024-04-12T05:56:05.490 INFO:tasks.workunit.client.1.smithi194.stdout:7/168: creat d3/d27/f39 x:0 0 0 2024-04-12T05:56:05.494 INFO:tasks.workunit.client.1.smithi194.stdout:2/335: read d7/d22/f63 [251871,101971] 0 2024-04-12T05:56:05.494 INFO:tasks.workunit.client.1.smithi194.stdout:3/478: rename d0/d1/d74/f32 to d0/d1/d10/d8c/fa0 0 2024-04-12T05:56:05.494 INFO:tasks.workunit.client.1.smithi194.stdout:9/358: write d1/d4e/f6d [252683,95831] 0 2024-04-12T05:56:05.495 INFO:tasks.workunit.client.1.smithi194.stdout:7/169: write d3/d27/d22/f29 [4746077,40191] 0 2024-04-12T05:56:05.499 INFO:tasks.workunit.client.1.smithi194.stdout:4/342: rename d2/d11/f39 to d2/d11/d1e/d25/f7a 0 2024-04-12T05:56:05.503 INFO:tasks.workunit.client.1.smithi194.stdout:3/479: write d0/dc/d1e/d81/d5b/f86 [615601,18538] 0 2024-04-12T05:56:05.504 INFO:tasks.workunit.client.1.smithi194.stdout:8/220: link d2/d8/f26 d2/f43 0 2024-04-12T05:56:05.508 INFO:tasks.workunit.client.1.smithi194.stdout:9/359: rename d1/d21/d39/f65 to d1/d19/d1f/d20/f71 0 2024-04-12T05:56:05.509 INFO:tasks.workunit.client.1.smithi194.stdout:1/445: dwrite d0/d11/d18/d19/d1a/d6a/d27/d2e/f33 [4194304,4194304] 0 2024-04-12T05:56:05.510 INFO:tasks.workunit.client.1.smithi194.stdout:8/221: write d2/f12 [3582287,92624] 0 2024-04-12T05:56:05.510 INFO:tasks.workunit.client.1.smithi194.stdout:4/343: unlink d2/d11/d46/d61/d63/c65 0 2024-04-12T05:56:05.517 INFO:tasks.workunit.client.1.smithi194.stdout:4/344: dread - d2/d11/d46/d61/f78 zero size 2024-04-12T05:56:05.520 INFO:tasks.workunit.client.1.smithi194.stdout:5/404: truncate d1/d4/df/d23/f56 735859 0 2024-04-12T05:56:05.520 INFO:tasks.workunit.client.1.smithi194.stdout:2/336: dwrite d7/d12/d25/f4f [0,4194304] 0 2024-04-12T05:56:05.524 INFO:tasks.workunit.client.1.smithi194.stdout:7/170: dwrite d3/d27/f39 [0,4194304] 0 2024-04-12T05:56:05.525 INFO:tasks.workunit.client.1.smithi194.stdout:5/405: write d1/d4/d4b/f7b [143368,112745] 0 2024-04-12T05:56:05.526 INFO:tasks.workunit.client.1.smithi194.stdout:1/446: link d0/d11/d18/f2d d0/f85 0 2024-04-12T05:56:05.531 INFO:tasks.workunit.client.1.smithi194.stdout:9/360: mkdir d1/d21/d39/d72 0 2024-04-12T05:56:05.535 INFO:tasks.workunit.client.1.smithi194.stdout:7/171: mkdir d3/d16/d21/d3a 0 2024-04-12T05:56:05.536 INFO:tasks.workunit.client.1.smithi194.stdout:1/447: chown d0/d11/d18/d19/d1a/d6a 35306 1 2024-04-12T05:56:05.536 INFO:tasks.workunit.client.1.smithi194.stdout:8/222: getdents d2/d8/d9/de/d38 0 2024-04-12T05:56:05.537 INFO:tasks.workunit.client.1.smithi194.stdout:9/361: creat d1/d19/d1f/f73 x:0 0 0 2024-04-12T05:56:05.538 INFO:tasks.workunit.client.1.smithi194.stdout:5/406: getdents d1/d4/d84 0 2024-04-12T05:56:05.539 INFO:tasks.workunit.client.1.smithi194.stdout:3/480: dwrite d0/dc/d1e/d81/d34/f71 [0,4194304] 0 2024-04-12T05:56:05.541 INFO:tasks.workunit.client.1.smithi194.stdout:2/337: getdents d7/d12/d25 0 2024-04-12T05:56:05.543 INFO:tasks.workunit.client.1.smithi194.stdout:1/448: creat d0/d3e/d54/d7b/f86 x:0 0 0 2024-04-12T05:56:05.545 INFO:tasks.workunit.client.1.smithi194.stdout:9/362: symlink d1/l74 0 2024-04-12T05:56:05.545 INFO:tasks.workunit.client.1.smithi194.stdout:2/338: read d7/d12/d1a/f44 [3183954,91554] 0 2024-04-12T05:56:05.551 INFO:tasks.workunit.client.1.smithi194.stdout:3/481: fdatasync d0/dc/d1e/d81/d34/f37 0 2024-04-12T05:56:05.552 INFO:tasks.workunit.client.1.smithi194.stdout:7/172: dwrite d3/d16/d23/f34 [0,4194304] 0 2024-04-12T05:56:05.556 INFO:tasks.workunit.client.1.smithi194.stdout:7/173: dread - d3/d27/d22/f38 zero size 2024-04-12T05:56:05.557 INFO:tasks.workunit.client.1.smithi194.stdout:1/449: creat d0/df/d68/f87 x:0 0 0 2024-04-12T05:56:05.560 INFO:tasks.workunit.client.1.smithi194.stdout:1/450: dread - d0/d11/f83 zero size 2024-04-12T05:56:05.561 INFO:tasks.workunit.client.1.smithi194.stdout:5/407: rename d1/d4/d6/l63 to d1/l87 0 2024-04-12T05:56:05.564 INFO:tasks.workunit.client.1.smithi194.stdout:7/174: mkdir d3/d27/d3b 0 2024-04-12T05:56:05.564 INFO:tasks.workunit.client.1.smithi194.stdout:1/451: fsync d0/d11/d18/d19/f4e 0 2024-04-12T05:56:05.565 INFO:tasks.workunit.client.1.smithi194.stdout:9/363: dwrite d1/d19/d1f/d20/f71 [4194304,4194304] 0 2024-04-12T05:56:05.565 INFO:tasks.workunit.client.1.smithi194.stdout:3/482: rename d0/d1/d10/d8c/fa0 to d0/dc/d1e/d81/d34/fa1 0 2024-04-12T05:56:05.567 INFO:tasks.workunit.client.1.smithi194.stdout:3/483: chown d0/dc/d1e/d81/d34/f41 29150675 1 2024-04-12T05:56:05.568 INFO:tasks.workunit.client.1.smithi194.stdout:3/484: write d0/dc/f12 [3973520,8969] 0 2024-04-12T05:56:05.570 INFO:tasks.workunit.client.1.smithi194.stdout:7/175: mknod d3/d16/c3c 0 2024-04-12T05:56:05.572 INFO:tasks.workunit.client.1.smithi194.stdout:7/176: fsync d3/d27/d22/f38 0 2024-04-12T05:56:05.572 INFO:tasks.workunit.client.1.smithi194.stdout:9/364: creat d1/d21/d39/d72/f75 x:0 0 0 2024-04-12T05:56:05.576 INFO:tasks.workunit.client.1.smithi194.stdout:3/485: dread d0/dc/f88 [4194304,4194304] 0 2024-04-12T05:56:05.577 INFO:tasks.workunit.client.1.smithi194.stdout:5/408: getdents d1/d4/df/d23 0 2024-04-12T05:56:05.581 INFO:tasks.workunit.client.1.smithi194.stdout:7/177: mkdir d3/d16/d23/d37/d3d 0 2024-04-12T05:56:05.583 INFO:tasks.workunit.client.1.smithi194.stdout:5/409: unlink d1/d4/d6/d34/d52/f5f 0 2024-04-12T05:56:05.585 INFO:tasks.workunit.client.1.smithi194.stdout:9/365: dwrite d1/d21/d39/d72/f75 [0,4194304] 0 2024-04-12T05:56:05.585 INFO:tasks.workunit.client.1.smithi194.stdout:7/178: write d3/d27/f1d [4261126,9719] 0 2024-04-12T05:56:05.588 INFO:tasks.workunit.client.1.smithi194.stdout:3/486: dread d0/f54 [0,4194304] 0 2024-04-12T05:56:05.592 INFO:tasks.workunit.client.1.smithi194.stdout:7/179: write d3/d27/f1f [2939341,48046] 0 2024-04-12T05:56:05.594 INFO:tasks.workunit.client.1.smithi194.stdout:9/366: rmdir d1/d21/d39 39 2024-04-12T05:56:05.596 INFO:tasks.workunit.client.1.smithi194.stdout:7/180: chown d3/l14 1359974 1 2024-04-12T05:56:05.597 INFO:tasks.workunit.client.1.smithi194.stdout:5/410: rename d1/d4/df/d23 to d1/d4/d6/d88 0 2024-04-12T05:56:05.599 INFO:tasks.workunit.client.1.smithi194.stdout:9/367: truncate d1/d4e/f55 163690 0 2024-04-12T05:56:05.600 INFO:tasks.workunit.client.1.smithi194.stdout:5/411: fdatasync d1/d4/d6/d34/d52/f65 0 2024-04-12T05:56:05.603 INFO:tasks.workunit.client.1.smithi194.stdout:7/181: mknod d3/d16/d23/d37/d3d/c3e 0 2024-04-12T05:56:05.603 INFO:tasks.workunit.client.1.smithi194.stdout:9/368: rmdir d1/d4e 39 2024-04-12T05:56:05.604 INFO:tasks.workunit.client.1.smithi194.stdout:7/182: truncate d3/d27/f2c 5236580 0 2024-04-12T05:56:05.606 INFO:tasks.workunit.client.1.smithi194.stdout:5/412: symlink d1/d4/d6/db/d3b/l89 0 2024-04-12T05:56:05.607 INFO:tasks.workunit.client.1.smithi194.stdout:9/369: chown d1/d21/d39/f51 43 1 2024-04-12T05:56:05.608 INFO:tasks.workunit.client.1.smithi194.stdout:7/183: chown d3/d16/d23/l24 0 1 2024-04-12T05:56:05.609 INFO:tasks.workunit.client.1.smithi194.stdout:5/413: mkdir d1/d4/d6/d8a 0 2024-04-12T05:56:05.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/370: mknod d1/d19/d1f/d5f/c76 0 2024-04-12T05:56:05.611 INFO:tasks.workunit.client.1.smithi194.stdout:7/184: write d3/d16/f19 [1669274,121719] 0 2024-04-12T05:56:05.612 INFO:tasks.workunit.client.1.smithi194.stdout:9/371: truncate d1/d19/f6c 1030879 0 2024-04-12T05:56:05.614 INFO:tasks.workunit.client.1.smithi194.stdout:5/414: rename d1/l7 to d1/d4/d6/db/d15/d26/l8b 0 2024-04-12T05:56:05.615 INFO:tasks.workunit.client.1.smithi194.stdout:5/415: dread - d1/d4/d75/f85 zero size 2024-04-12T05:56:05.617 INFO:tasks.workunit.client.1.smithi194.stdout:9/372: mkdir d1/d21/d77 0 2024-04-12T05:56:05.618 INFO:tasks.workunit.client.1.smithi194.stdout:5/416: mkdir d1/d4/d6/db/d15/d8c 0 2024-04-12T05:56:05.619 INFO:tasks.workunit.client.1.smithi194.stdout:9/373: write d1/f18 [1357172,93915] 0 2024-04-12T05:56:05.621 INFO:tasks.workunit.client.1.smithi194.stdout:5/417: read d1/d4/d6/d88/f56 [61269,102625] 0 2024-04-12T05:56:05.622 INFO:tasks.workunit.client.1.smithi194.stdout:7/185: truncate d3/d27/f2f 1761036 0 2024-04-12T05:56:05.623 INFO:tasks.workunit.client.1.smithi194.stdout:9/374: stat d1/d19/d1f/f4b 0 2024-04-12T05:56:05.630 INFO:tasks.workunit.client.1.smithi194.stdout:5/418: dwrite d1/d4/d6/db/d2b/f33 [0,4194304] 0 2024-04-12T05:56:05.631 INFO:tasks.workunit.client.1.smithi194.stdout:9/375: dwrite d1/d21/d39/f51 [0,4194304] 0 2024-04-12T05:56:05.637 INFO:tasks.workunit.client.1.smithi194.stdout:5/419: rename d1/d4/d6/f2d to d1/d4/d6/db/d15/d8c/f8d 0 2024-04-12T05:56:05.638 INFO:tasks.workunit.client.1.smithi194.stdout:5/420: dread - d1/d4/df/f72 zero size 2024-04-12T05:56:05.639 INFO:tasks.workunit.client.1.smithi194.stdout:5/421: chown d1/d4/d6/d34 4277958 1 2024-04-12T05:56:05.640 INFO:tasks.workunit.client.1.smithi194.stdout:9/376: link d1/d21/d49/c4c d1/d19/d1f/d20/d37/d60/c78 0 2024-04-12T05:56:05.643 INFO:tasks.workunit.client.1.smithi194.stdout:9/377: mknod d1/d4e/c79 0 2024-04-12T05:56:05.644 INFO:tasks.workunit.client.1.smithi194.stdout:7/186: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:05.646 INFO:tasks.workunit.client.1.smithi194.stdout:5/422: rename d1/d4/d6/db/d5b to d1/d4/d6/d8e 0 2024-04-12T05:56:05.647 INFO:tasks.workunit.client.1.smithi194.stdout:7/187: symlink d3/d16/d23/d37/l3f 0 2024-04-12T05:56:05.649 INFO:tasks.workunit.client.1.smithi194.stdout:5/423: unlink d1/d4/d4b/l74 0 2024-04-12T05:56:05.650 INFO:tasks.workunit.client.1.smithi194.stdout:7/188: mkdir d3/d16/d23/d40 0 2024-04-12T05:56:05.651 INFO:tasks.workunit.client.1.smithi194.stdout:9/378: rename d1/d19/d1f/d20/c48 to d1/d19/c7a 0 2024-04-12T05:56:05.652 INFO:tasks.workunit.client.1.smithi194.stdout:9/379: stat d1/f3f 0 2024-04-12T05:56:05.653 INFO:tasks.workunit.client.1.smithi194.stdout:5/424: mknod d1/d5c/c8f 0 2024-04-12T05:56:05.653 INFO:tasks.workunit.client.1.smithi194.stdout:9/380: chown d1/d21/f61 395 1 2024-04-12T05:56:05.655 INFO:tasks.workunit.client.1.smithi194.stdout:5/425: write d1/d4/d6/db/fc [8229291,93357] 0 2024-04-12T05:56:05.656 INFO:tasks.workunit.client.1.smithi194.stdout:9/381: unlink d1/c1a 0 2024-04-12T05:56:05.660 INFO:tasks.workunit.client.1.smithi194.stdout:9/382: dread d1/d19/f6c [0,4194304] 0 2024-04-12T05:56:05.663 INFO:tasks.workunit.client.1.smithi194.stdout:9/383: creat d1/d19/d1f/d20/f7b x:0 0 0 2024-04-12T05:56:05.663 INFO:tasks.workunit.client.1.smithi194.stdout:5/426: rmdir d1/d4/d6/d8e 39 2024-04-12T05:56:05.665 INFO:tasks.workunit.client.1.smithi194.stdout:9/384: creat d1/d19/d1f/d20/d37/d38/f7c x:0 0 0 2024-04-12T05:56:05.671 INFO:tasks.workunit.client.1.smithi194.stdout:7/189: write d3/d27/f1f [53009,95137] 0 2024-04-12T05:56:05.671 INFO:tasks.workunit.client.1.smithi194.stdout:5/427: dwrite d1/d4/d6/db/d38/f61 [0,4194304] 0 2024-04-12T05:56:05.673 INFO:tasks.workunit.client.1.smithi194.stdout:7/190: write d3/d27/f1f [235433,111240] 0 2024-04-12T05:56:05.678 INFO:tasks.workunit.client.1.smithi194.stdout:5/428: fdatasync d1/d4/d6/db/d15/d8c/f8d 0 2024-04-12T05:56:05.678 INFO:tasks.workunit.client.1.smithi194.stdout:5/429: readlink d1/l2 0 2024-04-12T05:56:05.678 INFO:tasks.workunit.client.1.smithi194.stdout:7/191: creat d3/d27/d3b/f41 x:0 0 0 2024-04-12T05:56:05.679 INFO:tasks.workunit.client.1.smithi194.stdout:9/385: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:56:05.682 INFO:tasks.workunit.client.1.smithi194.stdout:5/430: unlink d1/d4/d4b/l86 0 2024-04-12T05:56:05.683 INFO:tasks.workunit.client.1.smithi194.stdout:7/192: symlink d3/d16/d23/d37/d3d/l42 0 2024-04-12T05:56:05.684 INFO:tasks.workunit.client.1.smithi194.stdout:9/386: mkdir d1/d19/d1f/d20/d7d 0 2024-04-12T05:56:05.684 INFO:tasks.workunit.client.1.smithi194.stdout:5/431: chown d1/d4/d6/db/d15/d26/f7c 404108623 1 2024-04-12T05:56:05.686 INFO:tasks.workunit.client.1.smithi194.stdout:5/432: symlink d1/d4/d75/l90 0 2024-04-12T05:56:05.687 INFO:tasks.workunit.client.1.smithi194.stdout:9/387: rename d1/f3f to d1/d4e/f7e 0 2024-04-12T05:56:05.688 INFO:tasks.workunit.client.1.smithi194.stdout:9/388: truncate d1/d4e/f55 508866 0 2024-04-12T05:56:05.699 INFO:tasks.workunit.client.1.smithi194.stdout:5/433: dread d1/d4/f5 [0,4194304] 0 2024-04-12T05:56:05.700 INFO:tasks.workunit.client.1.smithi194.stdout:5/434: truncate d1/d4/d6/d6d/f6b 1099079 0 2024-04-12T05:56:05.701 INFO:tasks.workunit.client.1.smithi194.stdout:5/435: read - d1/d4/d6/db/f7a zero size 2024-04-12T05:56:05.707 INFO:tasks.workunit.client.1.smithi194.stdout:5/436: dwrite d1/f3 [4194304,4194304] 0 2024-04-12T05:56:05.721 INFO:tasks.workunit.client.1.smithi194.stdout:6/345: sync 2024-04-12T05:56:05.727 INFO:tasks.workunit.client.1.smithi194.stdout:9/389: truncate d1/d19/f6c 506489 0 2024-04-12T05:56:05.730 INFO:tasks.workunit.client.1.smithi194.stdout:6/346: link d3/d1a/f35 d3/d2f/d4e/d63/f67 0 2024-04-12T05:56:05.734 INFO:tasks.workunit.client.1.smithi194.stdout:7/193: truncate d3/d27/f7 757928 0 2024-04-12T05:56:05.737 INFO:tasks.workunit.client.1.smithi194.stdout:5/437: truncate d1/d4/d6/db/d15/d8c/f8d 634092 0 2024-04-12T05:56:05.740 INFO:tasks.workunit.client.1.smithi194.stdout:9/390: dread d1/d19/f6c [0,4194304] 0 2024-04-12T05:56:05.741 INFO:tasks.workunit.client.1.smithi194.stdout:5/438: unlink d1/d4/d6/db/d2b/f44 0 2024-04-12T05:56:05.746 INFO:tasks.workunit.client.1.smithi194.stdout:5/439: symlink d1/d4/d6/d6d/l91 0 2024-04-12T05:56:05.747 INFO:tasks.workunit.client.1.smithi194.stdout:5/440: write d1/f3 [6996745,6413] 0 2024-04-12T05:56:05.748 INFO:tasks.workunit.client.1.smithi194.stdout:9/391: mknod d1/d19/d1f/d20/d7d/c7f 0 2024-04-12T05:56:05.748 INFO:tasks.workunit.client.1.smithi194.stdout:9/392: fsync d1/d19/d1f/f73 0 2024-04-12T05:56:05.753 INFO:tasks.workunit.client.1.smithi194.stdout:5/441: rename d1/d4 to d1/d4/d92 22 2024-04-12T05:56:05.755 INFO:tasks.workunit.client.1.smithi194.stdout:9/393: write d1/d19/d1f/f2c [853009,53256] 0 2024-04-12T05:56:05.758 INFO:tasks.workunit.client.1.smithi194.stdout:7/194: truncate d3/d27/f7 58780 0 2024-04-12T05:56:05.761 INFO:tasks.workunit.client.1.smithi194.stdout:9/394: write d1/d19/d1f/f4b [183968,120745] 0 2024-04-12T05:56:05.766 INFO:tasks.workunit.client.1.smithi194.stdout:7/195: dwrite d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:05.772 INFO:tasks.workunit.client.1.smithi194.stdout:9/395: dwrite d1/d19/d1f/d20/f3d [0,4194304] 0 2024-04-12T05:56:05.776 INFO:tasks.workunit.client.1.smithi194.stdout:7/196: dwrite d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:05.779 INFO:tasks.workunit.client.1.smithi194.stdout:7/197: mkdir d3/d16/d21/d3a/d43 0 2024-04-12T05:56:05.782 INFO:tasks.workunit.client.1.smithi194.stdout:9/396: write d1/f1d [1049356,114894] 0 2024-04-12T05:56:05.783 INFO:tasks.workunit.client.1.smithi194.stdout:7/198: creat d3/d27/f44 x:0 0 0 2024-04-12T05:56:05.788 INFO:tasks.workunit.client.1.smithi194.stdout:7/199: unlink d3/d16/c30 0 2024-04-12T05:56:05.789 INFO:tasks.workunit.client.1.smithi194.stdout:9/397: dwrite d1/d21/f61 [0,4194304] 0 2024-04-12T05:56:05.791 INFO:tasks.workunit.client.1.smithi194.stdout:7/200: creat d3/d16/f45 x:0 0 0 2024-04-12T05:56:05.791 INFO:tasks.workunit.client.1.smithi194.stdout:9/398: write d1/f46 [2374461,116627] 0 2024-04-12T05:56:05.799 INFO:tasks.workunit.client.1.smithi194.stdout:7/201: creat d3/d27/f46 x:0 0 0 2024-04-12T05:56:05.801 INFO:tasks.workunit.client.1.smithi194.stdout:9/399: dwrite d1/f5b [0,4194304] 0 2024-04-12T05:56:05.804 INFO:tasks.workunit.client.1.smithi194.stdout:9/400: read d1/d41/f44 [174101,59064] 0 2024-04-12T05:56:05.805 INFO:tasks.workunit.client.1.smithi194.stdout:7/202: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:05.807 INFO:tasks.workunit.client.1.smithi194.stdout:9/401: symlink d1/d19/l80 0 2024-04-12T05:56:05.808 INFO:tasks.workunit.client.1.smithi194.stdout:7/203: mkdir d3/d16/d21/d3a/d47 0 2024-04-12T05:56:05.809 INFO:tasks.workunit.client.1.smithi194.stdout:9/402: dread d1/d41/f44 [0,4194304] 0 2024-04-12T05:56:05.814 INFO:tasks.workunit.client.1.smithi194.stdout:7/204: dwrite d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:05.823 INFO:tasks.workunit.client.1.smithi194.stdout:7/205: write d3/d27/fa [704169,118930] 0 2024-04-12T05:56:05.824 INFO:tasks.workunit.client.1.smithi194.stdout:7/206: read - d3/d16/f45 zero size 2024-04-12T05:56:05.827 INFO:tasks.workunit.client.1.smithi194.stdout:7/207: creat d3/f48 x:0 0 0 2024-04-12T05:56:05.828 INFO:tasks.workunit.client.1.smithi194.stdout:7/208: fdatasync d3/d27/f1f 0 2024-04-12T05:56:05.829 INFO:tasks.workunit.client.1.smithi194.stdout:7/209: chown d3/d27/f2f 0 1 2024-04-12T05:56:05.832 INFO:tasks.workunit.client.1.smithi194.stdout:9/403: dread d1/d19/d1f/d20/f35 [0,4194304] 0 2024-04-12T05:56:05.834 INFO:tasks.workunit.client.1.smithi194.stdout:9/404: read d1/f1d [1276674,40834] 0 2024-04-12T05:56:05.835 INFO:tasks.workunit.client.1.smithi194.stdout:7/210: getdents d3 0 2024-04-12T05:56:05.835 INFO:tasks.workunit.client.1.smithi194.stdout:9/405: read d1/d21/d39/d72/f75 [1394157,112905] 0 2024-04-12T05:56:05.838 INFO:tasks.workunit.client.1.smithi194.stdout:9/406: mknod d1/d21/d77/c81 0 2024-04-12T05:56:05.839 INFO:tasks.workunit.client.1.smithi194.stdout:9/407: rmdir d1/d21 39 2024-04-12T05:56:05.839 INFO:tasks.workunit.client.1.smithi194.stdout:7/211: dread d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:05.842 INFO:tasks.workunit.client.1.smithi194.stdout:9/408: symlink d1/d21/d39/d72/l82 0 2024-04-12T05:56:05.847 INFO:tasks.workunit.client.1.smithi194.stdout:7/212: dwrite d3/d27/d22/f29 [4194304,4194304] 0 2024-04-12T05:56:05.849 INFO:tasks.workunit.client.1.smithi194.stdout:7/213: write d3/d27/d22/f38 [281074,89010] 0 2024-04-12T05:56:05.850 INFO:tasks.workunit.client.1.smithi194.stdout:9/409: dwrite d1/f18 [0,4194304] 0 2024-04-12T05:56:05.853 INFO:tasks.workunit.client.1.smithi194.stdout:9/410: rmdir d1/d19/d1f/d20/d7d 39 2024-04-12T05:56:05.854 INFO:tasks.workunit.client.1.smithi194.stdout:7/214: mkdir d3/d16/d23/d40/d49 0 2024-04-12T05:56:05.855 INFO:tasks.workunit.client.1.smithi194.stdout:9/411: fsync d1/d19/f2e 0 2024-04-12T05:56:05.856 INFO:tasks.workunit.client.1.smithi194.stdout:7/215: write d3/d16/f19 [1669929,119685] 0 2024-04-12T05:56:05.858 INFO:tasks.workunit.client.1.smithi194.stdout:7/216: write d3/d27/f44 [784459,114742] 0 2024-04-12T05:56:05.859 INFO:tasks.workunit.client.1.smithi194.stdout:7/217: fdatasync d3/d27/f1f 0 2024-04-12T05:56:05.859 INFO:tasks.workunit.client.1.smithi194.stdout:7/218: readlink d3/d27/l20 0 2024-04-12T05:56:05.862 INFO:tasks.workunit.client.1.smithi194.stdout:7/219: symlink d3/d27/d3b/l4a 0 2024-04-12T05:56:05.924 INFO:tasks.workunit.client.1.smithi194.stdout:0/323: sync 2024-04-12T05:56:05.948 INFO:tasks.workunit.client.1.smithi194.stdout:7/220: chown d3/d16/d21/c36 65245 1 2024-04-12T05:56:05.949 INFO:tasks.workunit.client.1.smithi194.stdout:9/412: dwrite d1/f5 [4194304,4194304] 0 2024-04-12T05:56:05.951 INFO:tasks.workunit.client.1.smithi194.stdout:0/324: chown d0/da/d4a 185658290 1 2024-04-12T05:56:05.953 INFO:tasks.workunit.client.1.smithi194.stdout:0/325: fdatasync d0/da/d1e/fe 0 2024-04-12T05:56:05.957 INFO:tasks.workunit.client.1.smithi194.stdout:9/413: unlink d1/lc 0 2024-04-12T05:56:05.960 INFO:tasks.workunit.client.1.smithi194.stdout:0/326: mknod d0/da/d1e/d1a/d24/d56/d66/c6b 0 2024-04-12T05:56:05.960 INFO:tasks.workunit.client.1.smithi194.stdout:9/414: readlink d1/d19/d1f/d20/l23 0 2024-04-12T05:56:05.963 INFO:tasks.workunit.client.1.smithi194.stdout:0/327: truncate d0/f1d 2704084 0 2024-04-12T05:56:05.963 INFO:tasks.workunit.client.1.smithi194.stdout:7/221: rmdir d3/d27/d3b 39 2024-04-12T05:56:05.964 INFO:tasks.workunit.client.1.smithi194.stdout:9/415: chown d1/d19/d1f/d20/d37/d38 1505 1 2024-04-12T05:56:05.964 INFO:tasks.workunit.client.1.smithi194.stdout:7/222: stat d3/d16/c25 0 2024-04-12T05:56:05.965 INFO:tasks.workunit.client.1.smithi194.stdout:7/223: readlink d3/d16/d23/l24 0 2024-04-12T05:56:05.965 INFO:tasks.workunit.client.1.smithi194.stdout:7/224: chown d3/d27/d22 8036 1 2024-04-12T05:56:05.966 INFO:tasks.workunit.client.1.smithi194.stdout:9/416: unlink d1/f13 0 2024-04-12T05:56:05.969 INFO:tasks.workunit.client.1.smithi194.stdout:7/225: symlink d3/d16/d23/d40/d49/l4b 0 2024-04-12T05:56:05.970 INFO:tasks.workunit.client.1.smithi194.stdout:9/417: creat d1/d19/f83 x:0 0 0 2024-04-12T05:56:05.971 INFO:tasks.workunit.client.1.smithi194.stdout:7/226: mknod d3/d16/d23/c4c 0 2024-04-12T05:56:05.972 INFO:tasks.workunit.client.1.smithi194.stdout:9/418: chown d1/d41/c42 3 1 2024-04-12T05:56:05.974 INFO:tasks.workunit.client.1.smithi194.stdout:9/419: read d1/d19/d1f/f2c [947898,78273] 0 2024-04-12T05:56:05.978 INFO:tasks.workunit.client.1.smithi194.stdout:9/420: read d1/d19/f54 [1787068,65983] 0 2024-04-12T05:56:05.979 INFO:tasks.workunit.client.1.smithi194.stdout:7/227: dwrite d3/d27/f39 [0,4194304] 0 2024-04-12T05:56:05.981 INFO:tasks.workunit.client.1.smithi194.stdout:7/228: chown d3/d16/d23/c32 165 1 2024-04-12T05:56:05.981 INFO:tasks.workunit.client.1.smithi194.stdout:7/229: chown d3/d16/d21/d3a/d47 65799529 1 2024-04-12T05:56:05.982 INFO:tasks.workunit.client.1.smithi194.stdout:7/230: stat d3/d16/d23 0 2024-04-12T05:56:05.986 INFO:tasks.workunit.client.1.smithi194.stdout:0/328: dread d0/da/d1e/d1a/d24/f61 [8388608,4194304] 0 2024-04-12T05:56:05.988 INFO:tasks.workunit.client.1.smithi194.stdout:7/231: getdents d3/d16/d21/d3a 0 2024-04-12T05:56:05.989 INFO:tasks.workunit.client.1.smithi194.stdout:0/329: mknod d0/da/d1e/d1a/d24/d56/c6c 0 2024-04-12T05:56:05.994 INFO:tasks.workunit.client.1.smithi194.stdout:0/330: truncate d0/da/d1e/d1a/f44 488920 0 2024-04-12T05:56:05.996 INFO:tasks.workunit.client.1.smithi194.stdout:7/232: dwrite d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:05.996 INFO:tasks.workunit.client.1.smithi194.stdout:9/421: mkdir d1/d19/d1f/d20/d37/d84 0 2024-04-12T05:56:05.997 INFO:tasks.workunit.client.1.smithi194.stdout:0/331: dread - d0/da/d1e/d1a/d24/d56/d66/f68 zero size 2024-04-12T05:56:05.999 INFO:tasks.workunit.client.1.smithi194.stdout:0/332: read d0/f37 [974199,52337] 0 2024-04-12T05:56:05.999 INFO:tasks.workunit.client.1.smithi194.stdout:7/233: mknod d3/d16/d21/d3a/c4d 0 2024-04-12T05:56:06.000 INFO:tasks.workunit.client.1.smithi194.stdout:7/234: write d3/d27/d22/f38 [341261,24498] 0 2024-04-12T05:56:06.001 INFO:tasks.workunit.client.1.smithi194.stdout:9/422: link d1/d19/d1f/f2c d1/d19/d1f/d5f/f85 0 2024-04-12T05:56:06.006 INFO:tasks.workunit.client.1.smithi194.stdout:9/423: creat d1/d19/d1f/d5f/f86 x:0 0 0 2024-04-12T05:56:06.007 INFO:tasks.workunit.client.1.smithi194.stdout:7/235: symlink d3/d16/d21/d3a/d43/l4e 0 2024-04-12T05:56:06.009 INFO:tasks.workunit.client.1.smithi194.stdout:7/236: write d3/d27/fa [2186344,18366] 0 2024-04-12T05:56:06.010 INFO:tasks.workunit.client.1.smithi194.stdout:0/333: rename d0/da/d4a/f4f to d0/da/d1e/d1a/f6d 0 2024-04-12T05:56:06.010 INFO:tasks.workunit.client.1.smithi194.stdout:7/237: chown d3/d16/d23/c4c 4007 1 2024-04-12T05:56:06.011 INFO:tasks.workunit.client.1.smithi194.stdout:9/424: mknod d1/d21/d77/c87 0 2024-04-12T05:56:06.017 INFO:tasks.workunit.client.1.smithi194.stdout:9/425: dread d1/d21/f3e [0,4194304] 0 2024-04-12T05:56:06.026 INFO:tasks.workunit.client.1.smithi194.stdout:0/334: dwrite d0/da/d1e/d1a/f44 [0,4194304] 0 2024-04-12T05:56:06.032 INFO:tasks.workunit.client.1.smithi194.stdout:9/426: dread f0 [0,4194304] 0 2024-04-12T05:56:06.033 INFO:tasks.workunit.client.1.smithi194.stdout:0/335: dwrite d0/f53 [0,4194304] 0 2024-04-12T05:56:06.036 INFO:tasks.workunit.client.1.smithi194.stdout:9/427: creat d1/d19/d1f/f88 x:0 0 0 2024-04-12T05:56:06.044 INFO:tasks.workunit.client.1.smithi194.stdout:9/428: dwrite d1/f5b [0,4194304] 0 2024-04-12T05:56:06.047 INFO:tasks.workunit.client.1.smithi194.stdout:0/336: write d0/da/d1e/d1a/d24/d56/d66/f68 [140252,20720] 0 2024-04-12T05:56:06.048 INFO:tasks.workunit.client.1.smithi194.stdout:9/429: truncate d1/d21/d39/d72/f75 1590899 0 2024-04-12T05:56:06.049 INFO:tasks.workunit.client.1.smithi194.stdout:9/430: chown d1/d21/f3e 1163336 1 2024-04-12T05:56:06.051 INFO:tasks.workunit.client.1.smithi194.stdout:0/337: rename d0/da/d1e/d1a/d24/l59 to d0/da/d1e/d1a/d24/l6e 0 2024-04-12T05:56:06.052 INFO:tasks.workunit.client.1.smithi194.stdout:0/338: chown d0/da/d1e/d1a/l42 2322 1 2024-04-12T05:56:06.055 INFO:tasks.workunit.client.1.smithi194.stdout:9/431: creat d1/d21/d39/f89 x:0 0 0 2024-04-12T05:56:06.061 INFO:tasks.workunit.client.1.smithi194.stdout:0/339: dwrite d0/da/d1e/d1a/f64 [0,4194304] 0 2024-04-12T05:56:06.064 INFO:tasks.workunit.client.1.smithi194.stdout:9/432: dwrite d1/f69 [0,4194304] 0 2024-04-12T05:56:06.066 INFO:tasks.workunit.client.1.smithi194.stdout:9/433: chown d1/d21/d77/c87 871096 1 2024-04-12T05:56:06.074 INFO:tasks.workunit.client.1.smithi194.stdout:9/434: creat d1/d19/d1f/d20/d37/d60/f8a x:0 0 0 2024-04-12T05:56:06.077 INFO:tasks.workunit.client.1.smithi194.stdout:9/435: creat d1/d19/d1f/d20/f8b x:0 0 0 2024-04-12T05:56:06.077 INFO:tasks.workunit.client.1.smithi194.stdout:9/436: read - d1/d19/f83 zero size 2024-04-12T05:56:06.080 INFO:tasks.workunit.client.1.smithi194.stdout:0/340: write d0/da/d1e/f35 [1093826,47223] 0 2024-04-12T05:56:06.082 INFO:tasks.workunit.client.1.smithi194.stdout:0/341: write d0/da/f3e [2163009,60755] 0 2024-04-12T05:56:06.087 INFO:tasks.workunit.client.1.smithi194.stdout:9/437: dwrite d1/d4e/f6d [0,4194304] 0 2024-04-12T05:56:06.091 INFO:tasks.workunit.client.1.smithi194.stdout:0/342: dwrite d0/da/d4a/f5e [0,4194304] 0 2024-04-12T05:56:06.107 INFO:tasks.workunit.client.1.smithi194.stdout:4/345: sync 2024-04-12T05:56:06.111 INFO:tasks.workunit.client.1.smithi194.stdout:4/346: dread d2/d3/ff [0,4194304] 0 2024-04-12T05:56:06.113 INFO:tasks.workunit.client.1.smithi194.stdout:4/347: unlink d2/d11/c68 0 2024-04-12T05:56:06.116 INFO:tasks.workunit.client.1.smithi194.stdout:4/348: link d2/f67 d2/d3/d1b/d32/d3a/d3b/f7b 0 2024-04-12T05:56:06.117 INFO:tasks.workunit.client.1.smithi194.stdout:4/349: truncate d2/d11/d46/d61/f78 100206 0 2024-04-12T05:56:06.124 INFO:tasks.workunit.client.1.smithi194.stdout:4/350: dwrite d2/f21 [0,4194304] 0 2024-04-12T05:56:06.128 INFO:tasks.workunit.client.1.smithi194.stdout:4/351: rename f0 to d2/d11/d1e/d25/d2b/d5d/f7c 0 2024-04-12T05:56:06.134 INFO:tasks.workunit.client.1.smithi194.stdout:4/352: dwrite d2/d22/f34 [0,4194304] 0 2024-04-12T05:56:06.147 INFO:tasks.workunit.client.1.smithi194.stdout:4/353: dwrite d2/f67 [0,4194304] 0 2024-04-12T05:56:06.148 INFO:tasks.workunit.client.1.smithi194.stdout:4/354: write d2/d3/d1b/f5e [391150,50441] 0 2024-04-12T05:56:06.163 INFO:tasks.workunit.client.1.smithi194.stdout:4/355: truncate d2/d3/d58/f5c 2475586 0 2024-04-12T05:56:06.176 INFO:tasks.workunit.client.1.smithi194.stdout:4/356: dread d2/d3/d1b/d32/d3a/d3b/f41 [4194304,4194304] 0 2024-04-12T05:56:06.178 INFO:tasks.workunit.client.1.smithi194.stdout:4/357: mknod d2/d11/d46/c7d 0 2024-04-12T05:56:06.179 INFO:tasks.workunit.client.1.smithi194.stdout:4/358: mkdir d2/d3/d51/d7e 0 2024-04-12T05:56:06.182 INFO:tasks.workunit.client.1.smithi194.stdout:4/359: link d2/l26 d2/d11/d1e/d25/d2b/d5d/l7f 0 2024-04-12T05:56:06.184 INFO:tasks.workunit.client.1.smithi194.stdout:4/360: mkdir d2/d3/d50/d80 0 2024-04-12T05:56:06.190 INFO:tasks.workunit.client.1.smithi194.stdout:4/361: write d2/d3/d1b/d32/d3a/f3e [168753,11313] 0 2024-04-12T05:56:06.192 INFO:tasks.workunit.client.1.smithi194.stdout:4/362: mkdir d2/d11/d1f/d81 0 2024-04-12T05:56:06.194 INFO:tasks.workunit.client.1.smithi194.stdout:4/363: mkdir d2/d3/d58/d82 0 2024-04-12T05:56:06.195 INFO:tasks.workunit.client.1.smithi194.stdout:4/364: mknod d2/d11/d1f/c83 0 2024-04-12T05:56:06.196 INFO:tasks.workunit.client.1.smithi194.stdout:4/365: stat d2/d11/d1e/d2f 0 2024-04-12T05:56:06.202 INFO:tasks.workunit.client.1.smithi194.stdout:4/366: truncate d2/d11/d46/d61/f78 79349 0 2024-04-12T05:56:06.203 INFO:tasks.workunit.client.1.smithi194.stdout:4/367: fsync d2/d3/f13 0 2024-04-12T05:56:06.204 INFO:tasks.workunit.client.1.smithi194.stdout:4/368: creat d2/d11/d46/d61/d63/f84 x:0 0 0 2024-04-12T05:56:06.209 INFO:tasks.workunit.client.1.smithi194.stdout:4/369: dread d2/f4e [0,4194304] 0 2024-04-12T05:56:06.210 INFO:tasks.workunit.client.1.smithi194.stdout:4/370: write d2/d3/d1b/f5e [1309434,83354] 0 2024-04-12T05:56:06.216 INFO:tasks.workunit.client.1.smithi194.stdout:4/371: dwrite d2/d3/f18 [4194304,4194304] 0 2024-04-12T05:56:06.224 INFO:tasks.workunit.client.1.smithi194.stdout:4/372: dwrite d2/d3/ff [0,4194304] 0 2024-04-12T05:56:06.227 INFO:tasks.workunit.client.1.smithi194.stdout:4/373: mkdir d2/d85 0 2024-04-12T05:56:06.236 INFO:tasks.workunit.client.1.smithi194.stdout:4/374: dwrite d2/d3/d1b/d32/d3a/d3b/f7b [4194304,4194304] 0 2024-04-12T05:56:06.238 INFO:tasks.workunit.client.1.smithi194.stdout:4/375: write d2/d11/d46/d61/f70 [1043160,128455] 0 2024-04-12T05:56:06.240 INFO:tasks.workunit.client.1.smithi194.stdout:4/376: write d2/d11/d46/d61/d63/f84 [631097,67506] 0 2024-04-12T05:56:06.244 INFO:tasks.workunit.client.1.smithi194.stdout:4/377: creat d2/d3/d1b/d32/d3a/d3b/f86 x:0 0 0 2024-04-12T05:56:06.256 INFO:tasks.workunit.client.1.smithi194.stdout:4/378: dread d2/d3/d58/f5c [0,4194304] 0 2024-04-12T05:56:06.258 INFO:tasks.workunit.client.1.smithi194.stdout:4/379: unlink d2/d3/d1b/d32/f38 0 2024-04-12T05:56:06.263 INFO:tasks.workunit.client.1.smithi194.stdout:4/380: dwrite d2/d3/d1b/d32/d3a/d3b/f41 [4194304,4194304] 0 2024-04-12T05:56:06.265 INFO:tasks.workunit.client.1.smithi194.stdout:4/381: mknod d2/d3/d58/d82/c87 0 2024-04-12T05:56:06.266 INFO:tasks.workunit.client.1.smithi194.stdout:4/382: chown d2/d11/d1f 58661007 1 2024-04-12T05:56:06.273 INFO:tasks.workunit.client.1.smithi194.stdout:4/383: dwrite d2/d11/d1e/d25/d2b/d5d/f7c [0,4194304] 0 2024-04-12T05:56:06.281 INFO:tasks.workunit.client.1.smithi194.stdout:4/384: dwrite d2/d11/d46/d61/d63/f84 [0,4194304] 0 2024-04-12T05:56:06.282 INFO:tasks.workunit.client.1.smithi194.stdout:4/385: truncate d2/d3/f74 765806 0 2024-04-12T05:56:06.284 INFO:tasks.workunit.client.1.smithi194.stdout:4/386: write d2/f67 [3729593,118820] 0 2024-04-12T05:56:06.288 INFO:tasks.workunit.client.1.smithi194.stdout:4/387: dread d2/d3/f18 [4194304,4194304] 0 2024-04-12T05:56:06.290 INFO:tasks.workunit.client.1.smithi194.stdout:4/388: readlink d2/d3/d58/l60 0 2024-04-12T05:56:06.291 INFO:tasks.workunit.client.1.smithi194.stdout:4/389: write d2/d3/d1b/d32/d3a/d3b/f7b [1026490,119203] 0 2024-04-12T05:56:06.299 INFO:tasks.workunit.client.1.smithi194.stdout:4/390: write d2/d11/d46/d61/f78 [98723,115259] 0 2024-04-12T05:56:06.301 INFO:tasks.workunit.client.1.smithi194.stdout:4/391: creat d2/d11/f88 x:0 0 0 2024-04-12T05:56:06.302 INFO:tasks.workunit.client.1.smithi194.stdout:4/392: rename d2/d11 to d2/d11/d46/d89 22 2024-04-12T05:56:06.306 INFO:tasks.workunit.client.1.smithi194.stdout:4/393: dread d2/d3/d1b/d32/d3a/f3e [0,4194304] 0 2024-04-12T05:56:06.308 INFO:tasks.workunit.client.1.smithi194.stdout:4/394: mknod d2/d3/d51/c8a 0 2024-04-12T05:56:06.308 INFO:tasks.workunit.client.1.smithi194.stdout:4/395: dread - d2/d11/d1e/f6f zero size 2024-04-12T05:56:06.309 INFO:tasks.workunit.client.1.smithi194.stdout:4/396: mknod d2/c8b 0 2024-04-12T05:56:06.328 INFO:tasks.workunit.client.1.smithi194.stdout:8/223: sync 2024-04-12T05:56:06.331 INFO:tasks.workunit.client.1.smithi194.stdout:8/224: getdents d2/d8/d15/d35 0 2024-04-12T05:56:06.333 INFO:tasks.workunit.client.1.smithi194.stdout:8/225: symlink d2/d8/d9/de/d38/l44 0 2024-04-12T05:56:06.335 INFO:tasks.workunit.client.1.smithi194.stdout:8/226: mkdir d2/d8/d15/d33/d45 0 2024-04-12T05:56:06.341 INFO:tasks.workunit.client.1.smithi194.stdout:8/227: dwrite d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:06.342 INFO:tasks.workunit.client.1.smithi194.stdout:8/228: truncate d2/f3e 817960 0 2024-04-12T05:56:06.343 INFO:tasks.workunit.client.1.smithi194.stdout:8/229: fdatasync d2/d8/d15/d33/f27 0 2024-04-12T05:56:06.348 INFO:tasks.workunit.client.1.smithi194.stdout:8/230: dwrite d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:06.374 INFO:tasks.workunit.client.1.smithi194.stdout:8/231: fdatasync d2/d8/d9/de/d10/f1c 0 2024-04-12T05:56:06.377 INFO:tasks.workunit.client.1.smithi194.stdout:8/232: creat d2/f46 x:0 0 0 2024-04-12T05:56:06.378 INFO:tasks.workunit.client.1.smithi194.stdout:8/233: mknod d2/d8/d15/d3b/c47 0 2024-04-12T05:56:06.401 INFO:tasks.workunit.client.1.smithi194.stdout:8/234: dread d2/f12 [0,4194304] 0 2024-04-12T05:56:06.403 INFO:tasks.workunit.client.1.smithi194.stdout:8/235: creat d2/d8/d9/f48 x:0 0 0 2024-04-12T05:56:06.405 INFO:tasks.workunit.client.1.smithi194.stdout:8/236: mkdir d2/d8/d15/d33/d49 0 2024-04-12T05:56:06.407 INFO:tasks.workunit.client.1.smithi194.stdout:8/237: creat d2/d8/d9/de/d38/f4a x:0 0 0 2024-04-12T05:56:06.408 INFO:tasks.workunit.client.1.smithi194.stdout:8/238: dread d2/f43 [0,4194304] 0 2024-04-12T05:56:06.409 INFO:tasks.workunit.client.1.smithi194.stdout:8/239: read d2/d8/d9/de/d10/f1c [2066129,56704] 0 2024-04-12T05:56:06.412 INFO:tasks.workunit.client.1.smithi194.stdout:8/240: unlink d2/d8/d15/d35/f40 0 2024-04-12T05:56:06.413 INFO:tasks.workunit.client.1.smithi194.stdout:8/241: chown d2/d8/d9/de/f37 16 1 2024-04-12T05:56:06.413 INFO:tasks.workunit.client.1.smithi194.stdout:8/242: chown d2/f3e 182 1 2024-04-12T05:56:06.415 INFO:tasks.workunit.client.1.smithi194.stdout:8/243: getdents d2/d8/d15/d33/d49 0 2024-04-12T05:56:06.493 INFO:tasks.workunit.client.1.smithi194.stdout:1/452: sync 2024-04-12T05:56:06.493 INFO:tasks.workunit.client.1.smithi194.stdout:2/339: sync 2024-04-12T05:56:06.493 INFO:tasks.workunit.client.1.smithi194.stdout:1/453: readlink d0/d11/d18/l3f 0 2024-04-12T05:56:06.494 INFO:tasks.workunit.client.1.smithi194.stdout:2/340: write d7/fa [188562,30844] 0 2024-04-12T05:56:06.494 INFO:tasks.workunit.client.1.smithi194.stdout:1/454: fsync d0/d11/d18/d19/d1a/d6a/f30 0 2024-04-12T05:56:06.497 INFO:tasks.workunit.client.1.smithi194.stdout:2/341: symlink d7/d12/d1a/d68/l79 0 2024-04-12T05:56:06.499 INFO:tasks.workunit.client.1.smithi194.stdout:1/455: mknod d0/d11/d18/d19/d1a/d6a/d27/d6c/c88 0 2024-04-12T05:56:06.499 INFO:tasks.workunit.client.1.smithi194.stdout:1/456: chown d0/d3e/d54 1 1 2024-04-12T05:56:06.499 INFO:tasks.workunit.client.1.smithi194.stdout:2/342: truncate d7/d12/f3d 153898 0 2024-04-12T05:56:06.500 INFO:tasks.workunit.client.1.smithi194.stdout:2/343: fsync d7/fa 0 2024-04-12T05:56:06.506 INFO:tasks.workunit.client.1.smithi194.stdout:1/457: dwrite d0/d11/f26 [0,4194304] 0 2024-04-12T05:56:06.507 INFO:tasks.workunit.client.1.smithi194.stdout:1/458: chown d0/d11/d18/d35 858938 1 2024-04-12T05:56:06.512 INFO:tasks.workunit.client.1.smithi194.stdout:1/459: rename d0/d3e/d5b/l70 to d0/d11/d18/d19/d1a/d6a/d27/d48/l89 0 2024-04-12T05:56:06.513 INFO:tasks.workunit.client.1.smithi194.stdout:2/344: truncate d7/d12/f38 652289 0 2024-04-12T05:56:06.514 INFO:tasks.workunit.client.1.smithi194.stdout:1/460: write d0/d11/d18/d19/d1c/f22 [368484,66699] 0 2024-04-12T05:56:06.516 INFO:tasks.workunit.client.1.smithi194.stdout:2/345: read d7/d12/d25/f31 [1826964,119648] 0 2024-04-12T05:56:06.517 INFO:tasks.workunit.client.1.smithi194.stdout:2/346: truncate d7/fa 1591783 0 2024-04-12T05:56:06.518 INFO:tasks.workunit.client.1.smithi194.stdout:1/461: creat d0/d3e/d50/f8a x:0 0 0 2024-04-12T05:56:06.529 INFO:tasks.workunit.client.1.smithi194.stdout:2/347: dwrite d7/d12/d25/f64 [0,4194304] 0 2024-04-12T05:56:06.529 INFO:tasks.workunit.client.1.smithi194.stdout:1/462: dwrite d0/d3e/d50/f8a [0,4194304] 0 2024-04-12T05:56:06.530 INFO:tasks.workunit.client.1.smithi194.stdout:1/463: chown d0/d3e/d54/d7b 1257333 1 2024-04-12T05:56:06.530 INFO:tasks.workunit.client.1.smithi194.stdout:2/348: stat d7/d12/d34/c56 0 2024-04-12T05:56:06.534 INFO:tasks.workunit.client.1.smithi194.stdout:2/349: write d7/d12/d25/d5d/f51 [1473380,18905] 0 2024-04-12T05:56:06.536 INFO:tasks.workunit.client.1.smithi194.stdout:2/350: read d7/d12/f3b [357036,19169] 0 2024-04-12T05:56:06.538 INFO:tasks.workunit.client.1.smithi194.stdout:2/351: write d7/d22/f4d [1466039,57627] 0 2024-04-12T05:56:06.540 INFO:tasks.workunit.client.1.smithi194.stdout:2/352: fsync d7/f11 0 2024-04-12T05:56:06.543 INFO:tasks.workunit.client.1.smithi194.stdout:1/464: dwrite d0/d11/d18/d19/d1c/f6b [0,4194304] 0 2024-04-12T05:56:06.544 INFO:tasks.workunit.client.1.smithi194.stdout:2/353: mknod d7/d22/d42/c7a 0 2024-04-12T05:56:06.546 INFO:tasks.workunit.client.1.smithi194.stdout:2/354: mkdir d7/d12/d25/d5d/d47/d52/d7b 0 2024-04-12T05:56:06.550 INFO:tasks.workunit.client.1.smithi194.stdout:1/465: rename d0/d11/d18/c65 to d0/d3e/c8b 0 2024-04-12T05:56:06.560 INFO:tasks.workunit.client.1.smithi194.stdout:1/466: rename d0/d3e/d54 to d0/d11/d18/d19/d1c/d8c 0 2024-04-12T05:56:06.562 INFO:tasks.workunit.client.1.smithi194.stdout:1/467: mkdir d0/df/d8d 0 2024-04-12T05:56:06.563 INFO:tasks.workunit.client.1.smithi194.stdout:2/355: dwrite d7/d12/d25/d5d/d47/d52/f55 [0,4194304] 0 2024-04-12T05:56:06.564 INFO:tasks.workunit.client.1.smithi194.stdout:1/468: dread - d0/d11/f7c zero size 2024-04-12T05:56:06.569 INFO:tasks.workunit.client.1.smithi194.stdout:1/469: symlink d0/d11/d18/d35/d6f/l8e 0 2024-04-12T05:56:06.572 INFO:tasks.workunit.client.1.smithi194.stdout:2/356: link d7/l30 d7/d12/d25/d5d/l7c 0 2024-04-12T05:56:06.572 INFO:tasks.workunit.client.1.smithi194.stdout:1/470: read d0/d11/d18/d19/f39 [2241953,5411] 0 2024-04-12T05:56:06.572 INFO:tasks.workunit.client.1.smithi194.stdout:2/357: dread - d7/f5e zero size 2024-04-12T05:56:06.576 INFO:tasks.workunit.client.1.smithi194.stdout:1/471: symlink d0/d11/d18/d19/d1a/d6a/l8f 0 2024-04-12T05:56:06.578 INFO:tasks.workunit.client.1.smithi194.stdout:1/472: rmdir d0/df 39 2024-04-12T05:56:06.581 INFO:tasks.workunit.client.1.smithi194.stdout:1/473: mknod d0/d11/d18/d19/d1c/d8c/c90 0 2024-04-12T05:56:06.581 INFO:tasks.workunit.client.1.smithi194.stdout:1/474: readlink d0/d11/d18/d19/l2f 0 2024-04-12T05:56:06.583 INFO:tasks.workunit.client.1.smithi194.stdout:1/475: truncate d0/d11/d18/f76 873646 0 2024-04-12T05:56:06.584 INFO:tasks.workunit.client.1.smithi194.stdout:1/476: dread - d0/df/f82 zero size 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:0/343: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:4/397: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:7/238: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:8/244: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:5/442: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:6/347: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:9/438: sync 2024-04-12T05:56:06.586 INFO:tasks.workunit.client.1.smithi194.stdout:3/487: sync 2024-04-12T05:56:06.587 INFO:tasks.workunit.client.1.smithi194.stdout:0/344: fsync d0/d3b/f4c 0 2024-04-12T05:56:06.589 INFO:tasks.workunit.client.1.smithi194.stdout:4/398: unlink d2/d22/c35 0 2024-04-12T05:56:06.592 INFO:tasks.workunit.client.1.smithi194.stdout:8/245: creat d2/d8/d15/d33/f4b x:0 0 0 2024-04-12T05:56:06.592 INFO:tasks.workunit.client.1.smithi194.stdout:5/443: symlink d1/d4/d6/d34/d52/l93 0 2024-04-12T05:56:06.594 INFO:tasks.workunit.client.1.smithi194.stdout:2/358: dread d7/d22/d42/f77 [0,4194304] 0 2024-04-12T05:56:06.596 INFO:tasks.workunit.client.1.smithi194.stdout:0/345: creat d0/da/d1e/d1a/d24/d4b/f6f x:0 0 0 2024-04-12T05:56:06.597 INFO:tasks.workunit.client.1.smithi194.stdout:4/399: mknod d2/d11/d1e/d25/d2b/c8c 0 2024-04-12T05:56:06.597 INFO:tasks.workunit.client.1.smithi194.stdout:1/477: truncate d0/d11/d18/d19/d1a/d6a/d27/d2e/f33 8964240 0 2024-04-12T05:56:06.597 INFO:tasks.workunit.client.1.smithi194.stdout:8/246: chown d2/d8/d9/d1a/f30 134047883 1 2024-04-12T05:56:06.599 INFO:tasks.workunit.client.1.smithi194.stdout:0/346: readlink d0/da/l5d 0 2024-04-12T05:56:06.600 INFO:tasks.workunit.client.1.smithi194.stdout:5/444: dread - d1/d4/df/f6a zero size 2024-04-12T05:56:06.601 INFO:tasks.workunit.client.1.smithi194.stdout:8/247: readlink d2/d8/d9/de/l1f 0 2024-04-12T05:56:06.603 INFO:tasks.workunit.client.1.smithi194.stdout:4/400: symlink d2/d11/d1f/l8d 0 2024-04-12T05:56:06.605 INFO:tasks.workunit.client.1.smithi194.stdout:1/478: chown d0/d11/d18/d19/l7e 265 1 2024-04-12T05:56:06.607 INFO:tasks.workunit.client.1.smithi194.stdout:0/347: write d0/da/d1e/f32 [1050128,102962] 0 2024-04-12T05:56:06.607 INFO:tasks.workunit.client.1.smithi194.stdout:9/439: rename d1/d19/d1f/c5a to d1/d19/d1f/d20/d37/d60/c8c 0 2024-04-12T05:56:06.607 INFO:tasks.workunit.client.1.smithi194.stdout:6/348: dwrite d3/d15/d2b/d48/f54 [0,4194304] 0 2024-04-12T05:56:06.614 INFO:tasks.workunit.client.1.smithi194.stdout:8/248: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:06.614 INFO:tasks.workunit.client.1.smithi194.stdout:3/488: truncate d0/dc/d1e/d81/d34/d2d/f5a 336741 0 2024-04-12T05:56:06.614 INFO:tasks.workunit.client.1.smithi194.stdout:5/445: mknod d1/d4/d6/c94 0 2024-04-12T05:56:06.614 INFO:tasks.workunit.client.1.smithi194.stdout:2/359: dwrite d7/d12/d25/d5d/d47/d52/f55 [0,4194304] 0 2024-04-12T05:56:06.615 INFO:tasks.workunit.client.1.smithi194.stdout:9/440: fsync d1/d21/d39/f89 0 2024-04-12T05:56:06.617 INFO:tasks.workunit.client.1.smithi194.stdout:1/479: chown d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 869427070 1 2024-04-12T05:56:06.625 INFO:tasks.workunit.client.1.smithi194.stdout:9/441: symlink d1/d19/d1f/d5f/l8d 0 2024-04-12T05:56:06.626 INFO:tasks.workunit.client.1.smithi194.stdout:5/446: rename d1/d4/df to d1/d5c/d95 0 2024-04-12T05:56:06.634 INFO:tasks.workunit.client.1.smithi194.stdout:3/489: rmdir d0 39 2024-04-12T05:56:06.635 INFO:tasks.workunit.client.1.smithi194.stdout:4/401: dwrite d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:06.639 INFO:tasks.workunit.client.1.smithi194.stdout:5/447: write d1/d4/f1c [1635434,30629] 0 2024-04-12T05:56:06.640 INFO:tasks.workunit.client.1.smithi194.stdout:8/249: creat d2/d8/d15/d35/f4c x:0 0 0 2024-04-12T05:56:06.640 INFO:tasks.workunit.client.1.smithi194.stdout:9/442: symlink d1/l8e 0 2024-04-12T05:56:06.646 INFO:tasks.workunit.client.1.smithi194.stdout:0/348: dwrite d0/da/d1e/f41 [0,4194304] 0 2024-04-12T05:56:06.649 INFO:tasks.workunit.client.1.smithi194.stdout:1/480: dwrite d0/d11/d18/d19/f4e [0,4194304] 0 2024-04-12T05:56:06.649 INFO:tasks.workunit.client.1.smithi194.stdout:8/250: fdatasync d2/f12 0 2024-04-12T05:56:06.657 INFO:tasks.workunit.client.1.smithi194.stdout:0/349: write d0/da/d1e/f41 [2065214,55963] 0 2024-04-12T05:56:06.657 INFO:tasks.workunit.client.1.smithi194.stdout:6/349: truncate d3/d2d/f32 1373179 0 2024-04-12T05:56:06.658 INFO:tasks.workunit.client.1.smithi194.stdout:0/350: chown d0/l67 1 1 2024-04-12T05:56:06.662 INFO:tasks.workunit.client.1.smithi194.stdout:5/448: rmdir d1/d4/d6/d6d 39 2024-04-12T05:56:06.663 INFO:tasks.workunit.client.1.smithi194.stdout:3/490: mkdir d0/d1/d74/d39/da2 0 2024-04-12T05:56:06.670 INFO:tasks.workunit.client.1.smithi194.stdout:4/402: rename d2/d11/l6c to d2/d11/d1e/l8e 0 2024-04-12T05:56:06.670 INFO:tasks.workunit.client.1.smithi194.stdout:6/350: mkdir d3/d2f/d4e/d68 0 2024-04-12T05:56:06.670 INFO:tasks.workunit.client.1.smithi194.stdout:2/360: dread d7/d12/d1a/f44 [0,4194304] 0 2024-04-12T05:56:06.670 INFO:tasks.workunit.client.1.smithi194.stdout:8/251: dwrite d2/f12 [4194304,4194304] 0 2024-04-12T05:56:06.675 INFO:tasks.workunit.client.1.smithi194.stdout:5/449: symlink d1/d4/d6/d8e/l96 0 2024-04-12T05:56:06.679 INFO:tasks.workunit.client.1.smithi194.stdout:9/443: dwrite d1/d19/d1f/f70 [0,4194304] 0 2024-04-12T05:56:06.681 INFO:tasks.workunit.client.1.smithi194.stdout:4/403: creat d2/f8f x:0 0 0 2024-04-12T05:56:06.683 INFO:tasks.workunit.client.1.smithi194.stdout:6/351: dread - d3/d2d/d33/f4d zero size 2024-04-12T05:56:06.683 INFO:tasks.workunit.client.1.smithi194.stdout:5/450: chown d1/d4/d6/db/d38/f61 1248123284 1 2024-04-12T05:56:06.684 INFO:tasks.workunit.client.1.smithi194.stdout:2/361: mknod d7/d22/d42/c7d 0 2024-04-12T05:56:06.685 INFO:tasks.workunit.client.1.smithi194.stdout:4/404: fsync d2/f67 0 2024-04-12T05:56:06.691 INFO:tasks.workunit.client.1.smithi194.stdout:0/351: dwrite d0/f53 [4194304,4194304] 0 2024-04-12T05:56:06.695 INFO:tasks.workunit.client.1.smithi194.stdout:9/444: chown d1/d19/d1f/d20/f57 471977176 1 2024-04-12T05:56:06.695 INFO:tasks.workunit.client.1.smithi194.stdout:7/239: sync 2024-04-12T05:56:06.697 INFO:tasks.workunit.client.1.smithi194.stdout:3/491: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:06.703 INFO:tasks.workunit.client.1.smithi194.stdout:4/405: creat d2/d3/d1b/d32/d3a/d3b/f90 x:0 0 0 2024-04-12T05:56:06.706 INFO:tasks.workunit.client.1.smithi194.stdout:0/352: write d0/f4 [747820,115440] 0 2024-04-12T05:56:06.711 INFO:tasks.workunit.client.1.smithi194.stdout:5/451: unlink d1/d4/d6/d34/c3a 0 2024-04-12T05:56:06.711 INFO:tasks.workunit.client.1.smithi194.stdout:8/252: dwrite d2/d8/d15/d35/f4c [0,4194304] 0 2024-04-12T05:56:06.712 INFO:tasks.workunit.client.1.smithi194.stdout:4/406: chown d2/d3/d58/d82/c87 128 1 2024-04-12T05:56:06.712 INFO:tasks.workunit.client.1.smithi194.stdout:1/481: dwrite d0/f85 [0,4194304] 0 2024-04-12T05:56:06.726 INFO:tasks.workunit.client.1.smithi194.stdout:3/492: creat d0/d1/d4f/fa3 x:0 0 0 2024-04-12T05:56:06.726 INFO:tasks.workunit.client.1.smithi194.stdout:1/482: stat d0/d11/d18/l44 0 2024-04-12T05:56:06.727 INFO:tasks.workunit.client.1.smithi194.stdout:1/483: dread - d0/df/f82 zero size 2024-04-12T05:56:06.727 INFO:tasks.workunit.client.1.smithi194.stdout:6/352: dwrite d3/d34/f5d [0,4194304] 0 2024-04-12T05:56:06.727 INFO:tasks.workunit.client.1.smithi194.stdout:9/445: truncate d1/d41/f44 97932 0 2024-04-12T05:56:06.727 INFO:tasks.workunit.client.1.smithi194.stdout:3/493: rmdir d0/dc/d1e/d81/d5b 39 2024-04-12T05:56:06.727 INFO:tasks.workunit.client.1.smithi194.stdout:5/452: mknod d1/d4/d84/c97 0 2024-04-12T05:56:06.730 INFO:tasks.workunit.client.1.smithi194.stdout:7/240: dread d3/d27/fa [0,4194304] 0 2024-04-12T05:56:06.731 INFO:tasks.workunit.client.1.smithi194.stdout:5/453: fdatasync d1/d4/d6/f13 0 2024-04-12T05:56:06.739 INFO:tasks.workunit.client.1.smithi194.stdout:1/484: chown d0/d11/d18/d19/d1a/d6a/f30 36 1 2024-04-12T05:56:06.739 INFO:tasks.workunit.client.1.smithi194.stdout:0/353: dwrite d0/f48 [0,4194304] 0 2024-04-12T05:56:06.739 INFO:tasks.workunit.client.1.smithi194.stdout:8/253: link d2/d8/d9/d1a/l22 d2/d8/d9/d1a/l4d 0 2024-04-12T05:56:06.743 INFO:tasks.workunit.client.1.smithi194.stdout:5/454: symlink d1/d4/d6/d6d/l98 0 2024-04-12T05:56:06.744 INFO:tasks.workunit.client.1.smithi194.stdout:0/354: fdatasync d0/da/f3e 0 2024-04-12T05:56:06.747 INFO:tasks.workunit.client.1.smithi194.stdout:7/241: unlink d3/d27/d3b/f41 0 2024-04-12T05:56:06.748 INFO:tasks.workunit.client.1.smithi194.stdout:9/446: dwrite d1/fe [0,4194304] 0 2024-04-12T05:56:06.748 INFO:tasks.workunit.client.1.smithi194.stdout:5/455: stat d1/d4/d6/db/d15/d26/f4e 0 2024-04-12T05:56:06.750 INFO:tasks.workunit.client.1.smithi194.stdout:2/362: dread d7/d12/f3f [0,4194304] 0 2024-04-12T05:56:06.750 INFO:tasks.workunit.client.1.smithi194.stdout:8/254: getdents d2/d8/d15/d33/d32 0 2024-04-12T05:56:06.753 INFO:tasks.workunit.client.1.smithi194.stdout:5/456: stat d1/d4/d6/d34/d52/c7f 0 2024-04-12T05:56:06.755 INFO:tasks.workunit.client.1.smithi194.stdout:2/363: mkdir d7/d12/d1a/d69/d7e 0 2024-04-12T05:56:06.758 INFO:tasks.workunit.client.1.smithi194.stdout:5/457: unlink d1/d4/d84/c97 0 2024-04-12T05:56:06.758 INFO:tasks.workunit.client.1.smithi194.stdout:3/494: write d0/dc/d1e/d81/d34/f37 [28373,30507] 0 2024-04-12T05:56:06.761 INFO:tasks.workunit.client.1.smithi194.stdout:0/355: dread d0/da/f39 [0,4194304] 0 2024-04-12T05:56:06.766 INFO:tasks.workunit.client.1.smithi194.stdout:7/242: link d3/d16/d23/d37/l3f d3/d27/d3b/l4f 0 2024-04-12T05:56:06.767 INFO:tasks.workunit.client.1.smithi194.stdout:8/255: getdents d2/d8/d15/d33/d32 0 2024-04-12T05:56:06.767 INFO:tasks.workunit.client.1.smithi194.stdout:6/353: dwrite d3/d15/d2b/f5e [0,4194304] 0 2024-04-12T05:56:06.770 INFO:tasks.workunit.client.1.smithi194.stdout:5/458: rmdir d1/d4 39 2024-04-12T05:56:06.773 INFO:tasks.workunit.client.1.smithi194.stdout:1/485: dread d0/d11/f59 [0,4194304] 0 2024-04-12T05:56:06.775 INFO:tasks.workunit.client.1.smithi194.stdout:6/354: symlink d3/l69 0 2024-04-12T05:56:06.783 INFO:tasks.workunit.client.1.smithi194.stdout:8/256: rename d2/d8/d15/d33/d32 to d2/d8/d9/de/d10/d4e 0 2024-04-12T05:56:06.783 INFO:tasks.workunit.client.1.smithi194.stdout:3/495: getdents d0/dc/d1e/d81 0 2024-04-12T05:56:06.785 INFO:tasks.workunit.client.1.smithi194.stdout:9/447: truncate d1/d19/d1f/d20/d37/d38/f5c 1708965 0 2024-04-12T05:56:06.787 INFO:tasks.workunit.client.1.smithi194.stdout:8/257: write d2/d8/d9/d1a/f2b [4745789,14446] 0 2024-04-12T05:56:06.788 INFO:tasks.workunit.client.1.smithi194.stdout:1/486: dwrite d0/d11/d18/d19/d1a/d6a/d27/d2e/f33 [8388608,4194304] 0 2024-04-12T05:56:06.790 INFO:tasks.workunit.client.1.smithi194.stdout:9/448: dread - d1/d19/d1f/d20/d37/d60/f8a zero size 2024-04-12T05:56:06.790 INFO:tasks.workunit.client.1.smithi194.stdout:0/356: link d0/da/d1e/d1a/d24/d56/d66/c6b d0/da/d1e/d27/c70 0 2024-04-12T05:56:06.796 INFO:tasks.workunit.client.1.smithi194.stdout:2/364: dread f5 [4194304,4194304] 0 2024-04-12T05:56:06.798 INFO:tasks.workunit.client.1.smithi194.stdout:7/243: getdents d3/d27/d3b 0 2024-04-12T05:56:06.803 INFO:tasks.workunit.client.1.smithi194.stdout:9/449: readlink d1/l12 0 2024-04-12T05:56:06.804 INFO:tasks.workunit.client.1.smithi194.stdout:8/258: rmdir d2/d8/d9/de 39 2024-04-12T05:56:06.804 INFO:tasks.workunit.client.1.smithi194.stdout:6/355: dread d3/d15/d2b/d48/f49 [0,4194304] 0 2024-04-12T05:56:06.805 INFO:tasks.workunit.client.1.smithi194.stdout:7/244: creat d3/d16/d23/d37/d3d/f50 x:0 0 0 2024-04-12T05:56:06.806 INFO:tasks.workunit.client.1.smithi194.stdout:5/459: dwrite d1/fa [0,4194304] 0 2024-04-12T05:56:06.807 INFO:tasks.workunit.client.1.smithi194.stdout:1/487: mkdir d0/d11/d18/d19/d91 0 2024-04-12T05:56:06.809 INFO:tasks.workunit.client.1.smithi194.stdout:6/356: chown d3/d15/f21 1 1 2024-04-12T05:56:06.811 INFO:tasks.workunit.client.1.smithi194.stdout:0/357: creat d0/da/d1e/d27/d54/f71 x:0 0 0 2024-04-12T05:56:06.812 INFO:tasks.workunit.client.1.smithi194.stdout:2/365: link d7/d12/d6d/f72 d7/d12/d25/d5d/d47/f7f 0 2024-04-12T05:56:06.812 INFO:tasks.workunit.client.1.smithi194.stdout:2/366: fsync d7/d12/d25/d5d/d47/d52/f60 0 2024-04-12T05:56:06.813 INFO:tasks.workunit.client.1.smithi194.stdout:8/259: unlink d2/d8/d15/c2f 0 2024-04-12T05:56:06.814 INFO:tasks.workunit.client.1.smithi194.stdout:8/260: fsync d2/d8/d15/d33/f29 0 2024-04-12T05:56:06.816 INFO:tasks.workunit.client.1.smithi194.stdout:1/488: mkdir d0/df/d68/d92 0 2024-04-12T05:56:06.818 INFO:tasks.workunit.client.1.smithi194.stdout:9/450: dread d1/d19/d1f/f2c [0,4194304] 0 2024-04-12T05:56:06.821 INFO:tasks.workunit.client.1.smithi194.stdout:3/496: dread d0/d1/d74/f3f [0,4194304] 0 2024-04-12T05:56:06.825 INFO:tasks.workunit.client.1.smithi194.stdout:1/489: dwrite d0/f4 [0,4194304] 0 2024-04-12T05:56:06.827 INFO:tasks.workunit.client.1.smithi194.stdout:6/357: creat d3/d2f/d4e/d53/f6a x:0 0 0 2024-04-12T05:56:06.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/261: dread d2/fb [0,4194304] 0 2024-04-12T05:56:06.837 INFO:tasks.workunit.client.1.smithi194.stdout:8/262: dread - d2/d8/d9/f48 zero size 2024-04-12T05:56:06.837 INFO:tasks.workunit.client.1.smithi194.stdout:3/497: symlink d0/d1/la4 0 2024-04-12T05:56:06.838 INFO:tasks.workunit.client.1.smithi194.stdout:1/490: mknod d0/d11/d18/d19/d1c/c93 0 2024-04-12T05:56:06.839 INFO:tasks.workunit.client.1.smithi194.stdout:6/358: dread - d3/d1a/f23 zero size 2024-04-12T05:56:06.841 INFO:tasks.workunit.client.1.smithi194.stdout:8/263: chown d2/d8/d9/de/f2e 68283 1 2024-04-12T05:56:06.843 INFO:tasks.workunit.client.1.smithi194.stdout:2/367: getdents d7 0 2024-04-12T05:56:06.843 INFO:tasks.workunit.client.1.smithi194.stdout:6/359: symlink d3/d34/l6b 0 2024-04-12T05:56:06.844 INFO:tasks.workunit.client.1.smithi194.stdout:6/360: stat d3/d2d/d33/d52/d5c/f5f 0 2024-04-12T05:56:06.844 INFO:tasks.workunit.client.1.smithi194.stdout:3/498: rename d0/c5c to d0/d1/d10/d8c/d8f/ca5 0 2024-04-12T05:56:06.846 INFO:tasks.workunit.client.1.smithi194.stdout:6/361: write d3/f3a [6450699,3112] 0 2024-04-12T05:56:06.853 INFO:tasks.workunit.client.1.smithi194.stdout:8/264: write d2/f43 [258697,76295] 0 2024-04-12T05:56:06.855 INFO:tasks.workunit.client.1.smithi194.stdout:1/491: truncate d0/d11/d18/d19/d1c/f24 5184904 0 2024-04-12T05:56:06.858 INFO:tasks.workunit.client.1.smithi194.stdout:6/362: dwrite d3/d2d/d33/d52/f5a [0,4194304] 0 2024-04-12T05:56:06.858 INFO:tasks.workunit.client.1.smithi194.stdout:8/265: dread - d2/d8/d15/d33/f4b zero size 2024-04-12T05:56:06.861 INFO:tasks.workunit.client.1.smithi194.stdout:1/492: dread - d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 zero size 2024-04-12T05:56:06.862 INFO:tasks.workunit.client.1.smithi194.stdout:5/460: dwrite d1/d4/d6/d88/f56 [0,4194304] 0 2024-04-12T05:56:06.867 INFO:tasks.workunit.client.1.smithi194.stdout:1/493: fdatasync d0/d11/d18/d19/d1c/d8c/d7b/f86 0 2024-04-12T05:56:06.867 INFO:tasks.workunit.client.1.smithi194.stdout:3/499: dwrite d0/d1/d74/f3f [0,4194304] 0 2024-04-12T05:56:06.870 INFO:tasks.workunit.client.1.smithi194.stdout:5/461: unlink d1/d4/d6/db/d15/l36 0 2024-04-12T05:56:06.871 INFO:tasks.workunit.client.1.smithi194.stdout:5/462: write d1/f82 [925541,3926] 0 2024-04-12T05:56:06.871 INFO:tasks.workunit.client.1.smithi194.stdout:6/363: truncate d3/d2d/f32 1119496 0 2024-04-12T05:56:06.872 INFO:tasks.workunit.client.1.smithi194.stdout:3/500: read d0/dc/d3d/f90 [170483,67031] 0 2024-04-12T05:56:06.875 INFO:tasks.workunit.client.1.smithi194.stdout:8/266: getdents d2/d8/d9/d1a 0 2024-04-12T05:56:06.875 INFO:tasks.workunit.client.1.smithi194.stdout:6/364: chown d3/d15/f21 20 1 2024-04-12T05:56:06.876 INFO:tasks.workunit.client.1.smithi194.stdout:3/501: write d0/d1/d4f/fa3 [45427,15401] 0 2024-04-12T05:56:06.878 INFO:tasks.workunit.client.1.smithi194.stdout:8/267: dread - d2/f2d zero size 2024-04-12T05:56:06.878 INFO:tasks.workunit.client.1.smithi194.stdout:5/463: creat d1/d4/d84/f99 x:0 0 0 2024-04-12T05:56:06.883 INFO:tasks.workunit.client.1.smithi194.stdout:5/464: chown d1/d4/d6/d34/d52/l93 110 1 2024-04-12T05:56:06.883 INFO:tasks.workunit.client.1.smithi194.stdout:8/268: chown d2/d8/d9/d1a 124560739 1 2024-04-12T05:56:06.883 INFO:tasks.workunit.client.1.smithi194.stdout:3/502: symlink d0/dc/d1e/d33/la6 0 2024-04-12T05:56:06.883 INFO:tasks.workunit.client.1.smithi194.stdout:1/494: link d0/d11/d18/d19/d1a/d6a/d27/d48/l4a d0/d11/d18/d19/d1a/d6a/d27/l94 0 2024-04-12T05:56:06.886 INFO:tasks.workunit.client.1.smithi194.stdout:1/495: creat d0/df/f95 x:0 0 0 2024-04-12T05:56:06.887 INFO:tasks.workunit.client.1.smithi194.stdout:8/269: read d2/d8/d9/de/f2e [615306,61895] 0 2024-04-12T05:56:06.892 INFO:tasks.workunit.client.1.smithi194.stdout:5/465: rename d1/d4/d6/db/d38/f61 to d1/d4/d6/db/d2b/f9a 0 2024-04-12T05:56:06.893 INFO:tasks.workunit.client.1.smithi194.stdout:3/503: dread d0/d1/d10/f52 [0,4194304] 0 2024-04-12T05:56:06.894 INFO:tasks.workunit.client.1.smithi194.stdout:8/270: mknod d2/d8/d15/d33/c4f 0 2024-04-12T05:56:06.899 INFO:tasks.workunit.client.1.smithi194.stdout:3/504: mkdir d0/dc/d1e/d81/d5b/d64/d79/d87/da7 0 2024-04-12T05:56:06.900 INFO:tasks.workunit.client.1.smithi194.stdout:1/496: truncate d0/d11/d18/d19/d1a/d6a/d63/f72 593885 0 2024-04-12T05:56:06.901 INFO:tasks.workunit.client.1.smithi194.stdout:3/505: write d0/d1/f97 [1649914,53843] 0 2024-04-12T05:56:06.903 INFO:tasks.workunit.client.1.smithi194.stdout:5/466: rename d1/d5c/d95/l21 to d1/l9b 0 2024-04-12T05:56:06.904 INFO:tasks.workunit.client.1.smithi194.stdout:8/271: mknod d2/c50 0 2024-04-12T05:56:06.908 INFO:tasks.workunit.client.1.smithi194.stdout:5/467: creat d1/d4/d6/db/d38/f9c x:0 0 0 2024-04-12T05:56:06.909 INFO:tasks.workunit.client.1.smithi194.stdout:8/272: symlink d2/d8/d9/de/d38/l51 0 2024-04-12T05:56:06.910 INFO:tasks.workunit.client.1.smithi194.stdout:6/365: dread d3/d2d/d33/d52/f55 [0,4194304] 0 2024-04-12T05:56:06.910 INFO:tasks.workunit.client.1.smithi194.stdout:3/506: dread d0/d1/d74/f3f [0,4194304] 0 2024-04-12T05:56:06.910 INFO:tasks.workunit.client.1.smithi194.stdout:1/497: dwrite d0/d11/d18/d19/d1c/f20 [0,4194304] 0 2024-04-12T05:56:06.930 INFO:tasks.workunit.client.1.smithi194.stdout:1/498: dread d0/d3e/d50/f8a [0,4194304] 0 2024-04-12T05:56:06.930 INFO:tasks.workunit.client.1.smithi194.stdout:5/468: rename d1/d4/d6/db/d3b/l89 to d1/d4/d6/db/d3b/l9d 0 2024-04-12T05:56:06.933 INFO:tasks.workunit.client.1.smithi194.stdout:6/366: dwrite d3/d2d/d33/d52/f5a [0,4194304] 0 2024-04-12T05:56:06.934 INFO:tasks.workunit.client.1.smithi194.stdout:3/507: dwrite d0/dc/d3d/f66 [0,4194304] 0 2024-04-12T05:56:06.937 INFO:tasks.workunit.client.1.smithi194.stdout:6/367: unlink d3/f1e 0 2024-04-12T05:56:06.938 INFO:tasks.workunit.client.1.smithi194.stdout:6/368: readlink d3/d15/l2a 0 2024-04-12T05:56:06.939 INFO:tasks.workunit.client.1.smithi194.stdout:1/499: mkdir d0/df/d68/d96 0 2024-04-12T05:56:06.940 INFO:tasks.workunit.client.1.smithi194.stdout:6/369: write d3/d2f/d37/f3f [1211218,25672] 0 2024-04-12T05:56:06.940 INFO:tasks.workunit.client.1.smithi194.stdout:8/273: truncate d2/d8/d15/d33/f1b 516062 0 2024-04-12T05:56:06.942 INFO:tasks.workunit.client.1.smithi194.stdout:8/274: dread - d2/d8/d9/d1a/f3d zero size 2024-04-12T05:56:06.943 INFO:tasks.workunit.client.1.smithi194.stdout:3/508: rename d0/d1/d74/f9f to d0/d1/d10/fa8 0 2024-04-12T05:56:06.944 INFO:tasks.workunit.client.1.smithi194.stdout:5/469: mkdir d1/d4/d9e 0 2024-04-12T05:56:06.947 INFO:tasks.workunit.client.1.smithi194.stdout:1/500: creat d0/d11/d18/d19/d1a/d6a/d63/f97 x:0 0 0 2024-04-12T05:56:06.948 INFO:tasks.workunit.client.1.smithi194.stdout:6/370: rmdir d3/d2d/d33/d58 39 2024-04-12T05:56:06.949 INFO:tasks.workunit.client.1.smithi194.stdout:6/371: write d3/d15/d2b/d48/f4f [3772633,69340] 0 2024-04-12T05:56:06.952 INFO:tasks.workunit.client.1.smithi194.stdout:5/470: rename d1/d4/d6/db/d38/f60 to d1/d5c/d95/f9f 0 2024-04-12T05:56:06.953 INFO:tasks.workunit.client.1.smithi194.stdout:8/275: mkdir d2/d8/d15/d33/d45/d52 0 2024-04-12T05:56:06.953 INFO:tasks.workunit.client.1.smithi194.stdout:3/509: read d0/d1/d10/f2e [2340977,95998] 0 2024-04-12T05:56:06.953 INFO:tasks.workunit.client.1.smithi194.stdout:1/501: mknod d0/df/d68/d96/c98 0 2024-04-12T05:56:06.955 INFO:tasks.workunit.client.1.smithi194.stdout:5/471: creat d1/d4/d6/d34/fa0 x:0 0 0 2024-04-12T05:56:06.956 INFO:tasks.workunit.client.1.smithi194.stdout:6/372: symlink d3/d2d/d33/d52/d62/l6c 0 2024-04-12T05:56:06.956 INFO:tasks.workunit.client.1.smithi194.stdout:8/276: symlink d2/d8/d9/d1a/l53 0 2024-04-12T05:56:06.956 INFO:tasks.workunit.client.1.smithi194.stdout:5/472: fsync d1/f3 0 2024-04-12T05:56:06.958 INFO:tasks.workunit.client.1.smithi194.stdout:3/510: unlink d0/dc/d1e/d81/d5b/d64/c83 0 2024-04-12T05:56:06.961 INFO:tasks.workunit.client.1.smithi194.stdout:1/502: symlink d0/d11/d18/d19/d1c/d67/l99 0 2024-04-12T05:56:06.963 INFO:tasks.workunit.client.1.smithi194.stdout:3/511: truncate d0/dc/d53/f75 4952837 0 2024-04-12T05:56:06.970 INFO:tasks.workunit.client.1.smithi194.stdout:3/512: dread d0/d1/d4f/f59 [0,4194304] 0 2024-04-12T05:56:06.972 INFO:tasks.workunit.client.1.smithi194.stdout:1/503: creat d0/d11/d18/d19/d1c/f9a x:0 0 0 2024-04-12T05:56:06.974 INFO:tasks.workunit.client.1.smithi194.stdout:6/373: dwrite d3/d2f/d37/f4a [4194304,4194304] 0 2024-04-12T05:56:06.980 INFO:tasks.workunit.client.1.smithi194.stdout:3/513: dwrite d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:06.982 INFO:tasks.workunit.client.1.smithi194.stdout:3/514: chown d0/d1/d74/d39/f70 0 1 2024-04-12T05:56:06.982 INFO:tasks.workunit.client.1.smithi194.stdout:5/473: truncate d1/fa 682523 0 2024-04-12T05:56:06.987 INFO:tasks.workunit.client.1.smithi194.stdout:4/407: sync 2024-04-12T05:56:06.987 INFO:tasks.workunit.client.1.smithi194.stdout:5/474: creat d1/fa1 x:0 0 0 2024-04-12T05:56:06.988 INFO:tasks.workunit.client.1.smithi194.stdout:1/504: dread d0/d11/d18/d19/d1a/d6a/d63/f72 [0,4194304] 0 2024-04-12T05:56:06.990 INFO:tasks.workunit.client.1.smithi194.stdout:6/374: link d3/d15/d2b/c31 d3/d2f/d4e/d53/c6d 0 2024-04-12T05:56:07.000 INFO:tasks.workunit.client.1.smithi194.stdout:8/277: dread d2/d8/d15/d33/f27 [0,4194304] 0 2024-04-12T05:56:07.026 INFO:tasks.workunit.client.1.smithi194.stdout:4/408: fsync d2/d3/d58/f5c 0 2024-04-12T05:56:07.029 INFO:tasks.workunit.client.1.smithi194.stdout:1/505: creat d0/d3e/d5b/f9b x:0 0 0 2024-04-12T05:56:07.035 INFO:tasks.workunit.client.1.smithi194.stdout:1/506: readlink d0/d11/d18/l44 0 2024-04-12T05:56:07.041 INFO:tasks.workunit.client.1.smithi194.stdout:6/375: write d3/d15/f21 [602210,128708] 0 2024-04-12T05:56:07.041 INFO:tasks.workunit.client.1.smithi194.stdout:1/507: mknod d0/df/d68/c9c 0 2024-04-12T05:56:07.042 INFO:tasks.workunit.client.1.smithi194.stdout:8/278: creat d2/d8/f54 x:0 0 0 2024-04-12T05:56:07.051 INFO:tasks.workunit.client.1.smithi194.stdout:8/279: creat d2/d8/d15/d33/d45/f55 x:0 0 0 2024-04-12T05:56:07.053 INFO:tasks.workunit.client.1.smithi194.stdout:5/475: truncate d1/d4/d6/db/d2b/f33 691823 0 2024-04-12T05:56:07.055 INFO:tasks.workunit.client.1.smithi194.stdout:6/376: dwrite d3/d2f/f39 [0,4194304] 0 2024-04-12T05:56:07.055 INFO:tasks.workunit.client.1.smithi194.stdout:1/508: dwrite d0/d11/d18/d19/d1a/d6a/d27/d2e/f33 [8388608,4194304] 0 2024-04-12T05:56:07.056 INFO:tasks.workunit.client.1.smithi194.stdout:4/409: truncate d2/d3/d1b/d32/d3a/f3e 1075250 0 2024-04-12T05:56:07.057 INFO:tasks.workunit.client.1.smithi194.stdout:1/509: write d0/d11/f7c [714568,27375] 0 2024-04-12T05:56:07.059 INFO:tasks.workunit.client.1.smithi194.stdout:6/377: fsync d3/d15/f21 0 2024-04-12T05:56:07.060 INFO:tasks.workunit.client.1.smithi194.stdout:1/510: fsync d0/d11/d18/f2d 0 2024-04-12T05:56:07.060 INFO:tasks.workunit.client.1.smithi194.stdout:5/476: rename d1/d4/d6 to d1/d4/d6/d6d/da2 22 2024-04-12T05:56:07.066 INFO:tasks.workunit.client.1.smithi194.stdout:5/477: unlink d1/d4/d6/d88/c46 0 2024-04-12T05:56:07.069 INFO:tasks.workunit.client.1.smithi194.stdout:6/378: rename d3/d15/f28 to d3/d2d/d33/d52/f6e 0 2024-04-12T05:56:07.070 INFO:tasks.workunit.client.1.smithi194.stdout:8/280: creat d2/f56 x:0 0 0 2024-04-12T05:56:07.072 INFO:tasks.workunit.client.1.smithi194.stdout:8/281: dread - d2/f56 zero size 2024-04-12T05:56:07.073 INFO:tasks.workunit.client.1.smithi194.stdout:8/282: chown d2/d8/d9/de 61317 1 2024-04-12T05:56:07.073 INFO:tasks.workunit.client.1.smithi194.stdout:1/511: rename d0/d11/d18/d19/d1c/d8c to d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d 0 2024-04-12T05:56:07.075 INFO:tasks.workunit.client.1.smithi194.stdout:1/512: fsync d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/f64 0 2024-04-12T05:56:07.075 INFO:tasks.workunit.client.1.smithi194.stdout:4/410: link d2/d3/d51/f55 d2/d3/f91 0 2024-04-12T05:56:07.078 INFO:tasks.workunit.client.1.smithi194.stdout:4/411: chown d2/d3/f18 849147 1 2024-04-12T05:56:07.078 INFO:tasks.workunit.client.1.smithi194.stdout:5/478: creat d1/fa3 x:0 0 0 2024-04-12T05:56:07.079 INFO:tasks.workunit.client.1.smithi194.stdout:8/283: write d2/d8/f36 [1023394,45813] 0 2024-04-12T05:56:07.079 INFO:tasks.workunit.client.1.smithi194.stdout:6/379: read d3/d2d/d33/d52/f6e [1310569,128449] 0 2024-04-12T05:56:07.081 INFO:tasks.workunit.client.1.smithi194.stdout:8/284: fsync d2/d8/d9/d1a/f3d 0 2024-04-12T05:56:07.083 INFO:tasks.workunit.client.1.smithi194.stdout:4/412: read d2/d11/d1e/d25/f7a [1472681,79125] 0 2024-04-12T05:56:07.084 INFO:tasks.workunit.client.1.smithi194.stdout:1/513: mkdir d0/d11/d18/d19/d1a/d6a/d27/d9e 0 2024-04-12T05:56:07.086 INFO:tasks.workunit.client.1.smithi194.stdout:6/380: mkdir d3/d15/d2b/d48/d6f 0 2024-04-12T05:56:07.086 INFO:tasks.workunit.client.1.smithi194.stdout:1/514: dread - d0/df/f95 zero size 2024-04-12T05:56:07.086 INFO:tasks.workunit.client.1.smithi194.stdout:8/285: chown d2/d8/f36 154237 1 2024-04-12T05:56:07.090 INFO:tasks.workunit.client.1.smithi194.stdout:5/479: symlink d1/d4/d6/db/la4 0 2024-04-12T05:56:07.092 INFO:tasks.workunit.client.1.smithi194.stdout:4/413: write d2/f4e [1578692,86491] 0 2024-04-12T05:56:07.094 INFO:tasks.workunit.client.1.smithi194.stdout:1/515: creat d0/d11/f9f x:0 0 0 2024-04-12T05:56:07.094 INFO:tasks.workunit.client.1.smithi194.stdout:5/480: chown d1/d4/d6/f3d 4034 1 2024-04-12T05:56:07.095 INFO:tasks.workunit.client.1.smithi194.stdout:8/286: dread - d2/d8/d15/d35/f3f zero size 2024-04-12T05:56:07.095 INFO:tasks.workunit.client.1.smithi194.stdout:4/414: read - d2/d11/d1e/d25/f6e zero size 2024-04-12T05:56:07.096 INFO:tasks.workunit.client.1.smithi194.stdout:8/287: dread - d2/d8/f54 zero size 2024-04-12T05:56:07.102 INFO:tasks.workunit.client.1.smithi194.stdout:6/381: rename d3/d2f/d4e/d63/d64 to d3/d70 0 2024-04-12T05:56:07.107 INFO:tasks.workunit.client.1.smithi194.stdout:5/481: symlink d1/d4/d6/d88/la5 0 2024-04-12T05:56:07.107 INFO:tasks.workunit.client.1.smithi194.stdout:8/288: unlink d2/d8/d9/de/d38/l44 0 2024-04-12T05:56:07.108 INFO:tasks.workunit.client.1.smithi194.stdout:8/289: chown d2/d8/d9/f48 8202787 1 2024-04-12T05:56:07.111 INFO:tasks.workunit.client.1.smithi194.stdout:6/382: truncate d3/d1a/f35 1686308 0 2024-04-12T05:56:07.118 INFO:tasks.workunit.client.1.smithi194.stdout:8/290: dwrite d2/fb [0,4194304] 0 2024-04-12T05:56:07.119 INFO:tasks.workunit.client.1.smithi194.stdout:5/482: write d1/d4/d6/db/d2b/f66 [2306929,44720] 0 2024-04-12T05:56:07.119 INFO:tasks.workunit.client.1.smithi194.stdout:4/415: truncate d2/d11/d1e/d2f/f76 1084101 0 2024-04-12T05:56:07.120 INFO:tasks.workunit.client.1.smithi194.stdout:4/416: readlink d2/d11/d1e/l2d 0 2024-04-12T05:56:07.123 INFO:tasks.workunit.client.1.smithi194.stdout:4/417: creat d2/d11/d46/d61/f92 x:0 0 0 2024-04-12T05:56:07.127 INFO:tasks.workunit.client.1.smithi194.stdout:4/418: mkdir d2/d11/d93 0 2024-04-12T05:56:07.130 INFO:tasks.workunit.client.1.smithi194.stdout:6/383: truncate d3/d15/d2b/d48/f4f 2312903 0 2024-04-12T05:56:07.134 INFO:tasks.workunit.client.1.smithi194.stdout:4/419: dread d2/d11/d1e/d25/f7a [0,4194304] 0 2024-04-12T05:56:07.138 INFO:tasks.workunit.client.1.smithi194.stdout:6/384: rename d3/d15/d2b/c31 to d3/d2f/d37/c71 0 2024-04-12T05:56:07.138 INFO:tasks.workunit.client.1.smithi194.stdout:5/483: truncate d1/d4/d6/db/fc 2033114 0 2024-04-12T05:56:07.139 INFO:tasks.workunit.client.1.smithi194.stdout:5/484: readlink d1/d4/d75/l90 0 2024-04-12T05:56:07.140 INFO:tasks.workunit.client.1.smithi194.stdout:6/385: rename d3/d1a/l25 to d3/d2f/d4e/d68/l72 0 2024-04-12T05:56:07.141 INFO:tasks.workunit.client.1.smithi194.stdout:8/291: dread d2/d8/d15/d33/f18 [0,4194304] 0 2024-04-12T05:56:07.142 INFO:tasks.workunit.client.1.smithi194.stdout:6/386: readlink d3/l26 0 2024-04-12T05:56:07.145 INFO:tasks.workunit.client.1.smithi194.stdout:8/292: write d2/d8/d15/d33/f27 [5713028,104258] 0 2024-04-12T05:56:07.146 INFO:tasks.workunit.client.1.smithi194.stdout:4/420: dwrite d2/d11/d1e/d25/f6e [0,4194304] 0 2024-04-12T05:56:07.146 INFO:tasks.workunit.client.1.smithi194.stdout:8/293: dread - d2/d8/d15/d35/f3f zero size 2024-04-12T05:56:07.149 INFO:tasks.workunit.client.1.smithi194.stdout:5/485: creat d1/d4/d6/db/d3b/fa6 x:0 0 0 2024-04-12T05:56:07.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/387: symlink d3/d2f/d4e/d68/l73 0 2024-04-12T05:56:07.150 INFO:tasks.workunit.client.1.smithi194.stdout:6/388: readlink d3/d2d/d33/d52/l5b 0 2024-04-12T05:56:07.150 INFO:tasks.workunit.client.1.smithi194.stdout:4/421: mknod d2/d11/d46/c94 0 2024-04-12T05:56:07.155 INFO:tasks.workunit.client.1.smithi194.stdout:5/486: mkdir d1/d5c/da7 0 2024-04-12T05:56:07.157 INFO:tasks.workunit.client.1.smithi194.stdout:4/422: dread d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:07.157 INFO:tasks.workunit.client.1.smithi194.stdout:5/487: dread - d1/d4/d6/db/f7a zero size 2024-04-12T05:56:07.157 INFO:tasks.workunit.client.1.smithi194.stdout:6/389: creat d3/d15/d2b/d48/f74 x:0 0 0 2024-04-12T05:56:07.160 INFO:tasks.workunit.client.1.smithi194.stdout:6/390: fdatasync d3/d2f/d4e/d53/f6a 0 2024-04-12T05:56:07.162 INFO:tasks.workunit.client.1.smithi194.stdout:5/488: creat d1/d4/d6/d8e/fa8 x:0 0 0 2024-04-12T05:56:07.168 INFO:tasks.workunit.client.1.smithi194.stdout:8/294: rename d2/d8/d9/c34 to d2/d8/c57 0 2024-04-12T05:56:07.170 INFO:tasks.workunit.client.1.smithi194.stdout:5/489: mknod d1/d4/d6/ca9 0 2024-04-12T05:56:07.171 INFO:tasks.workunit.client.1.smithi194.stdout:4/423: dwrite d2/d3/d1b/d32/d3a/d3b/f86 [0,4194304] 0 2024-04-12T05:56:07.172 INFO:tasks.workunit.client.1.smithi194.stdout:6/391: dwrite d3/d2d/d33/d52/d5c/f5f [0,4194304] 0 2024-04-12T05:56:07.175 INFO:tasks.workunit.client.1.smithi194.stdout:8/295: mkdir d2/d8/d58 0 2024-04-12T05:56:07.179 INFO:tasks.workunit.client.1.smithi194.stdout:5/490: fdatasync d1/d4/d6/db/d3b/f3e 0 2024-04-12T05:56:07.180 INFO:tasks.workunit.client.1.smithi194.stdout:5/491: readlink d1/d4/d6/d6d/l91 0 2024-04-12T05:56:07.181 INFO:tasks.workunit.client.1.smithi194.stdout:4/424: link d2/d3/d1b/d32/d3a/d3b/f7b d2/d3/d50/f95 0 2024-04-12T05:56:07.182 INFO:tasks.workunit.client.1.smithi194.stdout:4/425: truncate d2/d3/f3f 570259 0 2024-04-12T05:56:07.183 INFO:tasks.workunit.client.1.smithi194.stdout:4/426: write d2/d11/f88 [804762,84921] 0 2024-04-12T05:56:07.187 INFO:tasks.workunit.client.1.smithi194.stdout:5/492: dread d1/d4/d6/f68 [0,4194304] 0 2024-04-12T05:56:07.188 INFO:tasks.workunit.client.1.smithi194.stdout:5/493: chown d1/d4/d6/db/d38/c5d 76985 1 2024-04-12T05:56:07.238 INFO:tasks.workunit.client.1.smithi194.stdout:7/245: sync 2024-04-12T05:56:07.238 INFO:tasks.workunit.client.1.smithi194.stdout:7/246: write d3/d27/f1b [5197711,65088] 0 2024-04-12T05:56:07.239 INFO:tasks.workunit.client.1.smithi194.stdout:7/247: write d3/d27/f46 [501644,59124] 0 2024-04-12T05:56:07.247 INFO:tasks.workunit.client.1.smithi194.stdout:7/248: dread d3/d16/f19 [0,4194304] 0 2024-04-12T05:56:07.268 INFO:tasks.workunit.client.1.smithi194.stdout:8/296: mknod d2/d8/d9/de/d38/c59 0 2024-04-12T05:56:07.270 INFO:tasks.workunit.client.1.smithi194.stdout:4/427: symlink d2/d11/d1e/d25/l96 0 2024-04-12T05:56:07.271 INFO:tasks.workunit.client.1.smithi194.stdout:5/494: creat d1/d4/d6/db/d15/faa x:0 0 0 2024-04-12T05:56:07.282 INFO:tasks.workunit.client.1.smithi194.stdout:6/392: rename d3/l26 to d3/d2d/d33/l75 0 2024-04-12T05:56:07.288 INFO:tasks.workunit.client.1.smithi194.stdout:7/249: unlink d3/d16/d21/c2d 0 2024-04-12T05:56:07.292 INFO:tasks.workunit.client.1.smithi194.stdout:6/393: rename d3/d2d/d33/d52/d62/l6c to d3/d2d/d33/d52/d5c/l76 0 2024-04-12T05:56:07.293 INFO:tasks.workunit.client.1.smithi194.stdout:4/428: creat d2/d11/d1e/d2f/d4a/f97 x:0 0 0 2024-04-12T05:56:07.294 INFO:tasks.workunit.client.1.smithi194.stdout:5/495: mknod d1/d4/d6/cab 0 2024-04-12T05:56:07.295 INFO:tasks.workunit.client.1.smithi194.stdout:4/429: chown d2/d3/f18 22948 1 2024-04-12T05:56:07.297 INFO:tasks.workunit.client.1.smithi194.stdout:7/250: dwrite d3/d27/d22/f38 [0,4194304] 0 2024-04-12T05:56:07.298 INFO:tasks.workunit.client.1.smithi194.stdout:8/297: getdents d2/d8/d15 0 2024-04-12T05:56:07.303 INFO:tasks.workunit.client.1.smithi194.stdout:7/251: write d3/d27/f1d [163058,112879] 0 2024-04-12T05:56:07.303 INFO:tasks.workunit.client.1.smithi194.stdout:8/298: getdents d2/d8/d15/d33/d49 0 2024-04-12T05:56:07.308 INFO:tasks.workunit.client.1.smithi194.stdout:8/299: fsync d2/d8/d9/de/d10/f3c 0 2024-04-12T05:56:07.311 INFO:tasks.workunit.client.1.smithi194.stdout:5/496: dwrite d1/d4/d6/d34/fa0 [0,4194304] 0 2024-04-12T05:56:07.311 INFO:tasks.workunit.client.1.smithi194.stdout:7/252: creat d3/d16/d21/d3a/d47/f51 x:0 0 0 2024-04-12T05:56:07.314 INFO:tasks.workunit.client.1.smithi194.stdout:8/300: creat d2/d8/d9/d1a/f5a x:0 0 0 2024-04-12T05:56:07.314 INFO:tasks.workunit.client.1.smithi194.stdout:5/497: fdatasync d1/d4/d6/db/d15/faa 0 2024-04-12T05:56:07.316 INFO:tasks.workunit.client.1.smithi194.stdout:6/394: dwrite d3/d15/d2b/d48/f49 [0,4194304] 0 2024-04-12T05:56:07.317 INFO:tasks.workunit.client.1.smithi194.stdout:8/301: read d2/fb [102412,86738] 0 2024-04-12T05:56:07.320 INFO:tasks.workunit.client.1.smithi194.stdout:6/395: mkdir d3/d2f/d4e/d53/d77 0 2024-04-12T05:56:07.320 INFO:tasks.workunit.client.1.smithi194.stdout:4/430: dread d2/d11/d43/f62 [0,4194304] 0 2024-04-12T05:56:07.322 INFO:tasks.workunit.client.1.smithi194.stdout:5/498: rename d1/d5c/d95/f6a to d1/d4/d6/db/d15/d26/fac 0 2024-04-12T05:56:07.323 INFO:tasks.workunit.client.1.smithi194.stdout:8/302: rename d2/d8 to d2/d8/d15/d33/d49/d5b 22 2024-04-12T05:56:07.326 INFO:tasks.workunit.client.1.smithi194.stdout:4/431: read d2/d3/f91 [693436,58219] 0 2024-04-12T05:56:07.327 INFO:tasks.workunit.client.1.smithi194.stdout:6/396: rename d3/d34/f5d to d3/d2f/d4e/d63/f78 0 2024-04-12T05:56:07.328 INFO:tasks.workunit.client.1.smithi194.stdout:5/499: mkdir d1/d4/dad 0 2024-04-12T05:56:07.329 INFO:tasks.workunit.client.1.smithi194.stdout:6/397: truncate d3/d15/f21 1317946 0 2024-04-12T05:56:07.330 INFO:tasks.workunit.client.1.smithi194.stdout:8/303: symlink d2/d8/d9/de/d10/l5c 0 2024-04-12T05:56:07.330 INFO:tasks.workunit.client.1.smithi194.stdout:6/398: chown d3/f3a 7502251 1 2024-04-12T05:56:07.332 INFO:tasks.workunit.client.1.smithi194.stdout:4/432: symlink d2/d11/d1e/d25/d2b/d5d/l98 0 2024-04-12T05:56:07.334 INFO:tasks.workunit.client.1.smithi194.stdout:5/500: chown d1/d4/d6/d88/f2c 6252842 1 2024-04-12T05:56:07.335 INFO:tasks.workunit.client.1.smithi194.stdout:4/433: mknod d2/d11/d1f/c99 0 2024-04-12T05:56:07.342 INFO:tasks.workunit.client.1.smithi194.stdout:5/501: unlink d1/d4/d6/db/d15/d26/c3f 0 2024-04-12T05:56:07.342 INFO:tasks.workunit.client.1.smithi194.stdout:4/434: write d2/d11/d1e/d25/f6e [5025840,26323] 0 2024-04-12T05:56:07.342 INFO:tasks.workunit.client.1.smithi194.stdout:8/304: dwrite d2/d8/d9/d1a/f3d [0,4194304] 0 2024-04-12T05:56:07.343 INFO:tasks.workunit.client.1.smithi194.stdout:7/253: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:07.350 INFO:tasks.workunit.client.1.smithi194.stdout:7/254: chown d3/d27/f7 7065 1 2024-04-12T05:56:07.351 INFO:tasks.workunit.client.1.smithi194.stdout:8/305: rename d2/c50 to d2/d8/d9/d1a/c5d 0 2024-04-12T05:56:07.352 INFO:tasks.workunit.client.1.smithi194.stdout:7/255: readlink d3/l14 0 2024-04-12T05:56:07.359 INFO:tasks.workunit.client.1.smithi194.stdout:5/502: dwrite d1/d4/d6/d8e/f81 [0,4194304] 0 2024-04-12T05:56:07.361 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:07 smithi008 ceph-mon[39722]: pgmap v29: 65 pgs: 65 active+clean; 908 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 19 MiB/s rd, 111 MiB/s wr, 373 op/s 2024-04-12T05:56:07.364 INFO:tasks.workunit.client.1.smithi194.stdout:4/435: dwrite d2/d11/d46/d61/d63/f84 [0,4194304] 0 2024-04-12T05:56:07.365 INFO:tasks.workunit.client.1.smithi194.stdout:6/399: dread d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:07.368 INFO:tasks.workunit.client.1.smithi194.stdout:5/503: stat d1/d4/d6/db/f4a 0 2024-04-12T05:56:07.369 INFO:tasks.workunit.client.1.smithi194.stdout:6/400: readlink d3/d2d/d33/d52/l5b 0 2024-04-12T05:56:07.370 INFO:tasks.workunit.client.1.smithi194.stdout:4/436: stat d2/d3/d1b/l29 0 2024-04-12T05:56:07.370 INFO:tasks.workunit.client.1.smithi194.stdout:7/256: dwrite d3/d16/f35 [0,4194304] 0 2024-04-12T05:56:07.371 INFO:tasks.workunit.client.1.smithi194.stdout:6/401: fdatasync d3/f1f 0 2024-04-12T05:56:07.372 INFO:tasks.workunit.client.1.smithi194.stdout:8/306: dread d2/d8/d15/d35/f4c [0,4194304] 0 2024-04-12T05:56:07.374 INFO:tasks.workunit.client.1.smithi194.stdout:4/437: dread d2/d3/f3f [0,4194304] 0 2024-04-12T05:56:07.378 INFO:tasks.workunit.client.1.smithi194.stdout:4/438: unlink d2/l7 0 2024-04-12T05:56:07.380 INFO:tasks.workunit.client.1.smithi194.stdout:5/504: chown d1/d4/d6/d8e/f81 164064834 1 2024-04-12T05:56:07.381 INFO:tasks.workunit.client.1.smithi194.stdout:6/402: rename d3/d2d/d33/d58/f59 to d3/d2f/d4e/d63/f79 0 2024-04-12T05:56:07.381 INFO:tasks.workunit.client.1.smithi194.stdout:8/307: getdents d2/d8/d9/de/d38 0 2024-04-12T05:56:07.382 INFO:tasks.workunit.client.1.smithi194.stdout:5/505: stat d1/d4/d6/d88 0 2024-04-12T05:56:07.385 INFO:tasks.workunit.client.1.smithi194.stdout:4/439: mknod d2/c9a 0 2024-04-12T05:56:07.386 INFO:tasks.workunit.client.1.smithi194.stdout:5/506: fsync d1/d4/d6/db/d15/d26/fac 0 2024-04-12T05:56:07.388 INFO:tasks.workunit.client.1.smithi194.stdout:6/403: truncate d3/d1a/f35 1009007 0 2024-04-12T05:56:07.390 INFO:tasks.workunit.client.1.smithi194.stdout:4/440: unlink d2/d11/d46/d61/f70 0 2024-04-12T05:56:07.394 INFO:tasks.workunit.client.1.smithi194.stdout:4/441: symlink d2/d11/d1e/l9b 0 2024-04-12T05:56:07.398 INFO:tasks.workunit.client.1.smithi194.stdout:6/404: mkdir d3/d15/d2b/d48/d6f/d7a 0 2024-04-12T05:56:07.399 INFO:tasks.workunit.client.1.smithi194.stdout:4/442: mkdir d2/d11/d1e/d2f/d4a/d9c 0 2024-04-12T05:56:07.402 INFO:tasks.workunit.client.1.smithi194.stdout:6/405: symlink d3/d2f/d4e/l7b 0 2024-04-12T05:56:07.403 INFO:tasks.workunit.client.1.smithi194.stdout:4/443: fsync d2/d3/f27 0 2024-04-12T05:56:07.404 INFO:tasks.workunit.client.1.smithi194.stdout:4/444: readlink d2/d22/l37 0 2024-04-12T05:56:07.405 INFO:tasks.workunit.client.1.smithi194.stdout:6/406: rename d3/l40 to d3/d2d/d33/d52/d62/l7c 0 2024-04-12T05:56:07.408 INFO:tasks.workunit.client.1.smithi194.stdout:6/407: creat d3/d2d/d33/d52/f7d x:0 0 0 2024-04-12T05:56:07.409 INFO:tasks.workunit.client.1.smithi194.stdout:5/507: dwrite d1/fa [0,4194304] 0 2024-04-12T05:56:07.412 INFO:tasks.workunit.client.1.smithi194.stdout:5/508: creat d1/d4/d6/d88/fae x:0 0 0 2024-04-12T05:56:07.414 INFO:tasks.workunit.client.1.smithi194.stdout:5/509: rmdir d1/d4/d4b 39 2024-04-12T05:56:07.440 INFO:tasks.workunit.client.1.smithi194.stdout:9/451: sync 2024-04-12T05:56:07.440 INFO:tasks.workunit.client.1.smithi194.stdout:0/358: sync 2024-04-12T05:56:07.440 INFO:tasks.workunit.client.1.smithi194.stdout:9/452: stat d1/c15 0 2024-04-12T05:56:07.448 INFO:tasks.workunit.client.1.smithi194.stdout:9/453: rename d1/d19/d1f/d20/d37/d60/f8a to d1/d19/d1f/d20/d37/d38/f8f 0 2024-04-12T05:56:07.449 INFO:tasks.workunit.client.1.smithi194.stdout:4/445: write d2/d3/f12 [185351,91042] 0 2024-04-12T05:56:07.451 INFO:tasks.workunit.client.1.smithi194.stdout:9/454: mknod d1/d21/d49/c90 0 2024-04-12T05:56:07.456 INFO:tasks.workunit.client.1.smithi194.stdout:0/359: dread d0/da/f3e [0,4194304] 0 2024-04-12T05:56:07.460 INFO:tasks.workunit.client.1.smithi194.stdout:6/408: dread d3/d2d/d33/d52/f6e [0,4194304] 0 2024-04-12T05:56:07.460 INFO:tasks.workunit.client.1.smithi194.stdout:0/360: creat d0/da/d1e/d1a/d24/d56/d66/f72 x:0 0 0 2024-04-12T05:56:07.463 INFO:tasks.workunit.client.1.smithi194.stdout:6/409: symlink d3/d15/d2b/d48/l7e 0 2024-04-12T05:56:07.463 INFO:tasks.workunit.client.1.smithi194.stdout:9/455: dread d1/d19/d1f/d20/d37/d38/f5c [0,4194304] 0 2024-04-12T05:56:07.465 INFO:tasks.workunit.client.1.smithi194.stdout:6/410: readlink d3/d15/d2b/l3e 0 2024-04-12T05:56:07.466 INFO:tasks.workunit.client.1.smithi194.stdout:4/446: dread d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:07.466 INFO:tasks.workunit.client.1.smithi194.stdout:9/456: creat d1/d19/d1f/d5f/f91 x:0 0 0 2024-04-12T05:56:07.467 INFO:tasks.workunit.client.1.smithi194.stdout:6/411: stat d3/d2f/d4e/l7b 0 2024-04-12T05:56:07.467 INFO:tasks.workunit.client.1.smithi194.stdout:9/457: fdatasync d1/d19/d1f/d5f/f91 0 2024-04-12T05:56:07.469 INFO:tasks.workunit.client.1.smithi194.stdout:9/458: truncate d1/d19/d1f/f4b 715142 0 2024-04-12T05:56:07.470 INFO:tasks.workunit.client.1.smithi194.stdout:6/412: creat d3/d2d/d33/d58/f7f x:0 0 0 2024-04-12T05:56:07.471 INFO:tasks.workunit.client.1.smithi194.stdout:9/459: chown d1/d19/d1f/d20/d37/d38/f8f 156597476 1 2024-04-12T05:56:07.473 INFO:tasks.workunit.client.1.smithi194.stdout:6/413: chown d3/d34/c66 23711497 1 2024-04-12T05:56:07.473 INFO:tasks.workunit.client.1.smithi194.stdout:0/361: truncate d0/da/d1e/f41 1996681 0 2024-04-12T05:56:07.474 INFO:tasks.workunit.client.1.smithi194.stdout:6/414: chown d3/d2d/d33/d52/d5c 48 1 2024-04-12T05:56:07.474 INFO:tasks.workunit.client.1.smithi194.stdout:6/415: stat d3 0 2024-04-12T05:56:07.475 INFO:tasks.workunit.client.1.smithi194.stdout:0/362: mkdir d0/d5f/d73 0 2024-04-12T05:56:07.476 INFO:tasks.workunit.client.1.smithi194.stdout:6/416: truncate d3/d2f/d4e/d63/f79 5058277 0 2024-04-12T05:56:07.477 INFO:tasks.workunit.client.1.smithi194.stdout:9/460: link d1/d19/f2e d1/d21/d77/f92 0 2024-04-12T05:56:07.478 INFO:tasks.workunit.client.1.smithi194.stdout:9/461: fdatasync d1/f5 0 2024-04-12T05:56:07.480 INFO:tasks.workunit.client.1.smithi194.stdout:9/462: chown d1/d21/f3e 36051 1 2024-04-12T05:56:07.480 INFO:tasks.workunit.client.1.smithi194.stdout:4/447: dread d2/d11/d1e/f42 [0,4194304] 0 2024-04-12T05:56:07.482 INFO:tasks.workunit.client.1.smithi194.stdout:9/463: write d1/d21/d39/f89 [665122,49651] 0 2024-04-12T05:56:07.483 INFO:tasks.workunit.client.1.smithi194.stdout:4/448: write d2/d11/d46/d61/f78 [369898,65853] 0 2024-04-12T05:56:07.487 INFO:tasks.workunit.client.1.smithi194.stdout:9/464: symlink d1/d19/d1f/d20/d37/l93 0 2024-04-12T05:56:07.488 INFO:tasks.workunit.client.1.smithi194.stdout:4/449: mknod d2/d11/d1e/d25/d2b/d5d/c9d 0 2024-04-12T05:56:07.490 INFO:tasks.workunit.client.1.smithi194.stdout:0/363: truncate d0/da/d1e/d1a/f23 1314911 0 2024-04-12T05:56:07.491 INFO:tasks.workunit.client.1.smithi194.stdout:9/465: link d1/l74 d1/d21/d49/l94 0 2024-04-12T05:56:07.494 INFO:tasks.workunit.client.1.smithi194.stdout:4/450: truncate d2/d3/f27 222138 0 2024-04-12T05:56:07.495 INFO:tasks.workunit.client.1.smithi194.stdout:0/364: creat d0/d3b/f74 x:0 0 0 2024-04-12T05:56:07.496 INFO:tasks.workunit.client.1.smithi194.stdout:6/417: dread d3/f3a [4194304,4194304] 0 2024-04-12T05:56:07.498 INFO:tasks.workunit.client.1.smithi194.stdout:6/418: write d3/f3b [1968054,101344] 0 2024-04-12T05:56:07.501 INFO:tasks.workunit.client.1.smithi194.stdout:4/451: creat d2/d3/d50/f9e x:0 0 0 2024-04-12T05:56:07.502 INFO:tasks.workunit.client.1.smithi194.stdout:4/452: read - d2/f8f zero size 2024-04-12T05:56:07.505 INFO:tasks.workunit.client.1.smithi194.stdout:6/419: write d3/d15/d2b/f5e [1687689,88578] 0 2024-04-12T05:56:07.508 INFO:tasks.workunit.client.1.smithi194.stdout:4/453: getdents d2/d11/d1f/d81 0 2024-04-12T05:56:07.509 INFO:tasks.workunit.client.1.smithi194.stdout:9/466: link d1/d19/d1f/d20/c2f d1/d19/d1f/d20/d37/d84/c95 0 2024-04-12T05:56:07.509 INFO:tasks.workunit.client.1.smithi194.stdout:4/454: chown d2/f67 56838080 1 2024-04-12T05:56:07.510 INFO:tasks.workunit.client.1.smithi194.stdout:4/455: stat d2/d11/d1f/c83 0 2024-04-12T05:56:07.514 INFO:tasks.workunit.client.1.smithi194.stdout:4/456: creat d2/d3/d1b/d32/d3a/d3b/f9f x:0 0 0 2024-04-12T05:56:07.516 INFO:tasks.workunit.client.1.smithi194.stdout:6/420: write d3/d1a/f35 [220739,1064] 0 2024-04-12T05:56:07.518 INFO:tasks.workunit.client.1.smithi194.stdout:4/457: read d2/d3/ff [3782090,117970] 0 2024-04-12T05:56:07.521 INFO:tasks.workunit.client.1.smithi194.stdout:0/365: dread d0/f30 [0,4194304] 0 2024-04-12T05:56:07.521 INFO:tasks.workunit.client.1.smithi194.stdout:9/467: dwrite d1/d19/d1f/f73 [0,4194304] 0 2024-04-12T05:56:07.522 INFO:tasks.workunit.client.1.smithi194.stdout:9/468: stat d1/d4e/f55 0 2024-04-12T05:56:07.527 INFO:tasks.workunit.client.1.smithi194.stdout:6/421: write d3/f18 [8333519,49488] 0 2024-04-12T05:56:07.529 INFO:tasks.workunit.client.1.smithi194.stdout:9/469: chown d1/l33 2526099 1 2024-04-12T05:56:07.529 INFO:tasks.workunit.client.1.smithi194.stdout:4/458: write d2/d11/d1e/d2f/f76 [1147462,39591] 0 2024-04-12T05:56:07.529 INFO:tasks.workunit.client.1.smithi194.stdout:0/366: symlink d0/da/d1e/d1a/l75 0 2024-04-12T05:56:07.530 INFO:tasks.workunit.client.1.smithi194.stdout:6/422: dread - d3/d2d/d33/d58/f7f zero size 2024-04-12T05:56:07.533 INFO:tasks.workunit.client.1.smithi194.stdout:0/367: unlink d0/f4 0 2024-04-12T05:56:07.534 INFO:tasks.workunit.client.1.smithi194.stdout:4/459: mkdir d2/d3/d50/da0 0 2024-04-12T05:56:07.536 INFO:tasks.workunit.client.1.smithi194.stdout:0/368: mkdir d0/d5f/d73/d76 0 2024-04-12T05:56:07.536 INFO:tasks.workunit.client.1.smithi194.stdout:9/470: dwrite d1/f5 [0,4194304] 0 2024-04-12T05:56:07.537 INFO:tasks.workunit.client.1.smithi194.stdout:4/460: chown d2/d11/d1e/d25/l96 14702 1 2024-04-12T05:56:07.539 INFO:tasks.workunit.client.1.smithi194.stdout:0/369: mknod d0/da/d1e/d1a/d24/c77 0 2024-04-12T05:56:07.546 INFO:tasks.workunit.client.1.smithi194.stdout:0/370: write d0/da/d1e/d1a/d24/d56/d66/f68 [598779,11276] 0 2024-04-12T05:56:07.546 INFO:tasks.workunit.client.1.smithi194.stdout:4/461: creat d2/d3/d58/fa1 x:0 0 0 2024-04-12T05:56:07.547 INFO:tasks.workunit.client.1.smithi194.stdout:9/471: rename d1/d21/f3e to d1/d19/d1f/d20/d37/d38/f96 0 2024-04-12T05:56:07.547 INFO:tasks.workunit.client.1.smithi194.stdout:4/462: chown d2/d11/d46/d61/f78 74915858 1 2024-04-12T05:56:07.547 INFO:tasks.workunit.client.1.smithi194.stdout:0/371: mknod d0/d5f/d73/d76/c78 0 2024-04-12T05:56:07.547 INFO:tasks.workunit.client.1.smithi194.stdout:0/372: chown d0/da/f2d 388 1 2024-04-12T05:56:07.550 INFO:tasks.workunit.client.1.smithi194.stdout:0/373: link d0/da/l5d d0/l79 0 2024-04-12T05:56:07.562 INFO:tasks.workunit.client.1.smithi194.stdout:9/472: write d1/d19/f2e [1318418,86472] 0 2024-04-12T05:56:07.562 INFO:tasks.workunit.client.1.smithi194.stdout:4/463: write d2/d3/f27 [468861,16229] 0 2024-04-12T05:56:07.565 INFO:tasks.workunit.client.1.smithi194.stdout:9/473: mkdir d1/d41/d97 0 2024-04-12T05:56:07.567 INFO:tasks.workunit.client.1.smithi194.stdout:9/474: mkdir d1/d21/d77/d98 0 2024-04-12T05:56:07.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:07 smithi194 ceph-mon[49771]: pgmap v29: 65 pgs: 65 active+clean; 908 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 19 MiB/s rd, 111 MiB/s wr, 373 op/s 2024-04-12T05:56:07.574 INFO:tasks.workunit.client.1.smithi194.stdout:4/464: dwrite d2/d11/d1e/d25/f7a [0,4194304] 0 2024-04-12T05:56:07.575 INFO:tasks.workunit.client.1.smithi194.stdout:0/374: dread d0/f11 [0,4194304] 0 2024-04-12T05:56:07.580 INFO:tasks.workunit.client.1.smithi194.stdout:4/465: mkdir d2/d11/d46/d61/d63/da2 0 2024-04-12T05:56:07.580 INFO:tasks.workunit.client.1.smithi194.stdout:0/375: write d0/da/d1e/f19 [5112204,91348] 0 2024-04-12T05:56:07.584 INFO:tasks.workunit.client.1.smithi194.stdout:0/376: readlink d0/l79 0 2024-04-12T05:56:07.586 INFO:tasks.workunit.client.1.smithi194.stdout:4/466: getdents d2/d11 0 2024-04-12T05:56:07.587 INFO:tasks.workunit.client.1.smithi194.stdout:0/377: creat d0/da/d1e/d1a/d3d/f7a x:0 0 0 2024-04-12T05:56:07.588 INFO:tasks.workunit.client.1.smithi194.stdout:4/467: dread - d2/d3/f77 zero size 2024-04-12T05:56:07.589 INFO:tasks.workunit.client.1.smithi194.stdout:9/475: dwrite d1/d21/d39/d72/f75 [0,4194304] 0 2024-04-12T05:56:07.594 INFO:tasks.workunit.client.1.smithi194.stdout:0/378: rename d0/da/d4a/f4e to d0/da/d1e/d27/d54/f7b 0 2024-04-12T05:56:07.595 INFO:tasks.workunit.client.1.smithi194.stdout:0/379: read d0/da/f3e [303015,4111] 0 2024-04-12T05:56:07.600 INFO:tasks.workunit.client.1.smithi194.stdout:4/468: dwrite d2/fc [0,4194304] 0 2024-04-12T05:56:07.601 INFO:tasks.workunit.client.1.smithi194.stdout:0/380: dread d0/da/f3e [0,4194304] 0 2024-04-12T05:56:07.602 INFO:tasks.workunit.client.1.smithi194.stdout:4/469: unlink d2/d11/d1e/l2d 0 2024-04-12T05:56:07.607 INFO:tasks.workunit.client.1.smithi194.stdout:4/470: mkdir d2/d3/d1b/d32/da3 0 2024-04-12T05:56:07.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/476: dread d1/f5b [0,4194304] 0 2024-04-12T05:56:07.612 INFO:tasks.workunit.client.1.smithi194.stdout:4/471: getdents d2/d11/d46 0 2024-04-12T05:56:07.612 INFO:tasks.workunit.client.1.smithi194.stdout:0/381: dwrite d0/da/d1e/d27/d54/f71 [0,4194304] 0 2024-04-12T05:56:07.616 INFO:tasks.workunit.client.1.smithi194.stdout:4/472: readlink d2/d11/d1f/l36 0 2024-04-12T05:56:07.624 INFO:tasks.workunit.client.1.smithi194.stdout:9/477: dwrite d1/d19/d1f/d20/d37/d38/f7c [0,4194304] 0 2024-04-12T05:56:07.629 INFO:tasks.workunit.client.1.smithi194.stdout:9/478: chown d1/d21/d49/c67 53326 1 2024-04-12T05:56:07.630 INFO:tasks.workunit.client.1.smithi194.stdout:9/479: readlink d1/d19/d1f/d20/d37/d38/l40 0 2024-04-12T05:56:07.630 INFO:tasks.workunit.client.1.smithi194.stdout:0/382: dwrite d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:07.634 INFO:tasks.workunit.client.1.smithi194.stdout:4/473: dwrite d2/d3/d58/f69 [0,4194304] 0 2024-04-12T05:56:07.635 INFO:tasks.workunit.client.1.smithi194.stdout:0/383: chown d0/d3b/f74 3570866 1 2024-04-12T05:56:07.636 INFO:tasks.workunit.client.1.smithi194.stdout:0/384: fdatasync d0/d3b/f4c 0 2024-04-12T05:56:07.641 INFO:tasks.workunit.client.1.smithi194.stdout:9/480: dwrite d1/fe [0,4194304] 0 2024-04-12T05:56:07.642 INFO:tasks.workunit.client.1.smithi194.stdout:4/474: rename d2/d3/d58/fa1 to d2/d22/d57/fa4 0 2024-04-12T05:56:07.643 INFO:tasks.workunit.client.1.smithi194.stdout:4/475: readlink d2/l59 0 2024-04-12T05:56:07.643 INFO:tasks.workunit.client.1.smithi194.stdout:9/481: read d1/f5b [3620730,84983] 0 2024-04-12T05:56:07.643 INFO:tasks.workunit.client.1.smithi194.stdout:9/482: chown d1/f69 1817908 1 2024-04-12T05:56:07.646 INFO:tasks.workunit.client.1.smithi194.stdout:9/483: write d1/d41/f5e [983982,43901] 0 2024-04-12T05:56:07.651 INFO:tasks.workunit.client.1.smithi194.stdout:0/385: write d0/da/d1e/f35 [987168,35202] 0 2024-04-12T05:56:07.720 INFO:tasks.workunit.client.1.smithi194.stdout:2/368: sync 2024-04-12T05:56:07.747 INFO:tasks.workunit.client.1.smithi194.stdout:0/386: dwrite d0/da/d1e/d27/f3f [0,4194304] 0 2024-04-12T05:56:07.749 INFO:tasks.workunit.client.1.smithi194.stdout:0/387: write d0/da/d1e/d1a/f64 [228028,61156] 0 2024-04-12T05:56:07.758 INFO:tasks.workunit.client.1.smithi194.stdout:2/369: dwrite d7/d12/d25/d5d/d47/d52/f60 [4194304,4194304] 0 2024-04-12T05:56:07.761 INFO:tasks.workunit.client.1.smithi194.stdout:0/388: dwrite d0/da/d1e/d1a/d24/d4b/f6f [0,4194304] 0 2024-04-12T05:56:07.763 INFO:tasks.workunit.client.1.smithi194.stdout:4/476: dread d2/d22/f34 [0,4194304] 0 2024-04-12T05:56:07.764 INFO:tasks.workunit.client.1.smithi194.stdout:4/477: write d2/d11/d1e/f6f [688049,103963] 0 2024-04-12T05:56:07.764 INFO:tasks.workunit.client.1.smithi194.stdout:2/370: creat d7/d12/f80 x:0 0 0 2024-04-12T05:56:07.765 INFO:tasks.workunit.client.1.smithi194.stdout:4/478: readlink d2/d3/d1b/l56 0 2024-04-12T05:56:07.766 INFO:tasks.workunit.client.1.smithi194.stdout:0/389: mkdir d0/d3b/d52/d7c 0 2024-04-12T05:56:07.766 INFO:tasks.workunit.client.1.smithi194.stdout:2/371: write d7/d12/f3f [3366091,102647] 0 2024-04-12T05:56:07.767 INFO:tasks.workunit.client.1.smithi194.stdout:4/479: rmdir d2/d3/d1b/d32 39 2024-04-12T05:56:07.776 INFO:tasks.workunit.client.1.smithi194.stdout:4/480: dread d2/d3/d58/f5c [0,4194304] 0 2024-04-12T05:56:07.776 INFO:tasks.workunit.client.1.smithi194.stdout:0/390: dwrite d0/da/d1e/f32 [0,4194304] 0 2024-04-12T05:56:07.781 INFO:tasks.workunit.client.1.smithi194.stdout:4/481: creat d2/d11/d46/d61/d63/da2/fa5 x:0 0 0 2024-04-12T05:56:07.782 INFO:tasks.workunit.client.1.smithi194.stdout:0/391: creat d0/da/d1e/d1a/f7d x:0 0 0 2024-04-12T05:56:07.782 INFO:tasks.workunit.client.1.smithi194.stdout:4/482: dread - d2/d11/d46/d61/d63/da2/fa5 zero size 2024-04-12T05:56:07.784 INFO:tasks.workunit.client.1.smithi194.stdout:0/392: creat d0/d3b/f7e x:0 0 0 2024-04-12T05:56:07.785 INFO:tasks.workunit.client.1.smithi194.stdout:4/483: write d2/d3/d50/f9e [26553,17228] 0 2024-04-12T05:56:07.787 INFO:tasks.workunit.client.1.smithi194.stdout:4/484: mkdir d2/d11/d93/da6 0 2024-04-12T05:56:07.788 INFO:tasks.workunit.client.1.smithi194.stdout:4/485: symlink d2/d11/d1e/d25/la7 0 2024-04-12T05:56:07.792 INFO:tasks.workunit.client.1.smithi194.stdout:4/486: dread d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:07.799 INFO:tasks.workunit.client.1.smithi194.stdout:4/487: dwrite d2/d11/d1e/f42 [0,4194304] 0 2024-04-12T05:56:07.802 INFO:tasks.workunit.client.1.smithi194.stdout:4/488: creat d2/d11/d1f/fa8 x:0 0 0 2024-04-12T05:56:07.806 INFO:tasks.workunit.client.1.smithi194.stdout:4/489: mknod d2/d3/d1b/d32/ca9 0 2024-04-12T05:56:07.808 INFO:tasks.workunit.client.1.smithi194.stdout:4/490: write d2/d22/f2a [2359178,104904] 0 2024-04-12T05:56:07.808 INFO:tasks.workunit.client.1.smithi194.stdout:4/491: chown d2/d11/d1f 41892 1 2024-04-12T05:56:07.809 INFO:tasks.workunit.client.1.smithi194.stdout:4/492: chown d2/d3/c3d 9141404 1 2024-04-12T05:56:07.809 INFO:tasks.workunit.client.1.smithi194.stdout:4/493: stat d2/d11/d1e/d25/c6a 0 2024-04-12T05:56:07.812 INFO:tasks.workunit.client.1.smithi194.stdout:4/494: rename d2/d3/d1b/d32/d3a/d3b/f7b to d2/d3/d50/d80/faa 0 2024-04-12T05:56:07.813 INFO:tasks.workunit.client.1.smithi194.stdout:4/495: dread - d2/d11/d1e/f5a zero size 2024-04-12T05:56:07.819 INFO:tasks.workunit.client.1.smithi194.stdout:4/496: dwrite d2/d11/d1e/f5a [0,4194304] 0 2024-04-12T05:56:07.824 INFO:tasks.workunit.client.1.smithi194.stdout:4/497: read d2/d3/d50/f9e [23099,93597] 0 2024-04-12T05:56:07.831 INFO:tasks.workunit.client.1.smithi194.stdout:4/498: dwrite d2/d11/d1f/fa8 [0,4194304] 0 2024-04-12T05:56:07.832 INFO:tasks.workunit.client.1.smithi194.stdout:4/499: truncate d2/d3/f1a 5290984 0 2024-04-12T05:56:07.836 INFO:tasks.workunit.client.1.smithi194.stdout:4/500: rename d2/d11/d1e/d25/d2b to d2/d11/d93/da6/dab 0 2024-04-12T05:56:07.853 INFO:tasks.workunit.client.1.smithi194.stdout:4/501: dread d2/d3/d50/f95 [4194304,4194304] 0 2024-04-12T05:56:07.856 INFO:tasks.workunit.client.1.smithi194.stdout:4/502: link d2/d11/d1e/d25/l96 d2/d3/d58/d82/lac 0 2024-04-12T05:56:07.858 INFO:tasks.workunit.client.1.smithi194.stdout:4/503: symlink d2/d3/d1b/lad 0 2024-04-12T05:56:07.864 INFO:tasks.workunit.client.1.smithi194.stdout:4/504: dwrite d2/d11/d1e/d25/f7a [0,4194304] 0 2024-04-12T05:56:07.865 INFO:tasks.workunit.client.1.smithi194.stdout:4/505: write d2/d11/d46/d61/f78 [247798,89228] 0 2024-04-12T05:56:07.866 INFO:tasks.workunit.client.1.smithi194.stdout:4/506: readlink d2/d3/d58/l60 0 2024-04-12T05:56:07.869 INFO:tasks.workunit.client.1.smithi194.stdout:4/507: chown d2/d11/d1e/f6f 43315 1 2024-04-12T05:56:07.872 INFO:tasks.workunit.client.1.smithi194.stdout:4/508: mkdir d2/d11/d93/da6/dab/d5d/dae 0 2024-04-12T05:56:07.873 INFO:tasks.workunit.client.1.smithi194.stdout:4/509: creat d2/faf x:0 0 0 2024-04-12T05:56:07.873 INFO:tasks.workunit.client.1.smithi194.stdout:4/510: stat d2/d3/f91 0 2024-04-12T05:56:07.874 INFO:tasks.workunit.client.1.smithi194.stdout:4/511: chown d2/d3/d1b/d32/d3a/d3b 159 1 2024-04-12T05:56:07.876 INFO:tasks.workunit.client.1.smithi194.stdout:4/512: rename d2/d3/c3d to d2/d11/d46/cb0 0 2024-04-12T05:56:07.877 INFO:tasks.workunit.client.1.smithi194.stdout:4/513: fsync d2/d11/d46/d61/f78 0 2024-04-12T05:56:07.883 INFO:tasks.workunit.client.1.smithi194.stdout:4/514: dwrite d2/faf [0,4194304] 0 2024-04-12T05:56:07.887 INFO:tasks.workunit.client.1.smithi194.stdout:4/515: rename d2/d3/d58/f5c to d2/d3/d51/fb1 0 2024-04-12T05:56:07.889 INFO:tasks.workunit.client.1.smithi194.stdout:4/516: read d2/f4e [553578,114065] 0 2024-04-12T05:56:07.890 INFO:tasks.workunit.client.1.smithi194.stdout:4/517: truncate d2/d3/d50/f9e 631774 0 2024-04-12T05:56:07.896 INFO:tasks.workunit.client.1.smithi194.stdout:4/518: write d2/d3/f3f [1251440,10837] 0 2024-04-12T05:56:07.899 INFO:tasks.workunit.client.1.smithi194.stdout:4/519: rename d2/d22 to d2/d11/d93/da6/dab/d5d/dae/db2 0 2024-04-12T05:56:07.902 INFO:tasks.workunit.client.1.smithi194.stdout:4/520: unlink d2/d11/d93/da6/dab/d5d/l7f 0 2024-04-12T05:56:07.904 INFO:tasks.workunit.client.1.smithi194.stdout:4/521: rmdir d2/d11/d93/da6/dab/d5d/dae 39 2024-04-12T05:56:07.905 INFO:tasks.workunit.client.1.smithi194.stdout:4/522: rmdir d2/d3/d50 39 2024-04-12T05:56:07.907 INFO:tasks.workunit.client.1.smithi194.stdout:4/523: write d2/d11/d1e/d25/f7a [1582283,52096] 0 2024-04-12T05:56:07.936 INFO:tasks.workunit.client.1.smithi194.stdout:3/515: sync 2024-04-12T05:56:07.936 INFO:tasks.workunit.client.1.smithi194.stdout:1/516: sync 2024-04-12T05:56:07.943 INFO:tasks.workunit.client.1.smithi194.stdout:3/516: dwrite d0/d1/d74/d39/f3b [4194304,4194304] 0 2024-04-12T05:56:07.945 INFO:tasks.workunit.client.1.smithi194.stdout:3/517: chown d0/d1/d74/d39 698219425 1 2024-04-12T05:56:07.946 INFO:tasks.workunit.client.1.smithi194.stdout:3/518: stat d0/dc/d3d/f69 0 2024-04-12T05:56:07.951 INFO:tasks.workunit.client.1.smithi194.stdout:3/519: mknod d0/d1/d10/ca9 0 2024-04-12T05:56:07.955 INFO:tasks.workunit.client.1.smithi194.stdout:3/520: creat d0/dc/d1e/d81/d5b/faa x:0 0 0 2024-04-12T05:56:07.958 INFO:tasks.workunit.client.1.smithi194.stdout:3/521: read d0/dc/d1e/d33/f4a [4082994,74618] 0 2024-04-12T05:56:07.968 INFO:tasks.workunit.client.1.smithi194.stdout:3/522: dwrite d0/dc/d3d/d6e/f77 [0,4194304] 0 2024-04-12T05:56:07.969 INFO:tasks.workunit.client.1.smithi194.stdout:3/523: readlink d0/d1/l8 0 2024-04-12T05:56:07.977 INFO:tasks.workunit.client.1.smithi194.stdout:1/517: dread d0/d11/f55 [0,4194304] 0 2024-04-12T05:56:07.978 INFO:tasks.workunit.client.1.smithi194.stdout:3/524: dwrite d0/dc/f1f [0,4194304] 0 2024-04-12T05:56:07.981 INFO:tasks.workunit.client.1.smithi194.stdout:1/518: mkdir d0/d11/d18/d19/d1a/d6a/d27/d6c/da0 0 2024-04-12T05:56:07.982 INFO:tasks.workunit.client.1.smithi194.stdout:3/525: mknod d0/dc/d1e/d33/d76/cab 0 2024-04-12T05:56:07.985 INFO:tasks.workunit.client.1.smithi194.stdout:3/526: fsync d0/dc/d1e/d81/d34/f41 0 2024-04-12T05:56:07.985 INFO:tasks.workunit.client.1.smithi194.stdout:1/519: mkdir d0/d11/d18/d19/d1a/d6a/d27/d2e/da1 0 2024-04-12T05:56:07.986 INFO:tasks.workunit.client.1.smithi194.stdout:1/520: chown d0/d11/d18/d35/d6f 171203 1 2024-04-12T05:56:07.995 INFO:tasks.workunit.client.1.smithi194.stdout:3/527: truncate d0/d1/d4f/f85 395059 0 2024-04-12T05:56:07.997 INFO:tasks.workunit.client.1.smithi194.stdout:1/521: dwrite d0/d11/f9f [0,4194304] 0 2024-04-12T05:56:08.002 INFO:tasks.workunit.client.1.smithi194.stdout:3/528: write d0/f27 [4532488,129547] 0 2024-04-12T05:56:08.004 INFO:tasks.workunit.client.1.smithi194.stdout:3/529: truncate d0/dc/d1e/d81/d34/f46 2498973 0 2024-04-12T05:56:08.008 INFO:tasks.workunit.client.1.smithi194.stdout:3/530: mkdir d0/d1/d10/d8c/d8f/dac 0 2024-04-12T05:56:08.010 INFO:tasks.workunit.client.1.smithi194.stdout:3/531: creat d0/dc/d1e/d81/d5b/d64/fad x:0 0 0 2024-04-12T05:56:08.011 INFO:tasks.workunit.client.1.smithi194.stdout:3/532: stat d0/d1/d74/d39/f3b 0 2024-04-12T05:56:08.015 INFO:tasks.workunit.client.1.smithi194.stdout:3/533: mkdir d0/dc/d3d/dae 0 2024-04-12T05:56:08.019 INFO:tasks.workunit.client.1.smithi194.stdout:3/534: creat d0/dc/d3d/dae/faf x:0 0 0 2024-04-12T05:56:08.020 INFO:tasks.workunit.client.1.smithi194.stdout:3/535: chown d0/d1/f60 6 1 2024-04-12T05:56:08.030 INFO:tasks.workunit.client.1.smithi194.stdout:1/522: dread d0/d11/d18/d19/f4e [0,4194304] 0 2024-04-12T05:56:08.031 INFO:tasks.workunit.client.1.smithi194.stdout:3/536: truncate d0/dc/f1f 2530888 0 2024-04-12T05:56:08.032 INFO:tasks.workunit.client.1.smithi194.stdout:1/523: chown d0/d11/d18/d19/f80 7722976 1 2024-04-12T05:56:08.034 INFO:tasks.workunit.client.1.smithi194.stdout:3/537: unlink d0/d1/d10/ca9 0 2024-04-12T05:56:08.035 INFO:tasks.workunit.client.1.smithi194.stdout:1/524: creat d0/d11/d18/fa2 x:0 0 0 2024-04-12T05:56:08.036 INFO:tasks.workunit.client.1.smithi194.stdout:1/525: dread - d0/d11/f83 zero size 2024-04-12T05:56:08.036 INFO:tasks.workunit.client.1.smithi194.stdout:3/538: fsync d0/dc/d1e/d81/d5b/faa 0 2024-04-12T05:56:08.037 INFO:tasks.workunit.client.1.smithi194.stdout:1/526: readlink d0/d11/d18/d19/d1a/l17 0 2024-04-12T05:56:08.044 INFO:tasks.workunit.client.1.smithi194.stdout:3/539: link d0/d1/d10/f2b d0/d1/d74/d39/fb0 0 2024-04-12T05:56:08.045 INFO:tasks.workunit.client.1.smithi194.stdout:1/527: dread d0/d11/f7c [0,4194304] 0 2024-04-12T05:56:08.049 INFO:tasks.workunit.client.1.smithi194.stdout:3/540: truncate d0/dc/d3d/f94 1935770 0 2024-04-12T05:56:08.053 INFO:tasks.workunit.client.1.smithi194.stdout:1/528: link d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/c90 d0/df/d8d/ca3 0 2024-04-12T05:56:08.055 INFO:tasks.workunit.client.1.smithi194.stdout:3/541: creat d0/d1/d74/fb1 x:0 0 0 2024-04-12T05:56:08.059 INFO:tasks.workunit.client.1.smithi194.stdout:1/529: unlink d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f7d 0 2024-04-12T05:56:08.065 INFO:tasks.workunit.client.1.smithi194.stdout:3/542: symlink d0/d1/d10/lb2 0 2024-04-12T05:56:08.066 INFO:tasks.workunit.client.1.smithi194.stdout:1/530: symlink d0/d11/d18/d19/d1a/d6a/d27/d5a/la4 0 2024-04-12T05:56:08.072 INFO:tasks.workunit.client.1.smithi194.stdout:3/543: creat d0/dc/d1e/d81/d5b/d64/d79/d87/fb3 x:0 0 0 2024-04-12T05:56:08.073 INFO:tasks.workunit.client.1.smithi194.stdout:3/544: mkdir d0/dc/db4 0 2024-04-12T05:56:08.076 INFO:tasks.workunit.client.1.smithi194.stdout:1/531: dwrite d0/d11/d18/d19/d1c/f46 [0,4194304] 0 2024-04-12T05:56:08.080 INFO:tasks.workunit.client.1.smithi194.stdout:3/545: getdents d0/d1/d10/d89 0 2024-04-12T05:56:08.081 INFO:tasks.workunit.client.1.smithi194.stdout:1/532: unlink d0/d11/d18/d19/d1a/d6a/d27/c81 0 2024-04-12T05:56:08.086 INFO:tasks.workunit.client.1.smithi194.stdout:1/533: rename d0/d11/d18/d19/d1a/d6a/d63 to d0/d11/d18/d35/da5 0 2024-04-12T05:56:08.089 INFO:tasks.workunit.client.1.smithi194.stdout:3/546: dread d0/dc/d1e/d81/d34/d2d/f3a [0,4194304] 0 2024-04-12T05:56:08.092 INFO:tasks.workunit.client.1.smithi194.stdout:8/308: sync 2024-04-12T05:56:08.093 INFO:tasks.workunit.client.1.smithi194.stdout:7/257: sync 2024-04-12T05:56:08.096 INFO:tasks.workunit.client.1.smithi194.stdout:7/258: read - d3/f48 zero size 2024-04-12T05:56:08.098 INFO:tasks.workunit.client.1.smithi194.stdout:1/534: dwrite d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [0,4194304] 0 2024-04-12T05:56:08.104 INFO:tasks.workunit.client.1.smithi194.stdout:7/259: symlink d3/d16/d21/d3a/d43/l52 0 2024-04-12T05:56:08.107 INFO:tasks.workunit.client.1.smithi194.stdout:7/260: symlink d3/d16/d21/d3a/l53 0 2024-04-12T05:56:08.109 INFO:tasks.workunit.client.1.smithi194.stdout:7/261: stat d3/d16/d23/d37/d3d/l42 0 2024-04-12T05:56:08.109 INFO:tasks.workunit.client.1.smithi194.stdout:1/535: dwrite d0/df/d68/f87 [0,4194304] 0 2024-04-12T05:56:08.110 INFO:tasks.workunit.client.1.smithi194.stdout:7/262: readlink d3/d16/d21/d3a/d43/l4e 0 2024-04-12T05:56:08.111 INFO:tasks.workunit.client.1.smithi194.stdout:1/536: chown d0/d3e/c4b 0 1 2024-04-12T05:56:08.112 INFO:tasks.workunit.client.1.smithi194.stdout:7/263: truncate d3/d27/d22/f2e 463482 0 2024-04-12T05:56:08.115 INFO:tasks.workunit.client.1.smithi194.stdout:7/264: write d3/d27/d22/f29 [6234640,7571] 0 2024-04-12T05:56:08.121 INFO:tasks.workunit.client.1.smithi194.stdout:1/537: dwrite d0/d11/d18/d19/f80 [0,4194304] 0 2024-04-12T05:56:08.125 INFO:tasks.workunit.client.1.smithi194.stdout:7/265: dread d3/d27/f44 [0,4194304] 0 2024-04-12T05:56:08.132 INFO:tasks.workunit.client.1.smithi194.stdout:1/538: dwrite d0/d3e/d5b/f9b [0,4194304] 0 2024-04-12T05:56:08.136 INFO:tasks.workunit.client.1.smithi194.stdout:1/539: creat d0/d11/d18/d19/d1a/fa6 x:0 0 0 2024-04-12T05:56:08.143 INFO:tasks.workunit.client.1.smithi194.stdout:7/266: dwrite d3/d27/f7 [0,4194304] 0 2024-04-12T05:56:08.146 INFO:tasks.workunit.client.1.smithi194.stdout:7/267: unlink d3/d16/d23/d37/d3d/f50 0 2024-04-12T05:56:08.147 INFO:tasks.workunit.client.1.smithi194.stdout:7/268: chown d3/d16/d23/f34 1105 1 2024-04-12T05:56:08.148 INFO:tasks.workunit.client.1.smithi194.stdout:7/269: chown d3/d27/f17 17 1 2024-04-12T05:56:08.148 INFO:tasks.workunit.client.1.smithi194.stdout:7/270: chown d3/d27/f7 422255 1 2024-04-12T05:56:08.152 INFO:tasks.workunit.client.1.smithi194.stdout:7/271: rename d3/d16/d23/f34 to d3/d16/d23/d37/f54 0 2024-04-12T05:56:08.158 INFO:tasks.workunit.client.1.smithi194.stdout:7/272: creat d3/d27/d3b/f55 x:0 0 0 2024-04-12T05:56:08.159 INFO:tasks.workunit.client.1.smithi194.stdout:7/273: rename d3/d16/d21/d3a/d47 to d3/d16/d23/d37/d3d/d56 0 2024-04-12T05:56:08.161 INFO:tasks.workunit.client.1.smithi194.stdout:7/274: mkdir d3/d16/d23/d57 0 2024-04-12T05:56:08.163 INFO:tasks.workunit.client.1.smithi194.stdout:7/275: mknod d3/d27/d22/d31/c58 0 2024-04-12T05:56:08.165 INFO:tasks.workunit.client.1.smithi194.stdout:1/540: dread d0/f4 [0,4194304] 0 2024-04-12T05:56:08.177 INFO:tasks.workunit.client.1.smithi194.stdout:7/276: dread d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:08.181 INFO:tasks.workunit.client.1.smithi194.stdout:7/277: getdents d3/d16/d21/d3a 0 2024-04-12T05:56:08.182 INFO:tasks.workunit.client.1.smithi194.stdout:7/278: read - d3/d16/f45 zero size 2024-04-12T05:56:08.187 INFO:tasks.workunit.client.1.smithi194.stdout:7/279: link d3/d27/c2a d3/d16/d23/d40/c59 0 2024-04-12T05:56:08.189 INFO:tasks.workunit.client.1.smithi194.stdout:7/280: mknod d3/d16/d21/d3a/c5a 0 2024-04-12T05:56:08.191 INFO:tasks.workunit.client.1.smithi194.stdout:7/281: symlink d3/d27/l5b 0 2024-04-12T05:56:08.191 INFO:tasks.workunit.client.1.smithi194.stdout:7/282: write d3/d27/f1f [2033626,27681] 0 2024-04-12T05:56:08.192 INFO:tasks.workunit.client.1.smithi194.stdout:7/283: write d3/d27/d22/f2e [668680,40044] 0 2024-04-12T05:56:08.206 INFO:tasks.workunit.client.1.smithi194.stdout:7/284: dread d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:08.212 INFO:tasks.workunit.client.1.smithi194.stdout:7/285: write d3/d27/d22/f38 [4606993,90508] 0 2024-04-12T05:56:08.214 INFO:tasks.workunit.client.1.smithi194.stdout:7/286: mkdir d3/d16/d21/d3a/d43/d5c 0 2024-04-12T05:56:08.227 INFO:tasks.workunit.client.1.smithi194.stdout:7/287: dread d3/d16/d23/d37/f54 [0,4194304] 0 2024-04-12T05:56:08.228 INFO:tasks.workunit.client.1.smithi194.stdout:7/288: mkdir d3/d16/d5d 0 2024-04-12T05:56:08.230 INFO:tasks.workunit.client.1.smithi194.stdout:7/289: mknod d3/d16/d21/d3a/d43/c5e 0 2024-04-12T05:56:08.231 INFO:tasks.workunit.client.1.smithi194.stdout:7/290: creat d3/d16/d23/f5f x:0 0 0 2024-04-12T05:56:08.232 INFO:tasks.workunit.client.1.smithi194.stdout:7/291: fdatasync d3/d16/f1a 0 2024-04-12T05:56:08.235 INFO:tasks.workunit.client.1.smithi194.stdout:7/292: getdents d3/d27/d22 0 2024-04-12T05:56:08.235 INFO:tasks.workunit.client.1.smithi194.stdout:7/293: dread - d3/f48 zero size 2024-04-12T05:56:08.237 INFO:tasks.workunit.client.1.smithi194.stdout:7/294: creat d3/d16/d21/d3a/f60 x:0 0 0 2024-04-12T05:56:08.238 INFO:tasks.workunit.client.1.smithi194.stdout:7/295: fdatasync d3/d16/d23/d37/d3d/d56/f51 0 2024-04-12T05:56:08.240 INFO:tasks.workunit.client.1.smithi194.stdout:7/296: unlink d3/l14 0 2024-04-12T05:56:08.242 INFO:tasks.workunit.client.1.smithi194.stdout:7/297: creat d3/d16/d23/d37/d3d/f61 x:0 0 0 2024-04-12T05:56:08.246 INFO:tasks.workunit.client.1.smithi194.stdout:7/298: dread d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:08.247 INFO:tasks.workunit.client.1.smithi194.stdout:7/299: readlink d3/d16/d23/l24 0 2024-04-12T05:56:08.247 INFO:tasks.workunit.client.1.smithi194.stdout:7/300: chown d3/d16/d23/d40/d49 0 1 2024-04-12T05:56:08.249 INFO:tasks.workunit.client.1.smithi194.stdout:7/301: mkdir d3/d27/d22/d62 0 2024-04-12T05:56:08.250 INFO:tasks.workunit.client.1.smithi194.stdout:7/302: mknod d3/d16/d23/d57/c63 0 2024-04-12T05:56:08.253 INFO:tasks.workunit.client.1.smithi194.stdout:7/303: creat d3/d27/f64 x:0 0 0 2024-04-12T05:56:08.253 INFO:tasks.workunit.client.1.smithi194.stdout:7/304: dread - d3/f48 zero size 2024-04-12T05:56:08.255 INFO:tasks.workunit.client.1.smithi194.stdout:7/305: creat d3/d16/d5d/f65 x:0 0 0 2024-04-12T05:56:08.260 INFO:tasks.workunit.client.1.smithi194.stdout:7/306: dwrite d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:08.261 INFO:tasks.workunit.client.1.smithi194.stdout:7/307: chown d3/d16/d23/d37/d3d/d56 6 1 2024-04-12T05:56:08.262 INFO:tasks.workunit.client.1.smithi194.stdout:7/308: fdatasync d3/d16/f45 0 2024-04-12T05:56:08.263 INFO:tasks.workunit.client.1.smithi194.stdout:7/309: write d3/d27/d22/f2e [316065,59347] 0 2024-04-12T05:56:08.265 INFO:tasks.workunit.client.1.smithi194.stdout:7/310: link d3/d16/d23/d37/f54 d3/d16/d23/f66 0 2024-04-12T05:56:08.266 INFO:tasks.workunit.client.1.smithi194.stdout:7/311: chown d3/d16/d23/d40 3 1 2024-04-12T05:56:08.268 INFO:tasks.workunit.client.1.smithi194.stdout:7/312: truncate d3/d16/d21/d3a/f60 422446 0 2024-04-12T05:56:08.270 INFO:tasks.workunit.client.1.smithi194.stdout:7/313: getdents d3/d27/d22/d62 0 2024-04-12T05:56:08.273 INFO:tasks.workunit.client.1.smithi194.stdout:7/314: rename d3/d27/d3b/l4a to d3/d16/d23/d57/l67 0 2024-04-12T05:56:08.329 INFO:tasks.workunit.client.1.smithi194.stdout:5/510: sync 2024-04-12T05:56:08.356 INFO:tasks.workunit.client.1.smithi194.stdout:7/315: rmdir d3/d16/d23/d57 39 2024-04-12T05:56:08.358 INFO:tasks.workunit.client.1.smithi194.stdout:5/511: creat d1/d4/d9e/faf x:0 0 0 2024-04-12T05:56:08.359 INFO:tasks.workunit.client.1.smithi194.stdout:7/316: rename d3/d16/f35 to d3/d16/d23/d40/f68 0 2024-04-12T05:56:08.360 INFO:tasks.workunit.client.1.smithi194.stdout:5/512: readlink d1/l87 0 2024-04-12T05:56:08.361 INFO:tasks.workunit.client.1.smithi194.stdout:5/513: chown d1/d4/d6/d88/la5 326020490 1 2024-04-12T05:56:08.363 INFO:tasks.workunit.client.1.smithi194.stdout:7/317: link d3/d16/f45 d3/d27/f69 0 2024-04-12T05:56:08.365 INFO:tasks.workunit.client.1.smithi194.stdout:7/318: truncate d3/d16/d21/d3a/f60 507109 0 2024-04-12T05:56:08.366 INFO:tasks.workunit.client.1.smithi194.stdout:5/514: rename d1/d4/d6/l43 to d1/d4/d9e/lb0 0 2024-04-12T05:56:08.368 INFO:tasks.workunit.client.1.smithi194.stdout:5/515: dread - d1/d4/d6/d34/d52/f77 zero size 2024-04-12T05:56:08.373 INFO:tasks.workunit.client.1.smithi194.stdout:7/319: dwrite d3/d27/d3b/f55 [0,4194304] 0 2024-04-12T05:56:08.379 INFO:tasks.workunit.client.1.smithi194.stdout:7/320: chown d3/d27/d22/c10 897702014 1 2024-04-12T05:56:08.379 INFO:tasks.workunit.client.1.smithi194.stdout:5/516: dwrite d1/d4/d6/d88/f56 [0,4194304] 0 2024-04-12T05:56:08.382 INFO:tasks.workunit.client.1.smithi194.stdout:5/517: creat d1/d5c/da7/fb1 x:0 0 0 2024-04-12T05:56:08.384 INFO:tasks.workunit.client.1.smithi194.stdout:7/321: rename d3/d27/d3b to d3/d27/d6a 0 2024-04-12T05:56:08.390 INFO:tasks.workunit.client.1.smithi194.stdout:7/322: dwrite d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:08.394 INFO:tasks.workunit.client.1.smithi194.stdout:7/323: unlink d3/d16/d21/d3a/d43/l52 0 2024-04-12T05:56:08.397 INFO:tasks.workunit.client.1.smithi194.stdout:7/324: rename d3/d27/l20 to d3/d16/d21/d3a/d43/l6b 0 2024-04-12T05:56:08.401 INFO:tasks.workunit.client.1.smithi194.stdout:7/325: link d3/d27/f69 d3/d27/d22/d62/f6c 0 2024-04-12T05:56:08.419 INFO:tasks.workunit.client.1.smithi194.stdout:7/326: dwrite d3/d16/f19 [0,4194304] 0 2024-04-12T05:56:08.423 INFO:tasks.workunit.client.1.smithi194.stdout:7/327: dread d3/d27/d6a/f55 [0,4194304] 0 2024-04-12T05:56:08.425 INFO:tasks.workunit.client.1.smithi194.stdout:7/328: mknod d3/d16/d23/c6d 0 2024-04-12T05:56:08.427 INFO:tasks.workunit.client.1.smithi194.stdout:7/329: mknod d3/d16/d21/c6e 0 2024-04-12T05:56:08.428 INFO:tasks.workunit.client.1.smithi194.stdout:7/330: chown d3/d16/d23/d57/c63 62 1 2024-04-12T05:56:08.446 INFO:tasks.workunit.client.1.smithi194.stdout:7/331: dread d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:08.452 INFO:tasks.workunit.client.1.smithi194.stdout:7/332: dwrite d3/d27/f7 [0,4194304] 0 2024-04-12T05:56:08.452 INFO:tasks.workunit.client.1.smithi194.stdout:7/333: fsync d3/d27/f1f 0 2024-04-12T05:56:08.496 INFO:tasks.workunit.client.1.smithi194.stdout:9/484: sync 2024-04-12T05:56:08.496 INFO:tasks.workunit.client.1.smithi194.stdout:6/423: sync 2024-04-12T05:56:08.497 INFO:tasks.workunit.client.1.smithi194.stdout:6/424: write d3/d2d/d33/f4d [844001,75942] 0 2024-04-12T05:56:08.498 INFO:tasks.workunit.client.1.smithi194.stdout:6/425: creat d3/d2f/d37/f80 x:0 0 0 2024-04-12T05:56:08.499 INFO:tasks.workunit.client.1.smithi194.stdout:9/485: getdents d1/d21/d39/d72 0 2024-04-12T05:56:08.500 INFO:tasks.workunit.client.1.smithi194.stdout:6/426: fsync d3/d1a/f23 0 2024-04-12T05:56:08.509 INFO:tasks.workunit.client.1.smithi194.stdout:6/427: dwrite d3/d2d/f32 [0,4194304] 0 2024-04-12T05:56:08.513 INFO:tasks.workunit.client.1.smithi194.stdout:6/428: getdents d3/d2f 0 2024-04-12T05:56:08.525 INFO:tasks.workunit.client.1.smithi194.stdout:6/429: dread d3/f3a [0,4194304] 0 2024-04-12T05:56:08.571 INFO:tasks.workunit.client.1.smithi194.stdout:6/430: fdatasync d3/d15/d2b/d48/f4f 0 2024-04-12T05:56:08.571 INFO:tasks.workunit.client.1.smithi194.stdout:6/431: fsync d3/f1f 0 2024-04-12T05:56:08.573 INFO:tasks.workunit.client.1.smithi194.stdout:6/432: creat d3/d2f/d37/f81 x:0 0 0 2024-04-12T05:56:08.574 INFO:tasks.workunit.client.1.smithi194.stdout:6/433: write d3/d15/d2b/d48/f49 [2121188,8554] 0 2024-04-12T05:56:08.580 INFO:tasks.workunit.client.1.smithi194.stdout:6/434: dwrite d3/d2d/d33/d52/f6e [0,4194304] 0 2024-04-12T05:56:08.582 INFO:tasks.workunit.client.1.smithi194.stdout:6/435: write d3/d2d/d33/d52/f6e [1823673,27972] 0 2024-04-12T05:56:08.582 INFO:tasks.workunit.client.1.smithi194.stdout:6/436: chown d3/d2d/d33/d52/d5c 15 1 2024-04-12T05:56:08.583 INFO:tasks.workunit.client.1.smithi194.stdout:6/437: chown d3/d15/d2b/d48/d6f/d7a 208393022 1 2024-04-12T05:56:08.590 INFO:tasks.workunit.client.1.smithi194.stdout:6/438: dwrite d3/d2f/d37/f4a [4194304,4194304] 0 2024-04-12T05:56:08.591 INFO:tasks.workunit.client.1.smithi194.stdout:6/439: chown d3/d15/d2b/f5e 57446 1 2024-04-12T05:56:08.596 INFO:tasks.workunit.client.1.smithi194.stdout:6/440: dread d3/f3a [4194304,4194304] 0 2024-04-12T05:56:08.599 INFO:tasks.workunit.client.1.smithi194.stdout:6/441: link d3/d2d/d33/d58/f7f d3/d15/d2b/d48/d6f/f82 0 2024-04-12T05:56:08.600 INFO:tasks.workunit.client.1.smithi194.stdout:6/442: mkdir d3/d2f/d4e/d63/d83 0 2024-04-12T05:56:08.602 INFO:tasks.workunit.client.1.smithi194.stdout:6/443: write d3/d2d/d33/d58/f7f [868745,112150] 0 2024-04-12T05:56:08.604 INFO:tasks.workunit.client.1.smithi194.stdout:6/444: write d3/d2d/d33/d52/f55 [2220653,86918] 0 2024-04-12T05:56:08.604 INFO:tasks.workunit.client.1.smithi194.stdout:2/372: sync 2024-04-12T05:56:08.607 INFO:tasks.workunit.client.1.smithi194.stdout:2/373: fsync d7/d12/f38 0 2024-04-12T05:56:08.617 INFO:tasks.workunit.client.1.smithi194.stdout:6/445: truncate d3/d15/f24 613629 0 2024-04-12T05:56:08.620 INFO:tasks.workunit.client.1.smithi194.stdout:6/446: creat d3/f84 x:0 0 0 2024-04-12T05:56:08.620 INFO:tasks.workunit.client.1.smithi194.stdout:2/374: dwrite d7/d22/f5c [4194304,4194304] 0 2024-04-12T05:56:08.624 INFO:tasks.workunit.client.1.smithi194.stdout:2/375: creat d7/d12/d34/d4a/f81 x:0 0 0 2024-04-12T05:56:08.624 INFO:tasks.workunit.client.1.smithi194.stdout:6/447: getdents d3/d15/d2b/d48/d6f/d7a 0 2024-04-12T05:56:08.626 INFO:tasks.workunit.client.1.smithi194.stdout:6/448: write d3/d15/d2b/d48/f54 [1963819,7566] 0 2024-04-12T05:56:08.626 INFO:tasks.workunit.client.1.smithi194.stdout:2/376: creat d7/d12/d1a/d69/f82 x:0 0 0 2024-04-12T05:56:08.628 INFO:tasks.workunit.client.1.smithi194.stdout:6/449: rmdir d3/d15/d2b/d48 39 2024-04-12T05:56:08.630 INFO:tasks.workunit.client.1.smithi194.stdout:2/377: unlink d7/d12/f3f 0 2024-04-12T05:56:08.631 INFO:tasks.workunit.client.1.smithi194.stdout:6/450: creat d3/d2f/d4e/d53/f85 x:0 0 0 2024-04-12T05:56:08.633 INFO:tasks.workunit.client.1.smithi194.stdout:2/378: creat d7/d12/d34/f83 x:0 0 0 2024-04-12T05:56:08.634 INFO:tasks.workunit.client.1.smithi194.stdout:6/451: creat d3/d70/f86 x:0 0 0 2024-04-12T05:56:08.634 INFO:tasks.workunit.client.1.smithi194.stdout:2/379: chown d7/d12/d6d/f72 2 1 2024-04-12T05:56:08.636 INFO:tasks.workunit.client.1.smithi194.stdout:2/380: write d7/d12/d25/d5d/f2f [1744585,99738] 0 2024-04-12T05:56:08.641 INFO:tasks.workunit.client.1.smithi194.stdout:6/452: dwrite d3/d2f/d4e/d63/f79 [4194304,4194304] 0 2024-04-12T05:56:08.728 INFO:tasks.workunit.client.1.smithi194.stdout:2/381: write d7/d12/d25/d5d/d47/f7f [4773606,10827] 0 2024-04-12T05:56:08.732 INFO:tasks.workunit.client.1.smithi194.stdout:2/382: getdents d7/d12/d25/d5d/d47/d52/d7b 0 2024-04-12T05:56:08.736 INFO:tasks.workunit.client.1.smithi194.stdout:2/383: symlink d7/d12/d1a/l84 0 2024-04-12T05:56:08.738 INFO:tasks.workunit.client.1.smithi194.stdout:2/384: getdents d7 0 2024-04-12T05:56:08.744 INFO:tasks.workunit.client.1.smithi194.stdout:6/453: dread d3/d15/f2e [0,4194304] 0 2024-04-12T05:56:08.744 INFO:tasks.workunit.client.1.smithi194.stdout:2/385: dread d7/fa [0,4194304] 0 2024-04-12T05:56:08.744 INFO:tasks.workunit.client.1.smithi194.stdout:6/454: write d3/d15/f21 [1933290,79873] 0 2024-04-12T05:56:08.747 INFO:tasks.workunit.client.1.smithi194.stdout:6/455: mknod d3/d2f/d4e/d53/c87 0 2024-04-12T05:56:08.749 INFO:tasks.workunit.client.1.smithi194.stdout:6/456: mknod d3/d2d/d33/d52/d5c/c88 0 2024-04-12T05:56:08.752 INFO:tasks.workunit.client.1.smithi194.stdout:2/386: write d7/d12/f41 [1802200,52194] 0 2024-04-12T05:56:08.754 INFO:tasks.workunit.client.1.smithi194.stdout:2/387: read d7/d22/d42/f77 [1944320,92657] 0 2024-04-12T05:56:08.756 INFO:tasks.workunit.client.1.smithi194.stdout:2/388: truncate d7/d22/d42/f54 133765 0 2024-04-12T05:56:08.757 INFO:tasks.workunit.client.1.smithi194.stdout:2/389: chown d7/c37 230366 1 2024-04-12T05:56:08.758 INFO:tasks.workunit.client.1.smithi194.stdout:2/390: symlink d7/d12/d34/d4a/l85 0 2024-04-12T05:56:08.760 INFO:tasks.workunit.client.1.smithi194.stdout:2/391: mkdir d7/d12/d1a/d86 0 2024-04-12T05:56:08.762 INFO:tasks.workunit.client.1.smithi194.stdout:2/392: creat d7/d22/f87 x:0 0 0 2024-04-12T05:56:08.763 INFO:tasks.workunit.client.1.smithi194.stdout:2/393: dread - d7/d12/f80 zero size 2024-04-12T05:56:08.764 INFO:tasks.workunit.client.1.smithi194.stdout:6/457: dread d3/d2f/d4e/d63/f78 [0,4194304] 0 2024-04-12T05:56:08.765 INFO:tasks.workunit.client.1.smithi194.stdout:6/458: chown d3/d15/d2b/l3e 243 1 2024-04-12T05:56:08.768 INFO:tasks.workunit.client.1.smithi194.stdout:2/394: dread d7/d12/d25/d5d/d47/d52/f60 [4194304,4194304] 0 2024-04-12T05:56:08.770 INFO:tasks.workunit.client.1.smithi194.stdout:2/395: creat d7/d12/d25/d5d/d47/f88 x:0 0 0 2024-04-12T05:56:08.774 INFO:tasks.workunit.client.1.smithi194.stdout:2/396: mknod d7/c89 0 2024-04-12T05:56:08.774 INFO:tasks.workunit.client.1.smithi194.stdout:2/397: write d7/d12/d1a/f24 [3653750,21992] 0 2024-04-12T05:56:08.775 INFO:tasks.workunit.client.1.smithi194.stdout:2/398: mkdir d7/d12/d34/d4a/d8a 0 2024-04-12T05:56:08.776 INFO:tasks.workunit.client.1.smithi194.stdout:2/399: write f5 [7225646,36121] 0 2024-04-12T05:56:08.778 INFO:tasks.workunit.client.1.smithi194.stdout:0/393: sync 2024-04-12T05:56:08.779 INFO:tasks.workunit.client.1.smithi194.stdout:2/400: symlink d7/d12/d1a/d68/l8b 0 2024-04-12T05:56:08.780 INFO:tasks.workunit.client.1.smithi194.stdout:6/459: dread d3/d2d/d33/d52/d5c/f5f [0,4194304] 0 2024-04-12T05:56:08.780 INFO:tasks.workunit.client.1.smithi194.stdout:2/401: write d7/d22/f4d [4229272,36204] 0 2024-04-12T05:56:08.781 INFO:tasks.workunit.client.1.smithi194.stdout:0/394: write d0/da/d1e/d27/d54/f7b [12100952,27877] 0 2024-04-12T05:56:08.783 INFO:tasks.workunit.client.1.smithi194.stdout:6/460: creat d3/d2d/d33/d52/f89 x:0 0 0 2024-04-12T05:56:08.784 INFO:tasks.workunit.client.1.smithi194.stdout:2/402: write d7/d12/d25/d5d/d47/d52/f60 [5993759,73060] 0 2024-04-12T05:56:08.786 INFO:tasks.workunit.client.1.smithi194.stdout:2/403: write d7/d12/d25/d5d/f51 [2493316,99472] 0 2024-04-12T05:56:08.786 INFO:tasks.workunit.client.1.smithi194.stdout:0/395: symlink d0/da/d1e/d27/d54/l7f 0 2024-04-12T05:56:08.790 INFO:tasks.workunit.client.1.smithi194.stdout:2/404: creat d7/d12/d1a/d68/f8c x:0 0 0 2024-04-12T05:56:08.791 INFO:tasks.workunit.client.1.smithi194.stdout:2/405: chown d7/d12/d6d/f72 12050 1 2024-04-12T05:56:08.793 INFO:tasks.workunit.client.1.smithi194.stdout:0/396: link d0/da/d1e/f41 d0/da/d1e/d1a/d58/f80 0 2024-04-12T05:56:08.797 INFO:tasks.workunit.client.1.smithi194.stdout:6/461: write d3/d15/f24 [1353119,65002] 0 2024-04-12T05:56:08.798 INFO:tasks.workunit.client.1.smithi194.stdout:2/406: dwrite d7/d12/f41 [0,4194304] 0 2024-04-12T05:56:08.812 INFO:tasks.workunit.client.1.smithi194.stdout:0/397: dwrite d0/da/d1e/d1a/d24/d56/d66/f72 [0,4194304] 0 2024-04-12T05:56:08.815 INFO:tasks.workunit.client.1.smithi194.stdout:0/398: chown d0/d3b/f74 482853966 1 2024-04-12T05:56:08.815 INFO:tasks.workunit.client.1.smithi194.stdout:6/462: dwrite d3/d2d/d33/d52/f89 [0,4194304] 0 2024-04-12T05:56:08.816 INFO:tasks.workunit.client.1.smithi194.stdout:2/407: dwrite d7/d12/d34/f83 [0,4194304] 0 2024-04-12T05:56:08.822 INFO:tasks.workunit.client.1.smithi194.stdout:0/399: dread d0/da/d1e/f32 [0,4194304] 0 2024-04-12T05:56:08.823 INFO:tasks.workunit.client.1.smithi194.stdout:0/400: chown d0/da/d1e/d1a/d24/c77 43 1 2024-04-12T05:56:08.927 INFO:tasks.workunit.client.1.smithi194.stdout:6/463: mknod d3/d70/c8a 0 2024-04-12T05:56:08.931 INFO:tasks.workunit.client.1.smithi194.stdout:0/401: chown d0/da/d1e/d1a/c22 3561 1 2024-04-12T05:56:08.934 INFO:tasks.workunit.client.1.smithi194.stdout:6/464: creat d3/d70/f8b x:0 0 0 2024-04-12T05:56:08.936 INFO:tasks.workunit.client.1.smithi194.stdout:2/408: creat d7/d12/f8d x:0 0 0 2024-04-12T05:56:08.938 INFO:tasks.workunit.client.1.smithi194.stdout:0/402: dread d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:08.938 INFO:tasks.workunit.client.1.smithi194.stdout:6/465: mkdir d3/d2d/d33/d52/d5c/d8c 0 2024-04-12T05:56:08.940 INFO:tasks.workunit.client.1.smithi194.stdout:2/409: symlink d7/d12/d25/d70/l8e 0 2024-04-12T05:56:08.942 INFO:tasks.workunit.client.1.smithi194.stdout:2/410: stat d7/d12/d25/d5d/d47 0 2024-04-12T05:56:08.948 INFO:tasks.workunit.client.1.smithi194.stdout:6/466: dwrite d3/f18 [4194304,4194304] 0 2024-04-12T05:56:08.961 INFO:tasks.workunit.client.1.smithi194.stdout:0/403: dread d0/da/d1e/d27/f2a [0,4194304] 0 2024-04-12T05:56:08.962 INFO:tasks.workunit.client.1.smithi194.stdout:0/404: stat d0/da/d1e/d1a/d58/l60 0 2024-04-12T05:56:08.966 INFO:tasks.workunit.client.1.smithi194.stdout:4/524: sync 2024-04-12T05:56:09.057 INFO:tasks.workunit.client.1.smithi194.stdout:3/547: sync 2024-04-12T05:56:09.057 INFO:tasks.workunit.client.1.smithi194.stdout:8/309: sync 2024-04-12T05:56:09.077 INFO:tasks.workunit.client.1.smithi194.stdout:2/411: creat d7/d12/d25/d5d/d47/f8f x:0 0 0 2024-04-12T05:56:09.078 INFO:tasks.workunit.client.1.smithi194.stdout:6/467: symlink d3/d2f/d4e/d68/l8d 0 2024-04-12T05:56:09.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:09 smithi008 ceph-mon[39722]: pgmap v30: 65 pgs: 65 active+clean; 908 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 MiB/s rd, 94 MiB/s wr, 249 op/s 2024-04-12T05:56:09.082 INFO:tasks.workunit.client.1.smithi194.stdout:0/405: mknod d0/da/d1e/d27/d54/c81 0 2024-04-12T05:56:09.083 INFO:tasks.workunit.client.1.smithi194.stdout:8/310: chown d2/d8/c57 56183 1 2024-04-12T05:56:09.087 INFO:tasks.workunit.client.1.smithi194.stdout:0/406: write d0/da/d1e/d27/f2a [2075145,23688] 0 2024-04-12T05:56:09.087 INFO:tasks.workunit.client.1.smithi194.stdout:3/548: fsync d0/d1/d10/f2b 0 2024-04-12T05:56:09.088 INFO:tasks.workunit.client.1.smithi194.stdout:8/311: symlink d2/d8/d15/d33/l5e 0 2024-04-12T05:56:09.088 INFO:tasks.workunit.client.1.smithi194.stdout:1/541: sync 2024-04-12T05:56:09.089 INFO:tasks.workunit.client.1.smithi194.stdout:5/518: sync 2024-04-12T05:56:09.089 INFO:tasks.workunit.client.1.smithi194.stdout:9/486: sync 2024-04-12T05:56:09.089 INFO:tasks.workunit.client.1.smithi194.stdout:7/334: sync 2024-04-12T05:56:09.090 INFO:tasks.workunit.client.1.smithi194.stdout:6/468: symlink d3/d15/d2b/l8e 0 2024-04-12T05:56:09.091 INFO:tasks.workunit.client.1.smithi194.stdout:1/542: stat d0/d11/d18/d35/da5 0 2024-04-12T05:56:09.092 INFO:tasks.workunit.client.1.smithi194.stdout:8/312: chown d2/d8/c57 39564373 1 2024-04-12T05:56:09.092 INFO:tasks.workunit.client.1.smithi194.stdout:5/519: dread - d1/d4/d6/d88/f2c zero size 2024-04-12T05:56:09.092 INFO:tasks.workunit.client.1.smithi194.stdout:0/407: write d0/da/d1e/d1a/d3d/f7a [205663,27514] 0 2024-04-12T05:56:09.093 INFO:tasks.workunit.client.1.smithi194.stdout:7/335: mkdir d3/d16/d23/d40/d6f 0 2024-04-12T05:56:09.096 INFO:tasks.workunit.client.1.smithi194.stdout:2/412: write d7/d22/d42/f1e [4952947,107311] 0 2024-04-12T05:56:09.099 INFO:tasks.workunit.client.1.smithi194.stdout:7/336: mknod d3/d27/d6a/c70 0 2024-04-12T05:56:09.099 INFO:tasks.workunit.client.1.smithi194.stdout:8/313: chown d2/d8/d15/d33/f1b 47178 1 2024-04-12T05:56:09.100 INFO:tasks.workunit.client.1.smithi194.stdout:0/408: dread - d0/da/d1e/d1a/d24/d4b/f65 zero size 2024-04-12T05:56:09.101 INFO:tasks.workunit.client.1.smithi194.stdout:1/543: mkdir d0/d11/d18/d19/d1a/d6a/d27/d48/da7 0 2024-04-12T05:56:09.103 INFO:tasks.workunit.client.1.smithi194.stdout:9/487: creat d1/d19/d1f/d20/d7d/f99 x:0 0 0 2024-04-12T05:56:09.103 INFO:tasks.workunit.client.1.smithi194.stdout:1/544: stat d0/df/l34 0 2024-04-12T05:56:09.104 INFO:tasks.workunit.client.1.smithi194.stdout:6/469: truncate d3/d15/d2b/d48/f49 4848651 0 2024-04-12T05:56:09.104 INFO:tasks.workunit.client.1.smithi194.stdout:5/520: chown d1/d4/d6/db/d3b/c62 15347 1 2024-04-12T05:56:09.105 INFO:tasks.workunit.client.1.smithi194.stdout:2/413: chown d7/d12/d1a/c40 74 1 2024-04-12T05:56:09.106 INFO:tasks.workunit.client.1.smithi194.stdout:3/549: dwrite d0/dc/d3d/d6e/f77 [0,4194304] 0 2024-04-12T05:56:09.106 INFO:tasks.workunit.client.1.smithi194.stdout:1/545: chown d0/le 215307 1 2024-04-12T05:56:09.108 INFO:tasks.workunit.client.1.smithi194.stdout:7/337: rename d3/d16/d23/f5f to d3/d16/d23/d40/d49/f71 0 2024-04-12T05:56:09.112 INFO:tasks.workunit.client.1.smithi194.stdout:9/488: mknod d1/d21/d39/d72/c9a 0 2024-04-12T05:56:09.114 INFO:tasks.workunit.client.1.smithi194.stdout:8/314: getdents d2/d8/d15/d33/d45/d52 0 2024-04-12T05:56:09.116 INFO:tasks.workunit.client.1.smithi194.stdout:2/414: unlink d7/c21 0 2024-04-12T05:56:09.117 INFO:tasks.workunit.client.1.smithi194.stdout:0/409: unlink d0/da/d1e/d1a/c26 0 2024-04-12T05:56:09.121 INFO:tasks.workunit.client.1.smithi194.stdout:0/410: fdatasync d0/da/d1e/fe 0 2024-04-12T05:56:09.121 INFO:tasks.workunit.client.1.smithi194.stdout:8/315: stat d2/d8/d9/de/c23 0 2024-04-12T05:56:09.121 INFO:tasks.workunit.client.1.smithi194.stdout:5/521: dwrite d1/fa3 [0,4194304] 0 2024-04-12T05:56:09.121 INFO:tasks.workunit.client.1.smithi194.stdout:9/489: creat d1/d19/d1f/d5f/f9b x:0 0 0 2024-04-12T05:56:09.122 INFO:tasks.workunit.client.1.smithi194.stdout:6/470: mkdir d3/d15/d2b/d48/d6f/d7a/d8f 0 2024-04-12T05:56:09.123 INFO:tasks.workunit.client.1.smithi194.stdout:0/411: stat d0/f48 0 2024-04-12T05:56:09.125 INFO:tasks.workunit.client.1.smithi194.stdout:0/412: stat d0 0 2024-04-12T05:56:09.126 INFO:tasks.workunit.client.1.smithi194.stdout:2/415: write d7/d12/d6d/f72 [1256666,7639] 0 2024-04-12T05:56:09.127 INFO:tasks.workunit.client.1.smithi194.stdout:3/550: dwrite d0/dc/d1e/d81/d34/f71 [0,4194304] 0 2024-04-12T05:56:09.127 INFO:tasks.workunit.client.1.smithi194.stdout:7/338: truncate d3/d27/f1f 1700521 0 2024-04-12T05:56:09.130 INFO:tasks.workunit.client.1.smithi194.stdout:1/546: rename d0/d3e to d0/d11/d18/d19/d1a/d6a/d27/da8 0 2024-04-12T05:56:09.131 INFO:tasks.workunit.client.1.smithi194.stdout:9/490: creat d1/d19/d1f/d20/d7d/f9c x:0 0 0 2024-04-12T05:56:09.132 INFO:tasks.workunit.client.1.smithi194.stdout:1/547: chown d0/d11/d18/d19/d1a/d6a/d27/da8 899994 1 2024-04-12T05:56:09.135 INFO:tasks.workunit.client.1.smithi194.stdout:8/316: creat d2/d8/d9/de/f5f x:0 0 0 2024-04-12T05:56:09.136 INFO:tasks.workunit.client.1.smithi194.stdout:0/413: write d0/da/d1e/d1a/d24/d56/d66/f68 [408062,93941] 0 2024-04-12T05:56:09.137 INFO:tasks.workunit.client.1.smithi194.stdout:9/491: read d1/d4e/f55 [215066,113532] 0 2024-04-12T05:56:09.139 INFO:tasks.workunit.client.1.smithi194.stdout:8/317: dread - d2/d8/d9/f48 zero size 2024-04-12T05:56:09.143 INFO:tasks.workunit.client.1.smithi194.stdout:9/492: write d1/d41/f6f [943939,55117] 0 2024-04-12T05:56:09.143 INFO:tasks.workunit.client.1.smithi194.stdout:3/551: mkdir d0/d1/d74/d39/d4d/d63/db5 0 2024-04-12T05:56:09.146 INFO:tasks.workunit.client.1.smithi194.stdout:2/416: symlink d7/d12/l90 0 2024-04-12T05:56:09.146 INFO:tasks.workunit.client.1.smithi194.stdout:6/471: dwrite d3/d2d/d33/d52/f6e [0,4194304] 0 2024-04-12T05:56:09.157 INFO:tasks.workunit.client.1.smithi194.stdout:1/548: dwrite d0/df/f82 [0,4194304] 0 2024-04-12T05:56:09.161 INFO:tasks.workunit.client.1.smithi194.stdout:7/339: dwrite d3/d16/d23/d40/f68 [4194304,4194304] 0 2024-04-12T05:56:09.173 INFO:tasks.workunit.client.1.smithi194.stdout:8/318: link d2/d8/f36 d2/d8/d9/de/d10/d4e/f60 0 2024-04-12T05:56:09.173 INFO:tasks.workunit.client.1.smithi194.stdout:3/552: rename d0/d1/l8 to d0/dc/d3d/dae/lb6 0 2024-04-12T05:56:09.175 INFO:tasks.workunit.client.1.smithi194.stdout:0/414: dread d0/da/d1e/d1a/d24/f62 [0,4194304] 0 2024-04-12T05:56:09.176 INFO:tasks.workunit.client.1.smithi194.stdout:6/472: dwrite d3/d15/f21 [0,4194304] 0 2024-04-12T05:56:09.178 INFO:tasks.workunit.client.1.smithi194.stdout:6/473: readlink d3/d2f/d4e/l7b 0 2024-04-12T05:56:09.182 INFO:tasks.workunit.client.1.smithi194.stdout:6/474: chown d3/d2d/d33/d52/f89 462365583 1 2024-04-12T05:56:09.182 INFO:tasks.workunit.client.1.smithi194.stdout:1/549: link d0/d11/d18/d19/d1c/f6b d0/d11/d18/d19/d1a/d6a/d27/da8/fa9 0 2024-04-12T05:56:09.184 INFO:tasks.workunit.client.1.smithi194.stdout:5/522: truncate d1/d4/d6/db/d15/d26/f7c 1864686 0 2024-04-12T05:56:09.187 INFO:tasks.workunit.client.1.smithi194.stdout:6/475: chown d3/d2f/d37/c71 307234581 1 2024-04-12T05:56:09.187 INFO:tasks.workunit.client.1.smithi194.stdout:8/319: dread d2/f43 [0,4194304] 0 2024-04-12T05:56:09.187 INFO:tasks.workunit.client.1.smithi194.stdout:2/417: write d7/d12/d25/d5d/d47/f57 [18065,59328] 0 2024-04-12T05:56:09.190 INFO:tasks.workunit.client.1.smithi194.stdout:0/415: getdents d0/da/d1e/d1a/d24/d56 0 2024-04-12T05:56:09.195 INFO:tasks.workunit.client.1.smithi194.stdout:3/553: dwrite d0/d1/f97 [0,4194304] 0 2024-04-12T05:56:09.196 INFO:tasks.workunit.client.1.smithi194.stdout:5/523: symlink d1/d4/d6/d34/d52/lb2 0 2024-04-12T05:56:09.198 INFO:tasks.workunit.client.1.smithi194.stdout:0/416: symlink d0/d5f/d73/l82 0 2024-04-12T05:56:09.199 INFO:tasks.workunit.client.1.smithi194.stdout:8/320: unlink d2/f46 0 2024-04-12T05:56:09.201 INFO:tasks.workunit.client.1.smithi194.stdout:6/476: symlink d3/l90 0 2024-04-12T05:56:09.202 INFO:tasks.workunit.client.1.smithi194.stdout:8/321: write d2/f56 [496796,67378] 0 2024-04-12T05:56:09.206 INFO:tasks.workunit.client.1.smithi194.stdout:5/524: read d1/f3 [8122375,80162] 0 2024-04-12T05:56:09.206 INFO:tasks.workunit.client.1.smithi194.stdout:9/493: dwrite d1/d19/d1f/f52 [0,4194304] 0 2024-04-12T05:56:09.207 INFO:tasks.workunit.client.1.smithi194.stdout:7/340: dread d3/d27/f17 [0,4194304] 0 2024-04-12T05:56:09.207 INFO:tasks.workunit.client.1.smithi194.stdout:1/550: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b [4194304,4194304] 0 2024-04-12T05:56:09.211 INFO:tasks.workunit.client.1.smithi194.stdout:0/417: unlink d0/da/d1e/d27/c57 0 2024-04-12T05:56:09.213 INFO:tasks.workunit.client.1.smithi194.stdout:3/554: write d0/f54 [825339,775] 0 2024-04-12T05:56:09.216 INFO:tasks.workunit.client.1.smithi194.stdout:6/477: symlink d3/d2d/d33/d52/l91 0 2024-04-12T05:56:09.217 INFO:tasks.workunit.client.1.smithi194.stdout:8/322: readlink d2/d8/d9/d1a/l4d 0 2024-04-12T05:56:09.222 INFO:tasks.workunit.client.1.smithi194.stdout:6/478: dread - d3/d2d/d33/d52/f7d zero size 2024-04-12T05:56:09.224 INFO:tasks.workunit.client.1.smithi194.stdout:2/418: getdents d7/d12/d1a 0 2024-04-12T05:56:09.228 INFO:tasks.workunit.client.1.smithi194.stdout:1/551: mknod d0/df/d8d/caa 0 2024-04-12T05:56:09.232 INFO:tasks.workunit.client.1.smithi194.stdout:9/494: dwrite d1/d19/f2e [0,4194304] 0 2024-04-12T05:56:09.233 INFO:tasks.workunit.client.1.smithi194.stdout:3/555: dread d0/d1/d4f/f59 [0,4194304] 0 2024-04-12T05:56:09.233 INFO:tasks.workunit.client.1.smithi194.stdout:8/323: creat d2/d8/d9/de/f61 x:0 0 0 2024-04-12T05:56:09.234 INFO:tasks.workunit.client.1.smithi194.stdout:0/418: dwrite d0/d3b/f7e [0,4194304] 0 2024-04-12T05:56:09.239 INFO:tasks.workunit.client.1.smithi194.stdout:3/556: write d0/dc/d1e/d81/f6b [1771022,62991] 0 2024-04-12T05:56:09.239 INFO:tasks.workunit.client.1.smithi194.stdout:5/525: dwrite d1/d4/d6/db/d2b/f66 [4194304,4194304] 0 2024-04-12T05:56:09.242 INFO:tasks.workunit.client.1.smithi194.stdout:9/495: read d1/d19/d1f/d20/d37/d38/f96 [87102,100049] 0 2024-04-12T05:56:09.244 INFO:tasks.workunit.client.1.smithi194.stdout:8/324: creat d2/d8/d15/f62 x:0 0 0 2024-04-12T05:56:09.244 INFO:tasks.workunit.client.1.smithi194.stdout:0/419: symlink d0/da/l83 0 2024-04-12T05:56:09.244 INFO:tasks.workunit.client.1.smithi194.stdout:7/341: getdents d3/d27 0 2024-04-12T05:56:09.245 INFO:tasks.workunit.client.1.smithi194.stdout:5/526: readlink d1/d4/l1b 0 2024-04-12T05:56:09.247 INFO:tasks.workunit.client.1.smithi194.stdout:2/419: creat d7/d22/f91 x:0 0 0 2024-04-12T05:56:09.250 INFO:tasks.workunit.client.1.smithi194.stdout:6/479: dwrite d3/d1a/f35 [0,4194304] 0 2024-04-12T05:56:09.250 INFO:tasks.workunit.client.1.smithi194.stdout:5/527: write d1/d5c/d95/f72 [185353,33791] 0 2024-04-12T05:56:09.255 INFO:tasks.workunit.client.1.smithi194.stdout:7/342: mkdir d3/d16/d23/d57/d72 0 2024-04-12T05:56:09.255 INFO:tasks.workunit.client.1.smithi194.stdout:1/552: creat d0/d11/d18/d19/d1a/d6a/d27/fab x:0 0 0 2024-04-12T05:56:09.255 INFO:tasks.workunit.client.1.smithi194.stdout:3/557: symlink d0/d1/d10/d8c/d8f/lb7 0 2024-04-12T05:56:09.258 INFO:tasks.workunit.client.1.smithi194.stdout:0/420: creat d0/d3b/d52/f84 x:0 0 0 2024-04-12T05:56:09.258 INFO:tasks.workunit.client.1.smithi194.stdout:8/325: mknod d2/c63 0 2024-04-12T05:56:09.258 INFO:tasks.workunit.client.1.smithi194.stdout:6/480: chown d3/d2f/c3c 1172029154 1 2024-04-12T05:56:09.262 INFO:tasks.workunit.client.1.smithi194.stdout:6/481: chown d3/d15/f22 60624 1 2024-04-12T05:56:09.263 INFO:tasks.workunit.client.1.smithi194.stdout:3/558: write d0/dc/d1e/d81/d5b/d64/fad [296740,78849] 0 2024-04-12T05:56:09.264 INFO:tasks.workunit.client.1.smithi194.stdout:9/496: mkdir d1/d41/d97/d9d 0 2024-04-12T05:56:09.266 INFO:tasks.workunit.client.1.smithi194.stdout:9/497: chown d1/d19/d1f 24910125 1 2024-04-12T05:56:09.266 INFO:tasks.workunit.client.1.smithi194.stdout:7/343: creat d3/d16/d21/f73 x:0 0 0 2024-04-12T05:56:09.267 INFO:tasks.workunit.client.1.smithi194.stdout:0/421: symlink d0/da/d1e/d1a/d24/d56/d66/l85 0 2024-04-12T05:56:09.268 INFO:tasks.workunit.client.1.smithi194.stdout:6/482: symlink d3/d34/l92 0 2024-04-12T05:56:09.269 INFO:tasks.workunit.client.1.smithi194.stdout:1/553: mkdir d0/d11/d18/d19/d1a/d6a/dac 0 2024-04-12T05:56:09.269 INFO:tasks.workunit.client.1.smithi194.stdout:0/422: readlink d0/da/l83 0 2024-04-12T05:56:09.269 INFO:tasks.workunit.client.1.smithi194.stdout:3/559: rmdir d0/dc/d1e/d81/d34 39 2024-04-12T05:56:09.270 INFO:tasks.workunit.client.1.smithi194.stdout:0/423: chown d0/f14 0 1 2024-04-12T05:56:09.272 INFO:tasks.workunit.client.1.smithi194.stdout:6/483: creat d3/d2d/d33/d52/d5c/d8c/f93 x:0 0 0 2024-04-12T05:56:09.273 INFO:tasks.workunit.client.1.smithi194.stdout:3/560: rmdir d0/dc/d1e/d33 39 2024-04-12T05:56:09.273 INFO:tasks.workunit.client.1.smithi194.stdout:0/424: write d0/da/d1e/d1a/f64 [3457925,87612] 0 2024-04-12T05:56:09.274 INFO:tasks.workunit.client.1.smithi194.stdout:3/561: chown d0/d1/d10/d8c/d8f/dac 65570 1 2024-04-12T05:56:09.275 INFO:tasks.workunit.client.1.smithi194.stdout:6/484: write d3/d15/d2b/d48/d6f/f82 [1817695,82233] 0 2024-04-12T05:56:09.278 INFO:tasks.workunit.client.1.smithi194.stdout:2/420: dread d7/d22/d42/f1d [0,4194304] 0 2024-04-12T05:56:09.279 INFO:tasks.workunit.client.1.smithi194.stdout:6/485: truncate d3/d2f/d37/f3f 1238527 0 2024-04-12T05:56:09.281 INFO:tasks.workunit.client.1.smithi194.stdout:1/554: creat d0/d11/d18/d19/d1a/d6a/d27/d9e/fad x:0 0 0 2024-04-12T05:56:09.285 INFO:tasks.workunit.client.1.smithi194.stdout:3/562: truncate d0/d1/d74/d39/fb0 295587 0 2024-04-12T05:56:09.289 INFO:tasks.workunit.client.1.smithi194.stdout:5/528: write d1/d4/d6/f35 [138579,42077] 0 2024-04-12T05:56:09.291 INFO:tasks.workunit.client.1.smithi194.stdout:6/486: dwrite d3/d2f/d4e/d53/f6a [0,4194304] 0 2024-04-12T05:56:09.293 INFO:tasks.workunit.client.1.smithi194.stdout:2/421: dwrite d7/d12/d25/d5d/d47/f57 [0,4194304] 0 2024-04-12T05:56:09.293 INFO:tasks.workunit.client.1.smithi194.stdout:9/498: truncate d1/d19/d1f/d20/d37/d38/f7c 4003579 0 2024-04-12T05:56:09.294 INFO:tasks.workunit.client.1.smithi194.stdout:3/563: creat d0/d1/d74/d39/fb8 x:0 0 0 2024-04-12T05:56:09.297 INFO:tasks.workunit.client.1.smithi194.stdout:5/529: write d1/d4/d6/d34/f54 [1834459,110925] 0 2024-04-12T05:56:09.302 INFO:tasks.workunit.client.1.smithi194.stdout:9/499: creat d1/d19/d1f/d5f/f9e x:0 0 0 2024-04-12T05:56:09.302 INFO:tasks.workunit.client.1.smithi194.stdout:6/487: symlink d3/d2f/l94 0 2024-04-12T05:56:09.305 INFO:tasks.workunit.client.1.smithi194.stdout:0/425: dwrite d0/da/d1e/f32 [0,4194304] 0 2024-04-12T05:56:09.308 INFO:tasks.workunit.client.1.smithi194.stdout:5/530: chown d1/d4/d6/db/d3b/l9d 3785 1 2024-04-12T05:56:09.309 INFO:tasks.workunit.client.1.smithi194.stdout:5/531: write d1/fa [1107766,67758] 0 2024-04-12T05:56:09.309 INFO:tasks.workunit.client.1.smithi194.stdout:0/426: rename d0/da/d1e/d27/d54/l7f to d0/l86 0 2024-04-12T05:56:09.310 INFO:tasks.workunit.client.1.smithi194.stdout:3/564: link d0/cf d0/d1/d10/cb9 0 2024-04-12T05:56:09.310 INFO:tasks.workunit.client.1.smithi194.stdout:9/500: mkdir d1/d21/d77/d98/d9f 0 2024-04-12T05:56:09.314 INFO:tasks.workunit.client.1.smithi194.stdout:0/427: mkdir d0/da/d1e/d27/d87 0 2024-04-12T05:56:09.316 INFO:tasks.workunit.client.1.smithi194.stdout:3/565: rename d0/dc/f91 to d0/d42/fba 0 2024-04-12T05:56:09.317 INFO:tasks.workunit.client.1.smithi194.stdout:9/501: symlink d1/d21/d77/d98/d9f/la0 0 2024-04-12T05:56:09.320 INFO:tasks.workunit.client.1.smithi194.stdout:5/532: symlink d1/d4/d6/db/d3b/lb3 0 2024-04-12T05:56:09.324 INFO:tasks.workunit.client.1.smithi194.stdout:0/428: write d0/da/d1e/d1a/d3d/f51 [1119799,97983] 0 2024-04-12T05:56:09.334 INFO:tasks.workunit.client.1.smithi194.stdout:0/429: rmdir d0/da/d1e/d1a/d24/d56 39 2024-04-12T05:56:09.335 INFO:tasks.workunit.client.1.smithi194.stdout:3/566: chown d0/dc/d1e/d81/d34/d2d/c58 47 1 2024-04-12T05:56:09.336 INFO:tasks.workunit.client.1.smithi194.stdout:9/502: link d1/f46 d1/d41/d97/d9d/fa1 0 2024-04-12T05:56:09.338 INFO:tasks.workunit.client.1.smithi194.stdout:0/430: symlink d0/da/d4a/l88 0 2024-04-12T05:56:09.341 INFO:tasks.workunit.client.1.smithi194.stdout:6/488: truncate d3/d2f/d37/f45 2092770 0 2024-04-12T05:56:09.342 INFO:tasks.workunit.client.1.smithi194.stdout:0/431: creat d0/da/f89 x:0 0 0 2024-04-12T05:56:09.344 INFO:tasks.workunit.client.1.smithi194.stdout:9/503: dread d1/d41/f6f [0,4194304] 0 2024-04-12T05:56:09.346 INFO:tasks.workunit.client.1.smithi194.stdout:0/432: chown d0/da/d1e/d1a/f23 174837197 1 2024-04-12T05:56:09.348 INFO:tasks.workunit.client.1.smithi194.stdout:3/567: dwrite d0/d1/d74/f3f [4194304,4194304] 0 2024-04-12T05:56:09.349 INFO:tasks.workunit.client.1.smithi194.stdout:9/504: mknod d1/d21/d49/ca2 0 2024-04-12T05:56:09.351 INFO:tasks.workunit.client.1.smithi194.stdout:0/433: mknod d0/da/d1e/d27/d87/c8a 0 2024-04-12T05:56:09.357 INFO:tasks.workunit.client.1.smithi194.stdout:0/434: unlink d0/da/d1e/d1a/d24/c28 0 2024-04-12T05:56:09.357 INFO:tasks.workunit.client.1.smithi194.stdout:5/533: dwrite d1/d4/d6/db/d2b/f73 [0,4194304] 0 2024-04-12T05:56:09.358 INFO:tasks.workunit.client.1.smithi194.stdout:9/505: getdents d1/d4e 0 2024-04-12T05:56:09.359 INFO:tasks.workunit.client.1.smithi194.stdout:5/534: write d1/d4/d6/f35 [446833,121597] 0 2024-04-12T05:56:09.364 INFO:tasks.workunit.client.1.smithi194.stdout:5/535: truncate d1/d4/d6/db/f53 490227 0 2024-04-12T05:56:09.364 INFO:tasks.workunit.client.1.smithi194.stdout:9/506: mkdir d1/d19/d1f/d20/d37/da3 0 2024-04-12T05:56:09.365 INFO:tasks.workunit.client.1.smithi194.stdout:9/507: stat d1/d19/d1f/d5f 0 2024-04-12T05:56:09.366 INFO:tasks.workunit.client.1.smithi194.stdout:9/508: fsync d1/d19/d1f/d5f/f85 0 2024-04-12T05:56:09.369 INFO:tasks.workunit.client.1.smithi194.stdout:9/509: creat d1/d21/d77/fa4 x:0 0 0 2024-04-12T05:56:09.369 INFO:tasks.workunit.client.1.smithi194.stdout:5/536: link d1/d4/d6/d34/d52/l93 d1/d4/d6/d8e/lb4 0 2024-04-12T05:56:09.371 INFO:tasks.workunit.client.1.smithi194.stdout:9/510: mknod d1/d19/d1f/ca5 0 2024-04-12T05:56:09.371 INFO:tasks.workunit.client.1.smithi194.stdout:5/537: creat d1/d4/d84/fb5 x:0 0 0 2024-04-12T05:56:09.373 INFO:tasks.workunit.client.1.smithi194.stdout:9/511: mkdir d1/da6 0 2024-04-12T05:56:09.375 INFO:tasks.workunit.client.1.smithi194.stdout:5/538: chown d1/d4/d6/db/f53 8972360 1 2024-04-12T05:56:09.377 INFO:tasks.workunit.client.1.smithi194.stdout:9/512: link d1/d21/d77/fa4 d1/d21/d39/d72/fa7 0 2024-04-12T05:56:09.380 INFO:tasks.workunit.client.1.smithi194.stdout:6/489: rename d3/d15/f4b to d3/d2d/d33/d52/f95 0 2024-04-12T05:56:09.382 INFO:tasks.workunit.client.1.smithi194.stdout:6/490: fdatasync d3/f3b 0 2024-04-12T05:56:09.385 INFO:tasks.workunit.client.1.smithi194.stdout:3/568: rename d0/dc/d3d/f90 to d0/d1/d74/d39/fbb 0 2024-04-12T05:56:09.386 INFO:tasks.workunit.client.1.smithi194.stdout:9/513: dwrite d1/d19/d1f/d5f/f86 [0,4194304] 0 2024-04-12T05:56:09.388 INFO:tasks.workunit.client.1.smithi194.stdout:6/491: symlink d3/d15/d2b/d48/l96 0 2024-04-12T05:56:09.392 INFO:tasks.workunit.client.1.smithi194.stdout:3/569: unlink d0/cf 0 2024-04-12T05:56:09.394 INFO:tasks.workunit.client.1.smithi194.stdout:0/435: dwrite d0/da/d1e/d1a/d58/f80 [0,4194304] 0 2024-04-12T05:56:09.396 INFO:tasks.workunit.client.1.smithi194.stdout:0/436: read d0/da/d1e/d1a/f31 [660181,36343] 0 2024-04-12T05:56:09.464 INFO:tasks.workunit.client.1.smithi194.stdout:6/492: mknod d3/d15/d2b/d48/c97 0 2024-04-12T05:56:09.472 INFO:tasks.workunit.client.1.smithi194.stdout:6/493: dwrite d3/d2f/d37/f4a [8388608,4194304] 0 2024-04-12T05:56:09.472 INFO:tasks.workunit.client.1.smithi194.stdout:3/570: mkdir d0/d1/d10/d8c/d8f/dbc 0 2024-04-12T05:56:09.475 INFO:tasks.workunit.client.1.smithi194.stdout:3/571: read - d0/d1/d74/fb1 zero size 2024-04-12T05:56:09.476 INFO:tasks.workunit.client.1.smithi194.stdout:9/514: creat d1/da6/fa8 x:0 0 0 2024-04-12T05:56:09.480 INFO:tasks.workunit.client.1.smithi194.stdout:5/539: dwrite d1/d4/d6/d88/f2c [0,4194304] 0 2024-04-12T05:56:09.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:09 smithi194 ceph-mon[49771]: pgmap v30: 65 pgs: 65 active+clean; 908 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 18 MiB/s rd, 94 MiB/s wr, 249 op/s 2024-04-12T05:56:09.752 INFO:tasks.workunit.client.1.smithi194.stdout:8/326: sync 2024-04-12T05:56:09.754 INFO:tasks.workunit.client.1.smithi194.stdout:8/327: readlink d2/d8/d9/d1a/l22 0 2024-04-12T05:56:09.850 INFO:tasks.workunit.client.1.smithi194.stdout:7/344: sync 2024-04-12T05:56:09.850 INFO:tasks.workunit.client.1.smithi194.stdout:2/422: sync 2024-04-12T05:56:09.850 INFO:tasks.workunit.client.1.smithi194.stdout:1/555: sync 2024-04-12T05:56:09.861 INFO:tasks.workunit.client.1.smithi194.stdout:2/423: truncate d7/d12/f80 813366 0 2024-04-12T05:56:09.881 INFO:tasks.workunit.client.1.smithi194.stdout:0/437: sync 2024-04-12T05:56:10.086 INFO:tasks.workunit.client.1.smithi194.stdout:9/515: symlink d1/d21/d77/d98/d9f/la9 0 2024-04-12T05:56:10.104 INFO:tasks.workunit.client.1.smithi194.stdout:9/516: dread d1/d21/f61 [0,4194304] 0 2024-04-12T05:56:10.118 INFO:tasks.workunit.client.1.smithi194.stdout:3/572: chown d0/dc/d1e/d81/d5b/d64/d79/d87/c9d 216528 1 2024-04-12T05:56:10.123 INFO:tasks.workunit.client.1.smithi194.stdout:5/540: creat d1/d4/d75/fb6 x:0 0 0 2024-04-12T05:56:10.125 INFO:tasks.workunit.client.1.smithi194.stdout:6/494: truncate d3/d2d/d33/d52/d5c/f5f 1467429 0 2024-04-12T05:56:10.139 INFO:tasks.workunit.client.1.smithi194.stdout:8/328: symlink d2/d8/d9/de/d10/l64 0 2024-04-12T05:56:10.140 INFO:tasks.workunit.client.1.smithi194.stdout:7/345: mkdir d3/d16/d23/d40/d49/d74 0 2024-04-12T05:56:10.141 INFO:tasks.workunit.client.1.smithi194.stdout:1/556: creat d0/d11/fae x:0 0 0 2024-04-12T05:56:10.142 INFO:tasks.workunit.client.1.smithi194.stdout:1/557: stat d0/d11/d18/d19/l2f 0 2024-04-12T05:56:10.144 INFO:tasks.workunit.client.1.smithi194.stdout:9/517: symlink d1/d19/d1f/d5f/laa 0 2024-04-12T05:56:10.145 INFO:tasks.workunit.client.1.smithi194.stdout:5/541: symlink d1/d4/d6/d8e/lb7 0 2024-04-12T05:56:10.148 INFO:tasks.workunit.client.1.smithi194.stdout:8/329: unlink d2/d8/d9/de/f5f 0 2024-04-12T05:56:10.148 INFO:tasks.workunit.client.1.smithi194.stdout:0/438: dwrite d0/da/d1e/d1a/f6d [0,4194304] 0 2024-04-12T05:56:10.152 INFO:tasks.workunit.client.1.smithi194.stdout:2/424: fsync d7/d22/d42/f54 0 2024-04-12T05:56:10.155 INFO:tasks.workunit.client.1.smithi194.stdout:6/495: mkdir d3/d15/d2b/d48/d6f/d7a/d8f/d98 0 2024-04-12T05:56:10.155 INFO:tasks.workunit.client.1.smithi194.stdout:9/518: mkdir d1/d21/d77/d98/dab 0 2024-04-12T05:56:10.157 INFO:tasks.workunit.client.1.smithi194.stdout:5/542: creat d1/d4/d6/d34/d52/fb8 x:0 0 0 2024-04-12T05:56:10.158 INFO:tasks.workunit.client.1.smithi194.stdout:0/439: mknod d0/d5f/d73/d76/c8b 0 2024-04-12T05:56:10.161 INFO:tasks.workunit.client.1.smithi194.stdout:8/330: rename d2/d8/d9/d1a/l53 to d2/d8/d9/de/d38/l65 0 2024-04-12T05:56:10.162 INFO:tasks.workunit.client.1.smithi194.stdout:8/331: stat d2/d8/d9/d1a/f25 0 2024-04-12T05:56:10.163 INFO:tasks.workunit.client.1.smithi194.stdout:5/543: dread d1/f82 [0,4194304] 0 2024-04-12T05:56:10.163 INFO:tasks.workunit.client.1.smithi194.stdout:2/425: mknod d7/c92 0 2024-04-12T05:56:10.164 INFO:tasks.workunit.client.1.smithi194.stdout:5/544: dread - d1/d4/d84/f99 zero size 2024-04-12T05:56:10.168 INFO:tasks.workunit.client.1.smithi194.stdout:3/573: unlink d0/d1/d10/d89/c99 0 2024-04-12T05:56:10.170 INFO:tasks.workunit.client.1.smithi194.stdout:6/496: fsync d3/d15/d2b/d48/f4f 0 2024-04-12T05:56:10.172 INFO:tasks.workunit.client.1.smithi194.stdout:6/497: fsync d3/d1a/f35 0 2024-04-12T05:56:10.172 INFO:tasks.workunit.client.1.smithi194.stdout:8/332: dwrite d2/d8/d9/de/d10/f41 [0,4194304] 0 2024-04-12T05:56:10.175 INFO:tasks.workunit.client.1.smithi194.stdout:9/519: write d1/d19/d1f/d20/f71 [2165689,87985] 0 2024-04-12T05:56:10.177 INFO:tasks.workunit.client.1.smithi194.stdout:7/346: write d3/d16/d23/d37/f54 [2617616,96795] 0 2024-04-12T05:56:10.178 INFO:tasks.workunit.client.1.smithi194.stdout:1/558: link d0/le d0/d11/d18/d19/d1a/laf 0 2024-04-12T05:56:10.178 INFO:tasks.workunit.client.1.smithi194.stdout:7/347: chown d3/d16/d23/d40/d49/d74 53161 1 2024-04-12T05:56:10.180 INFO:tasks.workunit.client.1.smithi194.stdout:0/440: dread d0/da/d1e/f15 [0,4194304] 0 2024-04-12T05:56:10.180 INFO:tasks.workunit.client.1.smithi194.stdout:5/545: mknod d1/d4/d6/d6d/cb9 0 2024-04-12T05:56:10.181 INFO:tasks.workunit.client.1.smithi194.stdout:7/348: readlink d3/d16/d21/d3a/d43/l4e 0 2024-04-12T05:56:10.186 INFO:tasks.workunit.client.1.smithi194.stdout:3/574: dread d0/dc/d1e/d81/d5b/d64/fad [0,4194304] 0 2024-04-12T05:56:10.192 INFO:tasks.workunit.client.1.smithi194.stdout:1/559: dwrite d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [4194304,4194304] 0 2024-04-12T05:56:10.197 INFO:tasks.workunit.client.1.smithi194.stdout:2/426: mkdir d7/d22/d93 0 2024-04-12T05:56:10.200 INFO:tasks.workunit.client.1.smithi194.stdout:5/546: dwrite d1/d4/d6/d34/fa0 [0,4194304] 0 2024-04-12T05:56:10.203 INFO:tasks.workunit.client.1.smithi194.stdout:0/441: mknod d0/da/d1e/d27/d87/c8c 0 2024-04-12T05:56:10.203 INFO:tasks.workunit.client.1.smithi194.stdout:3/575: dwrite d0/dc/f12 [0,4194304] 0 2024-04-12T05:56:10.205 INFO:tasks.workunit.client.1.smithi194.stdout:5/547: dread - d1/d4/d84/f99 zero size 2024-04-12T05:56:10.211 INFO:tasks.workunit.client.1.smithi194.stdout:6/498: fdatasync d3/d2d/d33/d52/d5c/f5f 0 2024-04-12T05:56:10.217 INFO:tasks.workunit.client.1.smithi194.stdout:0/442: chown d0/da/d1e/d1a/d24/c43 761455 1 2024-04-12T05:56:10.218 INFO:tasks.workunit.client.1.smithi194.stdout:8/333: dread d2/d8/d15/d33/f1b [0,4194304] 0 2024-04-12T05:56:10.219 INFO:tasks.workunit.client.1.smithi194.stdout:8/334: chown d2/f43 517923309 1 2024-04-12T05:56:10.222 INFO:tasks.workunit.client.1.smithi194.stdout:9/520: creat d1/d19/d1f/d20/fac x:0 0 0 2024-04-12T05:56:10.222 INFO:tasks.workunit.client.1.smithi194.stdout:6/499: symlink d3/d70/l99 0 2024-04-12T05:56:10.228 INFO:tasks.workunit.client.1.smithi194.stdout:1/560: dread d0/d11/d18/d19/f39 [0,4194304] 0 2024-04-12T05:56:10.231 INFO:tasks.workunit.client.1.smithi194.stdout:3/576: link d0/d1/d74/f3f d0/d1/d74/d39/da2/fbd 0 2024-04-12T05:56:10.234 INFO:tasks.workunit.client.1.smithi194.stdout:6/500: write d3/d15/f22 [3883873,9223] 0 2024-04-12T05:56:10.234 INFO:tasks.workunit.client.1.smithi194.stdout:9/521: creat d1/d19/d1f/d20/d7d/fad x:0 0 0 2024-04-12T05:56:10.234 INFO:tasks.workunit.client.1.smithi194.stdout:0/443: creat d0/da/d1e/d1a/d24/d56/f8d x:0 0 0 2024-04-12T05:56:10.236 INFO:tasks.workunit.client.1.smithi194.stdout:8/335: dwrite d2/d8/d9/d1a/f30 [0,4194304] 0 2024-04-12T05:56:10.237 INFO:tasks.workunit.client.1.smithi194.stdout:7/349: truncate d3/d16/f19 4004543 0 2024-04-12T05:56:10.240 INFO:tasks.workunit.client.1.smithi194.stdout:7/350: write d3/d27/d22/f2e [469075,20821] 0 2024-04-12T05:56:10.242 INFO:tasks.workunit.client.1.smithi194.stdout:2/427: truncate d7/d12/d25/d5d/d47/f7f 4338554 0 2024-04-12T05:56:10.245 INFO:tasks.workunit.client.1.smithi194.stdout:5/548: write d1/d4/d6/db/d2b/f33 [570105,30046] 0 2024-04-12T05:56:10.246 INFO:tasks.workunit.client.1.smithi194.stdout:0/444: unlink d0/da/d1e/d1a/d24/d4b/l63 0 2024-04-12T05:56:10.249 INFO:tasks.workunit.client.1.smithi194.stdout:3/577: write d0/d1/d74/d39/da2/fbd [5180106,19143] 0 2024-04-12T05:56:10.253 INFO:tasks.workunit.client.1.smithi194.stdout:3/578: write d0/d42/f51 [155617,50785] 0 2024-04-12T05:56:10.255 INFO:tasks.workunit.client.1.smithi194.stdout:9/522: dwrite d1/d21/f4d [0,4194304] 0 2024-04-12T05:56:10.257 INFO:tasks.workunit.client.1.smithi194.stdout:1/561: rename d0/d11/d18/d19/d1a/d6a/d27/d48/l89 to d0/d11/d18/d19/d1c/lb0 0 2024-04-12T05:56:10.257 INFO:tasks.workunit.client.1.smithi194.stdout:6/501: mkdir d3/d2f/d4e/d9a 0 2024-04-12T05:56:10.258 INFO:tasks.workunit.client.1.smithi194.stdout:3/579: write d0/d1/f97 [1856114,37940] 0 2024-04-12T05:56:10.258 INFO:tasks.workunit.client.1.smithi194.stdout:8/336: fsync d2/d8/d15/f16 0 2024-04-12T05:56:10.259 INFO:tasks.workunit.client.1.smithi194.stdout:6/502: stat d3/d15/d2b 0 2024-04-12T05:56:10.260 INFO:tasks.workunit.client.1.smithi194.stdout:9/523: mknod d1/da6/cae 0 2024-04-12T05:56:10.262 INFO:tasks.workunit.client.1.smithi194.stdout:9/524: readlink d1/d21/d77/d98/d9f/la0 0 2024-04-12T05:56:10.262 INFO:tasks.workunit.client.1.smithi194.stdout:6/503: write d3/d15/d2b/d48/f54 [2516189,128087] 0 2024-04-12T05:56:10.264 INFO:tasks.workunit.client.1.smithi194.stdout:0/445: rename d0/da/d1e/d1a/d24/d56/d66 to d0/da/d4a/d8e 0 2024-04-12T05:56:10.268 INFO:tasks.workunit.client.1.smithi194.stdout:2/428: creat d7/d22/f94 x:0 0 0 2024-04-12T05:56:10.274 INFO:tasks.workunit.client.1.smithi194.stdout:9/525: dwrite d1/d21/d39/f51 [0,4194304] 0 2024-04-12T05:56:10.291 INFO:tasks.workunit.client.1.smithi194.stdout:9/526: dwrite d1/d19/d1f/d5f/f85 [0,4194304] 0 2024-04-12T05:56:10.302 INFO:tasks.workunit.client.1.smithi194.stdout:9/527: write d1/d19/d1f/d20/f71 [150102,48576] 0 2024-04-12T05:56:10.303 INFO:tasks.workunit.client.1.smithi194.stdout:0/446: dread d0/f48 [0,4194304] 0 2024-04-12T05:56:10.308 INFO:tasks.workunit.client.1.smithi194.stdout:3/580: mkdir d0/dc/d3d/dae/dbe 0 2024-04-12T05:56:10.313 INFO:tasks.workunit.client.1.smithi194.stdout:1/562: creat d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fb1 x:0 0 0 2024-04-12T05:56:10.316 INFO:tasks.workunit.client.1.smithi194.stdout:7/351: truncate d3/d27/f1b 4060345 0 2024-04-12T05:56:10.317 INFO:tasks.workunit.client.1.smithi194.stdout:7/352: rename d3 to d3/d16/d75 22 2024-04-12T05:56:10.320 INFO:tasks.workunit.client.1.smithi194.stdout:6/504: write d3/d2f/f39 [2571397,54981] 0 2024-04-12T05:56:10.328 INFO:tasks.workunit.client.1.smithi194.stdout:2/429: creat d7/d12/d1a/d68/f95 x:0 0 0 2024-04-12T05:56:10.329 INFO:tasks.workunit.client.1.smithi194.stdout:2/430: chown d7/d12/d34/d4a/l85 2 1 2024-04-12T05:56:10.343 INFO:tasks.workunit.client.1.smithi194.stdout:1/563: rename d0/df/c3d to d0/d11/d18/d19/d1c/d67/cb2 0 2024-04-12T05:56:10.351 INFO:tasks.workunit.client.1.smithi194.stdout:2/431: write d7/d12/d25/f31 [518269,42720] 0 2024-04-12T05:56:10.353 INFO:tasks.workunit.client.1.smithi194.stdout:2/432: dread - d7/d12/d1a/d69/f82 zero size 2024-04-12T05:56:10.357 INFO:tasks.workunit.client.1.smithi194.stdout:7/353: creat d3/d16/d21/d3a/d43/d5c/f76 x:0 0 0 2024-04-12T05:56:10.357 INFO:tasks.workunit.client.1.smithi194.stdout:7/354: chown d3/d16/d21/d3a 180 1 2024-04-12T05:56:10.359 INFO:tasks.workunit.client.1.smithi194.stdout:7/355: fdatasync d3/d27/d22/f2e 0 2024-04-12T05:56:10.362 INFO:tasks.workunit.client.1.smithi194.stdout:9/528: creat d1/d21/d39/faf x:0 0 0 2024-04-12T05:56:10.363 INFO:tasks.workunit.client.1.smithi194.stdout:6/505: rmdir d3/d2f/d4e/d68 39 2024-04-12T05:56:10.363 INFO:tasks.workunit.client.1.smithi194.stdout:1/564: dread d0/d11/d18/d19/d1a/d6a/d27/d48/f62 [0,4194304] 0 2024-04-12T05:56:10.369 INFO:tasks.workunit.client.1.smithi194.stdout:1/565: dread d0/df/f82 [0,4194304] 0 2024-04-12T05:56:10.372 INFO:tasks.workunit.client.1.smithi194.stdout:1/566: stat d0/d11/d18/d19/d1c/d67/l99 0 2024-04-12T05:56:10.372 INFO:tasks.workunit.client.1.smithi194.stdout:9/529: mkdir d1/d21/d77/db0 0 2024-04-12T05:56:10.373 INFO:tasks.workunit.client.1.smithi194.stdout:9/530: stat d1/d21/d39/d72/l82 0 2024-04-12T05:56:10.374 INFO:tasks.workunit.client.1.smithi194.stdout:9/531: read d1/d21/d39/f51 [1380931,114790] 0 2024-04-12T05:56:10.375 INFO:tasks.workunit.client.1.smithi194.stdout:0/447: getdents d0/d3b 0 2024-04-12T05:56:10.376 INFO:tasks.workunit.client.1.smithi194.stdout:0/448: chown d0/d5f/d73/d76/c8b 2327398 1 2024-04-12T05:56:10.385 INFO:tasks.workunit.client.1.smithi194.stdout:9/532: rename d1/d21/d49/c67 to d1/d19/d1f/d20/d37/d84/cb1 0 2024-04-12T05:56:10.391 INFO:tasks.workunit.client.1.smithi194.stdout:1/567: creat d0/d11/d18/d19/d1a/d6a/d27/d2e/fb3 x:0 0 0 2024-04-12T05:56:10.394 INFO:tasks.workunit.client.1.smithi194.stdout:2/433: dread d7/d12/d6d/f72 [0,4194304] 0 2024-04-12T05:56:10.396 INFO:tasks.workunit.client.1.smithi194.stdout:0/449: mkdir d0/da/d1e/d8f 0 2024-04-12T05:56:10.397 INFO:tasks.workunit.client.1.smithi194.stdout:0/450: chown d0/f14 3844597 1 2024-04-12T05:56:10.398 INFO:tasks.workunit.client.1.smithi194.stdout:3/581: write d0/dc/d1e/d33/f68 [928839,90521] 0 2024-04-12T05:56:10.399 INFO:tasks.workunit.client.1.smithi194.stdout:1/568: mknod d0/d11/d18/d19/cb4 0 2024-04-12T05:56:10.400 INFO:tasks.workunit.client.1.smithi194.stdout:3/582: chown d0/d42/c80 2296 1 2024-04-12T05:56:10.400 INFO:tasks.workunit.client.1.smithi194.stdout:9/533: dwrite d1/d4e/f7e [4194304,4194304] 0 2024-04-12T05:56:10.401 INFO:tasks.workunit.client.1.smithi194.stdout:9/534: readlink d1/l33 0 2024-04-12T05:56:10.401 INFO:tasks.workunit.client.1.smithi194.stdout:2/434: mknod d7/d12/d25/d5d/d47/d52/d7b/c96 0 2024-04-12T05:56:10.404 INFO:tasks.workunit.client.1.smithi194.stdout:2/435: chown d7/d12/d25/d5d/d47/d52/d7b 0 1 2024-04-12T05:56:10.406 INFO:tasks.workunit.client.1.smithi194.stdout:2/436: chown d7/c45 5632 1 2024-04-12T05:56:10.409 INFO:tasks.workunit.client.1.smithi194.stdout:0/451: mknod d0/da/d1e/d1a/d24/c90 0 2024-04-12T05:56:10.410 INFO:tasks.workunit.client.1.smithi194.stdout:3/583: dread d0/d1/d10/f52 [0,4194304] 0 2024-04-12T05:56:10.412 INFO:tasks.workunit.client.1.smithi194.stdout:9/535: dwrite d1/d4e/f7e [4194304,4194304] 0 2024-04-12T05:56:10.412 INFO:tasks.workunit.client.1.smithi194.stdout:8/337: mkdir d2/d8/d66 0 2024-04-12T05:56:10.414 INFO:tasks.workunit.client.1.smithi194.stdout:9/536: write d1/d19/d1f/f70 [325650,92253] 0 2024-04-12T05:56:10.414 INFO:tasks.workunit.client.1.smithi194.stdout:0/452: creat d0/da/f91 x:0 0 0 2024-04-12T05:56:10.415 INFO:tasks.workunit.client.1.smithi194.stdout:6/506: write d3/d2f/d37/f45 [2292336,7358] 0 2024-04-12T05:56:10.418 INFO:tasks.workunit.client.1.smithi194.stdout:3/584: unlink d0/d1/la4 0 2024-04-12T05:56:10.419 INFO:tasks.workunit.client.1.smithi194.stdout:6/507: write d3/d15/d2b/d48/f74 [442837,48136] 0 2024-04-12T05:56:10.421 INFO:tasks.workunit.client.1.smithi194.stdout:2/437: creat d7/d12/d34/f97 x:0 0 0 2024-04-12T05:56:10.424 INFO:tasks.workunit.client.1.smithi194.stdout:0/453: mkdir d0/da/d1e/d27/d87/d92 0 2024-04-12T05:56:10.424 INFO:tasks.workunit.client.1.smithi194.stdout:8/338: mkdir d2/d8/d66/d67 0 2024-04-12T05:56:10.426 INFO:tasks.workunit.client.1.smithi194.stdout:6/508: rmdir d3/d2d/d33/d52/d5c/d8c 39 2024-04-12T05:56:10.427 INFO:tasks.workunit.client.1.smithi194.stdout:6/509: fsync d3/d2f/d4e/d63/f67 0 2024-04-12T05:56:10.428 INFO:tasks.workunit.client.1.smithi194.stdout:2/438: mknod d7/d22/d93/c98 0 2024-04-12T05:56:10.432 INFO:tasks.workunit.client.1.smithi194.stdout:9/537: rename d1/d19/d1f/d20/l6b to d1/d19/lb2 0 2024-04-12T05:56:10.435 INFO:tasks.workunit.client.1.smithi194.stdout:6/510: creat d3/d2f/d4e/d63/d83/f9b x:0 0 0 2024-04-12T05:56:10.438 INFO:tasks.workunit.client.1.smithi194.stdout:2/439: dwrite d7/d12/d25/d5d/f51 [0,4194304] 0 2024-04-12T05:56:10.439 INFO:tasks.workunit.client.1.smithi194.stdout:9/538: rename d1/d19/d1f/f2c to d1/d19/d1f/d20/d37/da3/fb3 0 2024-04-12T05:56:10.439 INFO:tasks.workunit.client.1.smithi194.stdout:2/440: write d7/d12/d25/d5d/d47/d52/f60 [438425,128526] 0 2024-04-12T05:56:10.440 INFO:tasks.workunit.client.1.smithi194.stdout:6/511: symlink d3/d2d/d33/d58/l9c 0 2024-04-12T05:56:10.441 INFO:tasks.workunit.client.1.smithi194.stdout:9/539: write d1/d19/d1f/f88 [613883,4695] 0 2024-04-12T05:56:10.442 INFO:tasks.workunit.client.1.smithi194.stdout:9/540: fdatasync d1/f18 0 2024-04-12T05:56:10.448 INFO:tasks.workunit.client.1.smithi194.stdout:6/512: symlink d3/d2d/d33/d52/l9d 0 2024-04-12T05:56:10.449 INFO:tasks.workunit.client.1.smithi194.stdout:2/441: mknod d7/d12/d1a/d69/c99 0 2024-04-12T05:56:10.453 INFO:tasks.workunit.client.1.smithi194.stdout:0/454: dwrite d0/da/d1e/f17 [0,4194304] 0 2024-04-12T05:56:10.457 INFO:tasks.workunit.client.1.smithi194.stdout:2/442: mkdir d7/d22/d93/d9a 0 2024-04-12T05:56:10.460 INFO:tasks.workunit.client.1.smithi194.stdout:9/541: creat d1/d19/d1f/d20/fb4 x:0 0 0 2024-04-12T05:56:10.461 INFO:tasks.workunit.client.1.smithi194.stdout:9/542: dread - d1/d19/d1f/d20/f8b zero size 2024-04-12T05:56:10.462 INFO:tasks.workunit.client.1.smithi194.stdout:6/513: creat d3/d2f/f9e x:0 0 0 2024-04-12T05:56:10.463 INFO:tasks.workunit.client.1.smithi194.stdout:9/543: dread - d1/d19/d1f/d20/d37/d38/f8f zero size 2024-04-12T05:56:10.463 INFO:tasks.workunit.client.1.smithi194.stdout:0/455: truncate d0/da/d1e/fe 2373778 0 2024-04-12T05:56:10.466 INFO:tasks.workunit.client.1.smithi194.stdout:2/443: dread d7/d12/f33 [0,4194304] 0 2024-04-12T05:56:10.466 INFO:tasks.workunit.client.1.smithi194.stdout:0/456: readlink d0/da/d1e/d1a/l33 0 2024-04-12T05:56:10.468 INFO:tasks.workunit.client.1.smithi194.stdout:6/514: unlink d3/d2f/d37/f80 0 2024-04-12T05:56:10.471 INFO:tasks.workunit.client.1.smithi194.stdout:6/515: creat d3/d15/f9f x:0 0 0 2024-04-12T05:56:10.472 INFO:tasks.workunit.client.1.smithi194.stdout:0/457: dread d0/d3b/f7e [0,4194304] 0 2024-04-12T05:56:10.473 INFO:tasks.workunit.client.1.smithi194.stdout:6/516: fdatasync d3/d15/d2b/d48/f74 0 2024-04-12T05:56:10.475 INFO:tasks.workunit.client.1.smithi194.stdout:2/444: rename d7/d22/d42/f54 to d7/d12/d25/d5d/d47/d52/f9b 0 2024-04-12T05:56:10.476 INFO:tasks.workunit.client.1.smithi194.stdout:2/445: write d7/d12/d25/f64 [3272283,71405] 0 2024-04-12T05:56:10.479 INFO:tasks.workunit.client.1.smithi194.stdout:0/458: creat d0/da/d1e/d1a/d24/d4b/f93 x:0 0 0 2024-04-12T05:56:10.480 INFO:tasks.workunit.client.1.smithi194.stdout:2/446: symlink d7/d12/d25/d5d/d47/d52/d7b/l9c 0 2024-04-12T05:56:10.481 INFO:tasks.workunit.client.1.smithi194.stdout:6/517: mknod d3/d2f/d4e/d63/ca0 0 2024-04-12T05:56:10.482 INFO:tasks.workunit.client.1.smithi194.stdout:0/459: truncate d0/f30 3375899 0 2024-04-12T05:56:10.485 INFO:tasks.workunit.client.1.smithi194.stdout:6/518: read d3/d2f/d37/f43 [3169777,106986] 0 2024-04-12T05:56:10.486 INFO:tasks.workunit.client.1.smithi194.stdout:2/447: rename d7/d12/d34/d4a/l58 to d7/d22/d93/d9a/l9d 0 2024-04-12T05:56:10.487 INFO:tasks.workunit.client.1.smithi194.stdout:2/448: rename d7/d12/d1a to d7/d12/d1a/d69/d9e 22 2024-04-12T05:56:10.488 INFO:tasks.workunit.client.1.smithi194.stdout:2/449: read - d7/d12/d25/d5d/d47/f88 zero size 2024-04-12T05:56:10.489 INFO:tasks.workunit.client.1.smithi194.stdout:6/519: creat d3/d2d/d33/fa1 x:0 0 0 2024-04-12T05:56:10.494 INFO:tasks.workunit.client.1.smithi194.stdout:0/460: truncate d0/f14 4333431 0 2024-04-12T05:56:10.497 INFO:tasks.workunit.client.1.smithi194.stdout:0/461: creat d0/f94 x:0 0 0 2024-04-12T05:56:10.500 INFO:tasks.workunit.client.1.smithi194.stdout:0/462: symlink d0/d5f/d73/l95 0 2024-04-12T05:56:10.501 INFO:tasks.workunit.client.1.smithi194.stdout:6/520: dwrite d3/d2d/d33/d52/d5c/d8c/f93 [0,4194304] 0 2024-04-12T05:56:10.505 INFO:tasks.workunit.client.1.smithi194.stdout:0/463: creat d0/da/f96 x:0 0 0 2024-04-12T05:56:10.507 INFO:tasks.workunit.client.1.smithi194.stdout:0/464: truncate d0/d3b/d52/f84 586465 0 2024-04-12T05:56:10.508 INFO:tasks.workunit.client.1.smithi194.stdout:0/465: chown d0/da/d1e/d1a/l42 0 1 2024-04-12T05:56:10.511 INFO:tasks.workunit.client.1.smithi194.stdout:6/521: rename d3/d2d/d33/d52/f95 to d3/d2f/d4e/d68/fa2 0 2024-04-12T05:56:10.512 INFO:tasks.workunit.client.1.smithi194.stdout:6/522: fdatasync d3/d2d/d33/d52/f55 0 2024-04-12T05:56:10.514 INFO:tasks.workunit.client.1.smithi194.stdout:6/523: write d3/d15/d2b/d48/d6f/f82 [2887937,62002] 0 2024-04-12T05:56:10.515 INFO:tasks.workunit.client.1.smithi194.stdout:0/466: symlink d0/da/d1e/d27/d87/d92/l97 0 2024-04-12T05:56:10.516 INFO:tasks.workunit.client.1.smithi194.stdout:0/467: write d0/da/d1e/d1a/d24/d4b/f93 [731046,81387] 0 2024-04-12T05:56:10.518 INFO:tasks.workunit.client.1.smithi194.stdout:6/524: creat d3/d15/d2b/d48/d6f/d7a/d8f/fa3 x:0 0 0 2024-04-12T05:56:10.520 INFO:tasks.workunit.client.1.smithi194.stdout:0/468: rename d0/l79 to d0/da/d1e/d1a/d58/d5b/l98 0 2024-04-12T05:56:10.524 INFO:tasks.workunit.client.1.smithi194.stdout:0/469: dread d0/da/d1e/d1a/f6d [0,4194304] 0 2024-04-12T05:56:10.531 INFO:tasks.workunit.client.1.smithi194.stdout:0/470: dwrite d0/da/d1e/f17 [0,4194304] 0 2024-04-12T05:56:10.535 INFO:tasks.workunit.client.1.smithi194.stdout:0/471: stat d0/da/d1e/d1a/d24/l50 0 2024-04-12T05:56:10.552 INFO:tasks.workunit.client.1.smithi194.stdout:4/525: mkdir d2/d3/d1b/d32/db3 0 2024-04-12T05:56:10.554 INFO:tasks.workunit.client.1.smithi194.stdout:0/472: dread d0/da/d1e/d1a/d24/d4b/f6f [0,4194304] 0 2024-04-12T05:56:10.556 INFO:tasks.workunit.client.1.smithi194.stdout:0/473: truncate d0/da/f96 492666 0 2024-04-12T05:56:10.556 INFO:tasks.workunit.client.1.smithi194.stdout:4/526: mkdir d2/d3/d1b/d32/db3/db4 0 2024-04-12T05:56:10.558 INFO:tasks.workunit.client.1.smithi194.stdout:0/474: mknod d0/da/d1e/d27/c99 0 2024-04-12T05:56:10.561 INFO:tasks.workunit.client.1.smithi194.stdout:4/527: symlink d2/d11/d1e/d25/lb5 0 2024-04-12T05:56:10.567 INFO:tasks.workunit.client.1.smithi194.stdout:4/528: write d2/d3/d58/f69 [259446,101354] 0 2024-04-12T05:56:10.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:10 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:10.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:10 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:10.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:10 smithi194 ceph-mon[49771]: pgmap v31: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 40 MiB/s rd, 158 MiB/s wr, 454 op/s 2024-04-12T05:56:10.574 INFO:tasks.workunit.client.1.smithi194.stdout:4/529: unlink d2/d11/d1e/d25/f6e 0 2024-04-12T05:56:10.575 INFO:tasks.workunit.client.1.smithi194.stdout:0/475: dread d0/da/d1e/d27/f3f [0,4194304] 0 2024-04-12T05:56:10.580 INFO:tasks.workunit.client.1.smithi194.stdout:4/530: symlink d2/d3/d1b/d32/db3/db4/lb6 0 2024-04-12T05:56:10.589 INFO:tasks.workunit.client.1.smithi194.stdout:4/531: getdents d2/d11/d1e 0 2024-04-12T05:56:10.590 INFO:tasks.workunit.client.1.smithi194.stdout:0/476: dwrite d0/da/d1e/d27/f3f [0,4194304] 0 2024-04-12T05:56:10.591 INFO:tasks.workunit.client.1.smithi194.stdout:0/477: read d0/da/d1e/f15 [1082229,16813] 0 2024-04-12T05:56:10.595 INFO:tasks.workunit.client.1.smithi194.stdout:0/478: unlink d0/da/d1e/fe 0 2024-04-12T05:56:10.605 INFO:tasks.workunit.client.1.smithi194.stdout:4/532: dread d2/d3/f13 [4194304,4194304] 0 2024-04-12T05:56:10.614 INFO:tasks.workunit.client.1.smithi194.stdout:0/479: dwrite d0/da/d1e/d1a/f31 [0,4194304] 0 2024-04-12T05:56:10.615 INFO:tasks.workunit.client.1.smithi194.stdout:4/533: dwrite d2/d3/d1b/f5e [0,4194304] 0 2024-04-12T05:56:10.619 INFO:tasks.workunit.client.1.smithi194.stdout:0/480: symlink d0/da/l9a 0 2024-04-12T05:56:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:10 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:10 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:10 smithi008 ceph-mon[39722]: pgmap v31: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 40 MiB/s rd, 158 MiB/s wr, 454 op/s 2024-04-12T05:56:10.721 INFO:tasks.workunit.client.1.smithi194.stdout:0/481: dwrite d0/da/d1e/f41 [4194304,4194304] 0 2024-04-12T05:56:10.726 INFO:tasks.workunit.client.1.smithi194.stdout:0/482: creat d0/da/d1e/d1a/d24/d56/f9b x:0 0 0 2024-04-12T05:56:10.727 INFO:tasks.workunit.client.1.smithi194.stdout:4/534: dread d2/d3/f18 [4194304,4194304] 0 2024-04-12T05:56:10.728 INFO:tasks.workunit.client.1.smithi194.stdout:0/483: mkdir d0/d3b/d9c 0 2024-04-12T05:56:10.730 INFO:tasks.workunit.client.1.smithi194.stdout:4/535: mknod d2/d3/d1b/d32/db3/db4/cb7 0 2024-04-12T05:56:10.730 INFO:tasks.workunit.client.1.smithi194.stdout:0/484: mkdir d0/da/d9d 0 2024-04-12T05:56:10.731 INFO:tasks.workunit.client.1.smithi194.stdout:0/485: write d0/f37 [784790,81984] 0 2024-04-12T05:56:10.735 INFO:tasks.workunit.client.1.smithi194.stdout:0/486: chown d0/da/d1e/d27/d54/f71 1 1 2024-04-12T05:56:10.737 INFO:tasks.workunit.client.1.smithi194.stdout:4/536: dwrite d2/d3/f74 [0,4194304] 0 2024-04-12T05:56:10.739 INFO:tasks.workunit.client.1.smithi194.stdout:4/537: fdatasync d2/d11/d46/d61/d63/da2/fa5 0 2024-04-12T05:56:10.739 INFO:tasks.workunit.client.1.smithi194.stdout:0/487: mknod d0/d3b/d52/d7c/c9e 0 2024-04-12T05:56:10.742 INFO:tasks.workunit.client.1.smithi194.stdout:0/488: creat d0/d3b/d52/d7c/f9f x:0 0 0 2024-04-12T05:56:10.743 INFO:tasks.workunit.client.1.smithi194.stdout:0/489: unlink d0/da/d1e/d1a/l33 0 2024-04-12T05:56:10.746 INFO:tasks.workunit.client.1.smithi194.stdout:0/490: creat d0/da/d1e/d27/fa0 x:0 0 0 2024-04-12T05:56:10.746 INFO:tasks.workunit.client.1.smithi194.stdout:4/538: truncate d2/d11/d93/da6/dab/d5d/dae/db2/d57/fa4 734144 0 2024-04-12T05:56:10.747 INFO:tasks.workunit.client.1.smithi194.stdout:0/491: readlink d0/da/l5d 0 2024-04-12T05:56:10.747 INFO:tasks.workunit.client.1.smithi194.stdout:0/492: fdatasync d0/da/d1e/f17 0 2024-04-12T05:56:10.749 INFO:tasks.workunit.client.1.smithi194.stdout:0/493: mknod d0/d5f/d73/ca1 0 2024-04-12T05:56:10.750 INFO:tasks.workunit.client.1.smithi194.stdout:0/494: mkdir d0/da/d4a/d8e/da2 0 2024-04-12T05:56:10.752 INFO:tasks.workunit.client.1.smithi194.stdout:0/495: mknod d0/da/d1e/d27/d87/ca3 0 2024-04-12T05:56:10.753 INFO:tasks.workunit.client.1.smithi194.stdout:0/496: dread - d0/da/d1e/d27/fa0 zero size 2024-04-12T05:56:10.755 INFO:tasks.workunit.client.1.smithi194.stdout:0/497: readlink d0/l86 0 2024-04-12T05:56:10.756 INFO:tasks.workunit.client.1.smithi194.stdout:4/539: dwrite d2/d3/f1a [4194304,4194304] 0 2024-04-12T05:56:10.757 INFO:tasks.workunit.client.1.smithi194.stdout:4/540: chown d2/f8f 8039419 1 2024-04-12T05:56:10.760 INFO:tasks.workunit.client.1.smithi194.stdout:4/541: write d2/d11/d1e/d2f/f76 [1909294,59880] 0 2024-04-12T05:56:10.766 INFO:tasks.workunit.client.1.smithi194.stdout:4/542: rmdir d2/d3/d51/d7e 0 2024-04-12T05:56:10.773 INFO:tasks.workunit.client.1.smithi194.stdout:4/543: dwrite d2/d11/d93/da6/dab/d5d/f7c [4194304,4194304] 0 2024-04-12T05:56:10.779 INFO:tasks.workunit.client.1.smithi194.stdout:4/544: creat d2/d11/d93/fb8 x:0 0 0 2024-04-12T05:56:10.790 INFO:tasks.workunit.client.1.smithi194.stdout:4/545: link d2/d11/d46/c94 d2/d3/d1b/d32/cb9 0 2024-04-12T05:56:10.794 INFO:tasks.workunit.client.1.smithi194.stdout:4/546: rmdir d2/d11/d1e/d2f 39 2024-04-12T05:56:10.795 INFO:tasks.workunit.client.1.smithi194.stdout:4/547: stat d2/d11/d1f 0 2024-04-12T05:56:10.795 INFO:tasks.workunit.client.1.smithi194.stdout:4/548: stat d2/d11/d1e/l8e 0 2024-04-12T05:56:10.797 INFO:tasks.workunit.client.1.smithi194.stdout:4/549: readlink d2/d11/d1e/d25/l96 0 2024-04-12T05:56:10.803 INFO:tasks.workunit.client.1.smithi194.stdout:4/550: dwrite d2/d11/d1e/d2f/f76 [0,4194304] 0 2024-04-12T05:56:10.806 INFO:tasks.workunit.client.1.smithi194.stdout:4/551: mkdir d2/d11/d1f/d81/dba 0 2024-04-12T05:56:10.808 INFO:tasks.workunit.client.1.smithi194.stdout:4/552: unlink d2/d11/d1e/l8e 0 2024-04-12T05:56:10.809 INFO:tasks.workunit.client.1.smithi194.stdout:4/553: symlink d2/d3/d58/d82/lbb 0 2024-04-12T05:56:10.824 INFO:tasks.workunit.client.1.smithi194.stdout:4/554: dwrite d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:10.825 INFO:tasks.workunit.client.1.smithi194.stdout:4/555: truncate d2/f8f 830601 0 2024-04-12T05:56:10.829 INFO:tasks.workunit.client.1.smithi194.stdout:4/556: mkdir d2/d3/d1b/d32/d3a/dbc 0 2024-04-12T05:56:10.829 INFO:tasks.workunit.client.1.smithi194.stdout:4/557: stat d2/d11/d1f/l36 0 2024-04-12T05:56:10.833 INFO:tasks.workunit.client.1.smithi194.stdout:4/558: rename d2/l2c to d2/d11/d93/da6/lbd 0 2024-04-12T05:56:10.834 INFO:tasks.workunit.client.1.smithi194.stdout:4/559: fdatasync d2/f67 0 2024-04-12T05:56:10.840 INFO:tasks.workunit.client.1.smithi194.stdout:4/560: rename d2/d11/d93/da6/dab/c8c to d2/d3/d50/da0/cbe 0 2024-04-12T05:56:10.843 INFO:tasks.workunit.client.1.smithi194.stdout:4/561: symlink d2/d3/d1b/lbf 0 2024-04-12T05:56:10.844 INFO:tasks.workunit.client.1.smithi194.stdout:4/562: write d2/d3/d50/f95 [2119368,110960] 0 2024-04-12T05:56:10.845 INFO:tasks.workunit.client.1.smithi194.stdout:4/563: write d2/f8f [550544,20637] 0 2024-04-12T05:56:10.845 INFO:tasks.workunit.client.1.smithi194.stdout:4/564: stat d2/c9a 0 2024-04-12T05:56:10.846 INFO:tasks.workunit.client.1.smithi194.stdout:4/565: stat d2/ce 0 2024-04-12T05:56:10.850 INFO:tasks.workunit.client.1.smithi194.stdout:4/566: rename d2/d11/d1e/d25/lb5 to d2/d3/d1b/d32/lc0 0 2024-04-12T05:56:10.854 INFO:tasks.workunit.client.1.smithi194.stdout:4/567: mknod d2/d11/d46/d61/d63/da2/cc1 0 2024-04-12T05:56:10.867 INFO:tasks.workunit.client.1.smithi194.stdout:4/568: dread d2/d3/d1b/d32/d3a/d3b/f86 [0,4194304] 0 2024-04-12T05:56:10.869 INFO:tasks.workunit.client.1.smithi194.stdout:4/569: mkdir d2/d85/dc2 0 2024-04-12T05:56:10.871 INFO:tasks.workunit.client.1.smithi194.stdout:4/570: symlink d2/d11/d1e/lc3 0 2024-04-12T05:56:10.873 INFO:tasks.workunit.client.1.smithi194.stdout:4/571: symlink d2/d11/d93/da6/dab/lc4 0 2024-04-12T05:56:10.874 INFO:tasks.workunit.client.1.smithi194.stdout:4/572: fdatasync d2/f4c 0 2024-04-12T05:56:10.875 INFO:tasks.workunit.client.1.smithi194.stdout:4/573: write d2/d11/d1e/d25/f7a [673153,30894] 0 2024-04-12T05:56:10.875 INFO:tasks.workunit.client.1.smithi194.stdout:4/574: chown d2/d3/d51/c8a 40325 1 2024-04-12T05:56:10.878 INFO:tasks.workunit.client.1.smithi194.stdout:4/575: read - d2/d11/d1e/d2f/f71 zero size 2024-04-12T05:56:10.881 INFO:tasks.workunit.client.1.smithi194.stdout:4/576: dread d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:10.885 INFO:tasks.workunit.client.1.smithi194.stdout:4/577: unlink d2/d11/d1e/l6d 0 2024-04-12T05:56:10.888 INFO:tasks.workunit.client.1.smithi194.stdout:4/578: mkdir d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5 0 2024-04-12T05:56:10.894 INFO:tasks.workunit.client.1.smithi194.stdout:4/579: symlink d2/d11/d1e/lc6 0 2024-04-12T05:56:10.900 INFO:tasks.workunit.client.1.smithi194.stdout:4/580: mkdir d2/d3/dc7 0 2024-04-12T05:56:10.905 INFO:tasks.workunit.client.1.smithi194.stdout:4/581: rename d2/d11/d46/d61/f78 to d2/d11/d46/d61/d63/fc8 0 2024-04-12T05:56:10.906 INFO:tasks.workunit.client.1.smithi194.stdout:4/582: read d2/faf [2153970,81187] 0 2024-04-12T05:56:10.911 INFO:tasks.workunit.client.1.smithi194.stdout:4/583: creat d2/d3/d1b/d32/d3a/d3b/fc9 x:0 0 0 2024-04-12T05:56:10.916 INFO:tasks.workunit.client.1.smithi194.stdout:4/584: creat d2/d85/dc2/fca x:0 0 0 2024-04-12T05:56:10.917 INFO:tasks.workunit.client.1.smithi194.stdout:4/585: write d2/d3/d1b/d32/d3a/d3b/f90 [167934,80862] 0 2024-04-12T05:56:10.921 INFO:tasks.workunit.client.1.smithi194.stdout:4/586: rmdir d2/d3/d51 39 2024-04-12T05:56:10.923 INFO:tasks.workunit.client.1.smithi194.stdout:4/587: fsync d2/d3/ff 0 2024-04-12T05:56:10.926 INFO:tasks.workunit.client.1.smithi194.stdout:4/588: symlink d2/d11/d93/da6/dab/d5d/dae/lcb 0 2024-04-12T05:56:10.926 INFO:tasks.workunit.client.1.smithi194.stdout:4/589: chown d2/d3/f18 505 1 2024-04-12T05:56:10.929 INFO:tasks.workunit.client.1.smithi194.stdout:4/590: creat d2/d11/d93/da6/fcc x:0 0 0 2024-04-12T05:56:10.929 INFO:tasks.workunit.client.1.smithi194.stdout:4/591: rename d2/d3 to d2/d3/d1b/d32/db3/dcd 22 2024-04-12T05:56:10.934 INFO:tasks.workunit.client.1.smithi194.stdout:4/592: creat d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/fce x:0 0 0 2024-04-12T05:56:10.936 INFO:tasks.workunit.client.1.smithi194.stdout:4/593: mkdir d2/dcf 0 2024-04-12T05:56:10.939 INFO:tasks.workunit.client.1.smithi194.stdout:4/594: rename d2/d11/d1f/d81 to d2/d3/d58/d82/dd0 0 2024-04-12T05:56:10.939 INFO:tasks.workunit.client.1.smithi194.stdout:4/595: fsync d2/d11/d1e/f5a 0 2024-04-12T05:56:10.940 INFO:tasks.workunit.client.1.smithi194.stdout:4/596: truncate d2/d11/d93/da6/fcc 941772 0 2024-04-12T05:56:10.959 INFO:tasks.workunit.client.1.smithi194.stdout:4/597: dwrite d2/d3/d51/f55 [0,4194304] 0 2024-04-12T05:56:10.960 INFO:tasks.workunit.client.1.smithi194.stdout:4/598: readlink d2/l5 0 2024-04-12T05:56:10.961 INFO:tasks.workunit.client.1.smithi194.stdout:4/599: write d2/d11/d93/da6/dab/d5d/dae/db2/f2a [746878,32452] 0 2024-04-12T05:56:11.038 INFO:tasks.workunit.client.1.smithi194.stdout:5/549: sync 2024-04-12T05:56:11.039 INFO:tasks.workunit.client.1.smithi194.stdout:5/550: stat d1/c80 0 2024-04-12T05:56:11.045 INFO:tasks.workunit.client.1.smithi194.stdout:5/551: dwrite d1/d4/f1c [0,4194304] 0 2024-04-12T05:56:11.046 INFO:tasks.workunit.client.1.smithi194.stdout:5/552: rmdir d1/d4/d6/db 39 2024-04-12T05:56:11.051 INFO:tasks.workunit.client.1.smithi194.stdout:5/553: getdents d1/d4/d6 0 2024-04-12T05:56:11.055 INFO:tasks.workunit.client.1.smithi194.stdout:5/554: link d1/d4/d6/db/d38/f5a d1/d4/d4b/fba 0 2024-04-12T05:56:11.056 INFO:tasks.workunit.client.1.smithi194.stdout:5/555: rmdir d1/d4/d6 39 2024-04-12T05:56:11.059 INFO:tasks.workunit.client.1.smithi194.stdout:5/556: mkdir d1/d4/d6/db/d15/d26/dbb 0 2024-04-12T05:56:11.071 INFO:tasks.workunit.client.1.smithi194.stdout:5/557: write d1/d5c/d95/f2e [1347242,80282] 0 2024-04-12T05:56:11.072 INFO:tasks.workunit.client.1.smithi194.stdout:5/558: truncate d1/d4/d6/f35 2345173 0 2024-04-12T05:56:11.074 INFO:tasks.workunit.client.1.smithi194.stdout:5/559: mknod d1/d4/d6/d88/cbc 0 2024-04-12T05:56:11.074 INFO:tasks.workunit.client.1.smithi194.stdout:5/560: chown d1/d4/d75/f85 107103 1 2024-04-12T05:56:11.079 INFO:tasks.workunit.client.1.smithi194.stdout:5/561: write d1/d4/d6/db/d15/d8c/f8d [843042,114201] 0 2024-04-12T05:56:11.083 INFO:tasks.workunit.client.1.smithi194.stdout:5/562: write d1/d4/d6/db/f53 [779557,9594] 0 2024-04-12T05:56:11.085 INFO:tasks.workunit.client.1.smithi194.stdout:5/563: symlink d1/d4/d6/d8e/lbd 0 2024-04-12T05:56:11.086 INFO:tasks.workunit.client.1.smithi194.stdout:5/564: write d1/d4/d6/db/d3b/fa6 [854088,42643] 0 2024-04-12T05:56:11.095 INFO:tasks.workunit.client.1.smithi194.stdout:5/565: dread d1/d4/f25 [0,4194304] 0 2024-04-12T05:56:11.109 INFO:tasks.workunit.client.1.smithi194.stdout:5/566: dread d1/d4/d6/db/d2b/f66 [0,4194304] 0 2024-04-12T05:56:11.112 INFO:tasks.workunit.client.1.smithi194.stdout:5/567: symlink d1/d4/d6/db/d3b/lbe 0 2024-04-12T05:56:11.115 INFO:tasks.workunit.client.1.smithi194.stdout:5/568: link d1/d4/d6/d8e/fa8 d1/d4/d75/fbf 0 2024-04-12T05:56:11.121 INFO:tasks.workunit.client.1.smithi194.stdout:5/569: rmdir d1/d4/d75 39 2024-04-12T05:56:11.121 INFO:tasks.workunit.client.1.smithi194.stdout:5/570: chown d1/d5c/d95/f9f 261888 1 2024-04-12T05:56:11.123 INFO:tasks.workunit.client.1.smithi194.stdout:5/571: creat d1/d4/d6/db/d38/fc0 x:0 0 0 2024-04-12T05:56:11.124 INFO:tasks.workunit.client.1.smithi194.stdout:5/572: mkdir d1/d4/d6/d88/dc1 0 2024-04-12T05:56:11.125 INFO:tasks.workunit.client.1.smithi194.stdout:5/573: dread - d1/d5c/da7/fb1 zero size 2024-04-12T05:56:11.126 INFO:tasks.workunit.client.1.smithi194.stdout:5/574: dread - d1/d4/d6/d88/fae zero size 2024-04-12T05:56:11.129 INFO:tasks.workunit.client.1.smithi194.stdout:5/575: write d1/d4/d6/db/f19 [865617,70393] 0 2024-04-12T05:56:11.131 INFO:tasks.workunit.client.1.smithi194.stdout:5/576: symlink d1/d4/d9e/lc2 0 2024-04-12T05:56:11.134 INFO:tasks.workunit.client.1.smithi194.stdout:5/577: write d1/d4/d6/f3d [4447393,13981] 0 2024-04-12T05:56:11.136 INFO:tasks.workunit.client.1.smithi194.stdout:5/578: creat d1/d5c/d95/fc3 x:0 0 0 2024-04-12T05:56:11.139 INFO:tasks.workunit.client.1.smithi194.stdout:5/579: mknod d1/d4/d75/cc4 0 2024-04-12T05:56:11.148 INFO:tasks.workunit.client.1.smithi194.stdout:5/580: dwrite d1/d4/d6/d34/fa0 [0,4194304] 0 2024-04-12T05:56:11.154 INFO:tasks.workunit.client.1.smithi194.stdout:5/581: truncate d1/d4/f42 24584 0 2024-04-12T05:56:11.155 INFO:tasks.workunit.client.1.smithi194.stdout:5/582: write d1/d4/d6/f76 [2182141,20465] 0 2024-04-12T05:56:11.183 INFO:tasks.workunit.client.1.smithi194.stdout:4/600: sync 2024-04-12T05:56:11.183 INFO:tasks.workunit.client.1.smithi194.stdout:9/544: sync 2024-04-12T05:56:11.183 INFO:tasks.workunit.client.1.smithi194.stdout:2/450: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:6/525: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:0/498: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:7/356: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:3/585: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:1/569: sync 2024-04-12T05:56:11.184 INFO:tasks.workunit.client.1.smithi194.stdout:8/339: sync 2024-04-12T05:56:11.187 INFO:tasks.workunit.client.1.smithi194.stdout:1/570: chown d0/d11/d18/d19/d1c/d67 6944 1 2024-04-12T05:56:11.191 INFO:tasks.workunit.client.1.smithi194.stdout:4/601: creat d2/d85/fd1 x:0 0 0 2024-04-12T05:56:11.193 INFO:tasks.workunit.client.1.smithi194.stdout:7/357: creat d3/d27/d22/f77 x:0 0 0 2024-04-12T05:56:11.198 INFO:tasks.workunit.client.1.smithi194.stdout:3/586: symlink d0/dc/d1e/d81/d5b/d64/d79/d87/lbf 0 2024-04-12T05:56:11.198 INFO:tasks.workunit.client.1.smithi194.stdout:8/340: write d2/d8/d9/de/d10/f41 [738463,1144] 0 2024-04-12T05:56:11.204 INFO:tasks.workunit.client.1.smithi194.stdout:7/358: write d3/d27/f39 [4647840,13409] 0 2024-04-12T05:56:11.205 INFO:tasks.workunit.client.1.smithi194.stdout:2/451: truncate d7/d12/d25/d5d/d47/f7f 2119350 0 2024-04-12T05:56:11.211 INFO:tasks.workunit.client.1.smithi194.stdout:4/602: unlink d2/d3/d50/f95 0 2024-04-12T05:56:11.211 INFO:tasks.workunit.client.1.smithi194.stdout:9/545: dwrite d1/d19/d1f/f73 [0,4194304] 0 2024-04-12T05:56:11.215 INFO:tasks.workunit.client.1.smithi194.stdout:6/526: truncate d3/d2f/d37/f4a 5467967 0 2024-04-12T05:56:11.219 INFO:tasks.workunit.client.1.smithi194.stdout:1/571: dwrite d0/d11/f83 [0,4194304] 0 2024-04-12T05:56:11.219 INFO:tasks.workunit.client.1.smithi194.stdout:8/341: chown d2/d8/d9/de/f2e 3121 1 2024-04-12T05:56:11.220 INFO:tasks.workunit.client.1.smithi194.stdout:7/359: symlink d3/d16/d21/l78 0 2024-04-12T05:56:11.223 INFO:tasks.workunit.client.1.smithi194.stdout:9/546: creat d1/d21/d39/d72/fb5 x:0 0 0 2024-04-12T05:56:11.226 INFO:tasks.workunit.client.1.smithi194.stdout:0/499: dwrite d0/da/f89 [0,4194304] 0 2024-04-12T05:56:11.229 INFO:tasks.workunit.client.1.smithi194.stdout:6/527: write d3/d2f/d4e/d68/fa2 [2661553,101901] 0 2024-04-12T05:56:11.229 INFO:tasks.workunit.client.1.smithi194.stdout:4/603: mknod d2/d11/d1e/cd2 0 2024-04-12T05:56:11.230 INFO:tasks.workunit.client.1.smithi194.stdout:2/452: getdents d7/d12/d25/d70 0 2024-04-12T05:56:11.232 INFO:tasks.workunit.client.1.smithi194.stdout:5/583: dread d1/d4/d6/db/d3b/f3e [0,4194304] 0 2024-04-12T05:56:11.234 INFO:tasks.workunit.client.1.smithi194.stdout:7/360: creat d3/d16/d21/d3a/d43/f79 x:0 0 0 2024-04-12T05:56:11.235 INFO:tasks.workunit.client.1.smithi194.stdout:8/342: truncate d2/d8/d9/de/d38/f4a 250440 0 2024-04-12T05:56:11.235 INFO:tasks.workunit.client.1.smithi194.stdout:4/604: stat d2/d11/d1e/d2f 0 2024-04-12T05:56:11.235 INFO:tasks.workunit.client.1.smithi194.stdout:9/547: unlink d1/d19/c7a 0 2024-04-12T05:56:11.237 INFO:tasks.workunit.client.1.smithi194.stdout:0/500: unlink d0/f37 0 2024-04-12T05:56:11.237 INFO:tasks.workunit.client.1.smithi194.stdout:9/548: readlink d1/l12 0 2024-04-12T05:56:11.241 INFO:tasks.workunit.client.1.smithi194.stdout:0/501: write d0/da/f89 [4775085,18159] 0 2024-04-12T05:56:11.241 INFO:tasks.workunit.client.1.smithi194.stdout:6/528: creat d3/d2f/d4e/d53/fa4 x:0 0 0 2024-04-12T05:56:11.244 INFO:tasks.workunit.client.1.smithi194.stdout:2/453: unlink d7/d22/l71 0 2024-04-12T05:56:11.245 INFO:tasks.workunit.client.1.smithi194.stdout:3/587: dwrite d0/dc/d1e/d33/f68 [0,4194304] 0 2024-04-12T05:56:11.245 INFO:tasks.workunit.client.1.smithi194.stdout:1/572: read d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/f64 [1469771,74935] 0 2024-04-12T05:56:11.245 INFO:tasks.workunit.client.1.smithi194.stdout:7/361: truncate d3/d16/d21/d3a/d43/d5c/f76 431261 0 2024-04-12T05:56:11.249 INFO:tasks.workunit.client.1.smithi194.stdout:6/529: write d3/f3b [4072541,94894] 0 2024-04-12T05:56:11.250 INFO:tasks.workunit.client.1.smithi194.stdout:9/549: fsync d1/d41/d97/d9d/fa1 0 2024-04-12T05:56:11.258 INFO:tasks.workunit.client.1.smithi194.stdout:4/605: rename d2/f4e to d2/d3/fd3 0 2024-04-12T05:56:11.258 INFO:tasks.workunit.client.1.smithi194.stdout:0/502: rmdir d0/da/d1e/d27/d87 39 2024-04-12T05:56:11.258 INFO:tasks.workunit.client.1.smithi194.stdout:9/550: write d1/d21/d39/faf [693294,102382] 0 2024-04-12T05:56:11.258 INFO:tasks.workunit.client.1.smithi194.stdout:2/454: mkdir d7/d22/d93/d9f 0 2024-04-12T05:56:11.259 INFO:tasks.workunit.client.1.smithi194.stdout:7/362: write d3/d27/d22/f77 [683499,127737] 0 2024-04-12T05:56:11.259 INFO:tasks.workunit.client.1.smithi194.stdout:8/343: link d2/l2c d2/d8/d15/d33/d45/d52/l68 0 2024-04-12T05:56:11.262 INFO:tasks.workunit.client.1.smithi194.stdout:9/551: read d1/d4e/f55 [157757,94869] 0 2024-04-12T05:56:11.265 INFO:tasks.workunit.client.1.smithi194.stdout:0/503: write d0/da/d1e/d27/d54/f71 [4413446,72627] 0 2024-04-12T05:56:11.265 INFO:tasks.workunit.client.1.smithi194.stdout:0/504: stat d0/d5f/d73 0 2024-04-12T05:56:11.266 INFO:tasks.workunit.client.1.smithi194.stdout:9/552: write d1/d19/d1f/d20/f8b [3006,68094] 0 2024-04-12T05:56:11.268 INFO:tasks.workunit.client.1.smithi194.stdout:7/363: write d3/d27/f1d [3918948,23735] 0 2024-04-12T05:56:11.271 INFO:tasks.workunit.client.1.smithi194.stdout:7/364: readlink d3/d27/l1e 0 2024-04-12T05:56:11.271 INFO:tasks.workunit.client.1.smithi194.stdout:9/553: read - d1/d19/d1f/d5f/f9e zero size 2024-04-12T05:56:11.272 INFO:tasks.workunit.client.1.smithi194.stdout:3/588: creat d0/dc/d3d/fc0 x:0 0 0 2024-04-12T05:56:11.273 INFO:tasks.workunit.client.1.smithi194.stdout:7/365: chown d3/d27/d6a/c70 28 1 2024-04-12T05:56:11.279 INFO:tasks.workunit.client.1.smithi194.stdout:2/455: fsync d7/d12/f3b 0 2024-04-12T05:56:11.280 INFO:tasks.workunit.client.1.smithi194.stdout:4/606: truncate d2/d11/d1e/f5a 149477 0 2024-04-12T05:56:11.280 INFO:tasks.workunit.client.1.smithi194.stdout:9/554: mkdir d1/d41/d97/db6 0 2024-04-12T05:56:11.280 INFO:tasks.workunit.client.1.smithi194.stdout:3/589: rmdir d0/dc/d1e 39 2024-04-12T05:56:11.280 INFO:tasks.workunit.client.1.smithi194.stdout:6/530: rename d3/d2d/f32 to d3/d2d/d33/d52/d5c/fa5 0 2024-04-12T05:56:11.290 INFO:tasks.workunit.client.1.smithi194.stdout:2/456: read d7/d12/f38 [482785,85727] 0 2024-04-12T05:56:11.290 INFO:tasks.workunit.client.1.smithi194.stdout:9/555: truncate d1/d21/d39/d72/fb5 451667 0 2024-04-12T05:56:11.290 INFO:tasks.workunit.client.1.smithi194.stdout:4/607: creat d2/d11/d1e/d25/fd4 x:0 0 0 2024-04-12T05:56:11.297 INFO:tasks.workunit.client.1.smithi194.stdout:8/344: rename d2/c4 to d2/d8/d15/d33/c69 0 2024-04-12T05:56:11.297 INFO:tasks.workunit.client.1.smithi194.stdout:7/366: truncate d3/d27/f39 2538613 0 2024-04-12T05:56:11.297 INFO:tasks.workunit.client.1.smithi194.stdout:9/556: mknod d1/d21/d77/d98/cb7 0 2024-04-12T05:56:11.298 INFO:tasks.workunit.client.1.smithi194.stdout:3/590: creat d0/dc/d1e/d33/d76/d9a/fc1 x:0 0 0 2024-04-12T05:56:11.298 INFO:tasks.workunit.client.1.smithi194.stdout:0/505: dread d0/da/f2d [0,4194304] 0 2024-04-12T05:56:11.300 INFO:tasks.workunit.client.1.smithi194.stdout:7/367: stat d3/d16/d23/c32 0 2024-04-12T05:56:11.301 INFO:tasks.workunit.client.1.smithi194.stdout:2/457: creat d7/d12/d25/d5d/d47/d52/fa0 x:0 0 0 2024-04-12T05:56:11.301 INFO:tasks.workunit.client.1.smithi194.stdout:1/573: dread d0/f32 [4194304,4194304] 0 2024-04-12T05:56:11.303 INFO:tasks.workunit.client.1.smithi194.stdout:4/608: write d2/d3/d58/f69 [3539936,16592] 0 2024-04-12T05:56:11.305 INFO:tasks.workunit.client.1.smithi194.stdout:0/506: mknod d0/da/d1e/d27/d54/ca4 0 2024-04-12T05:56:11.310 INFO:tasks.workunit.client.1.smithi194.stdout:2/458: write d7/d12/d1a/d68/f95 [115285,28994] 0 2024-04-12T05:56:11.311 INFO:tasks.workunit.client.1.smithi194.stdout:4/609: write d2/d11/d93/fb8 [540323,120455] 0 2024-04-12T05:56:11.313 INFO:tasks.workunit.client.1.smithi194.stdout:7/368: read d3/d27/f7 [1346930,59870] 0 2024-04-12T05:56:11.316 INFO:tasks.workunit.client.1.smithi194.stdout:8/345: link d2/fb d2/d8/f6a 0 2024-04-12T05:56:11.316 INFO:tasks.workunit.client.1.smithi194.stdout:2/459: chown d7/d12/d25 44215 1 2024-04-12T05:56:11.321 INFO:tasks.workunit.client.1.smithi194.stdout:7/369: write d3/d27/d22/f77 [19007,12704] 0 2024-04-12T05:56:11.321 INFO:tasks.workunit.client.1.smithi194.stdout:4/610: creat d2/d85/fd5 x:0 0 0 2024-04-12T05:56:11.328 INFO:tasks.workunit.client.1.smithi194.stdout:9/557: link d1/d19/d1f/d20/d37/d38/f5c d1/d19/d1f/d20/d37/d84/fb8 0 2024-04-12T05:56:11.329 INFO:tasks.workunit.client.1.smithi194.stdout:2/460: creat d7/d12/d25/d70/fa1 x:0 0 0 2024-04-12T05:56:11.329 INFO:tasks.workunit.client.1.smithi194.stdout:0/507: dwrite d0/da/d1e/f41 [8388608,4194304] 0 2024-04-12T05:56:11.343 INFO:tasks.workunit.client.1.smithi194.stdout:1/574: dwrite d0/d11/d18/d19/d1c/f22 [0,4194304] 0 2024-04-12T05:56:11.343 INFO:tasks.workunit.client.1.smithi194.stdout:3/591: dwrite d0/dc/d3d/f94 [0,4194304] 0 2024-04-12T05:56:11.346 INFO:tasks.workunit.client.1.smithi194.stdout:9/558: chown d1/d19/d1f/d20/d37/d60/c6e 595 1 2024-04-12T05:56:11.349 INFO:tasks.workunit.client.1.smithi194.stdout:0/508: creat d0/da/d1e/d1a/fa5 x:0 0 0 2024-04-12T05:56:11.350 INFO:tasks.workunit.client.1.smithi194.stdout:9/559: dread - d1/d19/d1f/d20/d7d/fad zero size 2024-04-12T05:56:11.352 INFO:tasks.workunit.client.1.smithi194.stdout:2/461: write d7/d12/d25/d70/fa1 [377025,93831] 0 2024-04-12T05:56:11.357 INFO:tasks.workunit.client.1.smithi194.stdout:1/575: rename d0/df to d0/d11/d18/d19/d1a/d6a/dac/db5 0 2024-04-12T05:56:11.357 INFO:tasks.workunit.client.1.smithi194.stdout:3/592: write d0/dc/d3d/f94 [2173031,1038] 0 2024-04-12T05:56:11.364 INFO:tasks.workunit.client.1.smithi194.stdout:4/611: dwrite d2/d11/d93/fb8 [0,4194304] 0 2024-04-12T05:56:11.370 INFO:tasks.workunit.client.1.smithi194.stdout:1/576: dread - d0/d11/d18/d19/d1a/d6a/d27/d9e/fad zero size 2024-04-12T05:56:11.370 INFO:tasks.workunit.client.1.smithi194.stdout:3/593: fsync d0/dc/d1e/d81/d34/f41 0 2024-04-12T05:56:11.370 INFO:tasks.workunit.client.1.smithi194.stdout:9/560: mkdir d1/d41/d97/d9d/db9 0 2024-04-12T05:56:11.374 INFO:tasks.workunit.client.1.smithi194.stdout:0/509: dwrite d0/da/d1e/d1a/d24/f61 [8388608,4194304] 0 2024-04-12T05:56:11.375 INFO:tasks.workunit.client.1.smithi194.stdout:2/462: creat d7/d12/fa2 x:0 0 0 2024-04-12T05:56:11.376 INFO:tasks.workunit.client.1.smithi194.stdout:8/346: dread d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:11.378 INFO:tasks.workunit.client.1.smithi194.stdout:7/370: truncate d3/d27/f2c 2107386 0 2024-04-12T05:56:11.382 INFO:tasks.workunit.client.1.smithi194.stdout:4/612: mknod d2/d11/d46/d61/d63/da2/cd6 0 2024-04-12T05:56:11.384 INFO:tasks.workunit.client.1.smithi194.stdout:8/347: stat d2/d8/d9/de/d38 0 2024-04-12T05:56:11.384 INFO:tasks.workunit.client.1.smithi194.stdout:9/561: rmdir d1/d19/d1f/d20/d37/d84 39 2024-04-12T05:56:11.386 INFO:tasks.workunit.client.1.smithi194.stdout:2/463: read - d7/d12/d1a/d68/f8c zero size 2024-04-12T05:56:11.388 INFO:tasks.workunit.client.1.smithi194.stdout:1/577: creat d0/d11/d18/d19/d1a/d6a/d27/d9e/fb6 x:0 0 0 2024-04-12T05:56:11.388 INFO:tasks.workunit.client.1.smithi194.stdout:4/613: chown d2/d11/d93/da6/dab/d5d/dae/db2/d57 6749714 1 2024-04-12T05:56:11.391 INFO:tasks.workunit.client.1.smithi194.stdout:0/510: write d0/d3b/f74 [256727,35822] 0 2024-04-12T05:56:11.391 INFO:tasks.workunit.client.1.smithi194.stdout:2/464: creat d7/fa3 x:0 0 0 2024-04-12T05:56:11.396 INFO:tasks.workunit.client.1.smithi194.stdout:4/614: creat d2/d11/d1e/d25/fd7 x:0 0 0 2024-04-12T05:56:11.399 INFO:tasks.workunit.client.1.smithi194.stdout:8/348: mknod d2/d8/d58/c6b 0 2024-04-12T05:56:11.404 INFO:tasks.workunit.client.1.smithi194.stdout:7/371: mknod d3/d16/d23/d37/c7a 0 2024-04-12T05:56:11.407 INFO:tasks.workunit.client.1.smithi194.stdout:3/594: truncate d0/dc/d1e/d81/d34/f41 3671553 0 2024-04-12T05:56:11.407 INFO:tasks.workunit.client.1.smithi194.stdout:4/615: chown d2/d3/d50/da0/cbe 2 1 2024-04-12T05:56:11.411 INFO:tasks.workunit.client.1.smithi194.stdout:0/511: dwrite d0/da/d1e/d27/d54/f71 [0,4194304] 0 2024-04-12T05:56:11.413 INFO:tasks.workunit.client.1.smithi194.stdout:8/349: unlink d2/d8/d9/de/d10/f3c 0 2024-04-12T05:56:11.414 INFO:tasks.workunit.client.1.smithi194.stdout:2/465: dwrite d7/d12/d1a/d68/f95 [0,4194304] 0 2024-04-12T05:56:11.414 INFO:tasks.workunit.client.1.smithi194.stdout:7/372: write d3/d16/d21/d3a/d43/d5c/f76 [1140176,10953] 0 2024-04-12T05:56:11.418 INFO:tasks.workunit.client.1.smithi194.stdout:1/578: dwrite d0/d11/d18/f76 [0,4194304] 0 2024-04-12T05:56:11.420 INFO:tasks.workunit.client.1.smithi194.stdout:8/350: write d2/d8/d15/f62 [874214,95882] 0 2024-04-12T05:56:11.421 INFO:tasks.workunit.client.1.smithi194.stdout:3/595: creat d0/dc/d1e/d81/d5b/fc2 x:0 0 0 2024-04-12T05:56:11.424 INFO:tasks.workunit.client.1.smithi194.stdout:0/512: mknod d0/d5f/d73/ca6 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:1/579: creat d0/d11/d18/d19/d1c/d67/fb7 x:0 0 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:3/596: creat d0/dc/d1e/d81/d5b/d64/fc3 x:0 0 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:0/513: mkdir d0/da/da7 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:8/351: truncate d2/d8/f36 449311 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:1/580: unlink d0/d11/d18/d19/d1a/l47 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:3/597: mkdir d0/dc/d1e/dc4 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:0/514: mkdir d0/da/d1e/d1a/d24/da8 0 2024-04-12T05:56:11.480 INFO:tasks.workunit.client.1.smithi194.stdout:8/352: mkdir d2/d8/d15/d35/d6c 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:8/353: rmdir d2 39 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:2/466: dwrite d7/d12/d1a/f44 [0,4194304] 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:0/515: creat d0/da/d1e/d1a/d58/d5b/fa9 x:0 0 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:3/598: mkdir d0/dc/d1e/d81/d5b/d64/dc5 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:7/373: dwrite d3/d27/f1b [4194304,4194304] 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:5/584: sync 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:1/581: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b [4194304,4194304] 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:2/467: rename d7/d12/d34/c56 to d7/d22/d93/d9f/ca4 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:2/468: chown d7/d12/d1a/f24 353 1 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:8/354: dread d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:7/374: mkdir d3/d16/d7b 0 2024-04-12T05:56:11.481 INFO:tasks.workunit.client.1.smithi194.stdout:0/516: creat d0/da/d1e/d27/d87/d92/faa x:0 0 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:5/585: unlink d1/d4/d6/db/d38/f9c 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/582: rename d0/d11/d18/d19/l7e to d0/d11/d18/d19/d1a/d6a/dac/lb8 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:5/586: dread d1/d4/d6/db/d3b/f3e [0,4194304] 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/583: dread - d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/d7b/f86 zero size 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:7/375: creat d3/d16/d23/d37/f7c x:0 0 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:0/517: creat d0/d3b/d52/d7c/fab x:0 0 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/584: stat d0/d11/d18/d19/d1c/d67/l99 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:2/469: rename d7/d22/f63 to d7/d12/d6d/fa5 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:8/355: symlink d2/d8/d15/d33/d49/l6d 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:7/376: fsync d3/d16/d23/d37/d3d/d56/f51 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/585: stat d0/d11/d18/d19/c2c 0 2024-04-12T05:56:11.482 INFO:tasks.workunit.client.1.smithi194.stdout:5/587: dwrite d1/d4/f1c [0,4194304] 0 2024-04-12T05:56:11.483 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:11 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:11.483 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:11 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:11.483 INFO:tasks.workunit.client.1.smithi194.stdout:7/377: symlink d3/d16/d7b/l7d 0 2024-04-12T05:56:11.490 INFO:tasks.workunit.client.1.smithi194.stdout:8/356: creat d2/d8/d15/d35/d6c/f6e x:0 0 0 2024-04-12T05:56:11.491 INFO:tasks.workunit.client.1.smithi194.stdout:7/378: readlink d3/d16/d23/d37/l3f 0 2024-04-12T05:56:11.492 INFO:tasks.workunit.client.1.smithi194.stdout:2/470: dwrite d7/d22/f91 [0,4194304] 0 2024-04-12T05:56:11.497 INFO:tasks.workunit.client.1.smithi194.stdout:0/518: getdents d0/da/d1e/d1a/d3d 0 2024-04-12T05:56:11.498 INFO:tasks.workunit.client.1.smithi194.stdout:5/588: mknod d1/d4/d6/d88/dc1/cc5 0 2024-04-12T05:56:11.498 INFO:tasks.workunit.client.1.smithi194.stdout:3/599: dread d0/d1/d10/f44 [0,4194304] 0 2024-04-12T05:56:11.498 INFO:tasks.workunit.client.1.smithi194.stdout:0/519: readlink d0/da/d4a/d8e/l85 0 2024-04-12T05:56:11.499 INFO:tasks.workunit.client.1.smithi194.stdout:7/379: stat d3/d16/d21/d3a/d43/c5e 0 2024-04-12T05:56:11.499 INFO:tasks.workunit.client.1.smithi194.stdout:6/531: sync 2024-04-12T05:56:11.501 INFO:tasks.workunit.client.1.smithi194.stdout:8/357: creat d2/d8/d9/d1a/f6f x:0 0 0 2024-04-12T05:56:11.502 INFO:tasks.workunit.client.1.smithi194.stdout:9/562: dread d1/d19/f2e [0,4194304] 0 2024-04-12T05:56:11.505 INFO:tasks.workunit.client.1.smithi194.stdout:7/380: chown d3/d16/d21/d3a/d43 271744 1 2024-04-12T05:56:11.510 INFO:tasks.workunit.client.1.smithi194.stdout:2/471: creat d7/d12/d25/d5d/d47/d52/d7b/fa6 x:0 0 0 2024-04-12T05:56:11.511 INFO:tasks.workunit.client.1.smithi194.stdout:0/520: symlink d0/da/d4a/d8e/lac 0 2024-04-12T05:56:11.512 INFO:tasks.workunit.client.1.smithi194.stdout:0/521: stat d0/f48 0 2024-04-12T05:56:11.512 INFO:tasks.workunit.client.1.smithi194.stdout:2/472: dread - d7/d12/d25/d5d/d47/d52/fa0 zero size 2024-04-12T05:56:11.514 INFO:tasks.workunit.client.1.smithi194.stdout:5/589: fsync d1/d4/d6/db/d15/d26/f50 0 2024-04-12T05:56:11.526 INFO:tasks.workunit.client.1.smithi194.stdout:0/522: creat d0/da/d1e/d27/d54/fad x:0 0 0 2024-04-12T05:56:11.527 INFO:tasks.workunit.client.1.smithi194.stdout:8/358: read d2/d8/d15/d33/f18 [201616,78262] 0 2024-04-12T05:56:11.530 INFO:tasks.workunit.client.1.smithi194.stdout:5/590: dread d1/f3 [0,4194304] 0 2024-04-12T05:56:11.531 INFO:tasks.workunit.client.1.smithi194.stdout:6/532: link d3/d2d/d33/fa1 d3/d2d/d33/fa6 0 2024-04-12T05:56:11.533 INFO:tasks.workunit.client.1.smithi194.stdout:1/586: dwrite d0/d11/d18/d19/f39 [0,4194304] 0 2024-04-12T05:56:11.536 INFO:tasks.workunit.client.1.smithi194.stdout:0/523: unlink d0/da/f2d 0 2024-04-12T05:56:11.536 INFO:tasks.workunit.client.1.smithi194.stdout:3/600: link d0/d1/d74/d39/d4d/d63/c9c d0/dc/d1e/dc4/cc6 0 2024-04-12T05:56:11.536 INFO:tasks.workunit.client.1.smithi194.stdout:5/591: truncate d1/d4/d84/fb5 772283 0 2024-04-12T05:56:11.537 INFO:tasks.workunit.client.1.smithi194.stdout:8/359: rmdir d2/d8 39 2024-04-12T05:56:11.538 INFO:tasks.workunit.client.1.smithi194.stdout:2/473: link d7/d12/c17 d7/ca7 0 2024-04-12T05:56:11.540 INFO:tasks.workunit.client.1.smithi194.stdout:5/592: chown d1/d4/d6/d34/c7d 247415 1 2024-04-12T05:56:11.542 INFO:tasks.workunit.client.1.smithi194.stdout:0/524: rmdir d0/da/d4a 39 2024-04-12T05:56:11.543 INFO:tasks.workunit.client.1.smithi194.stdout:6/533: creat d3/d2f/fa7 x:0 0 0 2024-04-12T05:56:11.545 INFO:tasks.workunit.client.1.smithi194.stdout:5/593: creat d1/d5c/d95/fc6 x:0 0 0 2024-04-12T05:56:11.546 INFO:tasks.workunit.client.1.smithi194.stdout:4/616: dread d2/d3/ff [0,4194304] 0 2024-04-12T05:56:11.546 INFO:tasks.workunit.client.1.smithi194.stdout:7/381: dread d3/d16/d23/f66 [0,4194304] 0 2024-04-12T05:56:11.549 INFO:tasks.workunit.client.1.smithi194.stdout:2/474: truncate d7/d12/d25/d5d/d47/f88 241721 0 2024-04-12T05:56:11.549 INFO:tasks.workunit.client.1.smithi194.stdout:8/360: creat d2/d8/d66/f70 x:0 0 0 2024-04-12T05:56:11.550 INFO:tasks.workunit.client.1.smithi194.stdout:9/563: dread d1/fe [0,4194304] 0 2024-04-12T05:56:11.555 INFO:tasks.workunit.client.1.smithi194.stdout:6/534: unlink d3/d15/d2b/d48/f49 0 2024-04-12T05:56:11.557 INFO:tasks.workunit.client.1.smithi194.stdout:7/382: write d3/d27/fa [1103657,27132] 0 2024-04-12T05:56:11.558 INFO:tasks.workunit.client.1.smithi194.stdout:9/564: write d1/d21/f4d [3101666,122568] 0 2024-04-12T05:56:11.558 INFO:tasks.workunit.client.1.smithi194.stdout:1/587: rename d0/d11/d18/d19/d1a/d6a/d27/da8/d50 to d0/d11/d18/d19/d1a/d6a/dac/db5/d8d/db9 0 2024-04-12T05:56:11.562 INFO:tasks.workunit.client.1.smithi194.stdout:6/535: write d3/d2f/f9e [650333,52858] 0 2024-04-12T05:56:11.564 INFO:tasks.workunit.client.1.smithi194.stdout:3/601: dwrite d0/dc/d3d/f94 [4194304,4194304] 0 2024-04-12T05:56:11.569 INFO:tasks.workunit.client.1.smithi194.stdout:9/565: write d1/d19/d1f/d20/f71 [4626618,123377] 0 2024-04-12T05:56:11.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:11 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:11.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:11 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:11.570 INFO:tasks.workunit.client.1.smithi194.stdout:8/361: read d2/d8/d9/de/d38/f3a [324429,72768] 0 2024-04-12T05:56:11.571 INFO:tasks.workunit.client.1.smithi194.stdout:4/617: rename d2/d11/d46/d61/d63/c72 to d2/d3/d1b/d32/d3a/cd8 0 2024-04-12T05:56:11.572 INFO:tasks.workunit.client.1.smithi194.stdout:5/594: link d1/c80 d1/d4/d6/db/d15/cc7 0 2024-04-12T05:56:11.572 INFO:tasks.workunit.client.1.smithi194.stdout:0/525: dread d0/f48 [0,4194304] 0 2024-04-12T05:56:11.581 INFO:tasks.workunit.client.1.smithi194.stdout:4/618: write d2/d3/d1b/d32/d3a/d3b/f9f [736840,18049] 0 2024-04-12T05:56:11.581 INFO:tasks.workunit.client.1.smithi194.stdout:6/536: creat d3/d2f/d4e/d53/fa8 x:0 0 0 2024-04-12T05:56:11.582 INFO:tasks.workunit.client.1.smithi194.stdout:9/566: mkdir d1/d41/dba 0 2024-04-12T05:56:11.585 INFO:tasks.workunit.client.1.smithi194.stdout:1/588: stat d0/d11/d18/d35/d6f/l8e 0 2024-04-12T05:56:11.590 INFO:tasks.workunit.client.1.smithi194.stdout:4/619: chown d2/d3/d1b/d32/d3a/d3b/f86 21055 1 2024-04-12T05:56:11.590 INFO:tasks.workunit.client.1.smithi194.stdout:3/602: creat d0/dc/d1e/d81/d5b/d64/fc7 x:0 0 0 2024-04-12T05:56:11.591 INFO:tasks.workunit.client.1.smithi194.stdout:8/362: dread d2/d8/d15/f62 [0,4194304] 0 2024-04-12T05:56:11.595 INFO:tasks.workunit.client.1.smithi194.stdout:9/567: mknod d1/d19/d1f/d20/d37/da3/cbb 0 2024-04-12T05:56:11.596 INFO:tasks.workunit.client.1.smithi194.stdout:3/603: dread - d0/d1/d74/fb1 zero size 2024-04-12T05:56:11.596 INFO:tasks.workunit.client.1.smithi194.stdout:4/620: symlink d2/d3/d51/ld9 0 2024-04-12T05:56:11.598 INFO:tasks.workunit.client.1.smithi194.stdout:0/526: rmdir d0/da/d9d 0 2024-04-12T05:56:11.598 INFO:tasks.workunit.client.1.smithi194.stdout:2/475: dwrite d7/d12/d25/d5d/d47/d52/f9b [0,4194304] 0 2024-04-12T05:56:11.599 INFO:tasks.workunit.client.1.smithi194.stdout:1/589: chown d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 12 1 2024-04-12T05:56:11.603 INFO:tasks.workunit.client.1.smithi194.stdout:5/595: dwrite d1/d4/d6/d8e/f81 [0,4194304] 0 2024-04-12T05:56:11.606 INFO:tasks.workunit.client.1.smithi194.stdout:4/621: mknod d2/d11/d93/cda 0 2024-04-12T05:56:11.608 INFO:tasks.workunit.client.1.smithi194.stdout:2/476: write d7/d22/d42/f1e [3349784,110194] 0 2024-04-12T05:56:11.611 INFO:tasks.workunit.client.1.smithi194.stdout:5/596: truncate d1/fa1 453555 0 2024-04-12T05:56:11.612 INFO:tasks.workunit.client.1.smithi194.stdout:6/537: dwrite d3/d15/d2b/d48/d6f/d7a/d8f/fa3 [0,4194304] 0 2024-04-12T05:56:11.612 INFO:tasks.workunit.client.1.smithi194.stdout:7/383: truncate d3/d27/f17 1152936 0 2024-04-12T05:56:11.613 INFO:tasks.workunit.client.1.smithi194.stdout:3/604: read d0/dc/d1e/d81/f48 [4052537,79614] 0 2024-04-12T05:56:11.619 INFO:tasks.workunit.client.1.smithi194.stdout:9/568: symlink d1/d19/d1f/d20/d37/d84/lbc 0 2024-04-12T05:56:11.620 INFO:tasks.workunit.client.1.smithi194.stdout:6/538: truncate d3/d1a/f23 371244 0 2024-04-12T05:56:11.623 INFO:tasks.workunit.client.1.smithi194.stdout:4/622: write d2/d11/f52 [4689310,19541] 0 2024-04-12T05:56:11.630 INFO:tasks.workunit.client.1.smithi194.stdout:1/590: chown d0/d11/d18/l3f 7531 1 2024-04-12T05:56:11.631 INFO:tasks.workunit.client.1.smithi194.stdout:8/363: dwrite d2/d8/d9/de/f2e [0,4194304] 0 2024-04-12T05:56:11.631 INFO:tasks.workunit.client.1.smithi194.stdout:0/527: dwrite d0/d3b/f74 [0,4194304] 0 2024-04-12T05:56:11.631 INFO:tasks.workunit.client.1.smithi194.stdout:5/597: truncate d1/d5c/d95/f69 3212814 0 2024-04-12T05:56:11.631 INFO:tasks.workunit.client.1.smithi194.stdout:4/623: stat d2/d3/d50 0 2024-04-12T05:56:11.636 INFO:tasks.workunit.client.1.smithi194.stdout:2/477: getdents d7/d22/d93 0 2024-04-12T05:56:11.639 INFO:tasks.workunit.client.1.smithi194.stdout:0/528: fsync d0/da/d1e/d1a/d3d/f7a 0 2024-04-12T05:56:11.641 INFO:tasks.workunit.client.1.smithi194.stdout:6/539: symlink d3/d2d/la9 0 2024-04-12T05:56:11.643 INFO:tasks.workunit.client.1.smithi194.stdout:4/624: fsync d2/d3/d1b/d32/d3a/d3b/f75 0 2024-04-12T05:56:11.645 INFO:tasks.workunit.client.1.smithi194.stdout:0/529: mknod d0/da/d1e/d1a/d58/cae 0 2024-04-12T05:56:11.647 INFO:tasks.workunit.client.1.smithi194.stdout:9/569: write d1/d19/d1f/d20/f57 [736501,55986] 0 2024-04-12T05:56:11.649 INFO:tasks.workunit.client.1.smithi194.stdout:1/591: rename d0/d11/d18/d19/d1a/d6a/d27/d2e/da1 to d0/d11/d18/d19/d1a/d6a/dac/dba 0 2024-04-12T05:56:11.651 INFO:tasks.workunit.client.1.smithi194.stdout:5/598: rmdir d1/d4/d6/d8a 0 2024-04-12T05:56:11.655 INFO:tasks.workunit.client.1.smithi194.stdout:7/384: dwrite d3/d16/d21/d3a/f60 [0,4194304] 0 2024-04-12T05:56:11.657 INFO:tasks.workunit.client.1.smithi194.stdout:6/540: creat d3/d2f/d4e/d63/faa x:0 0 0 2024-04-12T05:56:11.663 INFO:tasks.workunit.client.1.smithi194.stdout:7/385: rmdir d3/d16/d21/d3a/d43 39 2024-04-12T05:56:11.663 INFO:tasks.workunit.client.1.smithi194.stdout:1/592: chown d0/d11/d18/d19/d1a/d6a/d27/d48 24566496 1 2024-04-12T05:56:11.664 INFO:tasks.workunit.client.1.smithi194.stdout:6/541: rmdir d3/d2d/d33/d52 39 2024-04-12T05:56:11.665 INFO:tasks.workunit.client.1.smithi194.stdout:0/530: write d0/da/d1e/d1a/f6d [410614,112384] 0 2024-04-12T05:56:11.667 INFO:tasks.workunit.client.1.smithi194.stdout:4/625: creat d2/d3/d1b/d32/db3/fdb x:0 0 0 2024-04-12T05:56:11.670 INFO:tasks.workunit.client.1.smithi194.stdout:8/364: dwrite d2/d8/d9/de/d38/f4a [0,4194304] 0 2024-04-12T05:56:11.673 INFO:tasks.workunit.client.1.smithi194.stdout:4/626: dread - d2/d85/dc2/fca zero size 2024-04-12T05:56:11.675 INFO:tasks.workunit.client.1.smithi194.stdout:3/605: dwrite d0/d1/d10/f44 [0,4194304] 0 2024-04-12T05:56:11.675 INFO:tasks.workunit.client.1.smithi194.stdout:7/386: write d3/d16/d23/d37/d3d/d56/f51 [968670,57601] 0 2024-04-12T05:56:11.677 INFO:tasks.workunit.client.1.smithi194.stdout:1/593: symlink d0/d11/d18/d19/d1a/d6a/dac/db5/d8d/lbb 0 2024-04-12T05:56:11.679 INFO:tasks.workunit.client.1.smithi194.stdout:7/387: readlink d3/d27/l1e 0 2024-04-12T05:56:11.680 INFO:tasks.workunit.client.1.smithi194.stdout:9/570: dread d1/d19/d1f/f73 [0,4194304] 0 2024-04-12T05:56:11.682 INFO:tasks.workunit.client.1.smithi194.stdout:8/365: creat d2/d8/d66/d67/f71 x:0 0 0 2024-04-12T05:56:11.685 INFO:tasks.workunit.client.1.smithi194.stdout:5/599: creat d1/d4/d6/fc8 x:0 0 0 2024-04-12T05:56:11.687 INFO:tasks.workunit.client.1.smithi194.stdout:0/531: write d0/da/f39 [1229861,28698] 0 2024-04-12T05:56:11.687 INFO:tasks.workunit.client.1.smithi194.stdout:1/594: mknod d0/d11/d18/d19/d1a/d6a/dac/db5/cbc 0 2024-04-12T05:56:11.689 INFO:tasks.workunit.client.1.smithi194.stdout:7/388: mknod d3/d16/d21/d3a/d43/c7e 0 2024-04-12T05:56:11.690 INFO:tasks.workunit.client.1.smithi194.stdout:2/478: dread d7/f11 [4194304,4194304] 0 2024-04-12T05:56:11.694 INFO:tasks.workunit.client.1.smithi194.stdout:9/571: fsync d1/d19/d1f/d20/d37/d38/f7c 0 2024-04-12T05:56:11.694 INFO:tasks.workunit.client.1.smithi194.stdout:4/627: dwrite d2/d3/d58/f69 [0,4194304] 0 2024-04-12T05:56:11.699 INFO:tasks.workunit.client.1.smithi194.stdout:7/389: creat d3/d27/f7f x:0 0 0 2024-04-12T05:56:11.700 INFO:tasks.workunit.client.1.smithi194.stdout:6/542: dwrite d3/d2d/d33/d52/d5c/d8c/f93 [0,4194304] 0 2024-04-12T05:56:11.701 INFO:tasks.workunit.client.1.smithi194.stdout:4/628: mkdir d2/d3/d50/d80/ddc 0 2024-04-12T05:56:11.703 INFO:tasks.workunit.client.1.smithi194.stdout:9/572: chown d1/d19/d1f/d20/d37/d38/c4a 6075544 1 2024-04-12T05:56:11.704 INFO:tasks.workunit.client.1.smithi194.stdout:1/595: symlink d0/d11/d18/d35/d6f/lbd 0 2024-04-12T05:56:11.707 INFO:tasks.workunit.client.1.smithi194.stdout:4/629: fsync d2/d11/d1e/d25/f5f 0 2024-04-12T05:56:11.709 INFO:tasks.workunit.client.1.smithi194.stdout:8/366: dread d2/d8/d9/de/d38/f4a [0,4194304] 0 2024-04-12T05:56:11.711 INFO:tasks.workunit.client.1.smithi194.stdout:7/390: symlink d3/d16/d21/d3a/d43/l80 0 2024-04-12T05:56:11.711 INFO:tasks.workunit.client.1.smithi194.stdout:9/573: readlink d1/d19/d1f/d5f/l8d 0 2024-04-12T05:56:11.712 INFO:tasks.workunit.client.1.smithi194.stdout:4/630: write d2/d11/f88 [1036029,59051] 0 2024-04-12T05:56:11.713 INFO:tasks.workunit.client.1.smithi194.stdout:5/600: truncate d1/d4/f1c 1250672 0 2024-04-12T05:56:11.714 INFO:tasks.workunit.client.1.smithi194.stdout:2/479: dwrite d7/d12/f80 [0,4194304] 0 2024-04-12T05:56:11.715 INFO:tasks.workunit.client.1.smithi194.stdout:1/596: unlink d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/l78 0 2024-04-12T05:56:11.716 INFO:tasks.workunit.client.1.smithi194.stdout:7/391: truncate d3/d16/d21/f73 93034 0 2024-04-12T05:56:11.718 INFO:tasks.workunit.client.1.smithi194.stdout:8/367: chown d2/d8/d15/d33/c69 9165 1 2024-04-12T05:56:11.722 INFO:tasks.workunit.client.1.smithi194.stdout:9/574: truncate d1/f18 270932 0 2024-04-12T05:56:11.723 INFO:tasks.workunit.client.1.smithi194.stdout:4/631: truncate d2/d3/d58/f69 4619563 0 2024-04-12T05:56:11.723 INFO:tasks.workunit.client.1.smithi194.stdout:2/480: symlink d7/d22/d93/d9a/la8 0 2024-04-12T05:56:11.724 INFO:tasks.workunit.client.1.smithi194.stdout:1/597: chown d0/d11/d18/d35/da5/f72 7560980 1 2024-04-12T05:56:11.724 INFO:tasks.workunit.client.1.smithi194.stdout:7/392: write d3/d16/d23/d37/d3d/d56/f51 [182229,123188] 0 2024-04-12T05:56:11.724 INFO:tasks.workunit.client.1.smithi194.stdout:8/368: chown d2/d8/c57 163 1 2024-04-12T05:56:11.725 INFO:tasks.workunit.client.1.smithi194.stdout:5/601: symlink d1/d4/d6/db/d15/lc9 0 2024-04-12T05:56:11.730 INFO:tasks.workunit.client.1.smithi194.stdout:1/598: stat d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/d7b/f86 0 2024-04-12T05:56:11.731 INFO:tasks.workunit.client.1.smithi194.stdout:0/532: dread d0/da/d1e/d1a/f2e [0,4194304] 0 2024-04-12T05:56:11.732 INFO:tasks.workunit.client.1.smithi194.stdout:2/481: rmdir d7/d22/d93/d9a 39 2024-04-12T05:56:11.734 INFO:tasks.workunit.client.1.smithi194.stdout:5/602: creat d1/d5c/fca x:0 0 0 2024-04-12T05:56:11.735 INFO:tasks.workunit.client.1.smithi194.stdout:0/533: chown d0/d5f/d73/ca6 199171319 1 2024-04-12T05:56:11.736 INFO:tasks.workunit.client.1.smithi194.stdout:9/575: rename d1/d21/d39/l5d to d1/da6/lbd 0 2024-04-12T05:56:11.737 INFO:tasks.workunit.client.1.smithi194.stdout:5/603: dread - d1/d5c/d95/fc3 zero size 2024-04-12T05:56:11.737 INFO:tasks.workunit.client.1.smithi194.stdout:4/632: dread d2/d3/f12 [0,4194304] 0 2024-04-12T05:56:11.738 INFO:tasks.workunit.client.1.smithi194.stdout:1/599: creat d0/d11/d18/d35/da5/fbe x:0 0 0 2024-04-12T05:56:11.741 INFO:tasks.workunit.client.1.smithi194.stdout:2/482: dread - d7/d22/f61 zero size 2024-04-12T05:56:11.742 INFO:tasks.workunit.client.1.smithi194.stdout:8/369: getdents d2/d8/d15 0 2024-04-12T05:56:11.744 INFO:tasks.workunit.client.1.smithi194.stdout:4/633: write d2/d3/d1b/d32/d3a/d3b/f41 [7120518,96718] 0 2024-04-12T05:56:11.744 INFO:tasks.workunit.client.1.smithi194.stdout:9/576: mkdir d1/d19/d1f/d20/d7d/dbe 0 2024-04-12T05:56:11.745 INFO:tasks.workunit.client.1.smithi194.stdout:4/634: write d2/d11/d1e/d25/fd4 [805133,49024] 0 2024-04-12T05:56:11.747 INFO:tasks.workunit.client.1.smithi194.stdout:1/600: mknod d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/cbf 0 2024-04-12T05:56:11.748 INFO:tasks.workunit.client.1.smithi194.stdout:0/534: fsync d0/f30 0 2024-04-12T05:56:11.749 INFO:tasks.workunit.client.1.smithi194.stdout:4/635: creat d2/d3/d1b/d32/fdd x:0 0 0 2024-04-12T05:56:11.750 INFO:tasks.workunit.client.1.smithi194.stdout:0/535: chown d0/f53 13488252 1 2024-04-12T05:56:11.752 INFO:tasks.workunit.client.1.smithi194.stdout:7/393: truncate d3/d27/d22/f38 3704495 0 2024-04-12T05:56:11.753 INFO:tasks.workunit.client.1.smithi194.stdout:5/604: creat d1/d4/d6/d34/d52/fcb x:0 0 0 2024-04-12T05:56:11.753 INFO:tasks.workunit.client.1.smithi194.stdout:4/636: write d2/d3/f12 [1465358,24823] 0 2024-04-12T05:56:11.756 INFO:tasks.workunit.client.1.smithi194.stdout:2/483: unlink d7/d22/d42/c74 0 2024-04-12T05:56:11.758 INFO:tasks.workunit.client.1.smithi194.stdout:7/394: fdatasync d3/d27/d6a/f55 0 2024-04-12T05:56:11.762 INFO:tasks.workunit.client.1.smithi194.stdout:0/536: symlink d0/da/d4a/d8e/laf 0 2024-04-12T05:56:11.762 INFO:tasks.workunit.client.1.smithi194.stdout:5/605: creat d1/d4/d6/db/d15/d26/fcc x:0 0 0 2024-04-12T05:56:11.763 INFO:tasks.workunit.client.1.smithi194.stdout:4/637: write d2/d3/d1b/d32/d3a/d3b/f75 [4964619,59085] 0 2024-04-12T05:56:11.763 INFO:tasks.workunit.client.1.smithi194.stdout:2/484: stat d7/d12/d25/d5d/d47/d52/f55 0 2024-04-12T05:56:11.766 INFO:tasks.workunit.client.1.smithi194.stdout:1/601: truncate d0/d11/f59 4046082 0 2024-04-12T05:56:11.766 INFO:tasks.workunit.client.1.smithi194.stdout:8/370: dwrite d2/d8/d9/de/d38/f4a [0,4194304] 0 2024-04-12T05:56:11.768 INFO:tasks.workunit.client.1.smithi194.stdout:1/602: chown d0/d11/d18/d19/d1c/c31 814618795 1 2024-04-12T05:56:11.768 INFO:tasks.workunit.client.1.smithi194.stdout:0/537: dread - d0/f94 zero size 2024-04-12T05:56:11.768 INFO:tasks.workunit.client.1.smithi194.stdout:5/606: readlink d1/d4/d6/d34/l83 0 2024-04-12T05:56:11.769 INFO:tasks.workunit.client.1.smithi194.stdout:9/577: dread d1/d19/d1f/d20/d37/d38/f7c [0,4194304] 0 2024-04-12T05:56:11.769 INFO:tasks.workunit.client.1.smithi194.stdout:7/395: truncate d3/d16/d5d/f65 907595 0 2024-04-12T05:56:11.772 INFO:tasks.workunit.client.1.smithi194.stdout:4/638: chown d2/d11/d93/da6/dab/d5d 181792 1 2024-04-12T05:56:11.773 INFO:tasks.workunit.client.1.smithi194.stdout:2/485: mknod d7/d12/d25/d5d/ca9 0 2024-04-12T05:56:11.774 INFO:tasks.workunit.client.1.smithi194.stdout:0/538: creat d0/da/d4a/d8e/fb0 x:0 0 0 2024-04-12T05:56:11.775 INFO:tasks.workunit.client.1.smithi194.stdout:7/396: write d3/d16/d23/d37/f7c [49185,4520] 0 2024-04-12T05:56:11.783 INFO:tasks.workunit.client.1.smithi194.stdout:9/578: write d1/d19/d1f/d20/d37/d38/f96 [165343,25669] 0 2024-04-12T05:56:11.783 INFO:tasks.workunit.client.1.smithi194.stdout:4/639: creat d2/d11/d93/da6/dab/fde x:0 0 0 2024-04-12T05:56:11.787 INFO:tasks.workunit.client.1.smithi194.stdout:8/371: dwrite d2/f2d [0,4194304] 0 2024-04-12T05:56:11.787 INFO:tasks.workunit.client.1.smithi194.stdout:0/539: creat d0/d3b/d52/fb1 x:0 0 0 2024-04-12T05:56:11.789 INFO:tasks.workunit.client.1.smithi194.stdout:0/540: readlink d0/l67 0 2024-04-12T05:56:11.792 INFO:tasks.workunit.client.1.smithi194.stdout:4/640: fdatasync d2/d11/d1e/d2f/d4a/f97 0 2024-04-12T05:56:11.794 INFO:tasks.workunit.client.1.smithi194.stdout:5/607: dwrite d1/d4/d6/f35 [0,4194304] 0 2024-04-12T05:56:11.794 INFO:tasks.workunit.client.1.smithi194.stdout:2/486: getdents d7/d12/d34 0 2024-04-12T05:56:11.794 INFO:tasks.workunit.client.1.smithi194.stdout:7/397: rename d3/d16/d21 to d3/d16/d23/d57/d72/d81 0 2024-04-12T05:56:11.797 INFO:tasks.workunit.client.1.smithi194.stdout:8/372: mknod d2/d8/d15/d35/d6c/c72 0 2024-04-12T05:56:11.802 INFO:tasks.workunit.client.1.smithi194.stdout:2/487: fdatasync d7/d12/d25/f31 0 2024-04-12T05:56:11.803 INFO:tasks.workunit.client.1.smithi194.stdout:7/398: fdatasync d3/d27/f2c 0 2024-04-12T05:56:11.807 INFO:tasks.workunit.client.1.smithi194.stdout:2/488: mknod d7/d12/d25/caa 0 2024-04-12T05:56:11.807 INFO:tasks.workunit.client.1.smithi194.stdout:5/608: creat d1/d4/d6/db/d2b/fcd x:0 0 0 2024-04-12T05:56:11.808 INFO:tasks.workunit.client.1.smithi194.stdout:2/489: chown d7/l4e 99 1 2024-04-12T05:56:11.808 INFO:tasks.workunit.client.1.smithi194.stdout:9/579: dwrite d1/d19/d1f/d5f/f86 [4194304,4194304] 0 2024-04-12T05:56:11.809 INFO:tasks.workunit.client.1.smithi194.stdout:7/399: chown d3/d16/d23/d57/d72/d81/d3a/d43/d5c 17 1 2024-04-12T05:56:11.810 INFO:tasks.workunit.client.1.smithi194.stdout:5/609: chown d1/d4/d6/d34/f54 3567156 1 2024-04-12T05:56:11.810 INFO:tasks.workunit.client.1.smithi194.stdout:9/580: dread - d1/d19/d1f/d20/d7d/fad zero size 2024-04-12T05:56:11.813 INFO:tasks.workunit.client.1.smithi194.stdout:2/490: mknod d7/d12/d34/d4a/d8a/cab 0 2024-04-12T05:56:11.814 INFO:tasks.workunit.client.1.smithi194.stdout:5/610: truncate d1/d4/d6/d34/d52/fb8 250178 0 2024-04-12T05:56:11.814 INFO:tasks.workunit.client.1.smithi194.stdout:9/581: mkdir d1/d41/dbf 0 2024-04-12T05:56:11.815 INFO:tasks.workunit.client.1.smithi194.stdout:2/491: fdatasync d7/d12/d25/d5d/d47/d52/f9b 0 2024-04-12T05:56:11.816 INFO:tasks.workunit.client.1.smithi194.stdout:7/400: fdatasync d3/d27/f39 0 2024-04-12T05:56:11.818 INFO:tasks.workunit.client.1.smithi194.stdout:1/603: dread d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [4194304,4194304] 0 2024-04-12T05:56:11.820 INFO:tasks.workunit.client.1.smithi194.stdout:5/611: unlink d1/d4/d6/d6d/cb9 0 2024-04-12T05:56:11.821 INFO:tasks.workunit.client.1.smithi194.stdout:7/401: creat d3/d27/d6a/f82 x:0 0 0 2024-04-12T05:56:11.822 INFO:tasks.workunit.client.1.smithi194.stdout:2/492: mkdir d7/d12/d25/d5d/d47/dac 0 2024-04-12T05:56:11.825 INFO:tasks.workunit.client.1.smithi194.stdout:9/582: dwrite d1/d21/d77/f92 [0,4194304] 0 2024-04-12T05:56:11.827 INFO:tasks.workunit.client.1.smithi194.stdout:5/612: unlink d1/d4/d6/f35 0 2024-04-12T05:56:11.827 INFO:tasks.workunit.client.1.smithi194.stdout:9/583: fsync d1/d19/d1f/f52 0 2024-04-12T05:56:11.828 INFO:tasks.workunit.client.1.smithi194.stdout:2/493: creat d7/d12/d1a/fad x:0 0 0 2024-04-12T05:56:11.829 INFO:tasks.workunit.client.1.smithi194.stdout:7/402: creat d3/d16/d23/f83 x:0 0 0 2024-04-12T05:56:11.831 INFO:tasks.workunit.client.1.smithi194.stdout:1/604: dwrite d0/d11/d18/d19/d1a/d6a/dac/db5/d8d/db9/f8a [0,4194304] 0 2024-04-12T05:56:11.832 INFO:tasks.workunit.client.1.smithi194.stdout:7/403: chown d3/d16/d23/d37/d3d/d56 1587 1 2024-04-12T05:56:11.833 INFO:tasks.workunit.client.1.smithi194.stdout:2/494: symlink d7/d12/d34/d4a/lae 0 2024-04-12T05:56:11.834 INFO:tasks.workunit.client.1.smithi194.stdout:1/605: write d0/d11/f9f [537033,16095] 0 2024-04-12T05:56:11.835 INFO:tasks.workunit.client.1.smithi194.stdout:2/495: dread - d7/d12/d34/f97 zero size 2024-04-12T05:56:11.837 INFO:tasks.workunit.client.1.smithi194.stdout:5/613: dwrite d1/d4/d6/db/f19 [4194304,4194304] 0 2024-04-12T05:56:11.838 INFO:tasks.workunit.client.1.smithi194.stdout:1/606: readlink d0/d11/d18/l3f 0 2024-04-12T05:56:11.840 INFO:tasks.workunit.client.1.smithi194.stdout:7/404: truncate d3/d16/f19 4793542 0 2024-04-12T05:56:11.842 INFO:tasks.workunit.client.1.smithi194.stdout:5/614: unlink d1/d4/d75/f85 0 2024-04-12T05:56:11.846 INFO:tasks.workunit.client.1.smithi194.stdout:1/607: fsync d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 0 2024-04-12T05:56:11.847 INFO:tasks.workunit.client.1.smithi194.stdout:5/615: symlink d1/d4/d6/db/d2b/lce 0 2024-04-12T05:56:11.847 INFO:tasks.workunit.client.1.smithi194.stdout:2/496: dwrite d7/d22/f5c [8388608,4194304] 0 2024-04-12T05:56:11.849 INFO:tasks.workunit.client.1.smithi194.stdout:7/405: rename d3/d27/d22/f29 to d3/d16/d5d/f84 0 2024-04-12T05:56:11.852 INFO:tasks.workunit.client.1.smithi194.stdout:5/616: creat d1/d4/d6/d34/d52/fcf x:0 0 0 2024-04-12T05:56:11.852 INFO:tasks.workunit.client.1.smithi194.stdout:7/406: mknod d3/d16/d23/d40/c85 0 2024-04-12T05:56:11.858 INFO:tasks.workunit.client.1.smithi194.stdout:7/407: mknod d3/d16/d23/d40/d49/c86 0 2024-04-12T05:56:11.861 INFO:tasks.workunit.client.1.smithi194.stdout:5/617: link d1/d4/d6/db/d15/c78 d1/d5c/d95/cd0 0 2024-04-12T05:56:11.863 INFO:tasks.workunit.client.1.smithi194.stdout:5/618: chown d1/d4/d6/d34 9172 1 2024-04-12T05:56:11.866 INFO:tasks.workunit.client.1.smithi194.stdout:5/619: mknod d1/d4/d6/d6d/cd1 0 2024-04-12T05:56:11.867 INFO:tasks.workunit.client.1.smithi194.stdout:8/373: rmdir d2/d8/d15/d35 39 2024-04-12T05:56:11.867 INFO:tasks.workunit.client.1.smithi194.stdout:0/541: write d0/da/d1e/d1a/f44 [4009726,82580] 0 2024-04-12T05:56:11.871 INFO:tasks.workunit.client.1.smithi194.stdout:0/542: chown d0/da/d1e/d27/d87/d92/faa 0 1 2024-04-12T05:56:11.876 INFO:tasks.workunit.client.1.smithi194.stdout:0/543: rename d0/da to d0/da/d1e/d1a/d24/d56/db2 22 2024-04-12T05:56:11.878 INFO:tasks.workunit.client.1.smithi194.stdout:8/374: getdents d2/d8/d9/de/d10/d4e 0 2024-04-12T05:56:11.879 INFO:tasks.workunit.client.1.smithi194.stdout:5/620: dwrite d1/f3 [4194304,4194304] 0 2024-04-12T05:56:11.881 INFO:tasks.workunit.client.1.smithi194.stdout:4/641: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/f79 [0,4194304] 0 2024-04-12T05:56:11.883 INFO:tasks.workunit.client.1.smithi194.stdout:0/544: dread - d0/da/d1e/d1a/d24/d56/f9b zero size 2024-04-12T05:56:11.891 INFO:tasks.workunit.client.1.smithi194.stdout:5/621: creat d1/d4/d6/db/d2b/fd2 x:0 0 0 2024-04-12T05:56:11.893 INFO:tasks.workunit.client.1.smithi194.stdout:0/545: rename d0/da/l5d to d0/d3b/d9c/lb3 0 2024-04-12T05:56:11.895 INFO:tasks.workunit.client.1.smithi194.stdout:1/608: write d0/d11/d18/d35/da5/f72 [1465291,127037] 0 2024-04-12T05:56:11.895 INFO:tasks.workunit.client.1.smithi194.stdout:0/546: fdatasync d0/da/d4a/d8e/f72 0 2024-04-12T05:56:11.899 INFO:tasks.workunit.client.1.smithi194.stdout:5/622: symlink d1/d4/d6/db/d38/ld3 0 2024-04-12T05:56:11.900 INFO:tasks.workunit.client.1.smithi194.stdout:4/642: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/f2a [0,4194304] 0 2024-04-12T05:56:11.948 INFO:tasks.workunit.client.1.smithi194.stdout:1/609: read d0/d11/d18/d19/f80 [2912290,96239] 0 2024-04-12T05:56:11.948 INFO:tasks.workunit.client.1.smithi194.stdout:7/408: write d3/d27/f1f [105927,27555] 0 2024-04-12T05:56:11.948 INFO:tasks.workunit.client.1.smithi194.stdout:0/547: mknod d0/da/d1e/d1a/d58/cb4 0 2024-04-12T05:56:11.950 INFO:tasks.workunit.client.1.smithi194.stdout:4/643: rename d2/d11 to d2/d11/d43/ddf 22 2024-04-12T05:56:11.952 INFO:tasks.workunit.client.1.smithi194.stdout:5/623: creat d1/d4/d6/d34/fd4 x:0 0 0 2024-04-12T05:56:11.954 INFO:tasks.workunit.client.1.smithi194.stdout:7/409: creat d3/d16/d23/d37/d3d/d56/f87 x:0 0 0 2024-04-12T05:56:11.956 INFO:tasks.workunit.client.1.smithi194.stdout:1/610: mknod d0/d11/d18/d19/d1a/d6a/d27/d2e/cc0 0 2024-04-12T05:56:11.956 INFO:tasks.workunit.client.1.smithi194.stdout:8/375: dwrite d2/d8/d15/d33/f18 [0,4194304] 0 2024-04-12T05:56:11.960 INFO:tasks.workunit.client.1.smithi194.stdout:7/410: stat d3/d16/d23/d40/d49/f71 0 2024-04-12T05:56:11.960 INFO:tasks.workunit.client.1.smithi194.stdout:2/497: dread d7/d12/d1a/f24 [0,4194304] 0 2024-04-12T05:56:11.961 INFO:tasks.workunit.client.1.smithi194.stdout:5/624: write d1/d4/d6/d6d/f6b [1157640,71209] 0 2024-04-12T05:56:11.964 INFO:tasks.workunit.client.1.smithi194.stdout:1/611: write d0/d11/d18/f2d [8307647,19145] 0 2024-04-12T05:56:11.967 INFO:tasks.workunit.client.1.smithi194.stdout:5/625: fdatasync d1/d4/f1a 0 2024-04-12T05:56:11.967 INFO:tasks.workunit.client.1.smithi194.stdout:8/376: write d2/d8/d9/de/f61 [28335,30937] 0 2024-04-12T05:56:11.973 INFO:tasks.workunit.client.1.smithi194.stdout:0/548: dwrite d0/da/d1e/f17 [0,4194304] 0 2024-04-12T05:56:11.973 INFO:tasks.workunit.client.1.smithi194.stdout:1/612: fdatasync d0/d11/fae 0 2024-04-12T05:56:11.977 INFO:tasks.workunit.client.1.smithi194.stdout:5/626: symlink d1/ld5 0 2024-04-12T05:56:11.977 INFO:tasks.workunit.client.1.smithi194.stdout:0/549: write d0/f94 [812685,44713] 0 2024-04-12T05:56:11.977 INFO:tasks.workunit.client.1.smithi194.stdout:7/411: dread d3/d27/d22/f77 [0,4194304] 0 2024-04-12T05:56:11.979 INFO:tasks.workunit.client.1.smithi194.stdout:1/613: chown d0/d11/d18/d19/d1a/d6a/dac/db5/d8d/lbb 134382588 1 2024-04-12T05:56:11.980 INFO:tasks.workunit.client.1.smithi194.stdout:2/498: write d7/d12/f3d [581240,104171] 0 2024-04-12T05:56:11.981 INFO:tasks.workunit.client.1.smithi194.stdout:5/627: write d1/fa1 [750048,89038] 0 2024-04-12T05:56:11.981 INFO:tasks.workunit.client.1.smithi194.stdout:4/644: dwrite d2/d3/d50/f9e [0,4194304] 0 2024-04-12T05:56:11.986 INFO:tasks.workunit.client.1.smithi194.stdout:0/550: truncate d0/da/d1e/d27/d87/d92/faa 651832 0 2024-04-12T05:56:11.990 INFO:tasks.workunit.client.1.smithi194.stdout:0/551: read d0/da/d1e/d1a/d24/f61 [8312128,13861] 0 2024-04-12T05:56:11.990 INFO:tasks.workunit.client.1.smithi194.stdout:7/412: rename d3/d16/d23/d57/d72/d81/d3a/c5a to d3/d16/d23/d40/d49/d74/c88 0 2024-04-12T05:56:11.991 INFO:tasks.workunit.client.1.smithi194.stdout:4/645: creat d2/d11/d46/d61/fe0 x:0 0 0 2024-04-12T05:56:11.998 INFO:tasks.workunit.client.1.smithi194.stdout:1/614: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b [4194304,4194304] 0 2024-04-12T05:56:12.001 INFO:tasks.workunit.client.1.smithi194.stdout:5/628: getdents d1/d4/d9e 0 2024-04-12T05:56:12.002 INFO:tasks.workunit.client.1.smithi194.stdout:1/615: creat d0/d11/fc1 x:0 0 0 2024-04-12T05:56:12.004 INFO:tasks.workunit.client.1.smithi194.stdout:5/629: fsync d1/d4/d6/db/d15/faa 0 2024-04-12T05:56:12.005 INFO:tasks.workunit.client.1.smithi194.stdout:4/646: rename d2/d3/d50/d80 to d2/d11/d93/da6/dab/d5d/dae/de1 0 2024-04-12T05:56:12.010 INFO:tasks.workunit.client.1.smithi194.stdout:8/377: dread d2/fb [0,4194304] 0 2024-04-12T05:56:12.010 INFO:tasks.workunit.client.1.smithi194.stdout:5/630: dread - d1/d4/d6/d88/fae zero size 2024-04-12T05:56:12.011 INFO:tasks.workunit.client.1.smithi194.stdout:4/647: fsync d2/d3/d51/f55 0 2024-04-12T05:56:12.013 INFO:tasks.workunit.client.1.smithi194.stdout:1/616: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/cc2 0 2024-04-12T05:56:12.015 INFO:tasks.workunit.client.1.smithi194.stdout:0/552: dwrite d0/da/d1e/d1a/f64 [0,4194304] 0 2024-04-12T05:56:12.015 INFO:tasks.workunit.client.1.smithi194.stdout:2/499: dread d7/d12/d34/f83 [0,4194304] 0 2024-04-12T05:56:12.017 INFO:tasks.workunit.client.1.smithi194.stdout:5/631: write d1/d5c/da7/fb1 [139874,116267] 0 2024-04-12T05:56:12.019 INFO:tasks.workunit.client.1.smithi194.stdout:1/617: chown d0/d11/d18/d19/d1a/d6a/d27/d2e/f33 3 1 2024-04-12T05:56:12.019 INFO:tasks.workunit.client.1.smithi194.stdout:7/413: dwrite d3/d27/f39 [0,4194304] 0 2024-04-12T05:56:12.024 INFO:tasks.workunit.client.1.smithi194.stdout:7/414: truncate d3/d16/d23/d37/d3d/d56/f87 827907 0 2024-04-12T05:56:12.024 INFO:tasks.workunit.client.1.smithi194.stdout:5/632: rmdir d1/d5c/da7 39 2024-04-12T05:56:12.026 INFO:tasks.workunit.client.1.smithi194.stdout:0/553: dread d0/f94 [0,4194304] 0 2024-04-12T05:56:12.029 INFO:tasks.workunit.client.1.smithi194.stdout:4/648: mkdir d2/d3/d50/da0/de2 0 2024-04-12T05:56:12.030 INFO:tasks.workunit.client.1.smithi194.stdout:4/649: chown d2/d11/d1e/d25/l96 1376 1 2024-04-12T05:56:12.031 INFO:tasks.workunit.client.1.smithi194.stdout:4/650: stat d2/d11/f88 0 2024-04-12T05:56:12.033 INFO:tasks.workunit.client.1.smithi194.stdout:7/415: getdents d3/d27 0 2024-04-12T05:56:12.034 INFO:tasks.workunit.client.1.smithi194.stdout:0/554: truncate d0/da/d1e/d1a/f1c 3500356 0 2024-04-12T05:56:12.035 INFO:tasks.workunit.client.1.smithi194.stdout:7/416: dread - d3/d16/d23/d40/d49/f71 zero size 2024-04-12T05:56:12.039 INFO:tasks.workunit.client.1.smithi194.stdout:4/651: creat d2/d3/d58/d82/dd0/fe3 x:0 0 0 2024-04-12T05:56:12.041 INFO:tasks.workunit.client.1.smithi194.stdout:7/417: write d3/d16/d23/d57/d72/d81/d3a/d43/f79 [757169,22339] 0 2024-04-12T05:56:12.045 INFO:tasks.workunit.client.1.smithi194.stdout:0/555: dwrite d0/da/f3e [4194304,4194304] 0 2024-04-12T05:56:12.045 INFO:tasks.workunit.client.1.smithi194.stdout:5/633: dwrite d1/d4/d6/db/d3b/f3e [0,4194304] 0 2024-04-12T05:56:12.050 INFO:tasks.workunit.client.1.smithi194.stdout:5/634: chown d1/d4/d6/d34/f54 4 1 2024-04-12T05:56:12.056 INFO:tasks.workunit.client.1.smithi194.stdout:4/652: mkdir d2/d11/d93/da6/dab/d5d/dae/de1/ddc/de4 0 2024-04-12T05:56:12.057 INFO:tasks.workunit.client.1.smithi194.stdout:4/653: fdatasync d2/d85/dc2/fca 0 2024-04-12T05:56:12.059 INFO:tasks.workunit.client.1.smithi194.stdout:0/556: write d0/d3b/d52/f84 [1533180,60731] 0 2024-04-12T05:56:12.059 INFO:tasks.workunit.client.1.smithi194.stdout:7/418: link d3/d27/d6a/l4f d3/d16/d23/d37/d3d/d56/l89 0 2024-04-12T05:56:12.064 INFO:tasks.workunit.client.1.smithi194.stdout:4/654: creat d2/d11/d93/da6/dab/d5d/dae/db2/d57/fe5 x:0 0 0 2024-04-12T05:56:12.066 INFO:tasks.workunit.client.1.smithi194.stdout:7/419: write d3/d16/d5d/f84 [3317044,52794] 0 2024-04-12T05:56:12.069 INFO:tasks.workunit.client.1.smithi194.stdout:7/420: read d3/d27/f1d [2186777,14521] 0 2024-04-12T05:56:12.071 INFO:tasks.workunit.client.1.smithi194.stdout:5/635: dwrite d1/d4/d6/db/fc [0,4194304] 0 2024-04-12T05:56:12.075 INFO:tasks.workunit.client.1.smithi194.stdout:7/421: mkdir d3/d16/d23/d57/d8a 0 2024-04-12T05:56:12.076 INFO:tasks.workunit.client.1.smithi194.stdout:4/655: write d2/d11/d1e/f5a [1088342,22121] 0 2024-04-12T05:56:12.077 INFO:tasks.workunit.client.1.smithi194.stdout:5/636: symlink d1/d4/d6/db/d15/d8c/ld6 0 2024-04-12T05:56:12.077 INFO:tasks.workunit.client.1.smithi194.stdout:7/422: write d3/d27/d22/f2e [1666544,83416] 0 2024-04-12T05:56:12.078 INFO:tasks.workunit.client.1.smithi194.stdout:5/637: read - d1/d4/d84/f99 zero size 2024-04-12T05:56:12.079 INFO:tasks.workunit.client.1.smithi194.stdout:4/656: fdatasync d2/d11/d93/da6/dab/fde 0 2024-04-12T05:56:12.081 INFO:tasks.workunit.client.1.smithi194.stdout:0/557: dwrite d0/da/d1e/d1a/d24/f61 [4194304,4194304] 0 2024-04-12T05:56:12.082 INFO:tasks.workunit.client.1.smithi194.stdout:5/638: fdatasync d1/d4/d6/db/d3b/fa6 0 2024-04-12T05:56:12.083 INFO:tasks.workunit.client.1.smithi194.stdout:7/423: write d3/d27/f44 [574606,55649] 0 2024-04-12T05:56:12.084 INFO:tasks.workunit.client.1.smithi194.stdout:7/424: stat d3/d16/d23/c6d 0 2024-04-12T05:56:12.087 INFO:tasks.workunit.client.1.smithi194.stdout:4/657: creat d2/d11/d93/da6/dab/d5d/dae/db2/fe6 x:0 0 0 2024-04-12T05:56:12.087 INFO:tasks.workunit.client.1.smithi194.stdout:0/558: write d0/da/d1e/d27/fa0 [696433,54407] 0 2024-04-12T05:56:12.089 INFO:tasks.workunit.client.1.smithi194.stdout:5/639: write d1/d4/d6/d34/d52/f77 [609019,69280] 0 2024-04-12T05:56:12.090 INFO:tasks.workunit.client.1.smithi194.stdout:4/658: mkdir d2/d11/d1e/d2f/d4a/d9c/de7 0 2024-04-12T05:56:12.091 INFO:tasks.workunit.client.1.smithi194.stdout:4/659: dread - d2/d3/f77 zero size 2024-04-12T05:56:12.092 INFO:tasks.workunit.client.1.smithi194.stdout:5/640: mkdir d1/d4/dd7 0 2024-04-12T05:56:12.093 INFO:tasks.workunit.client.1.smithi194.stdout:4/660: creat d2/d3/d58/d82/dd0/fe8 x:0 0 0 2024-04-12T05:56:12.095 INFO:tasks.workunit.client.1.smithi194.stdout:5/641: rmdir d1/d4/d6/db/d3b 39 2024-04-12T05:56:12.095 INFO:tasks.workunit.client.1.smithi194.stdout:4/661: mkdir d2/d3/d51/de9 0 2024-04-12T05:56:12.098 INFO:tasks.workunit.client.1.smithi194.stdout:4/662: write d2/d3/d1b/d32/d3a/d3b/fc9 [139187,5045] 0 2024-04-12T05:56:12.103 INFO:tasks.workunit.client.1.smithi194.stdout:5/642: write d1/d4/d6/db/d15/d26/f50 [1596587,4830] 0 2024-04-12T05:56:12.105 INFO:tasks.workunit.client.1.smithi194.stdout:4/663: rename d2/c9a to d2/d11/d46/d61/d63/da2/cea 0 2024-04-12T05:56:12.106 INFO:tasks.workunit.client.1.smithi194.stdout:7/425: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:12.107 INFO:tasks.workunit.client.1.smithi194.stdout:5/643: unlink d1/d4/d6/db/d2b/f9a 0 2024-04-12T05:56:12.110 INFO:tasks.workunit.client.1.smithi194.stdout:4/664: unlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/fce 0 2024-04-12T05:56:12.112 INFO:tasks.workunit.client.1.smithi194.stdout:7/426: creat d3/d16/d23/d40/d6f/f8b x:0 0 0 2024-04-12T05:56:12.113 INFO:tasks.workunit.client.1.smithi194.stdout:5/644: creat d1/d4/d6/db/d15/d26/fd8 x:0 0 0 2024-04-12T05:56:12.114 INFO:tasks.workunit.client.1.smithi194.stdout:5/645: chown d1/d4/d6/db/f4a 13687 1 2024-04-12T05:56:12.114 INFO:tasks.workunit.client.1.smithi194.stdout:7/427: mkdir d3/d16/d5d/d8c 0 2024-04-12T05:56:12.115 INFO:tasks.workunit.client.1.smithi194.stdout:7/428: chown d3/d16/c3c 3 1 2024-04-12T05:56:12.115 INFO:tasks.workunit.client.1.smithi194.stdout:5/646: chown d1/d4/d6/db/d2b/fcd 10 1 2024-04-12T05:56:12.116 INFO:tasks.workunit.client.1.smithi194.stdout:4/665: rename d2/d11/d46/c94 to d2/d11/d43/ceb 0 2024-04-12T05:56:12.120 INFO:tasks.workunit.client.1.smithi194.stdout:5/647: link d1/d4/d6/db/d2b/f66 d1/d4/d4b/fd9 0 2024-04-12T05:56:12.122 INFO:tasks.workunit.client.1.smithi194.stdout:5/648: mknod d1/d4/d6/db/d38/cda 0 2024-04-12T05:56:12.123 INFO:tasks.workunit.client.1.smithi194.stdout:5/649: write d1/d4/d6/db/d2b/fcd [39853,50474] 0 2024-04-12T05:56:12.124 INFO:tasks.workunit.client.1.smithi194.stdout:7/429: truncate d3/d27/f44 701394 0 2024-04-12T05:56:12.125 INFO:tasks.workunit.client.1.smithi194.stdout:7/430: chown d3/d27/f64 11292 1 2024-04-12T05:56:12.129 INFO:tasks.workunit.client.1.smithi194.stdout:5/650: chown d1/d4/d6/db/d3b/fa6 1022450 1 2024-04-12T05:56:12.129 INFO:tasks.workunit.client.1.smithi194.stdout:5/651: fdatasync d1/d4/d6/d34/d52/fcb 0 2024-04-12T05:56:12.131 INFO:tasks.workunit.client.1.smithi194.stdout:7/431: symlink d3/d16/d23/d57/d72/d81/d3a/l8d 0 2024-04-12T05:56:12.133 INFO:tasks.workunit.client.1.smithi194.stdout:5/652: mkdir d1/d4/d9e/ddb 0 2024-04-12T05:56:12.134 INFO:tasks.workunit.client.1.smithi194.stdout:4/666: dread d2/d11/d1e/d2f/f76 [0,4194304] 0 2024-04-12T05:56:12.136 INFO:tasks.workunit.client.1.smithi194.stdout:5/653: symlink d1/d4/d6/d34/d52/ldc 0 2024-04-12T05:56:12.138 INFO:tasks.workunit.client.1.smithi194.stdout:4/667: symlink d2/d11/d46/d61/lec 0 2024-04-12T05:56:12.139 INFO:tasks.workunit.client.1.smithi194.stdout:7/432: getdents d3/d16/d23/d57/d72 0 2024-04-12T05:56:12.141 INFO:tasks.workunit.client.1.smithi194.stdout:5/654: readlink d1/d4/d6/d8e/lb7 0 2024-04-12T05:56:12.142 INFO:tasks.workunit.client.1.smithi194.stdout:4/668: unlink d2/d3/d1b/d32/db3/db4/cb7 0 2024-04-12T05:56:12.148 INFO:tasks.workunit.client.1.smithi194.stdout:5/655: creat d1/d4/d6/db/fdd x:0 0 0 2024-04-12T05:56:12.149 INFO:tasks.workunit.client.1.smithi194.stdout:4/669: link d2/d11/d1e/cd2 d2/d11/d1f/ced 0 2024-04-12T05:56:12.153 INFO:tasks.workunit.client.1.smithi194.stdout:5/656: mknod d1/d4/d6/d8e/cde 0 2024-04-12T05:56:12.153 INFO:tasks.workunit.client.1.smithi194.stdout:4/670: mkdir d2/d11/d1e/d2f/d4a/d9c/dee 0 2024-04-12T05:56:12.156 INFO:tasks.workunit.client.1.smithi194.stdout:4/671: chown d2/d11/d93/da6/dab/d5d/dae/de1/ddc/de4 423616 1 2024-04-12T05:56:12.159 INFO:tasks.workunit.client.1.smithi194.stdout:4/672: write d2/d3/f3f [1920302,23917] 0 2024-04-12T05:56:12.159 INFO:tasks.workunit.client.1.smithi194.stdout:5/657: mknod d1/d4/d6/db/d15/d26/dbb/cdf 0 2024-04-12T05:56:12.162 INFO:tasks.workunit.client.1.smithi194.stdout:4/673: fdatasync d2/d11/d1e/d25/fd7 0 2024-04-12T05:56:12.166 INFO:tasks.workunit.client.1.smithi194.stdout:7/433: dread d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:12.168 INFO:tasks.workunit.client.1.smithi194.stdout:5/658: link d1/d4/d6/d34/d52/l93 d1/d4/d6/db/d15/d26/le0 0 2024-04-12T05:56:12.186 INFO:tasks.workunit.client.1.smithi194.stdout:4/674: mknod d2/d3/d51/cef 0 2024-04-12T05:56:12.187 INFO:tasks.workunit.client.1.smithi194.stdout:3/606: sync 2024-04-12T05:56:12.189 INFO:tasks.workunit.client.1.smithi194.stdout:4/675: write d2/d11/d1e/d2f/f76 [1464941,1022] 0 2024-04-12T05:56:12.190 INFO:tasks.workunit.client.1.smithi194.stdout:5/659: rename d1/d4/d6/db/d3b/l9d to d1/d4/d6/d8e/le1 0 2024-04-12T05:56:12.193 INFO:tasks.workunit.client.1.smithi194.stdout:4/676: symlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/lf0 0 2024-04-12T05:56:12.193 INFO:tasks.workunit.client.1.smithi194.stdout:3/607: rename d0/d1/d10/f52 to d0/d1/d74/d39/da2/fc8 0 2024-04-12T05:56:12.196 INFO:tasks.workunit.client.1.smithi194.stdout:3/608: chown d0/dc/d1e/f5f 167999322 1 2024-04-12T05:56:12.199 INFO:tasks.workunit.client.1.smithi194.stdout:7/434: dread d3/d16/d23/d40/f68 [0,4194304] 0 2024-04-12T05:56:12.199 INFO:tasks.workunit.client.1.smithi194.stdout:4/677: dread d2/d3/d1b/d32/d3a/d3b/f75 [4194304,4194304] 0 2024-04-12T05:56:12.201 INFO:tasks.workunit.client.1.smithi194.stdout:7/435: readlink d3/d16/d23/d40/d49/l4b 0 2024-04-12T05:56:12.203 INFO:tasks.workunit.client.1.smithi194.stdout:3/609: dread d0/dc/d1e/d81/d34/d2d/f5a [0,4194304] 0 2024-04-12T05:56:12.203 INFO:tasks.workunit.client.1.smithi194.stdout:5/660: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:12.204 INFO:tasks.workunit.client.1.smithi194.stdout:5/661: stat d1/f82 0 2024-04-12T05:56:12.205 INFO:tasks.workunit.client.1.smithi194.stdout:7/436: read - d3/d27/d22/d62/f6c zero size 2024-04-12T05:56:12.205 INFO:tasks.workunit.client.1.smithi194.stdout:4/678: symlink d2/d3/d50/da0/de2/lf1 0 2024-04-12T05:56:12.208 INFO:tasks.workunit.client.1.smithi194.stdout:4/679: write d2/d11/d46/d61/f92 [350282,31599] 0 2024-04-12T05:56:12.212 INFO:tasks.workunit.client.1.smithi194.stdout:4/680: creat d2/d3/d58/d82/dd0/ff2 x:0 0 0 2024-04-12T05:56:12.215 INFO:tasks.workunit.client.1.smithi194.stdout:5/662: dwrite d1/d4/d6/db/fdd [0,4194304] 0 2024-04-12T05:56:12.220 INFO:tasks.workunit.client.1.smithi194.stdout:4/681: link d2/d3/ff d2/d11/d93/da6/dab/d5d/dae/de1/ddc/de4/ff3 0 2024-04-12T05:56:12.221 INFO:tasks.workunit.client.1.smithi194.stdout:5/663: creat d1/d4/dd7/fe2 x:0 0 0 2024-04-12T05:56:12.224 INFO:tasks.workunit.client.1.smithi194.stdout:5/664: write d1/d4/d6/db/d38/fc0 [405336,78995] 0 2024-04-12T05:56:12.226 INFO:tasks.workunit.client.1.smithi194.stdout:5/665: readlink d1/d4/d6/db/d15/d26/le0 0 2024-04-12T05:56:12.226 INFO:tasks.workunit.client.1.smithi194.stdout:4/682: creat d2/d3/d1b/d32/d3a/dbc/ff4 x:0 0 0 2024-04-12T05:56:12.229 INFO:tasks.workunit.client.1.smithi194.stdout:4/683: mknod d2/d3/d1b/d32/d3a/dbc/cf5 0 2024-04-12T05:56:12.230 INFO:tasks.workunit.client.1.smithi194.stdout:3/610: truncate d0/f27 373094 0 2024-04-12T05:56:12.230 INFO:tasks.workunit.client.1.smithi194.stdout:4/684: write d2/d3/f13 [2446582,462] 0 2024-04-12T05:56:12.233 INFO:tasks.workunit.client.1.smithi194.stdout:5/666: dread d1/d5c/d95/f2e [0,4194304] 0 2024-04-12T05:56:12.233 INFO:tasks.workunit.client.1.smithi194.stdout:3/611: mkdir d0/dc/d1e/dc4/dc9 0 2024-04-12T05:56:12.235 INFO:tasks.workunit.client.1.smithi194.stdout:5/667: creat d1/d5c/d95/fe3 x:0 0 0 2024-04-12T05:56:12.236 INFO:tasks.workunit.client.1.smithi194.stdout:3/612: creat d0/dc/d1e/d33/fca x:0 0 0 2024-04-12T05:56:12.240 INFO:tasks.workunit.client.1.smithi194.stdout:3/613: creat d0/d1/d74/d39/d4d/d63/db5/fcb x:0 0 0 2024-04-12T05:56:12.240 INFO:tasks.workunit.client.1.smithi194.stdout:5/668: link d1/d4/d6/d88/l37 d1/d4/d6/db/d15/d26/dbb/le4 0 2024-04-12T05:56:12.241 INFO:tasks.workunit.client.1.smithi194.stdout:3/614: fsync d0/d42/fba 0 2024-04-12T05:56:12.248 INFO:tasks.workunit.client.1.smithi194.stdout:3/615: write d0/dc/f88 [7149269,91924] 0 2024-04-12T05:56:12.249 INFO:tasks.workunit.client.1.smithi194.stdout:3/616: stat d0/fb 0 2024-04-12T05:56:12.249 INFO:tasks.workunit.client.1.smithi194.stdout:3/617: stat d0/d1/f57 0 2024-04-12T05:56:12.252 INFO:tasks.workunit.client.1.smithi194.stdout:3/618: creat d0/dc/d1e/d81/d5b/d64/dc5/fcc x:0 0 0 2024-04-12T05:56:12.290 INFO:tasks.workunit.client.1.smithi194.stdout:4/685: mkdir d2/d11/d93/da6/dab/d5d/dae/db2/df6 0 2024-04-12T05:56:12.294 INFO:tasks.workunit.client.1.smithi194.stdout:4/686: unlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/lf0 0 2024-04-12T05:56:12.296 INFO:tasks.workunit.client.1.smithi194.stdout:4/687: truncate d2/d11/d1e/d25/fd7 817698 0 2024-04-12T05:56:12.300 INFO:tasks.workunit.client.1.smithi194.stdout:3/619: dread d0/dc/d1e/d33/f68 [0,4194304] 0 2024-04-12T05:56:12.300 INFO:tasks.workunit.client.1.smithi194.stdout:4/688: getdents d2/d3/d1b/d32/db3/db4 0 2024-04-12T05:56:12.302 INFO:tasks.workunit.client.1.smithi194.stdout:3/620: creat d0/d1/d74/d39/d9e/fcd x:0 0 0 2024-04-12T05:56:12.303 INFO:tasks.workunit.client.1.smithi194.stdout:3/621: dread - d0/d1/f60 zero size 2024-04-12T05:56:12.307 INFO:tasks.workunit.client.1.smithi194.stdout:3/622: rmdir d0/dc/d1e/dc4/dc9 0 2024-04-12T05:56:12.311 INFO:tasks.workunit.client.1.smithi194.stdout:3/623: rename d0/dc/d1e/d81/d5b/d64/d79/d87/lbf to d0/dc/d1e/lce 0 2024-04-12T05:56:12.312 INFO:tasks.workunit.client.1.smithi194.stdout:3/624: mkdir d0/dcf 0 2024-04-12T05:56:12.320 INFO:tasks.workunit.client.1.smithi194.stdout:4/689: truncate d2/d3/f18 3491912 0 2024-04-12T05:56:12.326 INFO:tasks.workunit.client.1.smithi194.stdout:4/690: dwrite d2/d85/fd5 [0,4194304] 0 2024-04-12T05:56:12.330 INFO:tasks.workunit.client.1.smithi194.stdout:3/625: dread d0/d1/f97 [0,4194304] 0 2024-04-12T05:56:12.333 INFO:tasks.workunit.client.1.smithi194.stdout:3/626: chown d0/dc/d1e/d81/d34/f37 237019 1 2024-04-12T05:56:12.336 INFO:tasks.workunit.client.1.smithi194.stdout:4/691: dwrite d2/d3/f12 [0,4194304] 0 2024-04-12T05:56:12.336 INFO:tasks.workunit.client.1.smithi194.stdout:3/627: mknod d0/dc/db4/cd0 0 2024-04-12T05:56:12.340 INFO:tasks.workunit.client.1.smithi194.stdout:4/692: mknod d2/d11/d1e/cf7 0 2024-04-12T05:56:12.343 INFO:tasks.workunit.client.1.smithi194.stdout:4/693: stat d2/d3/d1b/d32/cb9 0 2024-04-12T05:56:12.348 INFO:tasks.workunit.client.1.smithi194.stdout:4/694: creat d2/d11/d46/d61/d63/ff8 x:0 0 0 2024-04-12T05:56:12.349 INFO:tasks.workunit.client.1.smithi194.stdout:4/695: chown d2/d3/d51/de9 1886846066 1 2024-04-12T05:56:12.356 INFO:tasks.workunit.client.1.smithi194.stdout:3/628: dread d0/f54 [0,4194304] 0 2024-04-12T05:56:12.358 INFO:tasks.workunit.client.1.smithi194.stdout:3/629: chown d0/d1/d10/d8c/c95 2990 1 2024-04-12T05:56:12.359 INFO:tasks.workunit.client.1.smithi194.stdout:3/630: write d0/d1/d10/f44 [2979815,90908] 0 2024-04-12T05:56:12.363 INFO:tasks.workunit.client.1.smithi194.stdout:4/696: dread d2/d3/d1b/d32/d3a/d3b/f75 [0,4194304] 0 2024-04-12T05:56:12.364 INFO:tasks.workunit.client.1.smithi194.stdout:3/631: link d0/dc/d3d/f94 d0/d1/d74/d39/fd1 0 2024-04-12T05:56:12.366 INFO:tasks.workunit.client.1.smithi194.stdout:4/697: mknod d2/d11/d1e/d2f/d4a/d9c/de7/cf9 0 2024-04-12T05:56:12.369 INFO:tasks.workunit.client.1.smithi194.stdout:3/632: rename d0/dc/db4/cd0 to d0/dc/d1e/d81/cd2 0 2024-04-12T05:56:12.369 INFO:tasks.workunit.client.1.smithi194.stdout:4/698: creat d2/d11/d93/da6/dab/d5d/dae/db2/df6/ffa x:0 0 0 2024-04-12T05:56:12.370 INFO:tasks.workunit.client.1.smithi194.stdout:3/633: readlink d0/d1/d74/d39/d4d/l8a 0 2024-04-12T05:56:12.371 INFO:tasks.workunit.client.1.smithi194.stdout:3/634: chown d0/dc/d1e/d81/f48 10281223 1 2024-04-12T05:56:12.377 INFO:tasks.workunit.client.1.smithi194.stdout:3/635: dwrite d0/d1/d10/f44 [0,4194304] 0 2024-04-12T05:56:12.379 INFO:tasks.workunit.client.1.smithi194.stdout:3/636: mkdir d0/dc/d1e/dd3 0 2024-04-12T05:56:12.383 INFO:tasks.workunit.client.1.smithi194.stdout:3/637: truncate d0/d1/d74/d39/fb0 154968 0 2024-04-12T05:56:12.383 INFO:tasks.workunit.client.1.smithi194.stdout:4/699: dread d2/d3/f1a [0,4194304] 0 2024-04-12T05:56:12.386 INFO:tasks.workunit.client.1.smithi194.stdout:3/638: creat d0/d1/d10/d89/fd4 x:0 0 0 2024-04-12T05:56:12.388 INFO:tasks.workunit.client.1.smithi194.stdout:4/700: mkdir d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb 0 2024-04-12T05:56:12.388 INFO:tasks.workunit.client.1.smithi194.stdout:3/639: dread d0/d1/d4f/fa3 [0,4194304] 0 2024-04-12T05:56:12.390 INFO:tasks.workunit.client.1.smithi194.stdout:3/640: creat d0/dc/d1e/d81/d5b/fd5 x:0 0 0 2024-04-12T05:56:12.392 INFO:tasks.workunit.client.1.smithi194.stdout:3/641: creat d0/dc/d1e/d81/d5b/d64/d79/d87/fd6 x:0 0 0 2024-04-12T05:56:12.395 INFO:tasks.workunit.client.1.smithi194.stdout:4/701: dread d2/d3/f1a [4194304,4194304] 0 2024-04-12T05:56:12.398 INFO:tasks.workunit.client.1.smithi194.stdout:4/702: creat d2/d3/d1b/d32/ffc x:0 0 0 2024-04-12T05:56:12.399 INFO:tasks.workunit.client.1.smithi194.stdout:4/703: readlink d2/d3/d50/da0/de2/lf1 0 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: pgmap v32: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 38 MiB/s rd, 129 MiB/s wr, 366 op/s 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: Standby manager daemon smithi008.ixmnlp restarted 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: Standby manager daemon smithi008.ixmnlp started 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/crt"}]: dispatch 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/key"}]: dispatch 2024-04-12T05:56:12.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:12 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:56:12.405 INFO:tasks.workunit.client.1.smithi194.stdout:3/642: dwrite d0/dc/d1e/d81/d34/fa1 [4194304,4194304] 0 2024-04-12T05:56:12.412 INFO:tasks.workunit.client.1.smithi194.stdout:3/643: dwrite d0/dc/d1e/d81/d5b/d64/dc5/fcc [0,4194304] 0 2024-04-12T05:56:12.421 INFO:tasks.workunit.client.1.smithi194.stdout:3/644: dwrite d0/dc/d3d/dae/faf [0,4194304] 0 2024-04-12T05:56:12.422 INFO:tasks.workunit.client.1.smithi194.stdout:3/645: dread - d0/dc/d1e/d33/fca zero size 2024-04-12T05:56:12.423 INFO:tasks.workunit.client.1.smithi194.stdout:3/646: readlink d0/dc/d1e/d81/d5b/l7d 0 2024-04-12T05:56:12.426 INFO:tasks.workunit.client.1.smithi194.stdout:3/647: getdents d0 0 2024-04-12T05:56:12.428 INFO:tasks.workunit.client.1.smithi194.stdout:3/648: fsync d0/d1/f60 0 2024-04-12T05:56:12.431 INFO:tasks.workunit.client.1.smithi194.stdout:6/543: sync 2024-04-12T05:56:12.431 INFO:tasks.workunit.client.1.smithi194.stdout:9/584: sync 2024-04-12T05:56:12.434 INFO:tasks.workunit.client.1.smithi194.stdout:6/544: write d3/d2f/d4e/d53/f85 [900150,95834] 0 2024-04-12T05:56:12.434 INFO:tasks.workunit.client.1.smithi194.stdout:9/585: chown d1/d19/c34 1227 1 2024-04-12T05:56:12.437 INFO:tasks.workunit.client.1.smithi194.stdout:9/586: chown d1/d19/d1f/d20/d37/d38/f7c 24112 1 2024-04-12T05:56:12.440 INFO:tasks.workunit.client.1.smithi194.stdout:6/545: dread d3/d15/d2b/d48/f74 [0,4194304] 0 2024-04-12T05:56:12.441 INFO:tasks.workunit.client.1.smithi194.stdout:3/649: dwrite d0/dc/d1e/d81/d5b/f86 [0,4194304] 0 2024-04-12T05:56:12.441 INFO:tasks.workunit.client.1.smithi194.stdout:9/587: symlink d1/d19/d1f/d20/d37/d84/lc0 0 2024-04-12T05:56:12.443 INFO:tasks.workunit.client.1.smithi194.stdout:6/546: creat d3/d70/fab x:0 0 0 2024-04-12T05:56:12.444 INFO:tasks.workunit.client.1.smithi194.stdout:3/650: mkdir d0/d1/d4f/dd7 0 2024-04-12T05:56:12.447 INFO:tasks.workunit.client.1.smithi194.stdout:3/651: creat d0/d1/d74/d39/d4d/d63/db5/fd8 x:0 0 0 2024-04-12T05:56:12.449 INFO:tasks.workunit.client.1.smithi194.stdout:9/588: rename d1/d19/d1f/d20/l56 to d1/d21/lc1 0 2024-04-12T05:56:12.450 INFO:tasks.workunit.client.1.smithi194.stdout:3/652: truncate d0/d1/d74/d39/f70 282592 0 2024-04-12T05:56:12.451 INFO:tasks.workunit.client.1.smithi194.stdout:3/653: chown d0/dc/d1e/d33/la6 67 1 2024-04-12T05:56:12.454 INFO:tasks.workunit.client.1.smithi194.stdout:9/589: dread d1/d19/d1f/d20/d37/d38/f96 [0,4194304] 0 2024-04-12T05:56:12.457 INFO:tasks.workunit.client.1.smithi194.stdout:6/547: dread d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:12.458 INFO:tasks.workunit.client.1.smithi194.stdout:9/590: fdatasync d1/f18 0 2024-04-12T05:56:12.459 INFO:tasks.workunit.client.1.smithi194.stdout:6/548: read - d3/d2f/d4e/d53/fa8 zero size 2024-04-12T05:56:12.460 INFO:tasks.workunit.client.1.smithi194.stdout:6/549: chown d3/d15/d2b/d48/d6f/d7a 0 1 2024-04-12T05:56:12.462 INFO:tasks.workunit.client.1.smithi194.stdout:9/591: rmdir d1/d21/d77/db0 0 2024-04-12T05:56:12.464 INFO:tasks.workunit.client.1.smithi194.stdout:6/550: rename d3/d2f/d4e/d68/l8d to d3/d15/d2b/d48/lac 0 2024-04-12T05:56:12.465 INFO:tasks.workunit.client.1.smithi194.stdout:9/592: mkdir d1/d41/dba/dc2 0 2024-04-12T05:56:12.467 INFO:tasks.workunit.client.1.smithi194.stdout:9/593: dread d1/d19/f6c [0,4194304] 0 2024-04-12T05:56:12.471 INFO:tasks.workunit.client.1.smithi194.stdout:9/594: creat d1/d41/dba/dc2/fc3 x:0 0 0 2024-04-12T05:56:12.473 INFO:tasks.workunit.client.1.smithi194.stdout:9/595: mkdir d1/d21/d39/dc4 0 2024-04-12T05:56:12.474 INFO:tasks.workunit.client.1.smithi194.stdout:9/596: chown d1/d21/d77/d98/d9f 1 1 2024-04-12T05:56:12.487 INFO:tasks.workunit.client.1.smithi194.stdout:9/597: dread d1/d4e/f7e [0,4194304] 0 2024-04-12T05:56:12.490 INFO:tasks.workunit.client.1.smithi194.stdout:9/598: creat d1/d21/d49/fc5 x:0 0 0 2024-04-12T05:56:12.490 INFO:tasks.workunit.client.1.smithi194.stdout:9/599: chown d1/d19/d1f 1283420496 1 2024-04-12T05:56:12.491 INFO:tasks.workunit.client.1.smithi194.stdout:9/600: chown d1/d19/d1f/f88 923 1 2024-04-12T05:56:12.500 INFO:tasks.workunit.client.1.smithi194.stdout:9/601: dwrite d1/d19/f83 [0,4194304] 0 2024-04-12T05:56:12.503 INFO:tasks.workunit.client.1.smithi194.stdout:9/602: read d1/d19/d1f/d5f/f85 [3999757,65740] 0 2024-04-12T05:56:12.505 INFO:tasks.workunit.client.1.smithi194.stdout:9/603: creat d1/d19/d1f/d5f/fc6 x:0 0 0 2024-04-12T05:56:12.516 INFO:tasks.workunit.client.1.smithi194.stdout:9/604: dread f0 [0,4194304] 0 2024-04-12T05:56:12.517 INFO:tasks.workunit.client.1.smithi194.stdout:9/605: readlink d1/d19/d1f/d20/d37/d84/lc0 0 2024-04-12T05:56:12.519 INFO:tasks.workunit.client.1.smithi194.stdout:9/606: mkdir d1/d19/d1f/d20/d37/da3/dc7 0 2024-04-12T05:56:12.522 INFO:tasks.workunit.client.1.smithi194.stdout:9/607: rename d1/d19/d1f/d20/fac to d1/d21/d77/d98/fc8 0 2024-04-12T05:56:12.523 INFO:tasks.workunit.client.1.smithi194.stdout:9/608: mkdir d1/d41/dc9 0 2024-04-12T05:56:12.524 INFO:tasks.workunit.client.1.smithi194.stdout:9/609: fdatasync d1/d19/d1f/d5f/f86 0 2024-04-12T05:56:12.526 INFO:tasks.workunit.client.1.smithi194.stdout:9/610: mknod d1/d21/d39/dc4/cca 0 2024-04-12T05:56:12.537 INFO:tasks.workunit.client.1.smithi194.stdout:9/611: dread d1/d19/f54 [0,4194304] 0 2024-04-12T05:56:12.539 INFO:tasks.workunit.client.1.smithi194.stdout:9/612: mkdir d1/d21/d49/dcb 0 2024-04-12T05:56:12.548 INFO:tasks.workunit.client.1.smithi194.stdout:9/613: dwrite d1/d21/d39/d72/fa7 [0,4194304] 0 2024-04-12T05:56:12.553 INFO:tasks.workunit.client.1.smithi194.stdout:9/614: link d1/d19/d1f/l6a d1/d41/d97/d9d/lcc 0 2024-04-12T05:56:12.560 INFO:tasks.workunit.client.1.smithi194.stdout:9/615: dwrite d1/d19/d1f/d20/f8b [0,4194304] 0 2024-04-12T05:56:12.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: pgmap v32: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 38 MiB/s rd, 129 MiB/s wr, 366 op/s 2024-04-12T05:56:12.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: Standby manager daemon smithi008.ixmnlp restarted 2024-04-12T05:56:12.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: Standby manager daemon smithi008.ixmnlp started 2024-04-12T05:56:12.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/crt"}]: dispatch 2024-04-12T05:56:12.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:56:12.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi008.ixmnlp/key"}]: dispatch 2024-04-12T05:56:12.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:12 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:56:12.573 INFO:tasks.workunit.client.1.smithi194.stdout:9/616: dread d1/d41/f6f [0,4194304] 0 2024-04-12T05:56:12.578 INFO:tasks.workunit.client.1.smithi194.stdout:9/617: truncate d1/fe 618798 0 2024-04-12T05:56:12.588 INFO:tasks.workunit.client.1.smithi194.stdout:9/618: fdatasync d1/d21/d39/d72/f75 0 2024-04-12T05:56:12.593 INFO:tasks.workunit.client.1.smithi194.stdout:9/619: dwrite d1/d19/f2e [0,4194304] 0 2024-04-12T05:56:12.596 INFO:tasks.workunit.client.1.smithi194.stdout:9/620: getdents d1/d41/d97/db6 0 2024-04-12T05:56:12.601 INFO:tasks.workunit.client.1.smithi194.stdout:9/621: creat d1/d19/d1f/d20/fcd x:0 0 0 2024-04-12T05:56:12.606 INFO:tasks.workunit.client.1.smithi194.stdout:9/622: link d1/d19/d1f/d20/d37/d38/l40 d1/d41/dbf/lce 0 2024-04-12T05:56:12.608 INFO:tasks.workunit.client.1.smithi194.stdout:9/623: write d1/f5b [2106736,94637] 0 2024-04-12T05:56:12.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/624: mknod d1/d41/dbf/ccf 0 2024-04-12T05:56:12.611 INFO:tasks.workunit.client.1.smithi194.stdout:9/625: write d1/f5b [4885683,9110] 0 2024-04-12T05:56:12.624 INFO:tasks.workunit.client.1.smithi194.stdout:9/626: dread d1/d19/d1f/f52 [0,4194304] 0 2024-04-12T05:56:12.626 INFO:tasks.workunit.client.1.smithi194.stdout:9/627: fsync d1/d41/f44 0 2024-04-12T05:56:12.628 INFO:tasks.workunit.client.1.smithi194.stdout:8/378: sync 2024-04-12T05:56:12.629 INFO:tasks.workunit.client.1.smithi194.stdout:9/628: mknod d1/d41/d97/d9d/cd0 0 2024-04-12T05:56:12.630 INFO:tasks.workunit.client.1.smithi194.stdout:8/379: write d2/d8/d9/de/d10/f41 [1705247,125307] 0 2024-04-12T05:56:12.636 INFO:tasks.workunit.client.1.smithi194.stdout:9/629: dwrite d1/d19/d1f/f73 [0,4194304] 0 2024-04-12T05:56:12.638 INFO:tasks.workunit.client.1.smithi194.stdout:8/380: truncate d2/d8/f26 290305 0 2024-04-12T05:56:12.639 INFO:tasks.workunit.client.1.smithi194.stdout:9/630: symlink d1/d41/dbf/ld1 0 2024-04-12T05:56:12.642 INFO:tasks.workunit.client.1.smithi194.stdout:9/631: unlink d1/f5 0 2024-04-12T05:56:12.643 INFO:tasks.workunit.client.1.smithi194.stdout:9/632: creat d1/d41/dba/fd2 x:0 0 0 2024-04-12T05:56:12.644 INFO:tasks.workunit.client.1.smithi194.stdout:9/633: dread - d1/d21/d49/fc5 zero size 2024-04-12T05:56:12.645 INFO:tasks.workunit.client.1.smithi194.stdout:9/634: fsync d1/d19/d1f/d5f/fc6 0 2024-04-12T05:56:12.646 INFO:tasks.workunit.client.1.smithi194.stdout:9/635: truncate d1/d41/dba/dc2/fc3 1003925 0 2024-04-12T05:56:12.646 INFO:tasks.workunit.client.1.smithi194.stdout:9/636: fdatasync d1/d41/dba/fd2 0 2024-04-12T05:56:12.650 INFO:tasks.workunit.client.1.smithi194.stdout:8/381: write d2/d8/d15/f62 [1437872,52830] 0 2024-04-12T05:56:12.652 INFO:tasks.workunit.client.1.smithi194.stdout:9/637: rename d1/c7 to d1/d41/dba/cd3 0 2024-04-12T05:56:12.652 INFO:tasks.workunit.client.1.smithi194.stdout:1/618: sync 2024-04-12T05:56:12.652 INFO:tasks.workunit.client.1.smithi194.stdout:0/559: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:2/500: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:4/704: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:7/437: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:5/669: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:6/551: sync 2024-04-12T05:56:12.653 INFO:tasks.workunit.client.1.smithi194.stdout:3/654: sync 2024-04-12T05:56:12.654 INFO:tasks.workunit.client.1.smithi194.stdout:6/552: read - d3/d70/fab zero size 2024-04-12T05:56:12.654 INFO:tasks.workunit.client.1.smithi194.stdout:9/638: unlink d1/f46 0 2024-04-12T05:56:12.657 INFO:tasks.workunit.client.1.smithi194.stdout:7/438: mkdir d3/d27/d22/d8e 0 2024-04-12T05:56:12.658 INFO:tasks.workunit.client.1.smithi194.stdout:2/501: write d7/d12/d1a/fad [339862,130653] 0 2024-04-12T05:56:12.660 INFO:tasks.workunit.client.1.smithi194.stdout:8/382: dread d2/fb [0,4194304] 0 2024-04-12T05:56:12.660 INFO:tasks.workunit.client.1.smithi194.stdout:3/655: chown d0/dc/d3d/l3e 61 1 2024-04-12T05:56:12.661 INFO:tasks.workunit.client.1.smithi194.stdout:6/553: mkdir d3/d2f/d4e/d68/dad 0 2024-04-12T05:56:12.664 INFO:tasks.workunit.client.1.smithi194.stdout:4/705: truncate d2/d11/d93/da6/dab/d5d/dae/db2/d57/fa4 1510567 0 2024-04-12T05:56:12.667 INFO:tasks.workunit.client.1.smithi194.stdout:3/656: mkdir d0/dc/db4/dd9 0 2024-04-12T05:56:12.667 INFO:tasks.workunit.client.1.smithi194.stdout:5/670: write d1/d4/f1c [1255520,37331] 0 2024-04-12T05:56:12.668 INFO:tasks.workunit.client.1.smithi194.stdout:3/657: stat d0/d1/f97 0 2024-04-12T05:56:12.669 INFO:tasks.workunit.client.1.smithi194.stdout:4/706: symlink d2/d11/d93/da6/dab/d5d/dae/db2/df6/lfd 0 2024-04-12T05:56:12.669 INFO:tasks.workunit.client.1.smithi194.stdout:3/658: dread - d0/dc/d1e/d81/d5b/fd5 zero size 2024-04-12T05:56:12.670 INFO:tasks.workunit.client.1.smithi194.stdout:2/502: readlink d7/d22/d42/l5f 0 2024-04-12T05:56:12.672 INFO:tasks.workunit.client.1.smithi194.stdout:0/560: link d0/da/d1e/d1a/f44 d0/d3b/fb5 0 2024-04-12T05:56:12.674 INFO:tasks.workunit.client.1.smithi194.stdout:1/619: rename d0/d11/d18/d19/d1a/d6a/d27/d9e/fb6 to d0/d11/d18/fc3 0 2024-04-12T05:56:12.675 INFO:tasks.workunit.client.1.smithi194.stdout:9/639: creat d1/fd4 x:0 0 0 2024-04-12T05:56:12.677 INFO:tasks.workunit.client.1.smithi194.stdout:4/707: symlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/lfe 0 2024-04-12T05:56:12.678 INFO:tasks.workunit.client.1.smithi194.stdout:2/503: dread d7/d12/f3b [0,4194304] 0 2024-04-12T05:56:12.678 INFO:tasks.workunit.client.1.smithi194.stdout:5/671: creat d1/d4/d84/fe5 x:0 0 0 2024-04-12T05:56:12.681 INFO:tasks.workunit.client.1.smithi194.stdout:7/439: dwrite d3/d27/f17 [0,4194304] 0 2024-04-12T05:56:12.682 INFO:tasks.workunit.client.1.smithi194.stdout:3/659: mknod d0/d1/d74/d39/d4d/d63/db5/cda 0 2024-04-12T05:56:12.684 INFO:tasks.workunit.client.1.smithi194.stdout:5/672: dread - d1/d5c/d95/fc3 zero size 2024-04-12T05:56:12.685 INFO:tasks.workunit.client.1.smithi194.stdout:2/504: chown d7/d12/d34/d4a/d8a 812 1 2024-04-12T05:56:12.685 INFO:tasks.workunit.client.1.smithi194.stdout:4/708: fsync d2/d11/d46/d61/d63/f84 0 2024-04-12T05:56:12.686 INFO:tasks.workunit.client.1.smithi194.stdout:7/440: write d3/d27/d22/f2e [4837,933] 0 2024-04-12T05:56:12.687 INFO:tasks.workunit.client.1.smithi194.stdout:5/673: stat d1/d5c/d95/f72 0 2024-04-12T05:56:12.689 INFO:tasks.workunit.client.1.smithi194.stdout:2/505: write d7/d12/d25/d70/fa1 [407950,123853] 0 2024-04-12T05:56:12.691 INFO:tasks.workunit.client.1.smithi194.stdout:9/640: creat d1/d41/dc9/fd5 x:0 0 0 2024-04-12T05:56:12.693 INFO:tasks.workunit.client.1.smithi194.stdout:1/620: dread d0/d11/f9f [0,4194304] 0 2024-04-12T05:56:12.694 INFO:tasks.workunit.client.1.smithi194.stdout:4/709: rename d2/d3/d1b/c1d to d2/d11/d93/da6/dab/d5d/dae/db2/df6/cff 0 2024-04-12T05:56:12.694 INFO:tasks.workunit.client.1.smithi194.stdout:6/554: link d3/d2d/d33/d52/d5c/c88 d3/d2d/d33/cae 0 2024-04-12T05:56:12.697 INFO:tasks.workunit.client.1.smithi194.stdout:5/674: write d1/d5c/d95/f9f [9035256,129697] 0 2024-04-12T05:56:12.700 INFO:tasks.workunit.client.1.smithi194.stdout:3/660: dread d0/f27 [0,4194304] 0 2024-04-12T05:56:12.700 INFO:tasks.workunit.client.1.smithi194.stdout:9/641: mknod d1/d41/dba/cd6 0 2024-04-12T05:56:12.700 INFO:tasks.workunit.client.1.smithi194.stdout:7/441: mkdir d3/d16/d23/d57/d72/d81/d3a/d8f 0 2024-04-12T05:56:12.703 INFO:tasks.workunit.client.1.smithi194.stdout:6/555: creat d3/d2d/d33/d58/faf x:0 0 0 2024-04-12T05:56:12.706 INFO:tasks.workunit.client.1.smithi194.stdout:4/710: write d2/faf [5071565,123386] 0 2024-04-12T05:56:12.707 INFO:tasks.workunit.client.1.smithi194.stdout:5/675: symlink d1/d4/d84/le6 0 2024-04-12T05:56:12.709 INFO:tasks.workunit.client.1.smithi194.stdout:9/642: write d1/d21/d39/d72/fb5 [947777,61904] 0 2024-04-12T05:56:12.709 INFO:tasks.workunit.client.1.smithi194.stdout:8/383: dwrite d2/d8/f36 [0,4194304] 0 2024-04-12T05:56:12.711 INFO:tasks.workunit.client.1.smithi194.stdout:5/676: write d1/d4/d84/f99 [954917,129548] 0 2024-04-12T05:56:12.712 INFO:tasks.workunit.client.1.smithi194.stdout:6/556: truncate d3/f3a 1471717 0 2024-04-12T05:56:12.714 INFO:tasks.workunit.client.1.smithi194.stdout:2/506: write d7/d12/d6d/f72 [1140838,78271] 0 2024-04-12T05:56:12.714 INFO:tasks.workunit.client.1.smithi194.stdout:1/621: getdents d0/d11/d18/d35/d6f 0 2024-04-12T05:56:12.715 INFO:tasks.workunit.client.1.smithi194.stdout:9/643: write d1/d41/dba/dc2/fc3 [993915,56261] 0 2024-04-12T05:56:12.716 INFO:tasks.workunit.client.1.smithi194.stdout:6/557: rmdir d3/d2f/d4e 39 2024-04-12T05:56:12.718 INFO:tasks.workunit.client.1.smithi194.stdout:5/677: symlink d1/d4/dd7/le7 0 2024-04-12T05:56:12.718 INFO:tasks.workunit.client.1.smithi194.stdout:7/442: dread d3/d16/d23/d40/f68 [0,4194304] 0 2024-04-12T05:56:12.719 INFO:tasks.workunit.client.1.smithi194.stdout:4/711: creat d2/d3/d1b/d32/d3a/d3b/f100 x:0 0 0 2024-04-12T05:56:12.722 INFO:tasks.workunit.client.1.smithi194.stdout:2/507: creat d7/d12/d34/d4a/d8a/faf x:0 0 0 2024-04-12T05:56:12.722 INFO:tasks.workunit.client.1.smithi194.stdout:8/384: rmdir d2/d8/d15/d3b 39 2024-04-12T05:56:12.724 INFO:tasks.workunit.client.1.smithi194.stdout:7/443: fdatasync d3/d27/d22/f2e 0 2024-04-12T05:56:12.729 INFO:tasks.workunit.client.1.smithi194.stdout:3/661: dwrite d0/d1/f57 [0,4194304] 0 2024-04-12T05:56:12.729 INFO:tasks.workunit.client.1.smithi194.stdout:2/508: write d7/d12/d25/f31 [2737590,25722] 0 2024-04-12T05:56:12.729 INFO:tasks.workunit.client.1.smithi194.stdout:4/712: truncate d2/d85/dc2/fca 418465 0 2024-04-12T05:56:12.729 INFO:tasks.workunit.client.1.smithi194.stdout:1/622: symlink d0/d11/d18/d19/d1a/d6a/d27/lc4 0 2024-04-12T05:56:12.733 INFO:tasks.workunit.client.1.smithi194.stdout:7/444: mknod d3/d16/c90 0 2024-04-12T05:56:12.733 INFO:tasks.workunit.client.1.smithi194.stdout:8/385: creat d2/d8/d9/de/d10/d4e/f73 x:0 0 0 2024-04-12T05:56:12.735 INFO:tasks.workunit.client.1.smithi194.stdout:9/644: rmdir d1/d19/d1f/d20/d7d/dbe 0 2024-04-12T05:56:12.736 INFO:tasks.workunit.client.1.smithi194.stdout:4/713: creat d2/d11/d46/d61/d63/f101 x:0 0 0 2024-04-12T05:56:12.737 INFO:tasks.workunit.client.1.smithi194.stdout:5/678: link d1/d4/d6/db/d38/fc0 d1/d4/d75/fe8 0 2024-04-12T05:56:12.741 INFO:tasks.workunit.client.1.smithi194.stdout:7/445: chown d3/d16/d23/d40/c59 12439 1 2024-04-12T05:56:12.741 INFO:tasks.workunit.client.1.smithi194.stdout:8/386: write d2/d8/d9/de/d10/f41 [1148000,121505] 0 2024-04-12T05:56:12.742 INFO:tasks.workunit.client.1.smithi194.stdout:4/714: mknod d2/d3/d51/c102 0 2024-04-12T05:56:12.745 INFO:tasks.workunit.client.1.smithi194.stdout:1/623: rename d0/d11/d18/d19/d1a/d6a/dac/db5/d8d to d0/d11/d18/d35/d6f/dc5 0 2024-04-12T05:56:12.751 INFO:tasks.workunit.client.1.smithi194.stdout:2/509: dwrite d7/d22/f87 [0,4194304] 0 2024-04-12T05:56:12.751 INFO:tasks.workunit.client.1.smithi194.stdout:4/715: readlink d2/d11/d93/da6/dab/l2e 0 2024-04-12T05:56:12.751 INFO:tasks.workunit.client.1.smithi194.stdout:8/387: creat d2/d8/d66/d67/f74 x:0 0 0 2024-04-12T05:56:12.755 INFO:tasks.workunit.client.1.smithi194.stdout:5/679: link d1/d4/d6/db/f53 d1/d4/d6/d88/dc1/fe9 0 2024-04-12T05:56:12.755 INFO:tasks.workunit.client.1.smithi194.stdout:6/558: dread d3/d2f/d4e/d63/f78 [0,4194304] 0 2024-04-12T05:56:12.757 INFO:tasks.workunit.client.1.smithi194.stdout:3/662: dwrite d0/dc/d1e/d33/d76/d9a/fc1 [0,4194304] 0 2024-04-12T05:56:12.757 INFO:tasks.workunit.client.1.smithi194.stdout:4/716: mkdir d2/d103 0 2024-04-12T05:56:12.758 INFO:tasks.workunit.client.1.smithi194.stdout:8/388: symlink d2/d8/d58/l75 0 2024-04-12T05:56:12.759 INFO:tasks.workunit.client.1.smithi194.stdout:1/624: chown d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/c6e 6383 1 2024-04-12T05:56:12.762 INFO:tasks.workunit.client.1.smithi194.stdout:7/446: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:12.765 INFO:tasks.workunit.client.1.smithi194.stdout:2/510: creat d7/d22/d93/d9f/fb0 x:0 0 0 2024-04-12T05:56:12.766 INFO:tasks.workunit.client.1.smithi194.stdout:6/559: mknod d3/d2f/d4e/d56/cb0 0 2024-04-12T05:56:12.769 INFO:tasks.workunit.client.1.smithi194.stdout:1/625: stat d0/d11/d18/d19/d1a/d6a/d27/da8/c4b 0 2024-04-12T05:56:12.769 INFO:tasks.workunit.client.1.smithi194.stdout:9/645: dread d1/d19/d1f/d20/f71 [0,4194304] 0 2024-04-12T05:56:12.769 INFO:tasks.workunit.client.1.smithi194.stdout:2/511: fsync d7/d12/d6d/f72 0 2024-04-12T05:56:12.775 INFO:tasks.workunit.client.1.smithi194.stdout:9/646: write d1/d19/d1f/d5f/f9e [7246,30509] 0 2024-04-12T05:56:12.775 INFO:tasks.workunit.client.1.smithi194.stdout:7/447: creat d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f91 x:0 0 0 2024-04-12T05:56:12.779 INFO:tasks.workunit.client.1.smithi194.stdout:5/680: dwrite d1/d4/d6/db/d2b/fcd [0,4194304] 0 2024-04-12T05:56:12.780 INFO:tasks.workunit.client.1.smithi194.stdout:7/448: write d3/d27/f7f [203646,130074] 0 2024-04-12T05:56:12.788 INFO:tasks.workunit.client.1.smithi194.stdout:6/560: dread d3/f3a [0,4194304] 0 2024-04-12T05:56:12.788 INFO:tasks.workunit.client.1.smithi194.stdout:1/626: unlink d0/d11/d18/d19/d1a/d6a/d27/d48/l4a 0 2024-04-12T05:56:12.788 INFO:tasks.workunit.client.1.smithi194.stdout:3/663: write d0/dc/d1e/d81/d5b/f8b [403561,21445] 0 2024-04-12T05:56:12.789 INFO:tasks.workunit.client.1.smithi194.stdout:1/627: rename d0/d11/c77 to d0/d11/d18/d35/d6f/dc5/db9/cc6 0 2024-04-12T05:56:12.790 INFO:tasks.workunit.client.1.smithi194.stdout:5/681: unlink d1/d4/d6/db/d2b/l70 0 2024-04-12T05:56:12.792 INFO:tasks.workunit.client.1.smithi194.stdout:5/682: write d1/d4/d6/fc8 [290415,67904] 0 2024-04-12T05:56:12.793 INFO:tasks.workunit.client.1.smithi194.stdout:7/449: truncate d3/d27/d22/f2e 947629 0 2024-04-12T05:56:12.797 INFO:tasks.workunit.client.1.smithi194.stdout:3/664: mkdir d0/d1/d10/d8c/d8f/ddb 0 2024-04-12T05:56:12.797 INFO:tasks.workunit.client.1.smithi194.stdout:1/628: unlink d0/f12 0 2024-04-12T05:56:12.798 INFO:tasks.workunit.client.1.smithi194.stdout:5/683: symlink d1/d4/d4b/lea 0 2024-04-12T05:56:12.798 INFO:tasks.workunit.client.1.smithi194.stdout:8/389: dwrite d2/d8/d15/d35/f4c [0,4194304] 0 2024-04-12T05:56:12.801 INFO:tasks.workunit.client.1.smithi194.stdout:2/512: truncate d7/d12/d1a/f3e 1913186 0 2024-04-12T05:56:12.802 INFO:tasks.workunit.client.1.smithi194.stdout:4/717: dread d2/f6 [0,4194304] 0 2024-04-12T05:56:12.810 INFO:tasks.workunit.client.1.smithi194.stdout:8/390: write d2/d8/d9/de/d10/d4e/f73 [765903,96058] 0 2024-04-12T05:56:12.810 INFO:tasks.workunit.client.1.smithi194.stdout:3/665: unlink d0/dc/d1e/d81/d5b/d64/fc3 0 2024-04-12T05:56:12.813 INFO:tasks.workunit.client.1.smithi194.stdout:3/666: chown d0/d1 59 1 2024-04-12T05:56:12.816 INFO:tasks.workunit.client.1.smithi194.stdout:6/561: dwrite d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:12.824 INFO:tasks.workunit.client.1.smithi194.stdout:4/718: dwrite d2/d3/d1b/d32/fdd [0,4194304] 0 2024-04-12T05:56:12.827 INFO:tasks.workunit.client.1.smithi194.stdout:6/562: mkdir d3/d2d/d33/db1 0 2024-04-12T05:56:12.827 INFO:tasks.workunit.client.1.smithi194.stdout:8/391: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:12.836 INFO:tasks.workunit.client.1.smithi194.stdout:1/629: dread - d0/d11/d18/d19/d1a/d6a/d27/fab zero size 2024-04-12T05:56:12.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/392: creat d2/d8/d58/f76 x:0 0 0 2024-04-12T05:56:12.837 INFO:tasks.workunit.client.1.smithi194.stdout:3/667: dread d0/dc/d1e/d81/d34/d2d/f3a [0,4194304] 0 2024-04-12T05:56:12.838 INFO:tasks.workunit.client.1.smithi194.stdout:6/563: mknod d3/d2f/d4e/d63/d83/cb2 0 2024-04-12T05:56:12.838 INFO:tasks.workunit.client.1.smithi194.stdout:2/513: getdents d7/d12/d1a/d69/d7e 0 2024-04-12T05:56:12.839 INFO:tasks.workunit.client.1.smithi194.stdout:5/684: link d1/d4/d6/db/d2b/f66 d1/d4/d6/db/d15/d26/feb 0 2024-04-12T05:56:12.840 INFO:tasks.workunit.client.1.smithi194.stdout:7/450: dread d3/d27/f39 [0,4194304] 0 2024-04-12T05:56:12.840 INFO:tasks.workunit.client.1.smithi194.stdout:4/719: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/d57/fe5 [0,4194304] 0 2024-04-12T05:56:12.845 INFO:tasks.workunit.client.1.smithi194.stdout:4/720: fsync d2/d3/f77 0 2024-04-12T05:56:12.846 INFO:tasks.workunit.client.1.smithi194.stdout:3/668: creat d0/d1/d10/d8c/fdc x:0 0 0 2024-04-12T05:56:12.846 INFO:tasks.workunit.client.1.smithi194.stdout:2/514: dread - d7/d12/d1a/d68/f8c zero size 2024-04-12T05:56:12.850 INFO:tasks.workunit.client.1.smithi194.stdout:1/630: mknod d0/d11/d18/d19/d1a/d6a/d27/d6c/da0/cc7 0 2024-04-12T05:56:12.855 INFO:tasks.workunit.client.1.smithi194.stdout:6/564: write d3/d2d/d33/d52/d5c/f5f [1130496,11929] 0 2024-04-12T05:56:12.861 INFO:tasks.workunit.client.1.smithi194.stdout:5/685: dwrite d1/d4/d75/fb6 [0,4194304] 0 2024-04-12T05:56:12.863 INFO:tasks.workunit.client.1.smithi194.stdout:2/515: dread d7/d12/d25/d70/fa1 [0,4194304] 0 2024-04-12T05:56:12.863 INFO:tasks.workunit.client.1.smithi194.stdout:7/451: dwrite d3/d16/d23/d37/d3d/f61 [0,4194304] 0 2024-04-12T05:56:12.865 INFO:tasks.workunit.client.1.smithi194.stdout:1/631: fdatasync d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fb1 0 2024-04-12T05:56:12.867 INFO:tasks.workunit.client.1.smithi194.stdout:2/516: fdatasync d7/d12/d25/d5d/d47/d52/f60 0 2024-04-12T05:56:12.876 INFO:tasks.workunit.client.1.smithi194.stdout:6/565: dwrite d3/d15/f9f [0,4194304] 0 2024-04-12T05:56:12.876 INFO:tasks.workunit.client.1.smithi194.stdout:7/452: creat d3/d16/d23/d57/d72/d81/f92 x:0 0 0 2024-04-12T05:56:12.876 INFO:tasks.workunit.client.1.smithi194.stdout:2/517: truncate d7/fa 897289 0 2024-04-12T05:56:12.883 INFO:tasks.workunit.client.1.smithi194.stdout:5/686: creat d1/d4/d6/db/d3b/fec x:0 0 0 2024-04-12T05:56:12.884 INFO:tasks.workunit.client.1.smithi194.stdout:2/518: truncate d7/d22/d42/f2a 3370877 0 2024-04-12T05:56:12.886 INFO:tasks.workunit.client.1.smithi194.stdout:5/687: stat d1/d4/d9e/lb0 0 2024-04-12T05:56:12.888 INFO:tasks.workunit.client.1.smithi194.stdout:2/519: creat d7/d12/d1a/d69/fb1 x:0 0 0 2024-04-12T05:56:12.891 INFO:tasks.workunit.client.1.smithi194.stdout:7/453: dwrite d3/d16/d23/d40/d49/f71 [0,4194304] 0 2024-04-12T05:56:12.892 INFO:tasks.workunit.client.1.smithi194.stdout:2/520: write d7/d12/d25/f31 [1423874,21009] 0 2024-04-12T05:56:12.895 INFO:tasks.workunit.client.1.smithi194.stdout:4/721: dread d2/d11/d93/da6/dab/d5d/f7c [4194304,4194304] 0 2024-04-12T05:56:12.897 INFO:tasks.workunit.client.1.smithi194.stdout:7/454: dread d3/d16/d23/d57/d72/d81/f73 [0,4194304] 0 2024-04-12T05:56:12.899 INFO:tasks.workunit.client.1.smithi194.stdout:5/688: dread d1/d4/d6/db/d38/f5a [0,4194304] 0 2024-04-12T05:56:12.908 INFO:tasks.workunit.client.1.smithi194.stdout:7/455: dwrite d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:12.909 INFO:tasks.workunit.client.1.smithi194.stdout:4/722: truncate d2/d3/d51/f55 2362796 0 2024-04-12T05:56:12.918 INFO:tasks.workunit.client.1.smithi194.stdout:5/689: dwrite d1/d4/d6/d34/f54 [0,4194304] 0 2024-04-12T05:56:12.921 INFO:tasks.workunit.client.1.smithi194.stdout:4/723: write d2/d11/d46/d61/d63/f101 [811446,35875] 0 2024-04-12T05:56:12.921 INFO:tasks.workunit.client.1.smithi194.stdout:2/521: dread d7/d12/d25/d5d/f2f [0,4194304] 0 2024-04-12T05:56:12.924 INFO:tasks.workunit.client.1.smithi194.stdout:6/566: dwrite d3/f3a [0,4194304] 0 2024-04-12T05:56:12.924 INFO:tasks.workunit.client.1.smithi194.stdout:7/456: unlink d3/d16/d23/d57/d72/d81/l78 0 2024-04-12T05:56:12.925 INFO:tasks.workunit.client.1.smithi194.stdout:7/457: chown d3/d16/d23/d57 403204 1 2024-04-12T05:56:12.930 INFO:tasks.workunit.client.1.smithi194.stdout:4/724: rmdir d2/d85 39 2024-04-12T05:56:12.933 INFO:tasks.workunit.client.1.smithi194.stdout:2/522: mkdir d7/d12/d25/d5d/d47/d52/d7b/db2 0 2024-04-12T05:56:12.933 INFO:tasks.workunit.client.1.smithi194.stdout:6/567: mkdir d3/d2f/d4e/d9a/db3 0 2024-04-12T05:56:12.933 INFO:tasks.workunit.client.1.smithi194.stdout:1/632: dread d0/f85 [4194304,4194304] 0 2024-04-12T05:56:12.939 INFO:tasks.workunit.client.1.smithi194.stdout:2/523: mkdir d7/d12/d25/d5d/d47/dac/db3 0 2024-04-12T05:56:12.939 INFO:tasks.workunit.client.1.smithi194.stdout:5/690: dread d1/d4/d6/f76 [0,4194304] 0 2024-04-12T05:56:12.941 INFO:tasks.workunit.client.1.smithi194.stdout:6/568: link d3/d2f/d4e/d63/d83/cb2 d3/d2d/d33/db1/cb4 0 2024-04-12T05:56:12.945 INFO:tasks.workunit.client.1.smithi194.stdout:2/524: write d7/d12/d25/d5d/d47/d52/f9b [1746766,125022] 0 2024-04-12T05:56:12.945 INFO:tasks.workunit.client.1.smithi194.stdout:5/691: rmdir d1/d4/d6/db/d15/d8c 39 2024-04-12T05:56:12.948 INFO:tasks.workunit.client.1.smithi194.stdout:6/569: creat d3/d15/d2b/d48/d6f/d7a/d8f/fb5 x:0 0 0 2024-04-12T05:56:12.948 INFO:tasks.workunit.client.1.smithi194.stdout:1/633: dwrite d0/d11/d18/d35/d6f/dc5/db9/f8a [0,4194304] 0 2024-04-12T05:56:12.950 INFO:tasks.workunit.client.1.smithi194.stdout:5/692: rmdir d1/d4/d4b 39 2024-04-12T05:56:12.953 INFO:tasks.workunit.client.1.smithi194.stdout:2/525: creat d7/d12/d25/d70/fb4 x:0 0 0 2024-04-12T05:56:12.958 INFO:tasks.workunit.client.1.smithi194.stdout:4/725: dwrite d2/d3/d1b/d32/d3a/f3e [0,4194304] 0 2024-04-12T05:56:12.959 INFO:tasks.workunit.client.1.smithi194.stdout:7/458: dwrite d3/d27/f69 [0,4194304] 0 2024-04-12T05:56:12.959 INFO:tasks.workunit.client.1.smithi194.stdout:2/526: write d7/d12/d25/d70/fa1 [292173,38550] 0 2024-04-12T05:56:12.962 INFO:tasks.workunit.client.1.smithi194.stdout:7/459: write d3/d27/f69 [421379,99892] 0 2024-04-12T05:56:12.962 INFO:tasks.workunit.client.1.smithi194.stdout:2/527: chown d7/d22/f94 84599 1 2024-04-12T05:56:12.964 INFO:tasks.workunit.client.1.smithi194.stdout:2/528: chown d7/d12/d1a/d68 77257 1 2024-04-12T05:56:12.966 INFO:tasks.workunit.client.1.smithi194.stdout:4/726: symlink d2/d11/d93/da6/dab/d5d/dae/de1/l104 0 2024-04-12T05:56:12.971 INFO:tasks.workunit.client.1.smithi194.stdout:2/529: rename d7/d12/d25/d5d/d47/f57 to d7/d22/d93/d9f/fb5 0 2024-04-12T05:56:12.972 INFO:tasks.workunit.client.1.smithi194.stdout:4/727: creat d2/d3/f105 x:0 0 0 2024-04-12T05:56:12.974 INFO:tasks.workunit.client.1.smithi194.stdout:0/561: sync 2024-04-12T05:56:12.974 INFO:tasks.workunit.client.1.smithi194.stdout:5/693: dread d1/d4/d6/db/f19 [4194304,4194304] 0 2024-04-12T05:56:12.976 INFO:tasks.workunit.client.1.smithi194.stdout:7/460: dwrite d3/d16/d23/d37/d3d/d56/f87 [0,4194304] 0 2024-04-12T05:56:12.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/530: truncate d7/d12/fa2 35026 0 2024-04-12T05:56:12.982 INFO:tasks.workunit.client.1.smithi194.stdout:2/531: write d7/d12/d25/d5d/d47/f88 [934637,123434] 0 2024-04-12T05:56:12.983 INFO:tasks.workunit.client.1.smithi194.stdout:4/728: creat d2/d85/f106 x:0 0 0 2024-04-12T05:56:12.985 INFO:tasks.workunit.client.1.smithi194.stdout:5/694: symlink d1/d4/d6/d34/led 0 2024-04-12T05:56:12.985 INFO:tasks.workunit.client.1.smithi194.stdout:2/532: read d7/d12/d25/d5d/f2f [3830468,99413] 0 2024-04-12T05:56:12.987 INFO:tasks.workunit.client.1.smithi194.stdout:1/634: dread d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 [0,4194304] 0 2024-04-12T05:56:12.989 INFO:tasks.workunit.client.1.smithi194.stdout:2/533: write d7/d12/d25/d70/fa1 [625897,10396] 0 2024-04-12T05:56:12.990 INFO:tasks.workunit.client.1.smithi194.stdout:4/729: unlink d2/d11/d93/da6/dab/d5d/dae/lcb 0 2024-04-12T05:56:12.991 INFO:tasks.workunit.client.1.smithi194.stdout:2/534: stat d7/d12/d25/d70/fb4 0 2024-04-12T05:56:12.992 INFO:tasks.workunit.client.1.smithi194.stdout:4/730: chown d2/d11/d1e/d25/f7a 234 1 2024-04-12T05:56:12.993 INFO:tasks.workunit.client.1.smithi194.stdout:1/635: dread - d0/d11/d18/d19/d1a/d6a/d27/d9e/fad zero size 2024-04-12T05:56:12.996 INFO:tasks.workunit.client.1.smithi194.stdout:4/731: write d2/d3/d58/d82/dd0/fe3 [971055,40636] 0 2024-04-12T05:56:12.996 INFO:tasks.workunit.client.1.smithi194.stdout:7/461: link d3/d16/d23/d37/l3f d3/d16/d23/d57/d72/d81/d3a/d8f/l93 0 2024-04-12T05:56:12.997 INFO:tasks.workunit.client.1.smithi194.stdout:1/636: mknod d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/cc8 0 2024-04-12T05:56:12.997 INFO:tasks.workunit.client.1.smithi194.stdout:2/535: read f5 [4625081,11504] 0 2024-04-12T05:56:12.997 INFO:tasks.workunit.client.1.smithi194.stdout:4/732: chown d2/d3/d1b/l29 30561531 1 2024-04-12T05:56:12.998 INFO:tasks.workunit.client.1.smithi194.stdout:2/536: dread - d7/d22/d93/d9f/fb0 zero size 2024-04-12T05:56:13.000 INFO:tasks.workunit.client.1.smithi194.stdout:4/733: symlink d2/d11/d1f/l107 0 2024-04-12T05:56:13.000 INFO:tasks.workunit.client.1.smithi194.stdout:1/637: symlink d0/d11/d18/d35/lc9 0 2024-04-12T05:56:13.001 INFO:tasks.workunit.client.1.smithi194.stdout:2/537: unlink d7/c45 0 2024-04-12T05:56:13.001 INFO:tasks.workunit.client.1.smithi194.stdout:7/462: read d3/d27/f7 [3731362,110371] 0 2024-04-12T05:56:13.002 INFO:tasks.workunit.client.1.smithi194.stdout:4/734: mkdir d2/d11/d46/d61/d108 0 2024-04-12T05:56:13.006 INFO:tasks.workunit.client.1.smithi194.stdout:7/463: write d3/d16/d23/d37/d3d/d56/f51 [1717711,41638] 0 2024-04-12T05:56:13.006 INFO:tasks.workunit.client.1.smithi194.stdout:4/735: mkdir d2/d11/d93/da6/dab/d5d/dae/de1/d109 0 2024-04-12T05:56:13.008 INFO:tasks.workunit.client.1.smithi194.stdout:2/538: getdents d7/d22/d93 0 2024-04-12T05:56:13.008 INFO:tasks.workunit.client.1.smithi194.stdout:1/638: dread d0/f32 [4194304,4194304] 0 2024-04-12T05:56:13.010 INFO:tasks.workunit.client.1.smithi194.stdout:0/562: truncate d0/da/d1e/f32 1905611 0 2024-04-12T05:56:13.010 INFO:tasks.workunit.client.1.smithi194.stdout:5/695: write d1/d4/d6/db/f4a [3252192,72814] 0 2024-04-12T05:56:13.013 INFO:tasks.workunit.client.1.smithi194.stdout:5/696: fsync d1/d5c/d95/fc6 0 2024-04-12T05:56:13.016 INFO:tasks.workunit.client.1.smithi194.stdout:5/697: chown d1/d4/d75/c79 0 1 2024-04-12T05:56:13.017 INFO:tasks.workunit.client.1.smithi194.stdout:1/639: rename d0/d11/d18/d19/d1a/d6a/dac/db5/d68/d96 to d0/d11/d18/d19/d1c/dca 0 2024-04-12T05:56:13.019 INFO:tasks.workunit.client.1.smithi194.stdout:4/736: dread d2/d11/d93/da6/dab/d5d/dae/db2/d57/fe5 [0,4194304] 0 2024-04-12T05:56:13.020 INFO:tasks.workunit.client.1.smithi194.stdout:5/698: creat d1/d4/d75/fee x:0 0 0 2024-04-12T05:56:13.020 INFO:tasks.workunit.client.1.smithi194.stdout:2/539: dread d7/d12/d25/f31 [0,4194304] 0 2024-04-12T05:56:13.021 INFO:tasks.workunit.client.1.smithi194.stdout:7/464: dwrite d3/d27/f7 [0,4194304] 0 2024-04-12T05:56:13.023 INFO:tasks.workunit.client.1.smithi194.stdout:2/540: dread - d7/d12/f8d zero size 2024-04-12T05:56:13.023 INFO:tasks.workunit.client.1.smithi194.stdout:5/699: symlink d1/d4/d84/lef 0 2024-04-12T05:56:13.026 INFO:tasks.workunit.client.1.smithi194.stdout:7/465: write d3/d27/d22/f77 [955336,123557] 0 2024-04-12T05:56:13.026 INFO:tasks.workunit.client.1.smithi194.stdout:2/541: write d7/d22/f5c [8009900,84331] 0 2024-04-12T05:56:13.026 INFO:tasks.workunit.client.1.smithi194.stdout:5/700: readlink d1/d4/d6/d34/l83 0 2024-04-12T05:56:13.027 INFO:tasks.workunit.client.1.smithi194.stdout:2/542: read - d7/d12/f8d zero size 2024-04-12T05:56:13.029 INFO:tasks.workunit.client.1.smithi194.stdout:4/737: rename d2/d3/d1b/d32/db3/db4/lb6 to d2/d3/d1b/d32/d3a/d3b/l10a 0 2024-04-12T05:56:13.029 INFO:tasks.workunit.client.1.smithi194.stdout:2/543: dread - d7/d12/d25/d5d/d47/d52/fa0 zero size 2024-04-12T05:56:13.030 INFO:tasks.workunit.client.1.smithi194.stdout:5/701: rmdir d1/d4/dd7 39 2024-04-12T05:56:13.033 INFO:tasks.workunit.client.1.smithi194.stdout:7/466: symlink d3/d27/l94 0 2024-04-12T05:56:13.034 INFO:tasks.workunit.client.1.smithi194.stdout:5/702: creat d1/d4/d6/db/d38/ff0 x:0 0 0 2024-04-12T05:56:13.034 INFO:tasks.workunit.client.1.smithi194.stdout:7/467: chown d3/d16/d23/d40/d6f 26548 1 2024-04-12T05:56:13.035 INFO:tasks.workunit.client.1.smithi194.stdout:7/468: stat d3/d16/d23/d40/c59 0 2024-04-12T05:56:13.037 INFO:tasks.workunit.client.1.smithi194.stdout:2/544: link d7/d12/d25/d5d/d47/f88 d7/d12/d34/fb6 0 2024-04-12T05:56:13.038 INFO:tasks.workunit.client.1.smithi194.stdout:5/703: chown d1/d4/d6/db/d15/d8c 458 1 2024-04-12T05:56:13.042 INFO:tasks.workunit.client.1.smithi194.stdout:4/738: getdents d2/d3 0 2024-04-12T05:56:13.043 INFO:tasks.workunit.client.1.smithi194.stdout:2/545: fsync d7/d12/d25/d5d/d47/d52/f55 0 2024-04-12T05:56:13.045 INFO:tasks.workunit.client.1.smithi194.stdout:7/469: dread d3/d16/f45 [0,4194304] 0 2024-04-12T05:56:13.047 INFO:tasks.workunit.client.1.smithi194.stdout:7/470: stat d3/d16/d23/d40/d49/c86 0 2024-04-12T05:56:13.048 INFO:tasks.workunit.client.1.smithi194.stdout:1/640: dread d0/d11/d18/d19/f4e [0,4194304] 0 2024-04-12T05:56:13.049 INFO:tasks.workunit.client.1.smithi194.stdout:2/546: creat d7/d22/d42/fb7 x:0 0 0 2024-04-12T05:56:13.052 INFO:tasks.workunit.client.1.smithi194.stdout:7/471: write d3/d16/d5d/f84 [4899788,52571] 0 2024-04-12T05:56:13.056 INFO:tasks.workunit.client.1.smithi194.stdout:1/641: symlink d0/d11/d18/d35/lcb 0 2024-04-12T05:56:13.058 INFO:tasks.workunit.client.1.smithi194.stdout:5/704: dwrite d1/d4/d6/d34/d52/fcf [0,4194304] 0 2024-04-12T05:56:13.061 INFO:tasks.workunit.client.1.smithi194.stdout:2/547: rmdir d7/d12/d34/d4a 39 2024-04-12T05:56:13.061 INFO:tasks.workunit.client.1.smithi194.stdout:5/705: fdatasync d1/d4/d6/db/d3b/fec 0 2024-04-12T05:56:13.063 INFO:tasks.workunit.client.1.smithi194.stdout:4/739: dwrite d2/d11/d46/d61/f92 [0,4194304] 0 2024-04-12T05:56:13.064 INFO:tasks.workunit.client.1.smithi194.stdout:7/472: rename d3/d16/d5d/d8c to d3/d16/d23/d37/d95 0 2024-04-12T05:56:13.070 INFO:tasks.workunit.client.1.smithi194.stdout:4/740: chown d2/d11/d93/da6/dab/l2e 12303765 1 2024-04-12T05:56:13.070 INFO:tasks.workunit.client.1.smithi194.stdout:2/548: fsync d7/f11 0 2024-04-12T05:56:13.071 INFO:tasks.workunit.client.1.smithi194.stdout:7/473: creat d3/d16/d23/d37/f96 x:0 0 0 2024-04-12T05:56:13.072 INFO:tasks.workunit.client.1.smithi194.stdout:1/642: creat d0/d11/d18/d19/fcc x:0 0 0 2024-04-12T05:56:13.074 INFO:tasks.workunit.client.1.smithi194.stdout:5/706: unlink d1/d4/d6/db/d38/c6c 0 2024-04-12T05:56:13.081 INFO:tasks.workunit.client.1.smithi194.stdout:1/643: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/ccd 0 2024-04-12T05:56:13.084 INFO:tasks.workunit.client.1.smithi194.stdout:1/644: fdatasync d0/f4 0 2024-04-12T05:56:13.086 INFO:tasks.workunit.client.1.smithi194.stdout:2/549: write d7/d12/d1a/f3e [2225276,116975] 0 2024-04-12T05:56:13.086 INFO:tasks.workunit.client.1.smithi194.stdout:4/741: dwrite d2/d3/d1b/d32/d3a/dbc/ff4 [0,4194304] 0 2024-04-12T05:56:13.092 INFO:tasks.workunit.client.1.smithi194.stdout:7/474: dwrite d3/d16/d23/d37/f54 [0,4194304] 0 2024-04-12T05:56:13.092 INFO:tasks.workunit.client.1.smithi194.stdout:5/707: dread d1/d4/d6/db/f19 [4194304,4194304] 0 2024-04-12T05:56:13.094 INFO:tasks.workunit.client.1.smithi194.stdout:2/550: fsync d7/d12/d25/d5d/d47/d52/d7b/fa6 0 2024-04-12T05:56:13.094 INFO:tasks.workunit.client.1.smithi194.stdout:4/742: write d2/d11/d1e/f6f [530968,28710] 0 2024-04-12T05:56:13.096 INFO:tasks.workunit.client.1.smithi194.stdout:1/645: creat d0/d11/d18/d19/d1a/d6a/dac/fce x:0 0 0 2024-04-12T05:56:13.096 INFO:tasks.workunit.client.1.smithi194.stdout:7/475: read d3/d27/d22/d62/f6c [4025784,64196] 0 2024-04-12T05:56:13.101 INFO:tasks.workunit.client.1.smithi194.stdout:7/476: creat d3/d27/f97 x:0 0 0 2024-04-12T05:56:13.102 INFO:tasks.workunit.client.1.smithi194.stdout:4/743: write d2/d11/d46/d61/d63/da2/fa5 [959914,30126] 0 2024-04-12T05:56:13.107 INFO:tasks.workunit.client.1.smithi194.stdout:1/646: write d0/d11/f26 [5020055,8302] 0 2024-04-12T05:56:13.109 INFO:tasks.workunit.client.1.smithi194.stdout:5/708: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:13.115 INFO:tasks.workunit.client.1.smithi194.stdout:4/744: rename d2/l5 to d2/d11/d93/da6/dab/l10b 0 2024-04-12T05:56:13.119 INFO:tasks.workunit.client.1.smithi194.stdout:7/477: dwrite d3/d16/d23/d37/d3d/d56/f87 [4194304,4194304] 0 2024-04-12T05:56:13.124 INFO:tasks.workunit.client.1.smithi194.stdout:5/709: link d1/d4/d6/db/d38/ff0 d1/d4/d6/db/d15/d26/dbb/ff1 0 2024-04-12T05:56:13.124 INFO:tasks.workunit.client.1.smithi194.stdout:7/478: fdatasync d3/d16/d23/d57/d72/d81/f73 0 2024-04-12T05:56:13.124 INFO:tasks.workunit.client.1.smithi194.stdout:1/647: mknod d0/d11/d18/d19/d1a/d6a/d27/d2e/ccf 0 2024-04-12T05:56:13.127 INFO:tasks.workunit.client.1.smithi194.stdout:7/479: write d3/d27/f69 [2164495,65812] 0 2024-04-12T05:56:13.131 INFO:tasks.workunit.client.1.smithi194.stdout:7/480: symlink d3/d27/d6a/l98 0 2024-04-12T05:56:13.133 INFO:tasks.workunit.client.1.smithi194.stdout:4/745: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/f2a [0,4194304] 0 2024-04-12T05:56:13.135 INFO:tasks.workunit.client.1.smithi194.stdout:7/481: mknod d3/d16/d23/d37/d95/c99 0 2024-04-12T05:56:13.138 INFO:tasks.workunit.client.1.smithi194.stdout:2/551: dread d7/d12/d1a/f24 [0,4194304] 0 2024-04-12T05:56:13.140 INFO:tasks.workunit.client.1.smithi194.stdout:4/746: read d2/d11/d93/da6/dab/d5d/dae/db2/d57/fa4 [1390397,90032] 0 2024-04-12T05:56:13.143 INFO:tasks.workunit.client.1.smithi194.stdout:2/552: fdatasync d7/d12/d34/f97 0 2024-04-12T05:56:13.143 INFO:tasks.workunit.client.1.smithi194.stdout:7/482: creat d3/d16/d23/d40/d6f/f9a x:0 0 0 2024-04-12T05:56:13.144 INFO:tasks.workunit.client.1.smithi194.stdout:7/483: fdatasync d3/d27/f2f 0 2024-04-12T05:56:13.148 INFO:tasks.workunit.client.1.smithi194.stdout:7/484: unlink d3/d16/d23/l24 0 2024-04-12T05:56:13.150 INFO:tasks.workunit.client.1.smithi194.stdout:2/553: getdents d7/d12/d25/d5d 0 2024-04-12T05:56:13.154 INFO:tasks.workunit.client.1.smithi194.stdout:4/747: unlink d2/d3/f3f 0 2024-04-12T05:56:13.154 INFO:tasks.workunit.client.1.smithi194.stdout:2/554: creat d7/d12/d25/d5d/d47/d52/d7b/fb8 x:0 0 0 2024-04-12T05:56:13.157 INFO:tasks.workunit.client.1.smithi194.stdout:1/648: dread d0/d11/d18/d19/d1a/d6a/dac/db5/d68/f87 [0,4194304] 0 2024-04-12T05:56:13.158 INFO:tasks.workunit.client.1.smithi194.stdout:4/748: write d2/d3/d1b/d32/d3a/dbc/ff4 [1636622,32394] 0 2024-04-12T05:56:13.162 INFO:tasks.workunit.client.1.smithi194.stdout:4/749: write d2/d11/d46/d61/d63/da2/fa5 [271442,56896] 0 2024-04-12T05:56:13.171 INFO:tasks.workunit.client.1.smithi194.stdout:4/750: rename d2/d3/d1b/d32/d3a/f3e to d2/d103/f10c 0 2024-04-12T05:56:13.181 INFO:tasks.workunit.client.1.smithi194.stdout:1/649: dwrite d0/d11/d18/d19/d1a/d6a/dac/db5/f7f [4194304,4194304] 0 2024-04-12T05:56:13.184 INFO:tasks.workunit.client.1.smithi194.stdout:7/485: dwrite d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:13.185 INFO:tasks.workunit.client.1.smithi194.stdout:7/486: fsync d3/d16/f45 0 2024-04-12T05:56:13.187 INFO:tasks.workunit.client.1.smithi194.stdout:4/751: link d2/d11/d93/da6/dab/lc4 d2/d85/l10d 0 2024-04-12T05:56:13.190 INFO:tasks.workunit.client.1.smithi194.stdout:1/650: readlink d0/d11/d18/d19/d1a/l5c 0 2024-04-12T05:56:13.190 INFO:tasks.workunit.client.1.smithi194.stdout:7/487: readlink d3/d16/d23/d37/l3f 0 2024-04-12T05:56:13.192 INFO:tasks.workunit.client.1.smithi194.stdout:4/752: creat d2/d11/d1f/f10e x:0 0 0 2024-04-12T05:56:13.193 INFO:tasks.workunit.client.1.smithi194.stdout:7/488: rmdir d3/d16/d23/d57/d72/d81/d3a 39 2024-04-12T05:56:13.194 INFO:tasks.workunit.client.1.smithi194.stdout:7/489: dread - d3/d16/d23/d40/d6f/f8b zero size 2024-04-12T05:56:13.196 INFO:tasks.workunit.client.1.smithi194.stdout:7/490: chown d3/d16/d23/f66 16535063 1 2024-04-12T05:56:13.203 INFO:tasks.workunit.client.1.smithi194.stdout:7/491: mkdir d3/d16/d23/d40/d9b 0 2024-04-12T05:56:13.212 INFO:tasks.workunit.client.1.smithi194.stdout:4/753: dwrite d2/d11/d1e/f5a [0,4194304] 0 2024-04-12T05:56:13.214 INFO:tasks.workunit.client.1.smithi194.stdout:1/651: dwrite d0/d11/d18/d19/d1c/f6b [0,4194304] 0 2024-04-12T05:56:13.222 INFO:tasks.workunit.client.1.smithi194.stdout:7/492: dwrite d3/d27/f2c [0,4194304] 0 2024-04-12T05:56:13.227 INFO:tasks.workunit.client.1.smithi194.stdout:9/647: sync 2024-04-12T05:56:13.231 INFO:tasks.workunit.client.1.smithi194.stdout:4/754: dwrite d2/d11/d93/da6/dab/d5d/f7c [0,4194304] 0 2024-04-12T05:56:13.236 INFO:tasks.workunit.client.1.smithi194.stdout:1/652: creat d0/d11/d18/d19/d1c/dca/fd0 x:0 0 0 2024-04-12T05:56:13.236 INFO:tasks.workunit.client.1.smithi194.stdout:9/648: link d1/d21/d39/d72/l82 d1/d21/d49/ld7 0 2024-04-12T05:56:13.282 INFO:tasks.workunit.client.1.smithi194.stdout:7/493: getdents d3/d27/d22 0 2024-04-12T05:56:13.288 INFO:tasks.workunit.client.1.smithi194.stdout:9/649: mknod d1/d21/d39/cd8 0 2024-04-12T05:56:13.292 INFO:tasks.workunit.client.1.smithi194.stdout:9/650: write d1/d19/f83 [2489416,125137] 0 2024-04-12T05:56:13.292 INFO:tasks.workunit.client.1.smithi194.stdout:1/653: dwrite d0/d11/f59 [0,4194304] 0 2024-04-12T05:56:13.293 INFO:tasks.workunit.client.1.smithi194.stdout:7/494: truncate d3/d27/f1f 304921 0 2024-04-12T05:56:13.296 INFO:tasks.workunit.client.1.smithi194.stdout:4/755: write d2/f6 [419209,93681] 0 2024-04-12T05:56:13.297 INFO:tasks.workunit.client.1.smithi194.stdout:1/654: creat d0/d11/d18/d19/d1c/d67/fd1 x:0 0 0 2024-04-12T05:56:13.301 INFO:tasks.workunit.client.1.smithi194.stdout:7/495: creat d3/d16/d23/d37/d95/f9c x:0 0 0 2024-04-12T05:56:13.302 INFO:tasks.workunit.client.1.smithi194.stdout:4/756: chown d2/d3/dc7 1 1 2024-04-12T05:56:13.309 INFO:tasks.workunit.client.1.smithi194.stdout:7/496: write d3/d27/d22/f77 [1447542,80933] 0 2024-04-12T05:56:13.309 INFO:tasks.workunit.client.1.smithi194.stdout:1/655: symlink d0/d11/d18/d19/d1a/d6a/d27/da8/ld2 0 2024-04-12T05:56:13.310 INFO:tasks.workunit.client.1.smithi194.stdout:4/757: symlink d2/d11/d93/da6/dab/d5d/dae/db2/df6/l10f 0 2024-04-12T05:56:13.312 INFO:tasks.workunit.client.1.smithi194.stdout:9/651: dread d1/d19/f54 [0,4194304] 0 2024-04-12T05:56:13.313 INFO:tasks.workunit.client.1.smithi194.stdout:1/656: chown d0/d11/d18/d19/d1a/d6a/dac/db5 282191135 1 2024-04-12T05:56:13.314 INFO:tasks.workunit.client.1.smithi194.stdout:4/758: write d2/d3/d1b/d32/fdd [4865675,121372] 0 2024-04-12T05:56:13.314 INFO:tasks.workunit.client.1.smithi194.stdout:7/497: unlink d3/d16/d23/d57/d72/d81/d3a/c4d 0 2024-04-12T05:56:13.318 INFO:tasks.workunit.client.1.smithi194.stdout:7/498: mknod d3/d16/c9d 0 2024-04-12T05:56:13.319 INFO:tasks.workunit.client.1.smithi194.stdout:4/759: mkdir d2/d11/d1f/d110 0 2024-04-12T05:56:13.320 INFO:tasks.workunit.client.1.smithi194.stdout:1/657: chown d0/d11/d18/d35/d6f/dc5/db9/cc6 85489510 1 2024-04-12T05:56:13.321 INFO:tasks.workunit.client.1.smithi194.stdout:9/652: mknod d1/d19/cd9 0 2024-04-12T05:56:13.323 INFO:tasks.workunit.client.1.smithi194.stdout:4/760: write d2/d3/f105 [678304,79661] 0 2024-04-12T05:56:13.326 INFO:tasks.workunit.client.1.smithi194.stdout:9/653: mknod d1/cda 0 2024-04-12T05:56:13.328 INFO:tasks.workunit.client.1.smithi194.stdout:7/499: getdents d3/d16/d23/d57 0 2024-04-12T05:56:13.329 INFO:tasks.workunit.client.1.smithi194.stdout:4/761: creat d2/d11/d43/f111 x:0 0 0 2024-04-12T05:56:13.331 INFO:tasks.workunit.client.1.smithi194.stdout:9/654: read d1/fe [252532,103513] 0 2024-04-12T05:56:13.332 INFO:tasks.workunit.client.1.smithi194.stdout:1/658: rename d0/d11/d18/d19/d1c/c93 to d0/d11/d18/d19/d1a/d6a/d27/d48/da7/cd3 0 2024-04-12T05:56:13.335 INFO:tasks.workunit.client.1.smithi194.stdout:1/659: write d0/d11/f55 [185882,51454] 0 2024-04-12T05:56:13.339 INFO:tasks.workunit.client.1.smithi194.stdout:1/660: read d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/f64 [2499346,6526] 0 2024-04-12T05:56:13.340 INFO:tasks.workunit.client.1.smithi194.stdout:7/500: write d3/d27/f1f [934846,43150] 0 2024-04-12T05:56:13.341 INFO:tasks.workunit.client.1.smithi194.stdout:1/661: chown d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/c5d 895679 1 2024-04-12T05:56:13.348 INFO:tasks.workunit.client.1.smithi194.stdout:9/655: dwrite d1/d21/d39/f89 [0,4194304] 0 2024-04-12T05:56:13.349 INFO:tasks.workunit.client.1.smithi194.stdout:1/662: link d0/d11/d18/l44 d0/d11/d18/d19/d1c/dca/ld4 0 2024-04-12T05:56:13.353 INFO:tasks.workunit.client.1.smithi194.stdout:7/501: dwrite d3/d16/d23/d37/d3d/d56/f51 [0,4194304] 0 2024-04-12T05:56:13.353 INFO:tasks.workunit.client.1.smithi194.stdout:9/656: write d1/d19/d1f/d20/d7d/f9c [60042,125263] 0 2024-04-12T05:56:13.355 INFO:tasks.workunit.client.1.smithi194.stdout:7/502: chown d3/d27/f2c 68 1 2024-04-12T05:56:13.360 INFO:tasks.workunit.client.1.smithi194.stdout:1/663: dread d0/d11/f26 [4194304,4194304] 0 2024-04-12T05:56:13.361 INFO:tasks.workunit.client.1.smithi194.stdout:1/664: chown d0/d11/d18/d19/d1a/d6a/d27/d2e 17 1 2024-04-12T05:56:13.362 INFO:tasks.workunit.client.1.smithi194.stdout:1/665: fsync d0/d11/d18/f76 0 2024-04-12T05:56:13.363 INFO:tasks.workunit.client.1.smithi194.stdout:1/666: chown d0/d11/d18/d19/d1c/c31 886 1 2024-04-12T05:56:13.372 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:13 smithi008 ceph-mon[39722]: mgrmap e26: smithi194.xbiyaj(active, since 58s), standbys: smithi008.ixmnlp 2024-04-12T05:56:13.432 INFO:tasks.workunit.client.1.smithi194.stdout:3/669: sync 2024-04-12T05:56:13.432 INFO:tasks.workunit.client.1.smithi194.stdout:8/393: sync 2024-04-12T05:56:13.433 INFO:tasks.workunit.client.1.smithi194.stdout:8/394: stat d2/d8/d9/de/d10/f1c 0 2024-04-12T05:56:13.437 INFO:tasks.workunit.client.1.smithi194.stdout:7/503: rename d3/d16/d23/d40/d49/f71 to d3/d16/d23/d40/d9b/f9e 0 2024-04-12T05:56:13.438 INFO:tasks.workunit.client.1.smithi194.stdout:7/504: fdatasync d3/d27/d6a/f55 0 2024-04-12T05:56:13.439 INFO:tasks.workunit.client.1.smithi194.stdout:9/657: write f0 [8456017,77958] 0 2024-04-12T05:56:13.441 INFO:tasks.workunit.client.1.smithi194.stdout:1/667: creat d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/fd5 x:0 0 0 2024-04-12T05:56:13.442 INFO:tasks.workunit.client.1.smithi194.stdout:3/670: creat d0/dc/d1e/d81/d34/fdd x:0 0 0 2024-04-12T05:56:13.443 INFO:tasks.workunit.client.1.smithi194.stdout:8/395: mknod d2/d8/d58/c77 0 2024-04-12T05:56:13.443 INFO:tasks.workunit.client.1.smithi194.stdout:1/668: read - d0/d11/d18/d19/d1a/fa6 zero size 2024-04-12T05:56:13.447 INFO:tasks.workunit.client.1.smithi194.stdout:7/505: rename d3/d27/d6a/c70 to d3/d16/c9f 0 2024-04-12T05:56:13.449 INFO:tasks.workunit.client.1.smithi194.stdout:3/671: truncate d0/d1/f60 955436 0 2024-04-12T05:56:13.451 INFO:tasks.workunit.client.1.smithi194.stdout:9/658: dwrite d1/d19/d1f/d5f/f9e [0,4194304] 0 2024-04-12T05:56:13.452 INFO:tasks.workunit.client.1.smithi194.stdout:8/396: creat d2/d8/d15/d3b/f78 x:0 0 0 2024-04-12T05:56:13.452 INFO:tasks.workunit.client.1.smithi194.stdout:3/672: chown d0/d1/f97 13899 1 2024-04-12T05:56:13.454 INFO:tasks.workunit.client.1.smithi194.stdout:8/397: truncate d2/fb 4559556 0 2024-04-12T05:56:13.455 INFO:tasks.workunit.client.1.smithi194.stdout:7/506: truncate d3/d16/d23/d37/f7c 553174 0 2024-04-12T05:56:13.455 INFO:tasks.workunit.client.1.smithi194.stdout:1/669: dread d0/d11/f9f [0,4194304] 0 2024-04-12T05:56:13.455 INFO:tasks.workunit.client.1.smithi194.stdout:8/398: fsync d2/f56 0 2024-04-12T05:56:13.455 INFO:tasks.workunit.client.1.smithi194.stdout:3/673: write d0/dc/d1e/d81/d5b/fd5 [388045,103187] 0 2024-04-12T05:56:13.456 INFO:tasks.workunit.client.1.smithi194.stdout:9/659: symlink d1/d19/d1f/d20/d37/da3/ldb 0 2024-04-12T05:56:13.459 INFO:tasks.workunit.client.1.smithi194.stdout:7/507: symlink d3/d16/d23/la0 0 2024-04-12T05:56:13.460 INFO:tasks.workunit.client.1.smithi194.stdout:9/660: creat d1/d21/d77/d98/d9f/fdc x:0 0 0 2024-04-12T05:56:13.461 INFO:tasks.workunit.client.1.smithi194.stdout:8/399: readlink d2/d8/d9/de/d38/l65 0 2024-04-12T05:56:13.461 INFO:tasks.workunit.client.1.smithi194.stdout:7/508: dread - d3/d16/d23/f83 zero size 2024-04-12T05:56:13.463 INFO:tasks.workunit.client.1.smithi194.stdout:3/674: rename d0/d1/d74/d39/d9e to d0/dcf/dde 0 2024-04-12T05:56:13.466 INFO:tasks.workunit.client.1.smithi194.stdout:1/670: dread d0/d11/d18/d19/d1a/d6a/dac/db5/f7f [0,4194304] 0 2024-04-12T05:56:13.466 INFO:tasks.workunit.client.1.smithi194.stdout:9/661: creat d1/d19/d1f/d20/d37/d84/fdd x:0 0 0 2024-04-12T05:56:13.467 INFO:tasks.workunit.client.1.smithi194.stdout:8/400: creat d2/d8/d9/d1a/f79 x:0 0 0 2024-04-12T05:56:13.471 INFO:tasks.workunit.client.1.smithi194.stdout:7/509: rename d3/d16/d23/d37/d3d/d56/l89 to d3/d16/d23/d37/d95/la1 0 2024-04-12T05:56:13.475 INFO:tasks.workunit.client.1.smithi194.stdout:1/671: write d0/d11/d18/d19/f4e [3253053,36124] 0 2024-04-12T05:56:13.478 INFO:tasks.workunit.client.1.smithi194.stdout:9/662: dwrite d1/d21/d39/d72/fa7 [4194304,4194304] 0 2024-04-12T05:56:13.479 INFO:tasks.workunit.client.1.smithi194.stdout:9/663: chown d1/d21/d77/f92 12 1 2024-04-12T05:56:13.480 INFO:tasks.workunit.client.1.smithi194.stdout:8/401: dwrite d2/d8/d9/d1a/f30 [0,4194304] 0 2024-04-12T05:56:13.480 INFO:tasks.workunit.client.1.smithi194.stdout:3/675: rename d0/c2f to d0/dc/d1e/d81/d5b/d64/dc5/cdf 0 2024-04-12T05:56:13.483 INFO:tasks.workunit.client.1.smithi194.stdout:7/510: mknod d3/d16/d23/d40/d9b/ca2 0 2024-04-12T05:56:13.486 INFO:tasks.workunit.client.1.smithi194.stdout:9/664: mkdir d1/d21/d77/d98/d9f/dde 0 2024-04-12T05:56:13.487 INFO:tasks.workunit.client.1.smithi194.stdout:1/672: creat d0/d11/d18/d19/d91/fd6 x:0 0 0 2024-04-12T05:56:13.490 INFO:tasks.workunit.client.1.smithi194.stdout:9/665: fsync d1/fe 0 2024-04-12T05:56:13.494 INFO:tasks.workunit.client.1.smithi194.stdout:7/511: dread d3/d16/d23/f66 [0,4194304] 0 2024-04-12T05:56:13.508 INFO:tasks.workunit.client.1.smithi194.stdout:1/673: write d0/d11/f26 [2320935,88342] 0 2024-04-12T05:56:13.510 INFO:tasks.workunit.client.1.smithi194.stdout:9/666: truncate d1/d21/f4d 3720306 0 2024-04-12T05:56:13.514 INFO:tasks.workunit.client.1.smithi194.stdout:1/674: rename d0/d11/c14 to d0/d11/d18/d19/d1a/d6a/d27/da8/cd7 0 2024-04-12T05:56:13.515 INFO:tasks.workunit.client.1.smithi194.stdout:9/667: mknod d1/d19/cdf 0 2024-04-12T05:56:13.518 INFO:tasks.workunit.client.1.smithi194.stdout:3/676: dwrite d0/dc/d3d/f66 [4194304,4194304] 0 2024-04-12T05:56:13.520 INFO:tasks.workunit.client.1.smithi194.stdout:8/402: dread d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:13.520 INFO:tasks.workunit.client.1.smithi194.stdout:1/675: mknod d0/d11/d18/d19/d1a/d6a/d27/d9e/cd8 0 2024-04-12T05:56:13.521 INFO:tasks.workunit.client.1.smithi194.stdout:9/668: creat d1/d19/d1f/d20/d37/d84/fe0 x:0 0 0 2024-04-12T05:56:13.523 INFO:tasks.workunit.client.1.smithi194.stdout:8/403: fdatasync d2/d8/d9/d1a/f2b 0 2024-04-12T05:56:13.524 INFO:tasks.workunit.client.1.smithi194.stdout:3/677: symlink d0/dc/d1e/dd3/le0 0 2024-04-12T05:56:13.525 INFO:tasks.workunit.client.1.smithi194.stdout:1/676: dread - d0/d11/d18/d19/d1a/d6a/dac/db5/f95 zero size 2024-04-12T05:56:13.527 INFO:tasks.workunit.client.1.smithi194.stdout:8/404: symlink d2/d8/d9/de/d10/d4e/l7a 0 2024-04-12T05:56:13.531 INFO:tasks.workunit.client.1.smithi194.stdout:9/669: symlink d1/d41/d97/d9d/db9/le1 0 2024-04-12T05:56:13.531 INFO:tasks.workunit.client.1.smithi194.stdout:8/405: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:13.532 INFO:tasks.workunit.client.1.smithi194.stdout:1/677: mknod d0/d11/d18/d19/d1a/d6a/d27/d6c/da0/cd9 0 2024-04-12T05:56:13.534 INFO:tasks.workunit.client.1.smithi194.stdout:3/678: dwrite d0/dc/d3d/dae/faf [0,4194304] 0 2024-04-12T05:56:13.539 INFO:tasks.workunit.client.1.smithi194.stdout:9/670: dread - d1/d41/dc9/fd5 zero size 2024-04-12T05:56:13.541 INFO:tasks.workunit.client.1.smithi194.stdout:3/679: readlink d0/dc/l20 0 2024-04-12T05:56:13.544 INFO:tasks.workunit.client.1.smithi194.stdout:8/406: creat d2/d8/d15/d33/d45/d52/f7b x:0 0 0 2024-04-12T05:56:13.545 INFO:tasks.workunit.client.1.smithi194.stdout:9/671: readlink d1/d19/d1f/d20/d37/d38/l40 0 2024-04-12T05:56:13.547 INFO:tasks.workunit.client.1.smithi194.stdout:1/678: creat d0/d11/d18/d19/fda x:0 0 0 2024-04-12T05:56:13.548 INFO:tasks.workunit.client.1.smithi194.stdout:8/407: creat d2/d8/d9/de/d10/f7c x:0 0 0 2024-04-12T05:56:13.551 INFO:tasks.workunit.client.1.smithi194.stdout:9/672: mkdir d1/d19/d1f/d20/de2 0 2024-04-12T05:56:13.552 INFO:tasks.workunit.client.1.smithi194.stdout:3/680: mkdir d0/d1/d4f/dd7/de1 0 2024-04-12T05:56:13.554 INFO:tasks.workunit.client.1.smithi194.stdout:9/673: truncate d1/d21/d39/faf 1019803 0 2024-04-12T05:56:13.555 INFO:tasks.workunit.client.1.smithi194.stdout:3/681: mknod d0/dc/ce2 0 2024-04-12T05:56:13.557 INFO:tasks.workunit.client.1.smithi194.stdout:9/674: chown d1/d21/d77/f92 891261 1 2024-04-12T05:56:13.557 INFO:tasks.workunit.client.1.smithi194.stdout:1/679: dwrite d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/d7b/f86 [0,4194304] 0 2024-04-12T05:56:13.558 INFO:tasks.workunit.client.1.smithi194.stdout:8/408: getdents d2/d8/d66 0 2024-04-12T05:56:13.558 INFO:tasks.workunit.client.1.smithi194.stdout:9/675: fdatasync d1/d21/d39/d72/fb5 0 2024-04-12T05:56:13.558 INFO:tasks.workunit.client.1.smithi194.stdout:3/682: chown d0/dc/d1e/d81/d5b/d64 4 1 2024-04-12T05:56:13.563 INFO:tasks.workunit.client.1.smithi194.stdout:3/683: chown d0/d1/d10/d8c/d8f/ca5 64369763 1 2024-04-12T05:56:13.564 INFO:tasks.workunit.client.1.smithi194.stdout:1/680: creat d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fdb x:0 0 0 2024-04-12T05:56:13.567 INFO:tasks.workunit.client.1.smithi194.stdout:8/409: dread d2/d8/d15/d33/f29 [0,4194304] 0 2024-04-12T05:56:13.567 INFO:tasks.workunit.client.1.smithi194.stdout:3/684: chown d0/f11 6067034 1 2024-04-12T05:56:13.568 INFO:tasks.workunit.client.1.smithi194.stdout:1/681: mknod d0/d11/d18/d19/d1c/cdc 0 2024-04-12T05:56:13.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:13 smithi194 ceph-mon[49771]: mgrmap e26: smithi194.xbiyaj(active, since 58s), standbys: smithi008.ixmnlp 2024-04-12T05:56:13.570 INFO:tasks.workunit.client.1.smithi194.stdout:8/410: mkdir d2/d8/d9/de/d10/d7d 0 2024-04-12T05:56:13.571 INFO:tasks.workunit.client.1.smithi194.stdout:9/676: creat d1/d19/fe3 x:0 0 0 2024-04-12T05:56:13.572 INFO:tasks.workunit.client.1.smithi194.stdout:1/682: rmdir d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/d7b 39 2024-04-12T05:56:13.572 INFO:tasks.workunit.client.1.smithi194.stdout:3/685: getdents d0/d1/d10/d8c/d8f/ddb 0 2024-04-12T05:56:13.573 INFO:tasks.workunit.client.1.smithi194.stdout:1/683: write d0/f32 [7210090,5032] 0 2024-04-12T05:56:13.578 INFO:tasks.workunit.client.1.smithi194.stdout:1/684: unlink d0/d11/d18/d19/d1a/d6a/d27/lc4 0 2024-04-12T05:56:13.579 INFO:tasks.workunit.client.1.smithi194.stdout:1/685: mkdir d0/d11/ddd 0 2024-04-12T05:56:13.580 INFO:tasks.workunit.client.1.smithi194.stdout:3/686: write d0/d1/f60 [118387,43208] 0 2024-04-12T05:56:13.582 INFO:tasks.workunit.client.1.smithi194.stdout:1/686: creat d0/d11/d18/d19/d1c/d67/fde x:0 0 0 2024-04-12T05:56:13.583 INFO:tasks.workunit.client.1.smithi194.stdout:3/687: mkdir d0/d1/d10/d8c/d8f/dac/de3 0 2024-04-12T05:56:13.584 INFO:tasks.workunit.client.1.smithi194.stdout:3/688: chown d0/d1/d74/d39/f3b 15240 1 2024-04-12T05:56:13.593 INFO:tasks.workunit.client.1.smithi194.stdout:1/687: rename d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/d7b to d0/d11/d18/d35/d6f/dc5/ddf 0 2024-04-12T05:56:13.594 INFO:tasks.workunit.client.1.smithi194.stdout:3/689: write d0/d1/d4f/f59 [645503,99104] 0 2024-04-12T05:56:13.595 INFO:tasks.workunit.client.1.smithi194.stdout:1/688: read d0/d11/d18/d19/d1c/f6b [4096060,31727] 0 2024-04-12T05:56:13.596 INFO:tasks.workunit.client.1.smithi194.stdout:1/689: dread - d0/d11/fae zero size 2024-04-12T05:56:13.600 INFO:tasks.workunit.client.1.smithi194.stdout:3/690: fsync d0/d1/d74/d39/d4d/f62 0 2024-04-12T05:56:13.601 INFO:tasks.workunit.client.1.smithi194.stdout:3/691: read - d0/d1/d10/d8c/fdc zero size 2024-04-12T05:56:13.602 INFO:tasks.workunit.client.1.smithi194.stdout:1/690: creat d0/d11/d18/d35/d6f/dc5/ddf/fe0 x:0 0 0 2024-04-12T05:56:13.602 INFO:tasks.workunit.client.1.smithi194.stdout:3/692: chown d0/dc/d3d/l3e 0 1 2024-04-12T05:56:13.606 INFO:tasks.workunit.client.1.smithi194.stdout:3/693: fsync d0/f27 0 2024-04-12T05:56:13.609 INFO:tasks.workunit.client.1.smithi194.stdout:1/691: rename d0/d11/d18/d19/d1a/d6a/f30 to d0/d11/d18/d19/d1a/d6a/d27/fe1 0 2024-04-12T05:56:13.611 INFO:tasks.workunit.client.1.smithi194.stdout:3/694: mknod d0/d1/d74/ce4 0 2024-04-12T05:56:13.612 INFO:tasks.workunit.client.1.smithi194.stdout:1/692: creat d0/d11/d18/d19/d1a/fe2 x:0 0 0 2024-04-12T05:56:13.614 INFO:tasks.workunit.client.1.smithi194.stdout:0/563: sync 2024-04-12T05:56:13.614 INFO:tasks.workunit.client.1.smithi194.stdout:6/570: sync 2024-04-12T05:56:13.614 INFO:tasks.workunit.client.1.smithi194.stdout:3/695: unlink d0/d1/d74/fb1 0 2024-04-12T05:56:13.616 INFO:tasks.workunit.client.1.smithi194.stdout:1/693: creat d0/d11/fe3 x:0 0 0 2024-04-12T05:56:13.617 INFO:tasks.workunit.client.1.smithi194.stdout:1/694: dread - d0/d11/d18/fc3 zero size 2024-04-12T05:56:13.620 INFO:tasks.workunit.client.1.smithi194.stdout:1/695: truncate d0/d11/d18/d19/f80 4938309 0 2024-04-12T05:56:13.621 INFO:tasks.workunit.client.1.smithi194.stdout:6/571: creat d3/d2f/d4e/d53/d77/fb6 x:0 0 0 2024-04-12T05:56:13.622 INFO:tasks.workunit.client.1.smithi194.stdout:6/572: chown d3/d2f/d4e/d63 54 1 2024-04-12T05:56:13.628 INFO:tasks.workunit.client.1.smithi194.stdout:3/696: write d0/dc/f1f [2850284,104414] 0 2024-04-12T05:56:13.630 INFO:tasks.workunit.client.1.smithi194.stdout:0/564: dread d0/da/d1e/d1a/f64 [0,4194304] 0 2024-04-12T05:56:13.630 INFO:tasks.workunit.client.1.smithi194.stdout:6/573: dwrite d3/d2d/d33/d52/f6e [4194304,4194304] 0 2024-04-12T05:56:13.631 INFO:tasks.workunit.client.1.smithi194.stdout:3/697: symlink d0/d1/d10/d8c/le5 0 2024-04-12T05:56:13.633 INFO:tasks.workunit.client.1.smithi194.stdout:0/565: creat d0/da/d1e/d1a/d58/d5b/fb6 x:0 0 0 2024-04-12T05:56:13.637 INFO:tasks.workunit.client.1.smithi194.stdout:3/698: mkdir d0/dc/d1e/d81/d5b/d64/d79/d87/da7/de6 0 2024-04-12T05:56:13.641 INFO:tasks.workunit.client.1.smithi194.stdout:0/566: dwrite d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:13.645 INFO:tasks.workunit.client.1.smithi194.stdout:3/699: dread d0/dc/d1e/f6d [0,4194304] 0 2024-04-12T05:56:13.651 INFO:tasks.workunit.client.1.smithi194.stdout:3/700: rename d0/dc/d1e/dd3/le0 to d0/dc/d1e/d81/d5b/d64/d79/d87/le7 0 2024-04-12T05:56:13.654 INFO:tasks.workunit.client.1.smithi194.stdout:0/567: dwrite d0/da/d1e/d1a/d58/f80 [0,4194304] 0 2024-04-12T05:56:13.655 INFO:tasks.workunit.client.1.smithi194.stdout:3/701: rename d0/dc/f1f to d0/dc/d1e/dd3/fe8 0 2024-04-12T05:56:13.656 INFO:tasks.workunit.client.1.smithi194.stdout:0/568: read d0/da/f39 [1074551,34511] 0 2024-04-12T05:56:13.657 INFO:tasks.workunit.client.1.smithi194.stdout:0/569: readlink d0/l67 0 2024-04-12T05:56:13.659 INFO:tasks.workunit.client.1.smithi194.stdout:0/570: write d0/da/d1e/d1a/d3d/f51 [5216,53677] 0 2024-04-12T05:56:13.659 INFO:tasks.workunit.client.1.smithi194.stdout:3/702: getdents d0/dc/d1e/d33/d76 0 2024-04-12T05:56:13.660 INFO:tasks.workunit.client.1.smithi194.stdout:3/703: dread - d0/d1/d10/d89/fd4 zero size 2024-04-12T05:56:13.662 INFO:tasks.workunit.client.1.smithi194.stdout:3/704: mkdir d0/dc/d3d/d6e/de9 0 2024-04-12T05:56:13.663 INFO:tasks.workunit.client.1.smithi194.stdout:0/571: truncate d0/da/d1e/d1a/f44 2692747 0 2024-04-12T05:56:13.663 INFO:tasks.workunit.client.1.smithi194.stdout:3/705: dread - d0/d1/d10/d8c/fdc zero size 2024-04-12T05:56:13.666 INFO:tasks.workunit.client.1.smithi194.stdout:3/706: mknod d0/d1/d10/d8c/d8f/dac/cea 0 2024-04-12T05:56:13.668 INFO:tasks.workunit.client.1.smithi194.stdout:3/707: read d0/dc/d3d/f94 [5296185,8276] 0 2024-04-12T05:56:13.678 INFO:tasks.workunit.client.1.smithi194.stdout:3/708: rename d0/dc to d0/d1/d74/d39/d4d/deb 0 2024-04-12T05:56:13.686 INFO:tasks.workunit.client.1.smithi194.stdout:3/709: unlink d0/d1/d74/d39/d4d/deb/c35 0 2024-04-12T05:56:13.688 INFO:tasks.workunit.client.1.smithi194.stdout:0/572: dwrite d0/da/d1e/d1a/f44 [0,4194304] 0 2024-04-12T05:56:13.689 INFO:tasks.workunit.client.1.smithi194.stdout:3/710: dread d0/d1/d4f/fa3 [0,4194304] 0 2024-04-12T05:56:13.690 INFO:tasks.workunit.client.1.smithi194.stdout:0/573: mknod d0/da/d1e/d27/d54/cb7 0 2024-04-12T05:56:13.692 INFO:tasks.workunit.client.1.smithi194.stdout:3/711: mkdir d0/d1/d74/d39/d4d/d63/db5/dec 0 2024-04-12T05:56:13.693 INFO:tasks.workunit.client.1.smithi194.stdout:0/574: write d0/da/f39 [2178959,124149] 0 2024-04-12T05:56:13.697 INFO:tasks.workunit.client.1.smithi194.stdout:0/575: rename d0/d5f/c69 to d0/d5f/d73/d76/cb8 0 2024-04-12T05:56:13.700 INFO:tasks.workunit.client.1.smithi194.stdout:0/576: write d0/da/d4a/d8e/f68 [724594,70578] 0 2024-04-12T05:56:13.700 INFO:tasks.workunit.client.1.smithi194.stdout:3/712: write d0/d42/f73 [397104,19590] 0 2024-04-12T05:56:13.702 INFO:tasks.workunit.client.1.smithi194.stdout:3/713: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d2d 73782749 1 2024-04-12T05:56:13.708 INFO:tasks.workunit.client.1.smithi194.stdout:0/577: dwrite d0/d3b/d52/d7c/f9f [0,4194304] 0 2024-04-12T05:56:13.710 INFO:tasks.workunit.client.1.smithi194.stdout:0/578: write d0/da/d1e/d1a/f44 [3846334,6999] 0 2024-04-12T05:56:13.714 INFO:tasks.workunit.client.1.smithi194.stdout:3/714: rename d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d2d/f3a to d0/d1/d74/d39/d4d/deb/db4/fed 0 2024-04-12T05:56:13.717 INFO:tasks.workunit.client.1.smithi194.stdout:0/579: symlink d0/da/d1e/d1a/d24/lb9 0 2024-04-12T05:56:13.718 INFO:tasks.workunit.client.1.smithi194.stdout:3/715: dread d0/d1/f60 [0,4194304] 0 2024-04-12T05:56:13.719 INFO:tasks.workunit.client.1.smithi194.stdout:3/716: chown d0/d1/d74/d39/d4d/deb/db4 483 1 2024-04-12T05:56:13.720 INFO:tasks.workunit.client.1.smithi194.stdout:3/717: write d0/d42/f73 [1142721,25165] 0 2024-04-12T05:56:13.725 INFO:tasks.workunit.client.1.smithi194.stdout:3/718: getdents d0/d1/d74/d39/d4d/deb 0 2024-04-12T05:56:13.726 INFO:tasks.workunit.client.1.smithi194.stdout:3/719: mkdir d0/d1/d4f/dd7/de1/dee 0 2024-04-12T05:56:13.729 INFO:tasks.workunit.client.1.smithi194.stdout:3/720: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/d34/def 0 2024-04-12T05:56:13.731 INFO:tasks.workunit.client.1.smithi194.stdout:3/721: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0 0 2024-04-12T05:56:13.734 INFO:tasks.workunit.client.1.smithi194.stdout:0/580: dread d0/f48 [0,4194304] 0 2024-04-12T05:56:13.735 INFO:tasks.workunit.client.1.smithi194.stdout:0/581: chown d0/da/d1e/d27/d87/c8c 19 1 2024-04-12T05:56:13.738 INFO:tasks.workunit.client.1.smithi194.stdout:0/582: creat d0/da/da7/fba x:0 0 0 2024-04-12T05:56:13.738 INFO:tasks.workunit.client.1.smithi194.stdout:0/583: stat d0/da/d4a/d8e 0 2024-04-12T05:56:13.739 INFO:tasks.workunit.client.1.smithi194.stdout:3/722: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/f84 [621351,96665] 0 2024-04-12T05:56:13.742 INFO:tasks.workunit.client.1.smithi194.stdout:3/723: mkdir d0/d1/d74/d39/d4d/deb/d1e/d33/d76/d9a/df1 0 2024-04-12T05:56:13.746 INFO:tasks.workunit.client.1.smithi194.stdout:0/584: dwrite d0/d3b/d52/fb1 [0,4194304] 0 2024-04-12T05:56:13.749 INFO:tasks.workunit.client.1.smithi194.stdout:3/724: getdents d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79 0 2024-04-12T05:56:13.754 INFO:tasks.workunit.client.1.smithi194.stdout:0/585: fdatasync d0/da/d1e/f32 0 2024-04-12T05:56:13.755 INFO:tasks.workunit.client.1.smithi194.stdout:0/586: mknod d0/da/d1e/d1a/d24/d4b/cbb 0 2024-04-12T05:56:13.762 INFO:tasks.workunit.client.1.smithi194.stdout:3/725: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/fb3 [549229,37937] 0 2024-04-12T05:56:13.762 INFO:tasks.workunit.client.1.smithi194.stdout:0/587: rmdir d0/da/d1e/d1a/d24/d4b 39 2024-04-12T05:56:13.765 INFO:tasks.workunit.client.1.smithi194.stdout:3/726: rmdir d0/d1/d74/d39/d4d/deb/d3d/dae 39 2024-04-12T05:56:13.767 INFO:tasks.workunit.client.1.smithi194.stdout:3/727: fsync d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f37 0 2024-04-12T05:56:13.768 INFO:tasks.workunit.client.1.smithi194.stdout:0/588: dwrite d0/d3b/d52/fb1 [0,4194304] 0 2024-04-12T05:56:13.769 INFO:tasks.workunit.client.1.smithi194.stdout:3/728: mknod d0/cf2 0 2024-04-12T05:56:13.772 INFO:tasks.workunit.client.1.smithi194.stdout:0/589: creat d0/da/da7/fbc x:0 0 0 2024-04-12T05:56:13.774 INFO:tasks.workunit.client.1.smithi194.stdout:3/729: mkdir d0/d1/d10/d8c/d8f/dac/de3/df3 0 2024-04-12T05:56:13.776 INFO:tasks.workunit.client.1.smithi194.stdout:3/730: write d0/d1/d74/d39/f70 [1242995,79173] 0 2024-04-12T05:56:13.776 INFO:tasks.workunit.client.1.smithi194.stdout:0/590: unlink d0/l67 0 2024-04-12T05:56:13.777 INFO:tasks.workunit.client.1.smithi194.stdout:3/731: readlink d0/d1/d10/d89/l92 0 2024-04-12T05:56:13.780 INFO:tasks.workunit.client.1.smithi194.stdout:3/732: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/df4 0 2024-04-12T05:56:13.792 INFO:tasks.workunit.client.1.smithi194.stdout:3/733: dread d0/d1/d10/f44 [0,4194304] 0 2024-04-12T05:56:13.894 INFO:tasks.workunit.client.1.smithi194.stdout:2/555: sync 2024-04-12T05:56:13.894 INFO:tasks.workunit.client.1.smithi194.stdout:5/710: sync 2024-04-12T05:56:13.895 INFO:tasks.workunit.client.1.smithi194.stdout:2/556: write d7/d22/d42/f77 [429180,51845] 0 2024-04-12T05:56:13.897 INFO:tasks.workunit.client.1.smithi194.stdout:2/557: readlink d7/d22/d42/l5f 0 2024-04-12T05:56:13.899 INFO:tasks.workunit.client.1.smithi194.stdout:5/711: rename d1/d4/d6/db/d3b to d1/d5c/d95/df2 0 2024-04-12T05:56:13.900 INFO:tasks.workunit.client.1.smithi194.stdout:5/712: stat d1/d4/dd7 0 2024-04-12T05:56:13.901 INFO:tasks.workunit.client.1.smithi194.stdout:2/558: symlink d7/d12/d1a/lb9 0 2024-04-12T05:56:13.902 INFO:tasks.workunit.client.1.smithi194.stdout:2/559: fsync d7/d22/f4d 0 2024-04-12T05:56:13.905 INFO:tasks.workunit.client.1.smithi194.stdout:2/560: write d7/d12/d25/d5d/f2f [4851712,6855] 0 2024-04-12T05:56:13.911 INFO:tasks.workunit.client.1.smithi194.stdout:2/561: truncate f5 218443 0 2024-04-12T05:56:13.914 INFO:tasks.workunit.client.1.smithi194.stdout:5/713: dwrite d1/d4/d6/db/d38/fc0 [0,4194304] 0 2024-04-12T05:56:13.919 INFO:tasks.workunit.client.1.smithi194.stdout:5/714: creat d1/d5c/da7/ff3 x:0 0 0 2024-04-12T05:56:13.920 INFO:tasks.workunit.client.1.smithi194.stdout:2/562: link d7/d12/d1a/l32 d7/d22/lba 0 2024-04-12T05:56:13.928 INFO:tasks.workunit.client.1.smithi194.stdout:5/715: dwrite d1/d4/d6/d34/fa0 [0,4194304] 0 2024-04-12T05:56:13.930 INFO:tasks.workunit.client.1.smithi194.stdout:2/563: dwrite d7/d12/d25/d5d/f51 [4194304,4194304] 0 2024-04-12T05:56:13.945 INFO:tasks.workunit.client.1.smithi194.stdout:5/716: truncate d1/d5c/d95/df2/f3e 620934 0 2024-04-12T05:56:13.945 INFO:tasks.workunit.client.1.smithi194.stdout:2/564: write d7/d12/d25/f4f [183255,28425] 0 2024-04-12T05:56:13.946 INFO:tasks.workunit.client.1.smithi194.stdout:5/717: chown d1/d4/d6/f76 37167622 1 2024-04-12T05:56:13.947 INFO:tasks.workunit.client.1.smithi194.stdout:2/565: mkdir d7/d22/dbb 0 2024-04-12T05:56:13.948 INFO:tasks.workunit.client.1.smithi194.stdout:5/718: read - d1/d4/d9e/faf zero size 2024-04-12T05:56:13.951 INFO:tasks.workunit.client.1.smithi194.stdout:2/566: link d7/d12/c53 d7/d22/d93/cbc 0 2024-04-12T05:56:13.954 INFO:tasks.workunit.client.1.smithi194.stdout:2/567: truncate d7/d12/d25/d5d/d47/f88 589716 0 2024-04-12T05:56:13.957 INFO:tasks.workunit.client.1.smithi194.stdout:2/568: symlink d7/d12/d1a/d69/d7e/lbd 0 2024-04-12T05:56:13.957 INFO:tasks.workunit.client.1.smithi194.stdout:2/569: stat d7/d12/d25/d5d/d47/f8f 0 2024-04-12T05:56:13.967 INFO:tasks.workunit.client.1.smithi194.stdout:2/570: link d7/f5e d7/d22/d93/fbe 0 2024-04-12T05:56:13.971 INFO:tasks.workunit.client.1.smithi194.stdout:2/571: rename d7/d22/dbb to d7/d12/d1a/dbf 0 2024-04-12T05:56:13.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/572: dread d7/fa [0,4194304] 0 2024-04-12T05:56:13.982 INFO:tasks.workunit.client.1.smithi194.stdout:2/573: dwrite d7/d12/d1a/d69/fb1 [0,4194304] 0 2024-04-12T05:56:13.984 INFO:tasks.workunit.client.1.smithi194.stdout:2/574: mknod d7/d12/d1a/d69/cc0 0 2024-04-12T05:56:13.986 INFO:tasks.workunit.client.1.smithi194.stdout:2/575: truncate d7/d12/d25/d5d/f51 8684678 0 2024-04-12T05:56:13.996 INFO:tasks.workunit.client.1.smithi194.stdout:2/576: truncate d7/d12/d1a/f24 340178 0 2024-04-12T05:56:13.998 INFO:tasks.workunit.client.1.smithi194.stdout:2/577: creat d7/d22/fc1 x:0 0 0 2024-04-12T05:56:14.000 INFO:tasks.workunit.client.1.smithi194.stdout:2/578: fdatasync d7/d22/f61 0 2024-04-12T05:56:14.004 INFO:tasks.workunit.client.1.smithi194.stdout:2/579: truncate d7/d12/d25/d5d/d47/d52/f55 2111460 0 2024-04-12T05:56:14.006 INFO:tasks.workunit.client.1.smithi194.stdout:2/580: mkdir d7/d12/d25/d5d/d47/dac/dc2 0 2024-04-12T05:56:14.008 INFO:tasks.workunit.client.1.smithi194.stdout:2/581: symlink d7/d22/d93/lc3 0 2024-04-12T05:56:14.010 INFO:tasks.workunit.client.1.smithi194.stdout:2/582: unlink d7/d22/f61 0 2024-04-12T05:56:14.011 INFO:tasks.workunit.client.1.smithi194.stdout:2/583: fsync d7/f48 0 2024-04-12T05:56:14.013 INFO:tasks.workunit.client.1.smithi194.stdout:2/584: write d7/d12/d1a/f44 [2985026,24421] 0 2024-04-12T05:56:14.017 INFO:tasks.workunit.client.1.smithi194.stdout:2/585: creat d7/d12/d34/d4a/fc4 x:0 0 0 2024-04-12T05:56:14.024 INFO:tasks.workunit.client.1.smithi194.stdout:2/586: dwrite d7/d12/d34/d4a/d8a/faf [0,4194304] 0 2024-04-12T05:56:14.026 INFO:tasks.workunit.client.1.smithi194.stdout:2/587: stat d7/d12/d1a/c50 0 2024-04-12T05:56:14.032 INFO:tasks.workunit.client.1.smithi194.stdout:2/588: dwrite d7/d12/d34/f83 [0,4194304] 0 2024-04-12T05:56:14.038 INFO:tasks.workunit.client.1.smithi194.stdout:2/589: rename d7/d12/f33 to d7/d12/d34/fc5 0 2024-04-12T05:56:14.040 INFO:tasks.workunit.client.1.smithi194.stdout:2/590: truncate d7/d12/d25/d5d/d47/f7f 3077485 0 2024-04-12T05:56:14.053 INFO:tasks.workunit.client.1.smithi194.stdout:2/591: dwrite d7/d12/d25/d5d/d47/d52/f55 [0,4194304] 0 2024-04-12T05:56:14.056 INFO:tasks.workunit.client.1.smithi194.stdout:2/592: mkdir d7/d12/d25/d5d/d47/dc6 0 2024-04-12T05:56:14.058 INFO:tasks.workunit.client.1.smithi194.stdout:2/593: mknod d7/cc7 0 2024-04-12T05:56:14.062 INFO:tasks.workunit.client.1.smithi194.stdout:2/594: rename d7/d12/d34/f83 to d7/d22/d93/d9a/fc8 0 2024-04-12T05:56:14.063 INFO:tasks.workunit.client.1.smithi194.stdout:2/595: chown d7/d12/d1a/d68/l79 2 1 2024-04-12T05:56:14.064 INFO:tasks.workunit.client.1.smithi194.stdout:2/596: write d7/d12/d34/d4a/d8a/faf [2334174,49928] 0 2024-04-12T05:56:14.066 INFO:tasks.workunit.client.1.smithi194.stdout:2/597: dread - d7/d22/d93/fbe zero size 2024-04-12T05:56:14.068 INFO:tasks.workunit.client.1.smithi194.stdout:2/598: mkdir d7/d22/dc9 0 2024-04-12T05:56:14.070 INFO:tasks.workunit.client.1.smithi194.stdout:2/599: write d7/d12/d25/d5d/f2f [1901329,99511] 0 2024-04-12T05:56:14.073 INFO:tasks.workunit.client.1.smithi194.stdout:2/600: rmdir d7/d12/d25/d5d/d47/dc6 0 2024-04-12T05:56:14.076 INFO:tasks.workunit.client.1.smithi194.stdout:2/601: mknod d7/d12/d25/d5d/d47/d52/d7b/db2/cca 0 2024-04-12T05:56:14.076 INFO:tasks.workunit.client.1.smithi194.stdout:2/602: chown d7/d12/f38 3140 1 2024-04-12T05:56:14.078 INFO:tasks.workunit.client.1.smithi194.stdout:2/603: chown d7/d22/d93/d9f/fb0 2849 1 2024-04-12T05:56:14.079 INFO:tasks.workunit.client.1.smithi194.stdout:2/604: chown d7/d12/d1a/d68/c6b 4 1 2024-04-12T05:56:14.084 INFO:tasks.workunit.client.1.smithi194.stdout:2/605: link d7/d12/d25/d5d/d47/f88 d7/d12/d25/d5d/d47/dac/fcb 0 2024-04-12T05:56:14.086 INFO:tasks.workunit.client.1.smithi194.stdout:2/606: mknod d7/d12/d34/ccc 0 2024-04-12T05:56:14.097 INFO:tasks.workunit.client.1.smithi194.stdout:2/607: dwrite d7/d12/f38 [0,4194304] 0 2024-04-12T05:56:14.100 INFO:tasks.workunit.client.1.smithi194.stdout:2/608: rename d7/d12/d34/d4a/lae to d7/d22/dc9/lcd 0 2024-04-12T05:56:14.104 INFO:tasks.workunit.client.1.smithi194.stdout:2/609: mknod d7/d22/d93/cce 0 2024-04-12T05:56:14.104 INFO:tasks.workunit.client.1.smithi194.stdout:7/512: sync 2024-04-12T05:56:14.105 INFO:tasks.workunit.client.1.smithi194.stdout:8/411: sync 2024-04-12T05:56:14.105 INFO:tasks.workunit.client.1.smithi194.stdout:4/762: sync 2024-04-12T05:56:14.107 INFO:tasks.workunit.client.1.smithi194.stdout:7/513: chown d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f91 13687176 1 2024-04-12T05:56:14.107 INFO:tasks.workunit.client.1.smithi194.stdout:2/610: getdents d7/d12/d25 0 2024-04-12T05:56:14.108 INFO:tasks.workunit.client.1.smithi194.stdout:7/514: chown d3/d16/d23/d37 4219182 1 2024-04-12T05:56:14.110 INFO:tasks.workunit.client.1.smithi194.stdout:4/763: rename d2/d3/f27 to d2/d11/d1e/d2f/f112 0 2024-04-12T05:56:14.112 INFO:tasks.workunit.client.1.smithi194.stdout:8/412: link d2/d8/d9/de/d38/c59 d2/d8/d15/d33/d45/c7e 0 2024-04-12T05:56:14.114 INFO:tasks.workunit.client.1.smithi194.stdout:7/515: creat d3/d27/d22/d62/fa3 x:0 0 0 2024-04-12T05:56:14.114 INFO:tasks.workunit.client.1.smithi194.stdout:8/413: mkdir d2/d8/d66/d7f 0 2024-04-12T05:56:14.119 INFO:tasks.workunit.client.1.smithi194.stdout:7/516: mkdir d3/d16/d23/d37/d3d/d56/da4 0 2024-04-12T05:56:14.119 INFO:tasks.workunit.client.1.smithi194.stdout:2/611: dread d7/d22/d42/f1e [4194304,4194304] 0 2024-04-12T05:56:14.120 INFO:tasks.workunit.client.1.smithi194.stdout:8/414: fdatasync d2/d8/d15/d33/d45/f55 0 2024-04-12T05:56:14.121 INFO:tasks.workunit.client.1.smithi194.stdout:4/764: dwrite d2/d3/d1b/d32/d3a/d3b/f9f [0,4194304] 0 2024-04-12T05:56:14.122 INFO:tasks.workunit.client.1.smithi194.stdout:8/415: chown d2/d8/d15 20461046 1 2024-04-12T05:56:14.125 INFO:tasks.workunit.client.1.smithi194.stdout:2/612: symlink d7/d12/d34/d4a/lcf 0 2024-04-12T05:56:14.126 INFO:tasks.workunit.client.1.smithi194.stdout:8/416: chown d2/d8/d15/d33/d45/d52/l68 73 1 2024-04-12T05:56:14.129 INFO:tasks.workunit.client.1.smithi194.stdout:4/765: creat d2/d11/d93/f113 x:0 0 0 2024-04-12T05:56:14.134 INFO:tasks.workunit.client.1.smithi194.stdout:2/613: symlink d7/d12/d34/d4a/d8a/ld0 0 2024-04-12T05:56:14.135 INFO:tasks.workunit.client.1.smithi194.stdout:4/766: symlink d2/d85/dc2/l114 0 2024-04-12T05:56:14.135 INFO:tasks.workunit.client.1.smithi194.stdout:8/417: mknod d2/d8/d9/de/d10/c80 0 2024-04-12T05:56:14.136 INFO:tasks.workunit.client.1.smithi194.stdout:7/517: dwrite d3/d16/d5d/f84 [0,4194304] 0 2024-04-12T05:56:14.142 INFO:tasks.workunit.client.1.smithi194.stdout:4/767: stat d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/lfe 0 2024-04-12T05:56:14.143 INFO:tasks.workunit.client.1.smithi194.stdout:4/768: mknod d2/d11/d1e/d25/c115 0 2024-04-12T05:56:14.143 INFO:tasks.workunit.client.1.smithi194.stdout:7/518: creat d3/d16/d23/d40/d9b/fa5 x:0 0 0 2024-04-12T05:56:14.144 INFO:tasks.workunit.client.1.smithi194.stdout:8/418: creat d2/d8/d9/de/d10/d7d/f81 x:0 0 0 2024-04-12T05:56:14.147 INFO:tasks.workunit.client.1.smithi194.stdout:7/519: symlink d3/d16/d23/d40/d9b/la6 0 2024-04-12T05:56:14.149 INFO:tasks.workunit.client.1.smithi194.stdout:4/769: write d2/d11/d46/d61/d63/da2/fa5 [841092,85203] 0 2024-04-12T05:56:14.155 INFO:tasks.workunit.client.1.smithi194.stdout:8/419: write d2/d8/d15/d33/f4b [866970,106871] 0 2024-04-12T05:56:14.157 INFO:tasks.workunit.client.1.smithi194.stdout:7/520: dwrite d3/d16/d23/d37/d3d/d56/f87 [4194304,4194304] 0 2024-04-12T05:56:14.168 INFO:tasks.workunit.client.1.smithi194.stdout:8/420: creat d2/d8/f82 x:0 0 0 2024-04-12T05:56:14.169 INFO:tasks.workunit.client.1.smithi194.stdout:2/614: dwrite d7/d12/d25/d5d/d47/f88 [0,4194304] 0 2024-04-12T05:56:14.169 INFO:tasks.workunit.client.1.smithi194.stdout:8/421: dread d2/d8/d15/d33/f29 [0,4194304] 0 2024-04-12T05:56:14.169 INFO:tasks.workunit.client.1.smithi194.stdout:4/770: link d2/d11/d93/da6/dab/d5d/dae/db2/f79 d2/d3/d1b/d32/d3a/dbc/f116 0 2024-04-12T05:56:14.175 INFO:tasks.workunit.client.1.smithi194.stdout:7/521: creat d3/d16/d23/d57/d72/d81/d3a/fa7 x:0 0 0 2024-04-12T05:56:14.177 INFO:tasks.workunit.client.1.smithi194.stdout:4/771: mknod d2/d11/d93/da6/dab/d5d/dae/db2/df6/c117 0 2024-04-12T05:56:14.177 INFO:tasks.workunit.client.1.smithi194.stdout:7/522: write d3/d27/d6a/f82 [767037,73716] 0 2024-04-12T05:56:14.179 INFO:tasks.workunit.client.1.smithi194.stdout:2/615: creat d7/d22/d93/d9f/fd1 x:0 0 0 2024-04-12T05:56:14.183 INFO:tasks.workunit.client.1.smithi194.stdout:2/616: readlink d7/d12/d25/d70/l78 0 2024-04-12T05:56:14.184 INFO:tasks.workunit.client.1.smithi194.stdout:8/422: mknod d2/d8/d15/c83 0 2024-04-12T05:56:14.184 INFO:tasks.workunit.client.1.smithi194.stdout:7/523: write d3/d16/d23/d37/f54 [5076516,117566] 0 2024-04-12T05:56:14.185 INFO:tasks.workunit.client.1.smithi194.stdout:2/617: symlink d7/d22/d93/d9a/ld2 0 2024-04-12T05:56:14.189 INFO:tasks.workunit.client.1.smithi194.stdout:8/423: mkdir d2/d8/d9/de/d10/d7d/d84 0 2024-04-12T05:56:14.190 INFO:tasks.workunit.client.1.smithi194.stdout:2/618: creat d7/d22/d42/fd3 x:0 0 0 2024-04-12T05:56:14.193 INFO:tasks.workunit.client.1.smithi194.stdout:2/619: chown d7/d12/f3d 27 1 2024-04-12T05:56:14.195 INFO:tasks.workunit.client.1.smithi194.stdout:3/734: sync 2024-04-12T05:56:14.195 INFO:tasks.workunit.client.1.smithi194.stdout:1/696: sync 2024-04-12T05:56:14.195 INFO:tasks.workunit.client.1.smithi194.stdout:6/574: sync 2024-04-12T05:56:14.195 INFO:tasks.workunit.client.1.smithi194.stdout:0/591: sync 2024-04-12T05:56:14.196 INFO:tasks.workunit.client.1.smithi194.stdout:9/677: sync 2024-04-12T05:56:14.207 INFO:tasks.workunit.client.1.smithi194.stdout:8/424: dwrite d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:14.213 INFO:tasks.workunit.client.1.smithi194.stdout:4/772: dread d2/d3/d1b/d32/d3a/d3b/f86 [0,4194304] 0 2024-04-12T05:56:14.215 INFO:tasks.workunit.client.1.smithi194.stdout:6/575: creat d3/d15/d2b/d48/d6f/fb7 x:0 0 0 2024-04-12T05:56:14.215 INFO:tasks.workunit.client.1.smithi194.stdout:0/592: dread d0/f94 [0,4194304] 0 2024-04-12T05:56:14.216 INFO:tasks.workunit.client.1.smithi194.stdout:8/425: mknod d2/d8/d15/d3b/c85 0 2024-04-12T05:56:14.218 INFO:tasks.workunit.client.1.smithi194.stdout:7/524: dread d3/d27/f17 [0,4194304] 0 2024-04-12T05:56:14.223 INFO:tasks.workunit.client.1.smithi194.stdout:9/678: link d1/d19/d1f/d5f/fc6 d1/d19/d1f/fe4 0 2024-04-12T05:56:14.223 INFO:tasks.workunit.client.1.smithi194.stdout:6/576: fdatasync d3/d2f/d4e/d63/d83/f9b 0 2024-04-12T05:56:14.223 INFO:tasks.workunit.client.1.smithi194.stdout:8/426: mkdir d2/d8/d9/de/d86 0 2024-04-12T05:56:14.225 INFO:tasks.workunit.client.1.smithi194.stdout:7/525: chown d3/d16/d23/d40/d49/d74 8365 1 2024-04-12T05:56:14.230 INFO:tasks.workunit.client.1.smithi194.stdout:3/735: truncate d0/d1/d10/f44 2923040 0 2024-04-12T05:56:14.231 INFO:tasks.workunit.client.1.smithi194.stdout:7/526: symlink d3/d16/d23/d57/la8 0 2024-04-12T05:56:14.236 INFO:tasks.workunit.client.1.smithi194.stdout:2/620: dwrite d7/d12/d25/d5d/d47/d52/d7b/fa6 [0,4194304] 0 2024-04-12T05:56:14.236 INFO:tasks.workunit.client.1.smithi194.stdout:0/593: stat d0/da/d1e/d1a/d24/d4b/f65 0 2024-04-12T05:56:14.236 INFO:tasks.workunit.client.1.smithi194.stdout:8/427: stat d2/d8/d9/d1a/c5d 0 2024-04-12T05:56:14.244 INFO:tasks.workunit.client.1.smithi194.stdout:7/527: write d3/d27/d22/d62/f6c [782165,21159] 0 2024-04-12T05:56:14.244 INFO:tasks.workunit.client.1.smithi194.stdout:6/577: mkdir d3/d2d/db8 0 2024-04-12T05:56:14.249 INFO:tasks.workunit.client.1.smithi194.stdout:9/679: dwrite d1/da6/fa8 [0,4194304] 0 2024-04-12T05:56:14.250 INFO:tasks.workunit.client.1.smithi194.stdout:3/736: fdatasync d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/fc7 0 2024-04-12T05:56:14.252 INFO:tasks.workunit.client.1.smithi194.stdout:1/697: dwrite d0/d11/d18/d19/d1c/f46 [0,4194304] 0 2024-04-12T05:56:14.260 INFO:tasks.workunit.client.1.smithi194.stdout:6/578: creat d3/d15/fb9 x:0 0 0 2024-04-12T05:56:14.262 INFO:tasks.workunit.client.1.smithi194.stdout:8/428: mknod d2/d8/d9/de/d10/c87 0 2024-04-12T05:56:14.262 INFO:tasks.workunit.client.1.smithi194.stdout:9/680: creat d1/d41/fe5 x:0 0 0 2024-04-12T05:56:14.274 INFO:tasks.workunit.client.1.smithi194.stdout:7/528: dread d3/d27/f17 [0,4194304] 0 2024-04-12T05:56:14.274 INFO:tasks.workunit.client.1.smithi194.stdout:4/773: dread d2/d3/ff [0,4194304] 0 2024-04-12T05:56:14.274 INFO:tasks.workunit.client.1.smithi194.stdout:8/429: rename d2/d8/d9/de/d10/f41 to d2/d8/d9/de/d10/f88 0 2024-04-12T05:56:14.274 INFO:tasks.workunit.client.1.smithi194.stdout:8/430: dread - d2/d8/f82 zero size 2024-04-12T05:56:14.279 INFO:tasks.workunit.client.1.smithi194.stdout:2/621: dwrite d7/d12/d1a/d68/f8c [0,4194304] 0 2024-04-12T05:56:14.282 INFO:tasks.workunit.client.1.smithi194.stdout:4/774: stat d2/d3/d58/d82/dd0/dba 0 2024-04-12T05:56:14.282 INFO:tasks.workunit.client.1.smithi194.stdout:1/698: dread d0/d11/d18/d19/f80 [0,4194304] 0 2024-04-12T05:56:14.282 INFO:tasks.workunit.client.1.smithi194.stdout:2/622: stat d7/c92 0 2024-04-12T05:56:14.285 INFO:tasks.workunit.client.1.smithi194.stdout:2/623: rmdir d7/d12/d1a 39 2024-04-12T05:56:14.290 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:14 smithi008 ceph-mon[39722]: pgmap v33: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 38 MiB/s rd, 129 MiB/s wr, 366 op/s 2024-04-12T05:56:14.290 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:14 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:14.292 INFO:tasks.workunit.client.1.smithi194.stdout:8/431: getdents d2/d8/d58 0 2024-04-12T05:56:14.296 INFO:tasks.workunit.client.1.smithi194.stdout:4/775: mknod d2/d85/c118 0 2024-04-12T05:56:14.297 INFO:tasks.workunit.client.1.smithi194.stdout:4/776: chown d2/d11/d1e/lc6 6794 1 2024-04-12T05:56:14.300 INFO:tasks.workunit.client.1.smithi194.stdout:1/699: symlink d0/d11/d18/d35/le4 0 2024-04-12T05:56:14.301 INFO:tasks.workunit.client.1.smithi194.stdout:3/737: dread d0/d1/d74/d39/d4d/deb/d3d/d6e/f77 [0,4194304] 0 2024-04-12T05:56:14.301 INFO:tasks.workunit.client.1.smithi194.stdout:8/432: truncate d2/f12 525341 0 2024-04-12T05:56:14.302 INFO:tasks.workunit.client.1.smithi194.stdout:7/529: link d3/d16/d23/d40/d49/d74/c88 d3/d27/d22/d62/ca9 0 2024-04-12T05:56:14.304 INFO:tasks.workunit.client.1.smithi194.stdout:7/530: fdatasync d3/d27/f97 0 2024-04-12T05:56:14.304 INFO:tasks.workunit.client.1.smithi194.stdout:8/433: read - d2/d8/d66/d67/f71 zero size 2024-04-12T05:56:14.307 INFO:tasks.workunit.client.1.smithi194.stdout:4/777: rename d2/d11/d46/c7d to d2/d11/d1e/d2f/d4a/d9c/dee/c119 0 2024-04-12T05:56:14.307 INFO:tasks.workunit.client.1.smithi194.stdout:2/624: creat d7/d12/d25/d5d/d47/dac/dc2/fd4 x:0 0 0 2024-04-12T05:56:14.307 INFO:tasks.workunit.client.1.smithi194.stdout:6/579: truncate d3/d2f/f39 3205255 0 2024-04-12T05:56:14.309 INFO:tasks.workunit.client.1.smithi194.stdout:9/681: write d1/d19/d1f/d20/d37/d38/f5c [2320443,58341] 0 2024-04-12T05:56:14.311 INFO:tasks.workunit.client.1.smithi194.stdout:7/531: creat d3/d27/d6a/faa x:0 0 0 2024-04-12T05:56:14.311 INFO:tasks.workunit.client.1.smithi194.stdout:1/700: unlink d0/d11/d18/d19/d1c/lb0 0 2024-04-12T05:56:14.314 INFO:tasks.workunit.client.1.smithi194.stdout:8/434: dread d2/d8/d15/d33/f29 [0,4194304] 0 2024-04-12T05:56:14.317 INFO:tasks.workunit.client.1.smithi194.stdout:6/580: read - d3/d2d/d33/fa1 zero size 2024-04-12T05:56:14.317 INFO:tasks.workunit.client.1.smithi194.stdout:1/701: symlink d0/d11/d18/le5 0 2024-04-12T05:56:14.322 INFO:tasks.workunit.client.1.smithi194.stdout:1/702: chown d0/d11/d18/d19/d1a/d6a/d27/d2e/fb3 33184 1 2024-04-12T05:56:14.323 INFO:tasks.workunit.client.1.smithi194.stdout:4/778: read d2/d11/d1e/d2f/f112 [108269,83257] 0 2024-04-12T05:56:14.323 INFO:tasks.workunit.client.1.smithi194.stdout:7/532: link d3/d27/f2c d3/d16/d23/d57/fab 0 2024-04-12T05:56:14.327 INFO:tasks.workunit.client.1.smithi194.stdout:3/738: rename d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/fd5 to d0/d1/d10/d8c/d8f/ddb/ff5 0 2024-04-12T05:56:14.330 INFO:tasks.workunit.client.1.smithi194.stdout:4/779: mknod d2/dcf/c11a 0 2024-04-12T05:56:14.334 INFO:tasks.workunit.client.1.smithi194.stdout:2/625: link d7/d12/d1a/d68/f8c d7/d22/d93/d9f/fd5 0 2024-04-12T05:56:14.335 INFO:tasks.workunit.client.1.smithi194.stdout:6/581: creat d3/d2d/d33/d58/fba x:0 0 0 2024-04-12T05:56:14.340 INFO:tasks.workunit.client.1.smithi194.stdout:9/682: mkdir d1/d21/d77/d98/d9f/de6 0 2024-04-12T05:56:14.340 INFO:tasks.workunit.client.1.smithi194.stdout:1/703: rmdir d0/d11/d18/d19/d1a/d6a/d27/da8 39 2024-04-12T05:56:14.342 INFO:tasks.workunit.client.1.smithi194.stdout:7/533: symlink d3/d16/d23/d40/d49/d74/lac 0 2024-04-12T05:56:14.342 INFO:tasks.workunit.client.1.smithi194.stdout:4/780: dwrite d2/d3/d58/d82/dd0/fe8 [0,4194304] 0 2024-04-12T05:56:14.343 INFO:tasks.workunit.client.1.smithi194.stdout:8/435: rename d2/d8/d9/d1a/c5d to d2/d8/d15/d35/d6c/c89 0 2024-04-12T05:56:14.344 INFO:tasks.workunit.client.1.smithi194.stdout:8/436: chown d2/d8/d9/f48 37245088 1 2024-04-12T05:56:14.345 INFO:tasks.workunit.client.1.smithi194.stdout:9/683: chown d1/d19/d1f/c66 1739434466 1 2024-04-12T05:56:14.346 INFO:tasks.workunit.client.1.smithi194.stdout:3/739: unlink d0/d1/d74/d39/fbb 0 2024-04-12T05:56:14.347 INFO:tasks.workunit.client.1.smithi194.stdout:2/626: mkdir d7/d12/d1a/d69/dd6 0 2024-04-12T05:56:14.349 INFO:tasks.workunit.client.1.smithi194.stdout:1/704: read d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 [21490,39163] 0 2024-04-12T05:56:14.351 INFO:tasks.workunit.client.1.smithi194.stdout:6/582: rename d3/d2f/d4e/d53/c6d to d3/d34/cbb 0 2024-04-12T05:56:14.354 INFO:tasks.workunit.client.1.smithi194.stdout:4/781: truncate d2/d11/d93/da6/dab/d5d/dae/de1/faa 2016960 0 2024-04-12T05:56:14.355 INFO:tasks.workunit.client.1.smithi194.stdout:9/684: unlink d1/d19/d1f/f73 0 2024-04-12T05:56:14.359 INFO:tasks.workunit.client.1.smithi194.stdout:2/627: dwrite d7/d12/d25/d5d/d47/f88 [0,4194304] 0 2024-04-12T05:56:14.362 INFO:tasks.workunit.client.1.smithi194.stdout:2/628: write d7/d22/f91 [83974,4674] 0 2024-04-12T05:56:14.362 INFO:tasks.workunit.client.1.smithi194.stdout:3/740: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f46 2761341 0 2024-04-12T05:56:14.364 INFO:tasks.workunit.client.1.smithi194.stdout:2/629: dread - d7/d12/f8d zero size 2024-04-12T05:56:14.364 INFO:tasks.workunit.client.1.smithi194.stdout:5/719: sync 2024-04-12T05:56:14.366 INFO:tasks.workunit.client.1.smithi194.stdout:4/782: rename d2/d103/f10c to d2/d11/d43/f11b 0 2024-04-12T05:56:14.371 INFO:tasks.workunit.client.1.smithi194.stdout:4/783: dwrite d2/d3/d1b/d32/d3a/d3b/f9f [0,4194304] 0 2024-04-12T05:56:14.415 INFO:tasks.workunit.client.1.smithi194.stdout:4/784: dwrite d2/d11/d1e/d2f/f76 [0,4194304] 0 2024-04-12T05:56:14.430 INFO:tasks.workunit.client.1.smithi194.stdout:6/583: link d3/d15/d2b/d48/d6f/d7a/d8f/fb5 d3/d2f/d4e/d9a/db3/fbc 0 2024-04-12T05:56:14.432 INFO:tasks.workunit.client.1.smithi194.stdout:2/630: rename d7/d12/d34 to d7/d12/d25/d5d/d47/dac/db3/dd7 0 2024-04-12T05:56:14.433 INFO:tasks.workunit.client.1.smithi194.stdout:7/534: dwrite d3/d16/d23/d57/d72/d81/f73 [0,4194304] 0 2024-04-12T05:56:14.441 INFO:tasks.workunit.client.1.smithi194.stdout:6/584: creat d3/d15/d2b/d48/d6f/d7a/d8f/fbd x:0 0 0 2024-04-12T05:56:14.442 INFO:tasks.workunit.client.1.smithi194.stdout:2/631: rename d7/d12/d1a/d68/l6a to d7/d12/d25/ld8 0 2024-04-12T05:56:14.444 INFO:tasks.workunit.client.1.smithi194.stdout:1/705: truncate d0/d11/d18/d19/f39 2294741 0 2024-04-12T05:56:14.446 INFO:tasks.workunit.client.1.smithi194.stdout:7/535: symlink d3/d16/d23/lad 0 2024-04-12T05:56:14.449 INFO:tasks.workunit.client.1.smithi194.stdout:7/536: stat d3/d16/c90 0 2024-04-12T05:56:14.450 INFO:tasks.workunit.client.1.smithi194.stdout:1/706: unlink d0/d11/d18/d19/d1a/fe2 0 2024-04-12T05:56:14.450 INFO:tasks.workunit.client.1.smithi194.stdout:4/785: dwrite d2/f21 [0,4194304] 0 2024-04-12T05:56:14.451 INFO:tasks.workunit.client.1.smithi194.stdout:9/685: dwrite d1/d19/d1f/fe4 [0,4194304] 0 2024-04-12T05:56:14.467 INFO:tasks.workunit.client.1.smithi194.stdout:2/632: getdents d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a 0 2024-04-12T05:56:14.467 INFO:tasks.workunit.client.1.smithi194.stdout:7/537: creat d3/d27/d22/d8e/fae x:0 0 0 2024-04-12T05:56:14.468 INFO:tasks.workunit.client.1.smithi194.stdout:1/707: rmdir d0/d11/d18/d19/d1a/d6a/d27/d48/da7 39 2024-04-12T05:56:14.471 INFO:tasks.workunit.client.1.smithi194.stdout:3/741: dwrite d0/d1/d74/d39/da2/fc8 [0,4194304] 0 2024-04-12T05:56:14.474 INFO:tasks.workunit.client.1.smithi194.stdout:1/708: fsync d0/d11/d18/d19/fda 0 2024-04-12T05:56:14.476 INFO:tasks.workunit.client.1.smithi194.stdout:7/538: chown d3/d16/d5d/f84 15384138 1 2024-04-12T05:56:14.477 INFO:tasks.workunit.client.1.smithi194.stdout:2/633: unlink d7/d12/d1a/d68/c6b 0 2024-04-12T05:56:14.478 INFO:tasks.workunit.client.1.smithi194.stdout:5/720: dwrite d1/d4/d6/db/d2b/f66 [0,4194304] 0 2024-04-12T05:56:14.479 INFO:tasks.workunit.client.1.smithi194.stdout:4/786: dwrite d2/d85/fd5 [0,4194304] 0 2024-04-12T05:56:14.482 INFO:tasks.workunit.client.1.smithi194.stdout:9/686: dwrite d1/d41/dba/fd2 [0,4194304] 0 2024-04-12T05:56:14.483 INFO:tasks.workunit.client.1.smithi194.stdout:6/585: dread d3/d2d/d33/d52/d5c/fa5 [0,4194304] 0 2024-04-12T05:56:14.487 INFO:tasks.workunit.client.1.smithi194.stdout:4/787: write d2/d3/d58/d82/dd0/fe3 [945451,89559] 0 2024-04-12T05:56:14.489 INFO:tasks.workunit.client.1.smithi194.stdout:2/634: readlink d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/l76 0 2024-04-12T05:56:14.491 INFO:tasks.workunit.client.1.smithi194.stdout:6/586: dread - d3/d2f/d4e/d53/fa8 zero size 2024-04-12T05:56:14.493 INFO:tasks.workunit.client.1.smithi194.stdout:4/788: chown d2/d3/d58/d82/dd0/fe3 192685206 1 2024-04-12T05:56:14.493 INFO:tasks.workunit.client.1.smithi194.stdout:3/742: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d2d/ff6 x:0 0 0 2024-04-12T05:56:14.493 INFO:tasks.workunit.client.1.smithi194.stdout:1/709: readlink d0/d11/d18/d19/d1a/d6a/d27/l94 0 2024-04-12T05:56:14.493 INFO:tasks.workunit.client.1.smithi194.stdout:7/539: symlink d3/d16/d23/d40/d9b/laf 0 2024-04-12T05:56:14.497 INFO:tasks.workunit.client.1.smithi194.stdout:7/540: chown d3/f48 43264624 1 2024-04-12T05:56:14.498 INFO:tasks.workunit.client.1.smithi194.stdout:3/743: dread - d0/d1/d10/d8c/fdc zero size 2024-04-12T05:56:14.498 INFO:tasks.workunit.client.1.smithi194.stdout:6/587: mknod d3/cbe 0 2024-04-12T05:56:14.500 INFO:tasks.workunit.client.1.smithi194.stdout:9/687: getdents d1/d19/d1f/d20/d37/d84 0 2024-04-12T05:56:14.504 INFO:tasks.workunit.client.1.smithi194.stdout:2/635: write d7/d12/d25/d5d/d47/dac/dc2/fd4 [768397,115609] 0 2024-04-12T05:56:14.506 INFO:tasks.workunit.client.1.smithi194.stdout:1/710: write d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [8266688,39396] 0 2024-04-12T05:56:14.508 INFO:tasks.workunit.client.1.smithi194.stdout:6/588: symlink d3/d70/lbf 0 2024-04-12T05:56:14.509 INFO:tasks.workunit.client.1.smithi194.stdout:5/721: dwrite d1/d5c/da7/ff3 [0,4194304] 0 2024-04-12T05:56:14.512 INFO:tasks.workunit.client.1.smithi194.stdout:3/744: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/dc5/fcc [4153617,45611] 0 2024-04-12T05:56:14.514 INFO:tasks.workunit.client.1.smithi194.stdout:6/589: chown d3/d2d/la9 4 1 2024-04-12T05:56:14.514 INFO:tasks.workunit.client.1.smithi194.stdout:4/789: getdents d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb 0 2024-04-12T05:56:14.520 INFO:tasks.workunit.client.1.smithi194.stdout:1/711: chown d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fb1 661 1 2024-04-12T05:56:14.520 INFO:tasks.workunit.client.1.smithi194.stdout:6/590: unlink d3/d2f/d4e/d63/d83/f9b 0 2024-04-12T05:56:14.528 INFO:tasks.workunit.client.1.smithi194.stdout:1/712: mkdir d0/d11/ddd/de6 0 2024-04-12T05:56:14.528 INFO:tasks.workunit.client.1.smithi194.stdout:2/636: dwrite d7/d22/f4d [4194304,4194304] 0 2024-04-12T05:56:14.529 INFO:tasks.workunit.client.1.smithi194.stdout:3/745: creat d0/d1/d10/d8c/d8f/ddb/ff7 x:0 0 0 2024-04-12T05:56:14.529 INFO:tasks.workunit.client.1.smithi194.stdout:4/790: symlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/l11c 0 2024-04-12T05:56:14.531 INFO:tasks.workunit.client.1.smithi194.stdout:7/541: dwrite d3/d16/d23/d57/d72/d81/f73 [0,4194304] 0 2024-04-12T05:56:14.532 INFO:tasks.workunit.client.1.smithi194.stdout:6/591: truncate d3/d2d/f3d 321505 0 2024-04-12T05:56:14.533 INFO:tasks.workunit.client.1.smithi194.stdout:5/722: write d1/d4/d6/db/d15/d26/dbb/ff1 [193251,95804] 0 2024-04-12T05:56:14.535 INFO:tasks.workunit.client.1.smithi194.stdout:6/592: stat d3/d34 0 2024-04-12T05:56:14.540 INFO:tasks.workunit.client.1.smithi194.stdout:9/688: dwrite d1/d19/d1f/d20/f7b [0,4194304] 0 2024-04-12T05:56:14.542 INFO:tasks.workunit.client.1.smithi194.stdout:3/746: write d0/d1/d10/d89/fd4 [663896,13685] 0 2024-04-12T05:56:14.543 INFO:tasks.workunit.client.1.smithi194.stdout:2/637: creat d7/d12/d25/d5d/d47/d52/fd9 x:0 0 0 2024-04-12T05:56:14.545 INFO:tasks.workunit.client.1.smithi194.stdout:1/713: rename d0/d11/d18/d35/d6f to d0/d11/d18/d19/d1a/d6a/d27/d5a/de7 0 2024-04-12T05:56:14.546 INFO:tasks.workunit.client.1.smithi194.stdout:7/542: write d3/d27/d22/f38 [4546292,54423] 0 2024-04-12T05:56:14.547 INFO:tasks.workunit.client.1.smithi194.stdout:3/747: truncate d0/d1/d74/d39/fb8 345627 0 2024-04-12T05:56:14.550 INFO:tasks.workunit.client.1.smithi194.stdout:2/638: creat d7/d12/d1a/dbf/fda x:0 0 0 2024-04-12T05:56:14.550 INFO:tasks.workunit.client.1.smithi194.stdout:4/791: getdents d2/d11/d1e/d2f/d4a/d9c 0 2024-04-12T05:56:14.555 INFO:tasks.workunit.client.1.smithi194.stdout:7/543: write d3/d27/f39 [2355242,83274] 0 2024-04-12T05:56:14.558 INFO:tasks.workunit.client.1.smithi194.stdout:4/792: mkdir d2/d85/d11d 0 2024-04-12T05:56:14.559 INFO:tasks.workunit.client.1.smithi194.stdout:3/748: dread - d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d2d/ff6 zero size 2024-04-12T05:56:14.561 INFO:tasks.workunit.client.1.smithi194.stdout:5/723: dwrite d1/d4/d75/fbf [0,4194304] 0 2024-04-12T05:56:14.562 INFO:tasks.workunit.client.1.smithi194.stdout:0/594: sync 2024-04-12T05:56:14.566 INFO:tasks.workunit.client.1.smithi194.stdout:1/714: write d0/d11/d18/d19/d1a/d6a/dac/db5/f82 [3624686,76718] 0 2024-04-12T05:56:14.567 INFO:tasks.workunit.client.1.smithi194.stdout:4/793: unlink d2/d11/d46/d61/d63/f101 0 2024-04-12T05:56:14.567 INFO:tasks.workunit.client.1.smithi194.stdout:9/689: dwrite d1/d19/d1f/d20/d37/d38/f8f [0,4194304] 0 2024-04-12T05:56:14.568 INFO:tasks.workunit.client.1.smithi194.stdout:2/639: rename d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/c66 to d7/cdb 0 2024-04-12T05:56:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:14 smithi194 ceph-mon[49771]: pgmap v33: 65 pgs: 65 active+clean; 1.2 GiB data, 3.9 GiB used, 532 GiB / 536 GiB avail; 38 MiB/s rd, 129 MiB/s wr, 366 op/s 2024-04-12T05:56:14.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:14 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:14.572 INFO:tasks.workunit.client.1.smithi194.stdout:0/595: stat d0/da/d4a/d8e/f72 0 2024-04-12T05:56:14.574 INFO:tasks.workunit.client.1.smithi194.stdout:5/724: symlink d1/d5c/d95/lf4 0 2024-04-12T05:56:14.574 INFO:tasks.workunit.client.1.smithi194.stdout:3/749: mknod d0/d1/d74/d39/d4d/deb/d8e/cf8 0 2024-04-12T05:56:14.577 INFO:tasks.workunit.client.1.smithi194.stdout:1/715: mknod d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/ce8 0 2024-04-12T05:56:14.578 INFO:tasks.workunit.client.1.smithi194.stdout:4/794: truncate d2/d11/d93/fb8 4376034 0 2024-04-12T05:56:14.579 INFO:tasks.workunit.client.1.smithi194.stdout:9/690: creat d1/d21/d77/d98/d9f/dde/fe7 x:0 0 0 2024-04-12T05:56:14.581 INFO:tasks.workunit.client.1.smithi194.stdout:2/640: unlink d7/d12/d25/d5d/d47/f88 0 2024-04-12T05:56:14.585 INFO:tasks.workunit.client.1.smithi194.stdout:4/795: creat d2/d11/d1e/d2f/f11e x:0 0 0 2024-04-12T05:56:14.586 INFO:tasks.workunit.client.1.smithi194.stdout:6/593: dread d3/d15/f21 [0,4194304] 0 2024-04-12T05:56:14.586 INFO:tasks.workunit.client.1.smithi194.stdout:2/641: write d7/d12/d6d/f72 [876643,116409] 0 2024-04-12T05:56:14.589 INFO:tasks.workunit.client.1.smithi194.stdout:1/716: symlink d0/d11/d18/d19/d1a/d6a/d27/d48/le9 0 2024-04-12T05:56:14.591 INFO:tasks.workunit.client.1.smithi194.stdout:0/596: dwrite d0/da/d1e/d1a/d24/d4b/f6f [0,4194304] 0 2024-04-12T05:56:14.596 INFO:tasks.workunit.client.1.smithi194.stdout:8/437: sync 2024-04-12T05:56:14.599 INFO:tasks.workunit.client.1.smithi194.stdout:4/796: chown d2/d3/d1b/d32/d3a/d3b/l10a 33399 1 2024-04-12T05:56:14.601 INFO:tasks.workunit.client.1.smithi194.stdout:6/594: creat d3/d15/d2b/d48/d6f/d7a/d8f/fc0 x:0 0 0 2024-04-12T05:56:14.602 INFO:tasks.workunit.client.1.smithi194.stdout:9/691: dwrite d1/d19/d1f/d20/fcd [0,4194304] 0 2024-04-12T05:56:14.602 INFO:tasks.workunit.client.1.smithi194.stdout:7/544: truncate d3/d16/f45 1431507 0 2024-04-12T05:56:14.606 INFO:tasks.workunit.client.1.smithi194.stdout:9/692: chown d1/d4e/c79 6716268 1 2024-04-12T05:56:14.607 INFO:tasks.workunit.client.1.smithi194.stdout:5/725: truncate d1/d4/d6/d34/fa0 1439004 0 2024-04-12T05:56:14.610 INFO:tasks.workunit.client.1.smithi194.stdout:6/595: truncate d3/d2f/fa7 95629 0 2024-04-12T05:56:14.613 INFO:tasks.workunit.client.1.smithi194.stdout:4/797: creat d2/d3/f11f x:0 0 0 2024-04-12T05:56:14.613 INFO:tasks.workunit.client.1.smithi194.stdout:7/545: write d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f76 [1280327,111852] 0 2024-04-12T05:56:14.616 INFO:tasks.workunit.client.1.smithi194.stdout:0/597: mknod d0/d5f/d73/cbd 0 2024-04-12T05:56:14.616 INFO:tasks.workunit.client.1.smithi194.stdout:3/750: dwrite d0/d1/d74/d39/d4d/deb/d3d/f69 [0,4194304] 0 2024-04-12T05:56:14.624 INFO:tasks.workunit.client.1.smithi194.stdout:4/798: dread d2/d11/d93/da6/dab/d5d/dae/db2/d57/fa4 [0,4194304] 0 2024-04-12T05:56:14.624 INFO:tasks.workunit.client.1.smithi194.stdout:8/438: creat d2/d8/d9/de/d10/d4e/f8a x:0 0 0 2024-04-12T05:56:14.624 INFO:tasks.workunit.client.1.smithi194.stdout:2/642: rename d7/d22/d93/lc3 to d7/d12/d25/d5d/d47/dac/ldc 0 2024-04-12T05:56:14.626 INFO:tasks.workunit.client.1.smithi194.stdout:4/799: stat d2/d3/d58/d82/dd0/ff2 0 2024-04-12T05:56:14.626 INFO:tasks.workunit.client.1.smithi194.stdout:5/726: creat d1/d4/d75/ff5 x:0 0 0 2024-04-12T05:56:14.627 INFO:tasks.workunit.client.1.smithi194.stdout:9/693: chown d1/d19/d1f/d20/d37/d84/c95 5755 1 2024-04-12T05:56:14.628 INFO:tasks.workunit.client.1.smithi194.stdout:6/596: unlink d3/f3a 0 2024-04-12T05:56:14.630 INFO:tasks.workunit.client.1.smithi194.stdout:1/717: dwrite d0/d11/d18/d19/d1a/d6a/dac/db5/d68/f87 [0,4194304] 0 2024-04-12T05:56:14.630 INFO:tasks.workunit.client.1.smithi194.stdout:7/546: stat d3/d27/f44 0 2024-04-12T05:56:14.632 INFO:tasks.workunit.client.1.smithi194.stdout:0/598: mkdir d0/d3b/d52/d7c/dbe 0 2024-04-12T05:56:14.633 INFO:tasks.workunit.client.1.smithi194.stdout:3/751: creat d0/d1/d10/d8c/ff9 x:0 0 0 2024-04-12T05:56:14.634 INFO:tasks.workunit.client.1.smithi194.stdout:1/718: fdatasync d0/d11/fae 0 2024-04-12T05:56:14.635 INFO:tasks.workunit.client.1.smithi194.stdout:6/597: dread - d3/d2f/d4e/d63/faa zero size 2024-04-12T05:56:14.635 INFO:tasks.workunit.client.1.smithi194.stdout:8/439: symlink d2/d8/d9/de/d10/d7d/l8b 0 2024-04-12T05:56:14.637 INFO:tasks.workunit.client.1.smithi194.stdout:4/800: chown d2/d3/d1b/d32/d3a/d3b/f90 49777496 1 2024-04-12T05:56:14.639 INFO:tasks.workunit.client.1.smithi194.stdout:5/727: dread d1/d4/d6/db/f53 [0,4194304] 0 2024-04-12T05:56:14.641 INFO:tasks.workunit.client.1.smithi194.stdout:7/547: creat d3/d27/d22/d8e/fb0 x:0 0 0 2024-04-12T05:56:14.644 INFO:tasks.workunit.client.1.smithi194.stdout:3/752: read d0/d1/d4f/fa3 [44920,72909] 0 2024-04-12T05:56:14.646 INFO:tasks.workunit.client.1.smithi194.stdout:1/719: read d0/d11/d18/d19/f80 [1801404,42927] 0 2024-04-12T05:56:14.648 INFO:tasks.workunit.client.1.smithi194.stdout:7/548: chown d3/d16/c9d 947005 1 2024-04-12T05:56:14.648 INFO:tasks.workunit.client.1.smithi194.stdout:4/801: creat d2/d11/d93/da6/dab/d5d/dae/db2/d57/f120 x:0 0 0 2024-04-12T05:56:14.652 INFO:tasks.workunit.client.1.smithi194.stdout:2/643: dread d7/d22/d42/f77 [0,4194304] 0 2024-04-12T05:56:14.652 INFO:tasks.workunit.client.1.smithi194.stdout:6/598: fdatasync d3/d2f/f39 0 2024-04-12T05:56:14.653 INFO:tasks.workunit.client.1.smithi194.stdout:3/753: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/f84 [2353398,100720] 0 2024-04-12T05:56:14.654 INFO:tasks.workunit.client.1.smithi194.stdout:9/694: dread d1/d19/d1f/f4b [0,4194304] 0 2024-04-12T05:56:14.657 INFO:tasks.workunit.client.1.smithi194.stdout:0/599: link d0/da/d1e/d1a/d24/d4b/f6f d0/d5f/d73/d76/fbf 0 2024-04-12T05:56:14.660 INFO:tasks.workunit.client.1.smithi194.stdout:4/802: write d2/d3/d1b/d32/d3a/d3b/f90 [376921,29167] 0 2024-04-12T05:56:14.661 INFO:tasks.workunit.client.1.smithi194.stdout:7/549: mknod d3/d27/d22/d8e/cb1 0 2024-04-12T05:56:14.662 INFO:tasks.workunit.client.1.smithi194.stdout:2/644: chown d7/d12/d25/d5d/d47/d52/d7b/c96 52432013 1 2024-04-12T05:56:14.663 INFO:tasks.workunit.client.1.smithi194.stdout:5/728: dwrite d1/d4/d6/d34/d52/fcb [0,4194304] 0 2024-04-12T05:56:14.668 INFO:tasks.workunit.client.1.smithi194.stdout:6/599: truncate d3/f1f 3462138 0 2024-04-12T05:56:14.669 INFO:tasks.workunit.client.1.smithi194.stdout:8/440: link d2/d8/d9/de/d10/f88 d2/d8/d15/f8c 0 2024-04-12T05:56:14.670 INFO:tasks.workunit.client.1.smithi194.stdout:3/754: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/f8b [976041,51228] 0 2024-04-12T05:56:14.674 INFO:tasks.workunit.client.1.smithi194.stdout:2/645: write d7/d12/d1a/f44 [1225749,49740] 0 2024-04-12T05:56:14.676 INFO:tasks.workunit.client.1.smithi194.stdout:4/803: creat d2/d3/d58/d82/dd0/f121 x:0 0 0 2024-04-12T05:56:14.677 INFO:tasks.workunit.client.1.smithi194.stdout:9/695: dwrite d1/d41/dba/fd2 [0,4194304] 0 2024-04-12T05:56:14.679 INFO:tasks.workunit.client.1.smithi194.stdout:3/755: read - d0/d1/d74/d39/d4d/deb/d3d/fc0 zero size 2024-04-12T05:56:14.679 INFO:tasks.workunit.client.1.smithi194.stdout:2/646: chown d7/d22/d93/d9f/fb0 117731 1 2024-04-12T05:56:14.680 INFO:tasks.workunit.client.1.smithi194.stdout:5/729: unlink d1/d4/d6/d34/d52/fcb 0 2024-04-12T05:56:14.681 INFO:tasks.workunit.client.1.smithi194.stdout:3/756: chown d0/d1/d74/d39/d4d/deb/d8e 9026 1 2024-04-12T05:56:14.684 INFO:tasks.workunit.client.1.smithi194.stdout:7/550: symlink d3/d16/d23/d37/d95/lb2 0 2024-04-12T05:56:14.686 INFO:tasks.workunit.client.1.smithi194.stdout:0/600: fsync d0/da/d1e/d1a/d24/d4b/f6f 0 2024-04-12T05:56:14.688 INFO:tasks.workunit.client.1.smithi194.stdout:1/720: dread d0/d11/d18/d19/d1c/f20 [0,4194304] 0 2024-04-12T05:56:14.692 INFO:tasks.workunit.client.1.smithi194.stdout:4/804: write d2/fc [2608418,97383] 0 2024-04-12T05:56:14.692 INFO:tasks.workunit.client.1.smithi194.stdout:4/805: dread - d2/f4c zero size 2024-04-12T05:56:14.693 INFO:tasks.workunit.client.1.smithi194.stdout:3/757: creat d0/d1/d74/d39/d4d/deb/d1e/d33/ffa x:0 0 0 2024-04-12T05:56:14.695 INFO:tasks.workunit.client.1.smithi194.stdout:7/551: creat d3/d16/d23/d37/fb3 x:0 0 0 2024-04-12T05:56:14.696 INFO:tasks.workunit.client.1.smithi194.stdout:8/441: creat d2/d8/d9/de/d86/f8d x:0 0 0 2024-04-12T05:56:14.700 INFO:tasks.workunit.client.1.smithi194.stdout:0/601: creat d0/d5f/fc0 x:0 0 0 2024-04-12T05:56:14.702 INFO:tasks.workunit.client.1.smithi194.stdout:8/442: write d2/d8/f82 [400222,75028] 0 2024-04-12T05:56:14.706 INFO:tasks.workunit.client.1.smithi194.stdout:9/696: getdents d1/d4e 0 2024-04-12T05:56:14.711 INFO:tasks.workunit.client.1.smithi194.stdout:0/602: mknod d0/da/da7/cc1 0 2024-04-12T05:56:14.711 INFO:tasks.workunit.client.1.smithi194.stdout:7/552: dwrite d3/d27/d22/f38 [0,4194304] 0 2024-04-12T05:56:14.711 INFO:tasks.workunit.client.1.smithi194.stdout:6/600: dread d3/d2d/d33/d52/f89 [0,4194304] 0 2024-04-12T05:56:14.712 INFO:tasks.workunit.client.1.smithi194.stdout:3/758: write d0/d1/d4f/fa3 [375793,122784] 0 2024-04-12T05:56:14.715 INFO:tasks.workunit.client.1.smithi194.stdout:4/806: dwrite d2/d11/d46/d61/f92 [4194304,4194304] 0 2024-04-12T05:56:14.715 INFO:tasks.workunit.client.1.smithi194.stdout:2/647: chown d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/faf 488655109 1 2024-04-12T05:56:14.721 INFO:tasks.workunit.client.1.smithi194.stdout:4/807: readlink d2/d3/d1b/l29 0 2024-04-12T05:56:14.722 INFO:tasks.workunit.client.1.smithi194.stdout:2/648: read - d7/d12/d25/d5d/d47/dac/db3/dd7/f97 zero size 2024-04-12T05:56:14.722 INFO:tasks.workunit.client.1.smithi194.stdout:9/697: chown d1/d19/d1f/d20/f3d 6476 1 2024-04-12T05:56:14.729 INFO:tasks.workunit.client.1.smithi194.stdout:4/808: truncate d2/d3/d58/d82/dd0/f121 304540 0 2024-04-12T05:56:14.730 INFO:tasks.workunit.client.1.smithi194.stdout:8/443: dwrite d2/f12 [0,4194304] 0 2024-04-12T05:56:14.734 INFO:tasks.workunit.client.1.smithi194.stdout:7/553: symlink d3/d16/d23/d37/d3d/d56/lb4 0 2024-04-12T05:56:14.735 INFO:tasks.workunit.client.1.smithi194.stdout:1/721: truncate d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b 7711707 0 2024-04-12T05:56:14.735 INFO:tasks.workunit.client.1.smithi194.stdout:8/444: read d2/d8/d15/d33/f18 [1975240,118465] 0 2024-04-12T05:56:14.738 INFO:tasks.workunit.client.1.smithi194.stdout:0/603: rename d0/da/d1e/d1a/d58/d5b/l98 to d0/da/d1e/d1a/d24/d56/lc2 0 2024-04-12T05:56:14.739 INFO:tasks.workunit.client.1.smithi194.stdout:2/649: mkdir d7/d12/d1a/d68/ddd 0 2024-04-12T05:56:14.740 INFO:tasks.workunit.client.1.smithi194.stdout:8/445: read d2/f56 [88575,36663] 0 2024-04-12T05:56:14.741 INFO:tasks.workunit.client.1.smithi194.stdout:3/759: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f71 784227 0 2024-04-12T05:56:14.744 INFO:tasks.workunit.client.1.smithi194.stdout:2/650: symlink d7/d12/d25/lde 0 2024-04-12T05:56:14.745 INFO:tasks.workunit.client.1.smithi194.stdout:6/601: getdents d3/d2f/d4e/d9a/db3 0 2024-04-12T05:56:14.747 INFO:tasks.workunit.client.1.smithi194.stdout:8/446: fdatasync d2/d8/d9/de/d38/f3a 0 2024-04-12T05:56:14.751 INFO:tasks.workunit.client.1.smithi194.stdout:9/698: getdents d1/d41/dba/dc2 0 2024-04-12T05:56:14.753 INFO:tasks.workunit.client.1.smithi194.stdout:1/722: dwrite d0/d11/d18/d19/d1a/d6a/dac/fce [0,4194304] 0 2024-04-12T05:56:14.757 INFO:tasks.workunit.client.1.smithi194.stdout:6/602: rmdir d3/d1a 39 2024-04-12T05:56:14.757 INFO:tasks.workunit.client.1.smithi194.stdout:8/447: dread d2/d8/d9/de/d38/f3a [0,4194304] 0 2024-04-12T05:56:14.758 INFO:tasks.workunit.client.1.smithi194.stdout:8/448: chown d2/d8/d15/d33/f1b 3205 1 2024-04-12T05:56:14.761 INFO:tasks.workunit.client.1.smithi194.stdout:7/554: write d3/d27/d22/f2e [1257711,3577] 0 2024-04-12T05:56:14.761 INFO:tasks.workunit.client.1.smithi194.stdout:4/809: write d2/f67 [3057610,97337] 0 2024-04-12T05:56:14.763 INFO:tasks.workunit.client.1.smithi194.stdout:1/723: mknod d0/d11/d18/d19/d91/cea 0 2024-04-12T05:56:14.765 INFO:tasks.workunit.client.1.smithi194.stdout:7/555: chown d3/d27/d22/d8e/fae 246499 1 2024-04-12T05:56:14.766 INFO:tasks.workunit.client.1.smithi194.stdout:9/699: rename d1/d19/d1f/d20/f71 to d1/d41/dba/fe8 0 2024-04-12T05:56:14.772 INFO:tasks.workunit.client.1.smithi194.stdout:4/810: rename d2/d11/d93/da6/dab/d5d/dae/db2/fe6 to d2/d85/d11d/f122 0 2024-04-12T05:56:14.772 INFO:tasks.workunit.client.1.smithi194.stdout:6/603: rmdir d3/d15 39 2024-04-12T05:56:14.774 INFO:tasks.workunit.client.1.smithi194.stdout:1/724: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ddf/ceb 0 2024-04-12T05:56:14.775 INFO:tasks.workunit.client.1.smithi194.stdout:0/604: dread d0/d3b/f4c [0,4194304] 0 2024-04-12T05:56:14.780 INFO:tasks.workunit.client.1.smithi194.stdout:6/604: write d3/d2d/d33/d52/d5c/fa5 [3622466,19117] 0 2024-04-12T05:56:14.781 INFO:tasks.workunit.client.1.smithi194.stdout:1/725: chown d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/fd5 3 1 2024-04-12T05:56:14.782 INFO:tasks.workunit.client.1.smithi194.stdout:0/605: creat d0/da/d1e/d27/d54/fc3 x:0 0 0 2024-04-12T05:56:14.782 INFO:tasks.workunit.client.1.smithi194.stdout:3/760: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/fad [0,4194304] 0 2024-04-12T05:56:14.783 INFO:tasks.workunit.client.1.smithi194.stdout:1/726: fsync d0/f32 0 2024-04-12T05:56:14.787 INFO:tasks.workunit.client.1.smithi194.stdout:2/651: write d7/d22/d42/f2a [4034607,27436] 0 2024-04-12T05:56:14.787 INFO:tasks.workunit.client.1.smithi194.stdout:1/727: dread - d0/d11/d18/d19/d1a/d6a/d27/d2e/fb3 zero size 2024-04-12T05:56:14.789 INFO:tasks.workunit.client.1.smithi194.stdout:4/811: getdents d2/d103 0 2024-04-12T05:56:14.790 INFO:tasks.workunit.client.1.smithi194.stdout:8/449: write d2/d8/d15/f8c [3872785,103056] 0 2024-04-12T05:56:14.794 INFO:tasks.workunit.client.1.smithi194.stdout:6/605: getdents d3/d2f/d4e/d68 0 2024-04-12T05:56:14.797 INFO:tasks.workunit.client.1.smithi194.stdout:7/556: truncate d3/d16/d23/d37/f54 4275026 0 2024-04-12T05:56:14.797 INFO:tasks.workunit.client.1.smithi194.stdout:9/700: dread d1/d19/d1f/f70 [0,4194304] 0 2024-04-12T05:56:14.798 INFO:tasks.workunit.client.1.smithi194.stdout:1/728: stat d0/d11/d18/d19/d1a/d6a/dac/db5/d68/d92 0 2024-04-12T05:56:14.802 INFO:tasks.workunit.client.1.smithi194.stdout:6/606: dread - d3/d2d/d33/fa1 zero size 2024-04-12T05:56:14.802 INFO:tasks.workunit.client.1.smithi194.stdout:0/606: dwrite d0/da/d1e/d1a/d3d/f7a [0,4194304] 0 2024-04-12T05:56:14.805 INFO:tasks.workunit.client.1.smithi194.stdout:6/607: stat d3/d70/l99 0 2024-04-12T05:56:14.805 INFO:tasks.workunit.client.1.smithi194.stdout:9/701: write d1/d21/d49/fc5 [457290,37695] 0 2024-04-12T05:56:14.812 INFO:tasks.workunit.client.1.smithi194.stdout:1/729: read d0/d11/d18/d19/d1c/f22 [862983,83363] 0 2024-04-12T05:56:14.812 INFO:tasks.workunit.client.1.smithi194.stdout:6/608: creat d3/d2f/d4e/d63/d83/fc1 x:0 0 0 2024-04-12T05:56:14.812 INFO:tasks.workunit.client.1.smithi194.stdout:2/652: dwrite d7/d22/f4d [0,4194304] 0 2024-04-12T05:56:14.813 INFO:tasks.workunit.client.1.smithi194.stdout:0/607: mkdir d0/da/dc4 0 2024-04-12T05:56:14.818 INFO:tasks.workunit.client.1.smithi194.stdout:6/609: dread - d3/d2f/d4e/d63/faa zero size 2024-04-12T05:56:14.823 INFO:tasks.workunit.client.1.smithi194.stdout:3/761: dread d0/fb [4194304,4194304] 0 2024-04-12T05:56:14.823 INFO:tasks.workunit.client.1.smithi194.stdout:2/653: creat d7/d22/d93/fdf x:0 0 0 2024-04-12T05:56:14.823 INFO:tasks.workunit.client.1.smithi194.stdout:7/557: dwrite d3/d16/d23/d57/d72/d81/d3a/f60 [0,4194304] 0 2024-04-12T05:56:14.823 INFO:tasks.workunit.client.1.smithi194.stdout:6/610: mknod d3/d2d/cc2 0 2024-04-12T05:56:14.823 INFO:tasks.workunit.client.1.smithi194.stdout:8/450: dwrite d2/d8/d15/d33/d45/f55 [0,4194304] 0 2024-04-12T05:56:14.828 INFO:tasks.workunit.client.1.smithi194.stdout:2/654: creat d7/d12/d1a/d69/d7e/fe0 x:0 0 0 2024-04-12T05:56:14.829 INFO:tasks.workunit.client.1.smithi194.stdout:1/730: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/fec x:0 0 0 2024-04-12T05:56:14.830 INFO:tasks.workunit.client.1.smithi194.stdout:8/451: fdatasync d2/d8/d15/d35/f3f 0 2024-04-12T05:56:14.832 INFO:tasks.workunit.client.1.smithi194.stdout:8/452: chown d2/d8/d9/de/f1d 49 1 2024-04-12T05:56:14.833 INFO:tasks.workunit.client.1.smithi194.stdout:7/558: link d3/d16/c9f d3/d16/d23/d37/d3d/d56/cb5 0 2024-04-12T05:56:14.834 INFO:tasks.workunit.client.1.smithi194.stdout:2/655: getdents d7/d22/dc9 0 2024-04-12T05:56:14.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/453: creat d2/d8/d15/d33/d49/f8e x:0 0 0 2024-04-12T05:56:14.838 INFO:tasks.workunit.client.1.smithi194.stdout:1/731: dwrite d0/d11/d18/d19/d1a/d6a/d27/d9e/fad [0,4194304] 0 2024-04-12T05:56:14.839 INFO:tasks.workunit.client.1.smithi194.stdout:2/656: creat d7/d12/d1a/d86/fe1 x:0 0 0 2024-04-12T05:56:14.839 INFO:tasks.workunit.client.1.smithi194.stdout:8/454: chown d2/d8/d15/d33/c69 1867 1 2024-04-12T05:56:14.841 INFO:tasks.workunit.client.1.smithi194.stdout:1/732: chown d0/d11/d18/d19/d1a/d6a/d27/fab 1293813 1 2024-04-12T05:56:14.846 INFO:tasks.workunit.client.1.smithi194.stdout:7/559: dwrite d3/d16/d23/d57/d72/d81/d3a/fa7 [0,4194304] 0 2024-04-12T05:56:14.852 INFO:tasks.workunit.client.1.smithi194.stdout:2/657: creat d7/d12/d25/fe2 x:0 0 0 2024-04-12T05:56:14.853 INFO:tasks.workunit.client.1.smithi194.stdout:2/658: write d7/d12/f38 [960137,104725] 0 2024-04-12T05:56:14.853 INFO:tasks.workunit.client.1.smithi194.stdout:1/733: mkdir d0/d11/d18/d19/d1a/ded 0 2024-04-12T05:56:14.855 INFO:tasks.workunit.client.1.smithi194.stdout:7/560: creat d3/d16/d23/d37/d95/fb6 x:0 0 0 2024-04-12T05:56:14.856 INFO:tasks.workunit.client.1.smithi194.stdout:8/455: mkdir d2/d8/d9/de/d38/d8f 0 2024-04-12T05:56:14.862 INFO:tasks.workunit.client.1.smithi194.stdout:7/561: mknod d3/d16/d7b/cb7 0 2024-04-12T05:56:14.872 INFO:tasks.workunit.client.1.smithi194.stdout:2/659: dwrite d7/d12/d25/d70/fa1 [0,4194304] 0 2024-04-12T05:56:14.879 INFO:tasks.workunit.client.1.smithi194.stdout:1/734: dwrite d0/d11/d18/d19/d1c/d67/fb7 [0,4194304] 0 2024-04-12T05:56:14.885 INFO:tasks.workunit.client.1.smithi194.stdout:7/562: dwrite d3/d16/d23/d57/d72/d81/d3a/fa7 [0,4194304] 0 2024-04-12T05:56:14.892 INFO:tasks.workunit.client.1.smithi194.stdout:4/812: dread d2/d3/d1b/d32/d3a/d3b/f41 [0,4194304] 0 2024-04-12T05:56:14.893 INFO:tasks.workunit.client.1.smithi194.stdout:8/456: dread d2/d8/d9/de/d10/d4e/f60 [0,4194304] 0 2024-04-12T05:56:14.893 INFO:tasks.workunit.client.1.smithi194.stdout:6/611: rmdir d3/d2f/d4e/d63/d83 39 2024-04-12T05:56:14.893 INFO:tasks.workunit.client.1.smithi194.stdout:2/660: dwrite d7/d12/d1a/d86/fe1 [0,4194304] 0 2024-04-12T05:56:14.896 INFO:tasks.workunit.client.1.smithi194.stdout:9/702: dread d1/f69 [0,4194304] 0 2024-04-12T05:56:14.896 INFO:tasks.workunit.client.1.smithi194.stdout:0/608: write d0/f53 [301667,59287] 0 2024-04-12T05:56:14.902 INFO:tasks.workunit.client.1.smithi194.stdout:8/457: write d2/d8/d66/d67/f71 [462531,125311] 0 2024-04-12T05:56:14.905 INFO:tasks.workunit.client.1.smithi194.stdout:6/612: fsync d3/d2d/d33/fa1 0 2024-04-12T05:56:14.906 INFO:tasks.workunit.client.1.smithi194.stdout:4/813: dread d2/d3/d1b/d32/d3a/d3b/f41 [0,4194304] 0 2024-04-12T05:56:14.911 INFO:tasks.workunit.client.1.smithi194.stdout:7/563: link d3/d16/d23/d37/f96 d3/d16/d23/d57/d72/d81/d3a/d43/fb8 0 2024-04-12T05:56:14.912 INFO:tasks.workunit.client.1.smithi194.stdout:3/762: write d0/d1/d74/d39/d4d/deb/d1e/d81/f48 [1776663,4101] 0 2024-04-12T05:56:14.912 INFO:tasks.workunit.client.1.smithi194.stdout:9/703: rename d1/d41/d97/d9d/db9 to d1/d41/dbf/de9 0 2024-04-12T05:56:14.913 INFO:tasks.workunit.client.1.smithi194.stdout:4/814: mknod d2/d103/c123 0 2024-04-12T05:56:14.914 INFO:tasks.workunit.client.1.smithi194.stdout:6/613: mkdir d3/d2d/dc3 0 2024-04-12T05:56:14.915 INFO:tasks.workunit.client.1.smithi194.stdout:0/609: dwrite d0/f11 [0,4194304] 0 2024-04-12T05:56:14.920 INFO:tasks.workunit.client.1.smithi194.stdout:6/614: chown d3/d2d/d33/d52/l5b 5966 1 2024-04-12T05:56:14.922 INFO:tasks.workunit.client.1.smithi194.stdout:9/704: rename d1/d41/d97/db6 to d1/d4e/dea 0 2024-04-12T05:56:14.922 INFO:tasks.workunit.client.1.smithi194.stdout:1/735: write d0/d11/d18/d19/f39 [3232649,54999] 0 2024-04-12T05:56:14.923 INFO:tasks.workunit.client.1.smithi194.stdout:0/610: mknod d0/da/d4a/d8e/cc5 0 2024-04-12T05:56:14.924 INFO:tasks.workunit.client.1.smithi194.stdout:8/458: write d2/d8/d15/d33/f27 [5386975,62498] 0 2024-04-12T05:56:14.928 INFO:tasks.workunit.client.1.smithi194.stdout:4/815: rename d2/d3/d50/da0/de2 to d2/d3/d1b/d32/d3a/dbc/d124 0 2024-04-12T05:56:14.934 INFO:tasks.workunit.client.1.smithi194.stdout:3/763: creat d0/d1/d74/d39/d4d/deb/d1e/ffb x:0 0 0 2024-04-12T05:56:14.937 INFO:tasks.workunit.client.1.smithi194.stdout:0/611: rename d0/da/f96 to d0/d3b/fc6 0 2024-04-12T05:56:14.940 INFO:tasks.workunit.client.1.smithi194.stdout:2/661: dwrite d7/d12/d25/d5d/d47/f8f [0,4194304] 0 2024-04-12T05:56:14.941 INFO:tasks.workunit.client.1.smithi194.stdout:6/615: dread d3/d15/d2b/d48/f74 [0,4194304] 0 2024-04-12T05:56:14.943 INFO:tasks.workunit.client.1.smithi194.stdout:9/705: write d1/d19/d1f/d20/d37/d38/f7c [2263418,68874] 0 2024-04-12T05:56:14.944 INFO:tasks.workunit.client.1.smithi194.stdout:7/564: truncate d3/d16/d5d/f84 7185825 0 2024-04-12T05:56:14.947 INFO:tasks.workunit.client.1.smithi194.stdout:8/459: write d2/d8/d9/f2a [759042,106556] 0 2024-04-12T05:56:14.950 INFO:tasks.workunit.client.1.smithi194.stdout:2/662: write d7/d12/d6d/fa5 [3247590,107961] 0 2024-04-12T05:56:14.951 INFO:tasks.workunit.client.1.smithi194.stdout:6/616: chown d3/d2f/d4e 1 1 2024-04-12T05:56:14.951 INFO:tasks.workunit.client.1.smithi194.stdout:4/816: dwrite d2/fc [0,4194304] 0 2024-04-12T05:56:14.952 INFO:tasks.workunit.client.1.smithi194.stdout:9/706: rmdir d1/d19 39 2024-04-12T05:56:14.959 INFO:tasks.workunit.client.1.smithi194.stdout:3/764: rename d0/d1/d74/d39/d4d/deb/d1e/dd3/fe8 to d0/d1/d74/d39/d4d/deb/d8e/ffc 0 2024-04-12T05:56:14.961 INFO:tasks.workunit.client.1.smithi194.stdout:7/565: write d3/d16/d23/d40/d6f/f8b [207022,118787] 0 2024-04-12T05:56:14.965 INFO:tasks.workunit.client.1.smithi194.stdout:6/617: dread - d3/f84 zero size 2024-04-12T05:56:14.965 INFO:tasks.workunit.client.1.smithi194.stdout:1/736: dread d0/d11/d18/d19/d1a/d6a/d27/da8/fa9 [0,4194304] 0 2024-04-12T05:56:14.965 INFO:tasks.workunit.client.1.smithi194.stdout:8/460: creat d2/d8/d9/de/d38/d8f/f90 x:0 0 0 2024-04-12T05:56:14.966 INFO:tasks.workunit.client.1.smithi194.stdout:0/612: dwrite d0/da/d1e/d1a/d3d/f51 [0,4194304] 0 2024-04-12T05:56:14.968 INFO:tasks.workunit.client.1.smithi194.stdout:6/618: stat d3/d2f/d4e/d63/f78 0 2024-04-12T05:56:14.968 INFO:tasks.workunit.client.1.smithi194.stdout:7/566: write d3/d16/d23/d40/d6f/f8b [174511,7385] 0 2024-04-12T05:56:14.969 INFO:tasks.workunit.client.1.smithi194.stdout:8/461: chown d2/d8/c57 57 1 2024-04-12T05:56:14.970 INFO:tasks.workunit.client.1.smithi194.stdout:4/817: creat d2/d3/f125 x:0 0 0 2024-04-12T05:56:14.970 INFO:tasks.workunit.client.1.smithi194.stdout:9/707: unlink d1/d19/d1f/d20/d37/d84/lc0 0 2024-04-12T05:56:14.972 INFO:tasks.workunit.client.1.smithi194.stdout:2/663: creat d7/d12/fe3 x:0 0 0 2024-04-12T05:56:14.973 INFO:tasks.workunit.client.1.smithi194.stdout:8/462: readlink d2/d8/d58/l75 0 2024-04-12T05:56:14.976 INFO:tasks.workunit.client.1.smithi194.stdout:7/567: read - d3/d27/d6a/faa zero size 2024-04-12T05:56:14.977 INFO:tasks.workunit.client.1.smithi194.stdout:3/765: link d0/d1/d10/d8c/le5 d0/d1/d74/d39/d4d/deb/d53/lfd 0 2024-04-12T05:56:14.980 INFO:tasks.workunit.client.1.smithi194.stdout:1/737: dwrite d0/d11/d18/d19/d1c/f46 [0,4194304] 0 2024-04-12T05:56:14.982 INFO:tasks.workunit.client.1.smithi194.stdout:4/818: chown d2/d3/d1b/d32/d3a/c48 10 1 2024-04-12T05:56:14.983 INFO:tasks.workunit.client.1.smithi194.stdout:6/619: write d3/d2d/d33/d52/d5c/fa5 [2161857,21455] 0 2024-04-12T05:56:14.988 INFO:tasks.workunit.client.1.smithi194.stdout:4/819: fdatasync d2/d3/d1b/d32/d3a/d3b/f100 0 2024-04-12T05:56:14.991 INFO:tasks.workunit.client.1.smithi194.stdout:0/613: dread d0/f11 [0,4194304] 0 2024-04-12T05:56:14.993 INFO:tasks.workunit.client.1.smithi194.stdout:3/766: rename d0/d1/d10/d8c/d8f/ca5 to d0/d1/d10/d8c/cfe 0 2024-04-12T05:56:14.994 INFO:tasks.workunit.client.1.smithi194.stdout:6/620: dread - d3/d2f/d4e/d63/d83/fc1 zero size 2024-04-12T05:56:14.994 INFO:tasks.workunit.client.1.smithi194.stdout:0/614: write d0/f53 [3749357,8869] 0 2024-04-12T05:56:14.997 INFO:tasks.workunit.client.1.smithi194.stdout:9/708: getdents d1/d21/d77/d98/d9f 0 2024-04-12T05:56:15.002 INFO:tasks.workunit.client.1.smithi194.stdout:1/738: write d0/d11/d18/d35/da5/f97 [1034395,81797] 0 2024-04-12T05:56:15.002 INFO:tasks.workunit.client.1.smithi194.stdout:3/767: chown d0/d1/d74/c3c 0 1 2024-04-12T05:56:15.004 INFO:tasks.workunit.client.1.smithi194.stdout:7/568: fdatasync d3/d16/d5d/f84 0 2024-04-12T05:56:15.005 INFO:tasks.workunit.client.1.smithi194.stdout:9/709: symlink d1/d41/leb 0 2024-04-12T05:56:15.006 INFO:tasks.workunit.client.1.smithi194.stdout:6/621: mkdir d3/d2f/d4e/d68/dad/dc4 0 2024-04-12T05:56:15.008 INFO:tasks.workunit.client.1.smithi194.stdout:2/664: dwrite d7/d12/d1a/f3e [0,4194304] 0 2024-04-12T05:56:15.009 INFO:tasks.workunit.client.1.smithi194.stdout:9/710: chown d1/d41/f6f 3142434 1 2024-04-12T05:56:15.014 INFO:tasks.workunit.client.1.smithi194.stdout:7/569: creat d3/d16/d7b/fb9 x:0 0 0 2024-04-12T05:56:15.014 INFO:tasks.workunit.client.1.smithi194.stdout:6/622: unlink d3/d2d/d33/d52/f6e 0 2024-04-12T05:56:15.015 INFO:tasks.workunit.client.1.smithi194.stdout:8/463: dread d2/d8/d9/d1a/f3d [0,4194304] 0 2024-04-12T05:56:15.016 INFO:tasks.workunit.client.1.smithi194.stdout:3/768: mkdir d0/dff 0 2024-04-12T05:56:15.017 INFO:tasks.workunit.client.1.smithi194.stdout:2/665: creat d7/d12/d1a/d69/d7e/fe4 x:0 0 0 2024-04-12T05:56:15.019 INFO:tasks.workunit.client.1.smithi194.stdout:9/711: unlink d1/d19/d1f/d20/d37/d38/f8f 0 2024-04-12T05:56:15.020 INFO:tasks.workunit.client.1.smithi194.stdout:4/820: dread d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:15.021 INFO:tasks.workunit.client.1.smithi194.stdout:7/570: write d3/d27/d22/d8e/fae [65335,70643] 0 2024-04-12T05:56:15.022 INFO:tasks.workunit.client.1.smithi194.stdout:9/712: chown d1/fd4 2 1 2024-04-12T05:56:15.022 INFO:tasks.workunit.client.1.smithi194.stdout:1/739: creat d0/d11/d18/d19/d1a/ded/fee x:0 0 0 2024-04-12T05:56:15.022 INFO:tasks.workunit.client.1.smithi194.stdout:2/666: mkdir d7/d22/dc9/de5 0 2024-04-12T05:56:15.023 INFO:tasks.workunit.client.1.smithi194.stdout:7/571: readlink d3/d16/d23/la0 0 2024-04-12T05:56:15.025 INFO:tasks.workunit.client.1.smithi194.stdout:3/769: truncate d0/d1/d10/d89/fd4 1133315 0 2024-04-12T05:56:15.026 INFO:tasks.workunit.client.1.smithi194.stdout:8/464: truncate d2/d8/d9/de/d10/f24 5724520 0 2024-04-12T05:56:15.028 INFO:tasks.workunit.client.1.smithi194.stdout:9/713: creat d1/d21/d77/d98/d9f/dde/fec x:0 0 0 2024-04-12T05:56:15.028 INFO:tasks.workunit.client.1.smithi194.stdout:0/615: dread d0/da/d1e/d1a/f31 [0,4194304] 0 2024-04-12T05:56:15.031 INFO:tasks.workunit.client.1.smithi194.stdout:7/572: creat d3/d16/d23/d57/fba x:0 0 0 2024-04-12T05:56:15.035 INFO:tasks.workunit.client.1.smithi194.stdout:3/770: symlink d0/d1/d4f/dd7/de1/l100 0 2024-04-12T05:56:15.036 INFO:tasks.workunit.client.1.smithi194.stdout:4/821: dwrite d2/d3/f105 [0,4194304] 0 2024-04-12T05:56:15.038 INFO:tasks.workunit.client.1.smithi194.stdout:0/616: fdatasync d0/da/f13 0 2024-04-12T05:56:15.041 INFO:tasks.workunit.client.1.smithi194.stdout:9/714: creat d1/d19/d1f/d20/fed x:0 0 0 2024-04-12T05:56:15.042 INFO:tasks.workunit.client.1.smithi194.stdout:2/667: rmdir d7/d12/d1a/d69/d7e 39 2024-04-12T05:56:15.044 INFO:tasks.workunit.client.1.smithi194.stdout:0/617: fsync d0/d3b/fb5 0 2024-04-12T05:56:15.044 INFO:tasks.workunit.client.1.smithi194.stdout:8/465: truncate d2/d8/d15/d33/d45/d52/f7b 23283 0 2024-04-12T05:56:15.046 INFO:tasks.workunit.client.1.smithi194.stdout:4/822: chown d2/d11/d93/da6/dab/d5d/dae/de1/ddc/de4/ff3 219984780 1 2024-04-12T05:56:15.047 INFO:tasks.workunit.client.1.smithi194.stdout:0/618: rename d0/da to d0/da/dc4/dc7 22 2024-04-12T05:56:15.050 INFO:tasks.workunit.client.1.smithi194.stdout:2/668: chown d7/d12/d25/d5d/d47/d52/d7b/l9c 6607 1 2024-04-12T05:56:15.050 INFO:tasks.workunit.client.1.smithi194.stdout:3/771: truncate d0/d42/f73 1889487 0 2024-04-12T05:56:15.051 INFO:tasks.workunit.client.1.smithi194.stdout:7/573: dwrite d3/d27/f39 [4194304,4194304] 0 2024-04-12T05:56:15.052 INFO:tasks.workunit.client.1.smithi194.stdout:9/715: creat d1/d19/d1f/d20/d37/d38/fee x:0 0 0 2024-04-12T05:56:15.052 INFO:tasks.workunit.client.1.smithi194.stdout:1/740: dread d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [0,4194304] 0 2024-04-12T05:56:15.052 INFO:tasks.workunit.client.1.smithi194.stdout:8/466: rmdir d2/d8/d66/d67 39 2024-04-12T05:56:15.053 INFO:tasks.workunit.client.1.smithi194.stdout:0/619: stat d0/da/d1e/d27/f2a 0 2024-04-12T05:56:15.054 INFO:tasks.workunit.client.1.smithi194.stdout:7/574: stat d3/d16/d23/lad 0 2024-04-12T05:56:15.055 INFO:tasks.workunit.client.1.smithi194.stdout:9/716: chown d1/d19/d1f/d20/d37/d38/l50 47520702 1 2024-04-12T05:56:15.056 INFO:tasks.workunit.client.1.smithi194.stdout:0/620: chown d0/da/d1e/d27/f2a 686054264 1 2024-04-12T05:56:15.056 INFO:tasks.workunit.client.1.smithi194.stdout:4/823: read d2/d3/d1b/d32/d3a/d3b/f9f [2741686,20071] 0 2024-04-12T05:56:15.058 INFO:tasks.workunit.client.1.smithi194.stdout:2/669: symlink d7/d12/d1a/d69/dd6/le6 0 2024-04-12T05:56:15.065 INFO:tasks.workunit.client.1.smithi194.stdout:3/772: link d0/d1/l2 d0/d42/l101 0 2024-04-12T05:56:15.065 INFO:tasks.workunit.client.1.smithi194.stdout:9/717: write d1/d21/d39/d72/fa7 [4160789,39271] 0 2024-04-12T05:56:15.074 INFO:tasks.workunit.client.1.smithi194.stdout:4/824: mknod d2/d11/d93/da6/dab/d5d/dae/db2/c126 0 2024-04-12T05:56:15.076 INFO:tasks.workunit.client.1.smithi194.stdout:1/741: dwrite d0/d11/d18/d35/da5/fbe [0,4194304] 0 2024-04-12T05:56:15.077 INFO:tasks.workunit.client.1.smithi194.stdout:2/670: mknod d7/d22/d93/ce7 0 2024-04-12T05:56:15.077 INFO:tasks.workunit.client.1.smithi194.stdout:9/718: write d1/d21/d39/d72/fb5 [1707637,37711] 0 2024-04-12T05:56:15.080 INFO:tasks.workunit.client.1.smithi194.stdout:7/575: dwrite d3/d16/d23/d37/fb3 [0,4194304] 0 2024-04-12T05:56:15.091 INFO:tasks.workunit.client.1.smithi194.stdout:5/730: sync 2024-04-12T05:56:15.091 INFO:tasks.workunit.client.1.smithi194.stdout:2/671: creat d7/d22/d42/fe8 x:0 0 0 2024-04-12T05:56:15.092 INFO:tasks.workunit.client.1.smithi194.stdout:7/576: creat d3/d27/fbb x:0 0 0 2024-04-12T05:56:15.092 INFO:tasks.workunit.client.1.smithi194.stdout:0/621: dwrite d0/da/f91 [0,4194304] 0 2024-04-12T05:56:15.096 INFO:tasks.workunit.client.1.smithi194.stdout:3/773: dwrite d0/d1/d4f/fa3 [0,4194304] 0 2024-04-12T05:56:15.105 INFO:tasks.workunit.client.1.smithi194.stdout:2/672: write d7/d22/d42/fb7 [163743,4334] 0 2024-04-12T05:56:15.108 INFO:tasks.workunit.client.1.smithi194.stdout:5/731: read d1/d4/d6/db/d15/d26/feb [8129401,120869] 0 2024-04-12T05:56:15.112 INFO:tasks.workunit.client.1.smithi194.stdout:1/742: rename d0/d11/d18/d19/d1a/d6a/dac/db5/f82 to d0/d11/d18/d19/d1a/d6a/dac/dba/fef 0 2024-04-12T05:56:15.114 INFO:tasks.workunit.client.1.smithi194.stdout:2/673: mkdir d7/d12/d1a/de9 0 2024-04-12T05:56:15.117 INFO:tasks.workunit.client.1.smithi194.stdout:7/577: dwrite d3/d27/fa [0,4194304] 0 2024-04-12T05:56:15.119 INFO:tasks.workunit.client.1.smithi194.stdout:1/743: chown d0/d11/c16 28883261 1 2024-04-12T05:56:15.119 INFO:tasks.workunit.client.1.smithi194.stdout:9/719: dwrite d1/d21/d77/d98/d9f/fdc [0,4194304] 0 2024-04-12T05:56:15.123 INFO:tasks.workunit.client.1.smithi194.stdout:0/622: link d0/d3b/d9c/lb3 d0/da/dc4/lc8 0 2024-04-12T05:56:15.123 INFO:tasks.workunit.client.1.smithi194.stdout:3/774: link d0/f27 d0/d1/d74/d39/d4d/deb/db4/dd9/f102 0 2024-04-12T05:56:15.129 INFO:tasks.workunit.client.1.smithi194.stdout:4/825: dread d2/d11/d93/da6/dab/d5d/dae/db2/f79 [0,4194304] 0 2024-04-12T05:56:15.130 INFO:tasks.workunit.client.1.smithi194.stdout:9/720: mkdir d1/d41/dba/dc2/def 0 2024-04-12T05:56:15.132 INFO:tasks.workunit.client.1.smithi194.stdout:1/744: mkdir d0/d11/d18/d19/d1c/df0 0 2024-04-12T05:56:15.136 INFO:tasks.workunit.client.1.smithi194.stdout:7/578: dwrite d3/d27/f7 [0,4194304] 0 2024-04-12T05:56:15.136 INFO:tasks.workunit.client.1.smithi194.stdout:9/721: dread - d1/d19/d1f/d20/d37/d84/fdd zero size 2024-04-12T05:56:15.137 INFO:tasks.workunit.client.1.smithi194.stdout:4/826: read - d2/d11/d46/d61/fe0 zero size 2024-04-12T05:56:15.144 INFO:tasks.workunit.client.1.smithi194.stdout:7/579: chown d3/d27/d6a/l4f 1481 1 2024-04-12T05:56:15.145 INFO:tasks.workunit.client.1.smithi194.stdout:4/827: fsync d2/d3/d58/d82/dd0/fe3 0 2024-04-12T05:56:15.145 INFO:tasks.workunit.client.1.smithi194.stdout:3/775: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f103 x:0 0 0 2024-04-12T05:56:15.146 INFO:tasks.workunit.client.1.smithi194.stdout:2/674: creat d7/d12/d25/d5d/d47/dac/fea x:0 0 0 2024-04-12T05:56:15.147 INFO:tasks.workunit.client.1.smithi194.stdout:9/722: rmdir d1/d19/d1f/d20/d37/d38 39 2024-04-12T05:56:15.147 INFO:tasks.workunit.client.1.smithi194.stdout:7/580: symlink d3/d16/d7b/lbc 0 2024-04-12T05:56:15.149 INFO:tasks.workunit.client.1.smithi194.stdout:9/723: chown d1/d19/d1f/d5f/f85 0 1 2024-04-12T05:56:15.150 INFO:tasks.workunit.client.1.smithi194.stdout:4/828: rmdir d2/d3/d1b 39 2024-04-12T05:56:15.152 INFO:tasks.workunit.client.1.smithi194.stdout:0/623: getdents d0/da 0 2024-04-12T05:56:15.153 INFO:tasks.workunit.client.1.smithi194.stdout:3/776: symlink d0/d1/d10/d8c/d8f/dbc/l104 0 2024-04-12T05:56:15.153 INFO:tasks.workunit.client.1.smithi194.stdout:7/581: creat d3/d16/d5d/fbd x:0 0 0 2024-04-12T05:56:15.154 INFO:tasks.workunit.client.1.smithi194.stdout:1/745: getdents d0/d11/d18/d19/d1a/d6a/d27 0 2024-04-12T05:56:15.162 INFO:tasks.workunit.client.1.smithi194.stdout:3/777: truncate d0/d1/d74/d39/fd1 381370 0 2024-04-12T05:56:15.168 INFO:tasks.workunit.client.1.smithi194.stdout:2/675: dwrite d7/d12/d25/d5d/d47/d52/f60 [0,4194304] 0 2024-04-12T05:56:15.171 INFO:tasks.workunit.client.1.smithi194.stdout:0/624: fsync d0/da/d1e/d1a/d24/d4b/f6f 0 2024-04-12T05:56:15.187 INFO:tasks.workunit.client.1.smithi194.stdout:3/778: rename d0/d1/d74/d39/d4d/deb/d1e/d33/d76/d9a/fc1 to d0/d1/d74/d39/d4d/deb/d53/f105 0 2024-04-12T05:56:15.188 INFO:tasks.workunit.client.1.smithi194.stdout:2/676: dwrite d7/d22/f87 [0,4194304] 0 2024-04-12T05:56:15.189 INFO:tasks.workunit.client.1.smithi194.stdout:9/724: dwrite d1/d19/d1f/d20/f3d [4194304,4194304] 0 2024-04-12T05:56:15.203 INFO:tasks.workunit.client.1.smithi194.stdout:1/746: dwrite d0/d11/f55 [0,4194304] 0 2024-04-12T05:56:15.204 INFO:tasks.workunit.client.1.smithi194.stdout:4/829: dwrite d2/d3/d1b/d32/f64 [0,4194304] 0 2024-04-12T05:56:15.207 INFO:tasks.workunit.client.1.smithi194.stdout:3/779: fsync d0/d1/d74/d39/d4d/d63/db5/fd8 0 2024-04-12T05:56:15.212 INFO:tasks.workunit.client.1.smithi194.stdout:4/830: write d2/d85/fd5 [3609264,107871] 0 2024-04-12T05:56:15.212 INFO:tasks.workunit.client.1.smithi194.stdout:7/582: dread d3/d27/d6a/f55 [0,4194304] 0 2024-04-12T05:56:15.222 INFO:tasks.workunit.client.1.smithi194.stdout:9/725: dread d1/d19/d1f/d20/f57 [0,4194304] 0 2024-04-12T05:56:15.225 INFO:tasks.workunit.client.1.smithi194.stdout:4/831: rename d2/d11/d93/da6/dab/d5d/dae/de1/ddc/de4/ff3 to d2/d3/d50/da0/f127 0 2024-04-12T05:56:15.230 INFO:tasks.workunit.client.1.smithi194.stdout:9/726: symlink d1/d19/d1f/d20/d7d/lf0 0 2024-04-12T05:56:15.230 INFO:tasks.workunit.client.1.smithi194.stdout:2/677: truncate d7/d12/d1a/d69/d7e/fe0 466914 0 2024-04-12T05:56:15.231 INFO:tasks.workunit.client.1.smithi194.stdout:4/832: write d2/d11/d93/f113 [852164,16365] 0 2024-04-12T05:56:15.232 INFO:tasks.workunit.client.1.smithi194.stdout:3/780: creat d0/d1/d74/d39/d4d/deb/d3d/d6e/de9/f106 x:0 0 0 2024-04-12T05:56:15.236 INFO:tasks.workunit.client.1.smithi194.stdout:2/678: symlink d7/d12/d25/leb 0 2024-04-12T05:56:15.239 INFO:tasks.workunit.client.1.smithi194.stdout:0/625: dwrite d0/da/d1e/d1a/f7d [0,4194304] 0 2024-04-12T05:56:15.240 INFO:tasks.workunit.client.1.smithi194.stdout:9/727: mkdir d1/d21/d39/df1 0 2024-04-12T05:56:15.243 INFO:tasks.workunit.client.1.smithi194.stdout:0/626: readlink d0/d5f/d73/l82 0 2024-04-12T05:56:15.245 INFO:tasks.workunit.client.1.smithi194.stdout:3/781: mkdir d0/d1/d10/d8c/d8f/dac/de3/d107 0 2024-04-12T05:56:15.246 INFO:tasks.workunit.client.1.smithi194.stdout:4/833: rmdir d2/d3/d51 39 2024-04-12T05:56:15.246 INFO:tasks.workunit.client.1.smithi194.stdout:2/679: dwrite d7/d12/f80 [0,4194304] 0 2024-04-12T05:56:15.248 INFO:tasks.workunit.client.1.smithi194.stdout:4/834: dread - d2/d3/d1b/d32/db3/fdb zero size 2024-04-12T05:56:15.249 INFO:tasks.workunit.client.1.smithi194.stdout:9/728: mkdir d1/d19/d1f/d20/d37/d84/df2 0 2024-04-12T05:56:15.252 INFO:tasks.workunit.client.1.smithi194.stdout:4/835: chown d2/d3/d1b/d32/d3a/c48 216808936 1 2024-04-12T05:56:15.257 INFO:tasks.workunit.client.1.smithi194.stdout:1/747: getdents d0/d11/d18/d19/d1a/d6a/d27/d48/da7 0 2024-04-12T05:56:15.257 INFO:tasks.workunit.client.1.smithi194.stdout:0/627: rmdir d0/da/dc4 39 2024-04-12T05:56:15.257 INFO:tasks.workunit.client.1.smithi194.stdout:3/782: mknod d0/d1/c108 0 2024-04-12T05:56:15.263 INFO:tasks.workunit.client.1.smithi194.stdout:2/680: mknod d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/cec 0 2024-04-12T05:56:15.265 INFO:tasks.workunit.client.1.smithi194.stdout:7/583: dread d3/d16/d23/d57/fab [0,4194304] 0 2024-04-12T05:56:15.267 INFO:tasks.workunit.client.1.smithi194.stdout:4/836: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/d57/fe5 [0,4194304] 0 2024-04-12T05:56:15.267 INFO:tasks.workunit.client.1.smithi194.stdout:1/748: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ff1 x:0 0 0 2024-04-12T05:56:15.279 INFO:tasks.workunit.client.1.smithi194.stdout:4/837: chown d2/d3/d50/da0/f127 0 1 2024-04-12T05:56:15.281 INFO:tasks.workunit.client.1.smithi194.stdout:4/838: read d2/f67 [2980095,46974] 0 2024-04-12T05:56:15.283 INFO:tasks.workunit.client.1.smithi194.stdout:7/584: mknod d3/d16/cbe 0 2024-04-12T05:56:15.283 INFO:tasks.workunit.client.1.smithi194.stdout:4/839: write d2/f8f [1289125,124504] 0 2024-04-12T05:56:15.285 INFO:tasks.workunit.client.1.smithi194.stdout:7/585: dread - d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f91 zero size 2024-04-12T05:56:15.288 INFO:tasks.workunit.client.1.smithi194.stdout:3/783: dread d0/d1/d10/f2b [0,4194304] 0 2024-04-12T05:56:15.295 INFO:tasks.workunit.client.1.smithi194.stdout:4/840: rmdir d2/d11/d1f/d110 0 2024-04-12T05:56:15.301 INFO:tasks.workunit.client.1.smithi194.stdout:7/586: dread d3/d16/d23/d37/f7c [0,4194304] 0 2024-04-12T05:56:15.302 INFO:tasks.workunit.client.1.smithi194.stdout:4/841: fsync d2/d3/f13 0 2024-04-12T05:56:15.306 INFO:tasks.workunit.client.1.smithi194.stdout:7/587: write d3/d16/d7b/fb9 [927828,357] 0 2024-04-12T05:56:15.310 INFO:tasks.workunit.client.1.smithi194.stdout:4/842: creat d2/d3/d1b/d32/d3a/d3b/f128 x:0 0 0 2024-04-12T05:56:15.312 INFO:tasks.workunit.client.1.smithi194.stdout:2/681: rename d7/d12/d6d/f72 to d7/d12/d1a/de9/fed 0 2024-04-12T05:56:15.317 INFO:tasks.workunit.client.1.smithi194.stdout:1/749: write d0/d11/d18/d35/da5/fbe [4274616,106222] 0 2024-04-12T05:56:15.321 INFO:tasks.workunit.client.1.smithi194.stdout:7/588: dwrite d3/d27/d22/d8e/fae [0,4194304] 0 2024-04-12T05:56:15.327 INFO:tasks.workunit.client.1.smithi194.stdout:7/589: chown d3/d16/d23/d37/f96 176698 1 2024-04-12T05:56:15.329 INFO:tasks.workunit.client.1.smithi194.stdout:2/682: creat d7/d22/dc9/fee x:0 0 0 2024-04-12T05:56:15.338 INFO:tasks.workunit.client.1.smithi194.stdout:4/843: dwrite d2/d11/d93/f113 [0,4194304] 0 2024-04-12T05:56:15.341 INFO:tasks.workunit.client.1.smithi194.stdout:6/623: sync 2024-04-12T05:56:15.345 INFO:tasks.workunit.client.1.smithi194.stdout:4/844: readlink d2/d11/d1f/l73 0 2024-04-12T05:56:15.345 INFO:tasks.workunit.client.1.smithi194.stdout:6/624: chown d3/d2d/d33/d58/fba 1277 1 2024-04-12T05:56:15.347 INFO:tasks.workunit.client.1.smithi194.stdout:3/784: link d0/d42/l50 d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/l109 0 2024-04-12T05:56:15.348 INFO:tasks.workunit.client.1.smithi194.stdout:6/625: write d3/d2f/d4e/d63/f67 [3231397,79631] 0 2024-04-12T05:56:15.348 INFO:tasks.workunit.client.1.smithi194.stdout:0/628: getdents d0/da/d1e/d27/d54 0 2024-04-12T05:56:15.350 INFO:tasks.workunit.client.1.smithi194.stdout:3/785: write d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/fc2 [105,82717] 0 2024-04-12T05:56:15.353 INFO:tasks.workunit.client.1.smithi194.stdout:2/683: symlink d7/d22/dc9/lef 0 2024-04-12T05:56:15.354 INFO:tasks.workunit.client.1.smithi194.stdout:2/684: write d7/d22/d42/fb7 [305389,23687] 0 2024-04-12T05:56:15.355 INFO:tasks.workunit.client.1.smithi194.stdout:4/845: rename d2/d3/d1b/f5e to d2/d11/d1e/d2f/d4a/d9c/dee/f129 0 2024-04-12T05:56:15.364 INFO:tasks.workunit.client.1.smithi194.stdout:4/846: dread - d2/d85/d11d/f122 zero size 2024-04-12T05:56:15.373 INFO:tasks.workunit.client.1.smithi194.stdout:9/729: creat d1/d19/d1f/d20/d37/ff3 x:0 0 0 2024-04-12T05:56:15.374 INFO:tasks.workunit.client.1.smithi194.stdout:2/685: mkdir d7/d12/d1a/d69/df0 0 2024-04-12T05:56:15.375 INFO:tasks.workunit.client.1.smithi194.stdout:1/750: getdents d0/d11/d18/d19/d1a/d6a/d27/d5a 0 2024-04-12T05:56:15.377 INFO:tasks.workunit.client.1.smithi194.stdout:9/730: chown d1/d19/d1f/d20/f57 13325 1 2024-04-12T05:56:15.378 INFO:tasks.workunit.client.1.smithi194.stdout:0/629: dread d0/da/d1e/f15 [0,4194304] 0 2024-04-12T05:56:15.379 INFO:tasks.workunit.client.1.smithi194.stdout:3/786: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/f8b [0,4194304] 0 2024-04-12T05:56:15.380 INFO:tasks.workunit.client.1.smithi194.stdout:4/847: rename d2/d3/d1b/d32/d3a/d3b/f86 to d2/d11/d93/da6/dab/f12a 0 2024-04-12T05:56:15.382 INFO:tasks.workunit.client.1.smithi194.stdout:2/686: stat f5 0 2024-04-12T05:56:15.385 INFO:tasks.workunit.client.1.smithi194.stdout:2/687: stat d7/d12/d1a/c50 0 2024-04-12T05:56:15.385 INFO:tasks.workunit.client.1.smithi194.stdout:4/848: write d2/d11/d93/f113 [1863757,51312] 0 2024-04-12T05:56:15.388 INFO:tasks.workunit.client.1.smithi194.stdout:9/731: mkdir d1/d19/d1f/d5f/df4 0 2024-04-12T05:56:15.389 INFO:tasks.workunit.client.1.smithi194.stdout:4/849: chown d2/d11/d1f/fa8 40 1 2024-04-12T05:56:15.390 INFO:tasks.workunit.client.1.smithi194.stdout:7/590: write d3/d16/f19 [4723834,24374] 0 2024-04-12T05:56:15.391 INFO:tasks.workunit.client.1.smithi194.stdout:9/732: chown d1/d41/leb 12623 1 2024-04-12T05:56:15.391 INFO:tasks.workunit.client.1.smithi194.stdout:3/787: creat d0/d1/d74/d39/d4d/deb/d1e/dc4/f10a x:0 0 0 2024-04-12T05:56:15.394 INFO:tasks.workunit.client.1.smithi194.stdout:0/630: dread d0/da/d1e/f15 [0,4194304] 0 2024-04-12T05:56:15.395 INFO:tasks.workunit.client.1.smithi194.stdout:4/850: write d2/d11/d1e/d2f/f11e [391432,25748] 0 2024-04-12T05:56:15.399 INFO:tasks.workunit.client.1.smithi194.stdout:3/788: write d0/d1/d4f/fa3 [1363315,130659] 0 2024-04-12T05:56:15.400 INFO:tasks.workunit.client.1.smithi194.stdout:8/467: sync 2024-04-12T05:56:15.401 INFO:tasks.workunit.client.1.smithi194.stdout:6/626: dread d3/d2f/d4e/d53/f6a [0,4194304] 0 2024-04-12T05:56:15.403 INFO:tasks.workunit.client.1.smithi194.stdout:4/851: chown d2/d11/d43/f11b 36 1 2024-04-12T05:56:15.404 INFO:tasks.workunit.client.1.smithi194.stdout:9/733: symlink d1/d19/d1f/d20/d37/da3/dc7/lf5 0 2024-04-12T05:56:15.406 INFO:tasks.workunit.client.1.smithi194.stdout:2/688: dwrite d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/fc4 [0,4194304] 0 2024-04-12T05:56:15.407 INFO:tasks.workunit.client.1.smithi194.stdout:0/631: unlink d0/da/d1e/d1a/d24/d56/f9b 0 2024-04-12T05:56:15.412 INFO:tasks.workunit.client.1.smithi194.stdout:9/734: creat d1/d41/dbf/ff6 x:0 0 0 2024-04-12T05:56:15.412 INFO:tasks.workunit.client.1.smithi194.stdout:0/632: write d0/d3b/fc6 [902547,126211] 0 2024-04-12T05:56:15.415 INFO:tasks.workunit.client.1.smithi194.stdout:2/689: mkdir d7/d12/d25/d70/df1 0 2024-04-12T05:56:15.416 INFO:tasks.workunit.client.1.smithi194.stdout:1/751: dwrite d0/d11/d18/d19/d1a/d6a/dac/db5/f95 [0,4194304] 0 2024-04-12T05:56:15.422 INFO:tasks.workunit.client.1.smithi194.stdout:6/627: read d3/d15/d2b/d48/d6f/d7a/d8f/fa3 [1873012,58072] 0 2024-04-12T05:56:15.425 INFO:tasks.workunit.client.1.smithi194.stdout:0/633: symlink d0/d5f/lc9 0 2024-04-12T05:56:15.430 INFO:tasks.workunit.client.1.smithi194.stdout:7/591: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:15.430 INFO:tasks.workunit.client.1.smithi194.stdout:3/789: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f10b x:0 0 0 2024-04-12T05:56:15.430 INFO:tasks.workunit.client.1.smithi194.stdout:1/752: fdatasync d0/d11/d18/d19/d1a/d6a/dac/db5/d68/f87 0 2024-04-12T05:56:15.432 INFO:tasks.workunit.client.1.smithi194.stdout:2/690: dwrite d7/d12/d25/d5d/d47/f8f [0,4194304] 0 2024-04-12T05:56:15.432 INFO:tasks.workunit.client.1.smithi194.stdout:4/852: dread d2/d11/d93/da6/dab/d5d/dae/db2/f79 [0,4194304] 0 2024-04-12T05:56:15.434 INFO:tasks.workunit.client.1.smithi194.stdout:0/634: creat d0/da/d4a/fca x:0 0 0 2024-04-12T05:56:15.434 INFO:tasks.workunit.client.1.smithi194.stdout:9/735: dread d1/fe [0,4194304] 0 2024-04-12T05:56:15.439 INFO:tasks.workunit.client.1.smithi194.stdout:1/753: read d0/d11/d18/d19/d1a/d6a/dac/fce [2508425,55104] 0 2024-04-12T05:56:15.444 INFO:tasks.workunit.client.1.smithi194.stdout:2/691: rmdir d7/d12/d25/d5d/d47/dac 39 2024-04-12T05:56:15.444 INFO:tasks.workunit.client.1.smithi194.stdout:2/692: chown d7 78954605 1 2024-04-12T05:56:15.445 INFO:tasks.workunit.client.1.smithi194.stdout:7/592: write d3/d16/d23/d37/f96 [632555,120784] 0 2024-04-12T05:56:15.446 INFO:tasks.workunit.client.1.smithi194.stdout:6/628: link d3/d2f/d4e/d9a/db3/fbc d3/d15/d2b/fc5 0 2024-04-12T05:56:15.447 INFO:tasks.workunit.client.1.smithi194.stdout:4/853: getdents d2/d11/d46/d61 0 2024-04-12T05:56:15.450 INFO:tasks.workunit.client.1.smithi194.stdout:9/736: symlink d1/d21/d39/lf7 0 2024-04-12T05:56:15.450 INFO:tasks.workunit.client.1.smithi194.stdout:6/629: write d3/d2f/fa7 [533464,102682] 0 2024-04-12T05:56:15.453 INFO:tasks.workunit.client.1.smithi194.stdout:9/737: chown d1/d41/dba/cd6 280 1 2024-04-12T05:56:15.453 INFO:tasks.workunit.client.1.smithi194.stdout:0/635: dwrite d0/da/f39 [0,4194304] 0 2024-04-12T05:56:15.457 INFO:tasks.workunit.client.1.smithi194.stdout:3/790: mknod d0/d1/d74/d39/d4d/deb/d1e/d81/df4/c10c 0 2024-04-12T05:56:15.459 INFO:tasks.workunit.client.1.smithi194.stdout:6/630: unlink d3/d2d/d33/d52/d5c/c60 0 2024-04-12T05:56:15.460 INFO:tasks.workunit.client.1.smithi194.stdout:0/636: creat d0/d5f/fcb x:0 0 0 2024-04-12T05:56:15.461 INFO:tasks.workunit.client.1.smithi194.stdout:2/693: mknod d7/d12/cf2 0 2024-04-12T05:56:15.467 INFO:tasks.workunit.client.1.smithi194.stdout:7/593: creat d3/d16/d23/d57/d72/d81/d3a/fbf x:0 0 0 2024-04-12T05:56:15.467 INFO:tasks.workunit.client.1.smithi194.stdout:4/854: dwrite d2/d11/d93/da6/dab/f12a [0,4194304] 0 2024-04-12T05:56:15.470 INFO:tasks.workunit.client.1.smithi194.stdout:1/754: link d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/lbd d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/lf2 0 2024-04-12T05:56:15.471 INFO:tasks.workunit.client.1.smithi194.stdout:3/791: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d10d 0 2024-04-12T05:56:15.471 INFO:tasks.workunit.client.1.smithi194.stdout:9/738: dwrite d1/d19/d1f/d20/fcd [0,4194304] 0 2024-04-12T05:56:15.475 INFO:tasks.workunit.client.1.smithi194.stdout:0/637: rename d0/da/d1e/d1a/d24/d56/c6c to d0/d3b/ccc 0 2024-04-12T05:56:15.480 INFO:tasks.workunit.client.1.smithi194.stdout:3/792: write d0/d1/d74/d39/d4d/deb/d1e/d81/d34/fa1 [4426935,97964] 0 2024-04-12T05:56:15.481 INFO:tasks.workunit.client.1.smithi194.stdout:4/855: mknod d2/d11/d1e/d2f/c12b 0 2024-04-12T05:56:15.483 INFO:tasks.workunit.client.1.smithi194.stdout:2/694: getdents d7/d12/d1a 0 2024-04-12T05:56:15.484 INFO:tasks.workunit.client.1.smithi194.stdout:9/739: creat d1/d21/d77/ff8 x:0 0 0 2024-04-12T05:56:15.484 INFO:tasks.workunit.client.1.smithi194.stdout:6/631: dwrite d3/d2f/d4e/d63/f67 [0,4194304] 0 2024-04-12T05:56:15.489 INFO:tasks.workunit.client.1.smithi194.stdout:4/856: read d2/f21 [276637,112517] 0 2024-04-12T05:56:15.489 INFO:tasks.workunit.client.1.smithi194.stdout:1/755: mkdir d0/d11/d18/d19/d1a/d6a/d27/d48/da7/df3 0 2024-04-12T05:56:15.489 INFO:tasks.workunit.client.1.smithi194.stdout:9/740: mknod d1/d19/cf9 0 2024-04-12T05:56:15.489 INFO:tasks.workunit.client.1.smithi194.stdout:4/857: readlink d2/d3/d58/l60 0 2024-04-12T05:56:15.490 INFO:tasks.workunit.client.1.smithi194.stdout:7/594: dread d3/d27/f44 [0,4194304] 0 2024-04-12T05:56:15.490 INFO:tasks.workunit.client.1.smithi194.stdout:2/695: mknod d7/d12/d1a/dbf/cf3 0 2024-04-12T05:56:15.491 INFO:tasks.workunit.client.1.smithi194.stdout:4/858: chown d2/faf 46 1 2024-04-12T05:56:15.493 INFO:tasks.workunit.client.1.smithi194.stdout:3/793: mkdir d0/d1/d10/d8c/d8f/dac/de3/d10e 0 2024-04-12T05:56:15.493 INFO:tasks.workunit.client.1.smithi194.stdout:1/756: read d0/f85 [1271497,115403] 0 2024-04-12T05:56:15.494 INFO:tasks.workunit.client.1.smithi194.stdout:6/632: symlink d3/d2f/d4e/d68/dad/lc6 0 2024-04-12T05:56:15.495 INFO:tasks.workunit.client.1.smithi194.stdout:0/638: mknod d0/da/d1e/d8f/ccd 0 2024-04-12T05:56:15.495 INFO:tasks.workunit.client.1.smithi194.stdout:4/859: write d2/f21 [3143132,12007] 0 2024-04-12T05:56:15.499 INFO:tasks.workunit.client.1.smithi194.stdout:2/696: mkdir d7/d12/d25/d70/df4 0 2024-04-12T05:56:15.500 INFO:tasks.workunit.client.1.smithi194.stdout:9/741: unlink d1/d19/d1f/d20/d37/d84/c95 0 2024-04-12T05:56:15.503 INFO:tasks.workunit.client.1.smithi194.stdout:6/633: creat d3/d2f/d4e/d63/d83/fc7 x:0 0 0 2024-04-12T05:56:15.505 INFO:tasks.workunit.client.1.smithi194.stdout:1/757: fdatasync d0/d11/d18/d19/d1a/d6a/dac/fce 0 2024-04-12T05:56:15.506 INFO:tasks.workunit.client.1.smithi194.stdout:6/634: write d3/d2f/d4e/d63/faa [280330,5004] 0 2024-04-12T05:56:15.507 INFO:tasks.workunit.client.1.smithi194.stdout:7/595: mknod d3/d16/d23/d57/d72/d81/d3a/d8f/cc0 0 2024-04-12T05:56:15.508 INFO:tasks.workunit.client.1.smithi194.stdout:9/742: creat d1/d19/d1f/d5f/ffa x:0 0 0 2024-04-12T05:56:15.508 INFO:tasks.workunit.client.1.smithi194.stdout:2/697: dread d7/d12/f3d [0,4194304] 0 2024-04-12T05:56:15.508 INFO:tasks.workunit.client.1.smithi194.stdout:1/758: dread - d0/d11/d18/fa2 zero size 2024-04-12T05:56:15.512 INFO:tasks.workunit.client.1.smithi194.stdout:1/759: chown d0/d11/d18/d19/d1c/c1d 328 1 2024-04-12T05:56:15.513 INFO:tasks.workunit.client.1.smithi194.stdout:6/635: mkdir d3/d15/dc8 0 2024-04-12T05:56:15.515 INFO:tasks.workunit.client.1.smithi194.stdout:4/860: write d2/d3/d1b/d32/d3a/d3b/f41 [3784322,95135] 0 2024-04-12T05:56:15.516 INFO:tasks.workunit.client.1.smithi194.stdout:2/698: unlink d7/d12/d25/d5d/c75 0 2024-04-12T05:56:15.516 INFO:tasks.workunit.client.1.smithi194.stdout:6/636: fdatasync d3/d2f/d4e/d63/f67 0 2024-04-12T05:56:15.521 INFO:tasks.workunit.client.1.smithi194.stdout:4/861: unlink d2/d3/f11f 0 2024-04-12T05:56:15.521 INFO:tasks.workunit.client.1.smithi194.stdout:3/794: write d0/d1/d74/d39/d4d/deb/d53/f75 [3549691,116988] 0 2024-04-12T05:56:15.522 INFO:tasks.workunit.client.1.smithi194.stdout:6/637: read d3/d15/d2b/d48/f4f [1493936,554] 0 2024-04-12T05:56:15.523 INFO:tasks.workunit.client.1.smithi194.stdout:9/743: dwrite d1/d19/d1f/d20/f3d [4194304,4194304] 0 2024-04-12T05:56:15.525 INFO:tasks.workunit.client.1.smithi194.stdout:2/699: symlink d7/d12/lf5 0 2024-04-12T05:56:15.532 INFO:tasks.workunit.client.1.smithi194.stdout:0/639: dread d0/da/d4a/d8e/f72 [0,4194304] 0 2024-04-12T05:56:15.532 INFO:tasks.workunit.client.1.smithi194.stdout:6/638: chown d3/d15/d2b/d48/d6f/f82 5014997 1 2024-04-12T05:56:15.532 INFO:tasks.workunit.client.1.smithi194.stdout:4/862: write d2/d3/d58/d82/dd0/f121 [272327,28779] 0 2024-04-12T05:56:15.534 INFO:tasks.workunit.client.1.smithi194.stdout:9/744: write d1/d19/d1f/d5f/f86 [8505407,117245] 0 2024-04-12T05:56:15.536 INFO:tasks.workunit.client.1.smithi194.stdout:3/795: creat d0/d1/d10/d8c/d8f/dac/de3/d10e/f10f x:0 0 0 2024-04-12T05:56:15.538 INFO:tasks.workunit.client.1.smithi194.stdout:7/596: dwrite d3/d16/d23/d37/f7c [0,4194304] 0 2024-04-12T05:56:15.541 INFO:tasks.workunit.client.1.smithi194.stdout:1/760: dwrite d0/d11/d18/d19/d1a/d6a/dac/dba/fef [0,4194304] 0 2024-04-12T05:56:15.541 INFO:tasks.workunit.client.1.smithi194.stdout:0/640: write d0/da/d4a/d8e/fb0 [952706,710] 0 2024-04-12T05:56:15.541 INFO:tasks.workunit.client.1.smithi194.stdout:4/863: creat d2/d3/d1b/d32/f12c x:0 0 0 2024-04-12T05:56:15.545 INFO:tasks.workunit.client.1.smithi194.stdout:6/639: rmdir d3/d2d/d33/d52 39 2024-04-12T05:56:15.548 INFO:tasks.workunit.client.1.smithi194.stdout:6/640: write d3/d70/fab [914305,115461] 0 2024-04-12T05:56:15.550 INFO:tasks.workunit.client.1.smithi194.stdout:7/597: chown d3/d16/d23/d57/d72/d81/f92 264660870 1 2024-04-12T05:56:15.551 INFO:tasks.workunit.client.1.smithi194.stdout:9/745: creat d1/d19/d1f/d20/d37/d60/ffb x:0 0 0 2024-04-12T05:56:15.552 INFO:tasks.workunit.client.1.smithi194.stdout:3/796: fdatasync d0/d1/d74/d39/d4d/f62 0 2024-04-12T05:56:15.560 INFO:tasks.workunit.client.1.smithi194.stdout:6/641: truncate d3/d70/f86 316235 0 2024-04-12T05:56:15.562 INFO:tasks.workunit.client.1.smithi194.stdout:7/598: write d3/d16/f45 [826037,9025] 0 2024-04-12T05:56:15.563 INFO:tasks.workunit.client.1.smithi194.stdout:4/864: dwrite d2/d3/d50/f9e [0,4194304] 0 2024-04-12T05:56:15.565 INFO:tasks.workunit.client.1.smithi194.stdout:0/641: dwrite d0/da/d1e/d1a/d24/d56/f8d [0,4194304] 0 2024-04-12T05:56:15.565 INFO:tasks.workunit.client.1.smithi194.stdout:1/761: link d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/f8a d0/d11/d18/d19/d1a/ded/ff4 0 2024-04-12T05:56:15.568 INFO:tasks.workunit.client.1.smithi194.stdout:4/865: chown d2/d3/d1b/d32 1954 1 2024-04-12T05:56:15.568 INFO:tasks.workunit.client.1.smithi194.stdout:6/642: chown d3/d2d/d33/d52/d5c 107185821 1 2024-04-12T05:56:15.570 INFO:tasks.workunit.client.1.smithi194.stdout:7/599: unlink d3/d16/d23/d37/d95/fb6 0 2024-04-12T05:56:15.575 INFO:tasks.workunit.client.1.smithi194.stdout:1/762: symlink d0/d11/d18/d19/d1c/df0/lf5 0 2024-04-12T05:56:15.578 INFO:tasks.workunit.client.1.smithi194.stdout:0/642: creat d0/da/d1e/d1a/d58/d5b/fce x:0 0 0 2024-04-12T05:56:15.579 INFO:tasks.workunit.client.1.smithi194.stdout:4/866: read d2/d11/d1e/d2f/f112 [38125,5266] 0 2024-04-12T05:56:15.579 INFO:tasks.workunit.client.1.smithi194.stdout:7/600: symlink d3/d16/d23/lc1 0 2024-04-12T05:56:15.583 INFO:tasks.workunit.client.1.smithi194.stdout:2/700: dread d7/d22/d42/f2a [0,4194304] 0 2024-04-12T05:56:15.584 INFO:tasks.workunit.client.1.smithi194.stdout:6/643: dwrite d3/d15/d2b/d48/f54 [0,4194304] 0 2024-04-12T05:56:15.587 INFO:tasks.workunit.client.1.smithi194.stdout:9/746: dread d1/d21/d39/f89 [0,4194304] 0 2024-04-12T05:56:15.590 INFO:tasks.workunit.client.1.smithi194.stdout:4/867: rename d2/d85/dc2 to d2/d85/d11d/d12d 0 2024-04-12T05:56:15.593 INFO:tasks.workunit.client.1.smithi194.stdout:9/747: chown d1/d19/d1f/d20/d7d/f9c 244 1 2024-04-12T05:56:15.594 INFO:tasks.workunit.client.1.smithi194.stdout:7/601: dwrite d3/d16/f1a [0,4194304] 0 2024-04-12T05:56:15.594 INFO:tasks.workunit.client.1.smithi194.stdout:1/763: mkdir d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/df6 0 2024-04-12T05:56:15.594 INFO:tasks.workunit.client.1.smithi194.stdout:3/797: truncate d0/d1/d74/d39/da2/fc8 2773875 0 2024-04-12T05:56:15.595 INFO:tasks.workunit.client.1.smithi194.stdout:2/701: fsync d7/d12/d25/f31 0 2024-04-12T05:56:15.598 INFO:tasks.workunit.client.1.smithi194.stdout:0/643: rmdir d0/da/d4a/d8e/da2 0 2024-04-12T05:56:15.599 INFO:tasks.workunit.client.1.smithi194.stdout:6/644: symlink d3/d15/d2b/d48/d6f/d7a/d8f/d98/lc9 0 2024-04-12T05:56:15.609 INFO:tasks.workunit.client.1.smithi194.stdout:7/602: mknod d3/d16/d23/d57/d72/d81/d3a/d8f/cc2 0 2024-04-12T05:56:15.611 INFO:tasks.workunit.client.1.smithi194.stdout:3/798: truncate d0/d1/d74/d39/d4d/deb/d1e/ffb 652932 0 2024-04-12T05:56:15.619 INFO:tasks.workunit.client.1.smithi194.stdout:3/799: mkdir d0/d42/d110 0 2024-04-12T05:56:15.619 INFO:tasks.workunit.client.1.smithi194.stdout:4/868: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/f79 [0,4194304] 0 2024-04-12T05:56:15.624 INFO:tasks.workunit.client.1.smithi194.stdout:2/702: dwrite d7/d22/d42/f1e [4194304,4194304] 0 2024-04-12T05:56:15.627 INFO:tasks.workunit.client.1.smithi194.stdout:9/748: getdents d1/d19/d1f 0 2024-04-12T05:56:15.628 INFO:tasks.workunit.client.1.smithi194.stdout:1/764: write d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 [35787,102909] 0 2024-04-12T05:56:15.631 INFO:tasks.workunit.client.1.smithi194.stdout:9/749: stat d1/d4e/dea 0 2024-04-12T05:56:15.631 INFO:tasks.workunit.client.1.smithi194.stdout:7/603: dwrite d3/d27/d22/f77 [0,4194304] 0 2024-04-12T05:56:15.631 INFO:tasks.workunit.client.1.smithi194.stdout:1/765: write d0/f85 [1111817,79517] 0 2024-04-12T05:56:15.631 INFO:tasks.workunit.client.1.smithi194.stdout:5/732: sync 2024-04-12T05:56:15.633 INFO:tasks.workunit.client.1.smithi194.stdout:2/703: rmdir d7/d12/d6d 39 2024-04-12T05:56:15.641 INFO:tasks.workunit.client.1.smithi194.stdout:4/869: mkdir d2/d11/d93/da6/dab/d5d/dae/de1/d109/d12e 0 2024-04-12T05:56:15.641 INFO:tasks.workunit.client.1.smithi194.stdout:3/800: unlink d0/d1/d10/d8c/d8f/dac/de3/d10e/f10f 0 2024-04-12T05:56:15.641 INFO:tasks.workunit.client.1.smithi194.stdout:9/750: chown d1/d19/d1f/d20/f57 948881490 1 2024-04-12T05:56:15.641 INFO:tasks.workunit.client.1.smithi194.stdout:4/870: chown d2/f6 293 1 2024-04-12T05:56:15.641 INFO:tasks.workunit.client.1.smithi194.stdout:5/733: stat d1/d4/d84/fe5 0 2024-04-12T05:56:15.642 INFO:tasks.workunit.client.1.smithi194.stdout:1/766: chown d0/d11/d18/d19/d1c/f24 92 1 2024-04-12T05:56:15.643 INFO:tasks.workunit.client.1.smithi194.stdout:6/645: dwrite d3/f18 [4194304,4194304] 0 2024-04-12T05:56:15.645 INFO:tasks.workunit.client.1.smithi194.stdout:9/751: rmdir d1/d21/d77 39 2024-04-12T05:56:15.650 INFO:tasks.workunit.client.1.smithi194.stdout:5/734: write d1/d4/d6/db/fdd [560645,76225] 0 2024-04-12T05:56:15.651 INFO:tasks.workunit.client.1.smithi194.stdout:7/604: dwrite d3/d16/d5d/fbd [0,4194304] 0 2024-04-12T05:56:15.653 INFO:tasks.workunit.client.1.smithi194.stdout:7/605: stat d3/d16/d7b/l7d 0 2024-04-12T05:56:15.653 INFO:tasks.workunit.client.1.smithi194.stdout:4/871: symlink d2/d11/d46/d61/d63/l12f 0 2024-04-12T05:56:15.654 INFO:tasks.workunit.client.1.smithi194.stdout:9/752: mkdir d1/d4e/dea/dfc 0 2024-04-12T05:56:15.655 INFO:tasks.workunit.client.1.smithi194.stdout:6/646: rename d3/d70/lbf to d3/d2f/d4e/d68/dad/dc4/lca 0 2024-04-12T05:56:15.692 INFO:tasks.workunit.client.1.smithi194.stdout:2/704: rmdir d7/d12/d25/d5d/d47/dac/dc2 39 2024-04-12T05:56:15.692 INFO:tasks.workunit.client.1.smithi194.stdout:1/767: mknod d0/d11/d18/d19/d1a/d6a/d27/d48/cf7 0 2024-04-12T05:56:15.694 INFO:tasks.workunit.client.1.smithi194.stdout:4/872: symlink d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/l130 0 2024-04-12T05:56:15.696 INFO:tasks.workunit.client.1.smithi194.stdout:1/768: chown d0 5692078 1 2024-04-12T05:56:15.696 INFO:tasks.workunit.client.1.smithi194.stdout:9/753: mknod d1/d19/d1f/cfd 0 2024-04-12T05:56:15.697 INFO:tasks.workunit.client.1.smithi194.stdout:6/647: fdatasync d3/d2f/f39 0 2024-04-12T05:56:15.698 INFO:tasks.workunit.client.1.smithi194.stdout:7/606: dread d3/d27/d22/d8e/fae [0,4194304] 0 2024-04-12T05:56:15.699 INFO:tasks.workunit.client.1.smithi194.stdout:1/769: write d0/d11/d18/d19/fda [376704,41519] 0 2024-04-12T05:56:15.701 INFO:tasks.workunit.client.1.smithi194.stdout:9/754: truncate d1/d19/d1f/fe4 5196034 0 2024-04-12T05:56:15.704 INFO:tasks.workunit.client.1.smithi194.stdout:6/648: rename d3/d2f/d37/f43 to d3/d2d/fcb 0 2024-04-12T05:56:15.706 INFO:tasks.workunit.client.1.smithi194.stdout:2/705: creat d7/d12/d25/d5d/d47/dac/ff6 x:0 0 0 2024-04-12T05:56:15.708 INFO:tasks.workunit.client.1.smithi194.stdout:7/607: dread d3/d16/d23/d37/f96 [0,4194304] 0 2024-04-12T05:56:15.709 INFO:tasks.workunit.client.1.smithi194.stdout:3/801: dwrite d0/d1/d74/d39/d4d/deb/d1e/f6d [0,4194304] 0 2024-04-12T05:56:15.711 INFO:tasks.workunit.client.1.smithi194.stdout:9/755: readlink d1/d19/d1f/l6a 0 2024-04-12T05:56:15.711 INFO:tasks.workunit.client.1.smithi194.stdout:7/608: fsync d3/d16/d7b/fb9 0 2024-04-12T05:56:15.717 INFO:tasks.workunit.client.1.smithi194.stdout:6/649: creat d3/d2d/d33/d52/fcc x:0 0 0 2024-04-12T05:56:15.718 INFO:tasks.workunit.client.1.smithi194.stdout:7/609: write d3/d27/f2c [1961926,123568] 0 2024-04-12T05:56:15.721 INFO:tasks.workunit.client.1.smithi194.stdout:3/802: rmdir d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87 39 2024-04-12T05:56:15.724 INFO:tasks.workunit.client.1.smithi194.stdout:1/770: dwrite d0/d11/d18/fc3 [0,4194304] 0 2024-04-12T05:56:15.724 INFO:tasks.workunit.client.1.smithi194.stdout:6/650: rename d3/cbe to d3/d34/ccd 0 2024-04-12T05:56:15.724 INFO:tasks.workunit.client.1.smithi194.stdout:4/873: dread d2/d3/f74 [0,4194304] 0 2024-04-12T05:56:15.725 INFO:tasks.workunit.client.1.smithi194.stdout:7/610: mkdir d3/d16/d7b/dc3 0 2024-04-12T05:56:15.728 INFO:tasks.workunit.client.1.smithi194.stdout:3/803: write d0/d1/d74/d39/d4d/deb/d3d/f69 [4738243,10726] 0 2024-04-12T05:56:15.730 INFO:tasks.workunit.client.1.smithi194.stdout:7/611: write d3/d27/f7f [127975,36679] 0 2024-04-12T05:56:15.732 INFO:tasks.workunit.client.1.smithi194.stdout:9/756: rename d1/d19/f83 to d1/d21/d77/d98/dab/ffe 0 2024-04-12T05:56:15.735 INFO:tasks.workunit.client.1.smithi194.stdout:4/874: mkdir d2/d11/d46/d61/d63/d131 0 2024-04-12T05:56:15.736 INFO:tasks.workunit.client.1.smithi194.stdout:4/875: write d2/d3/d58/f69 [234467,1454] 0 2024-04-12T05:56:15.737 INFO:tasks.workunit.client.1.smithi194.stdout:9/757: rmdir d1/d19/d1f/d20/d7d 39 2024-04-12T05:56:15.737 INFO:tasks.workunit.client.1.smithi194.stdout:2/706: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/f81 [137044,126253] 0 2024-04-12T05:56:15.738 INFO:tasks.workunit.client.1.smithi194.stdout:6/651: rename d3/d15/fb9 to d3/d2d/d33/d58/fce 0 2024-04-12T05:56:15.741 INFO:tasks.workunit.client.1.smithi194.stdout:6/652: readlink d3/d2d/d33/d52/l91 0 2024-04-12T05:56:15.744 INFO:tasks.workunit.client.1.smithi194.stdout:3/804: rename d0/d1/d74/d39/d4d/deb/d1e/d81/df4/c10c to d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/c111 0 2024-04-12T05:56:15.746 INFO:tasks.workunit.client.1.smithi194.stdout:1/771: getdents d0/d11/d18/d19/d1a/d6a/dac/db5 0 2024-04-12T05:56:15.747 INFO:tasks.workunit.client.1.smithi194.stdout:7/612: getdents d3/d16/d7b 0 2024-04-12T05:56:15.747 INFO:tasks.workunit.client.1.smithi194.stdout:4/876: dwrite d2/d11/d43/f11b [0,4194304] 0 2024-04-12T05:56:15.749 INFO:tasks.workunit.client.1.smithi194.stdout:3/805: stat d0/d1/d74/d39/f7e 0 2024-04-12T05:56:15.752 INFO:tasks.workunit.client.1.smithi194.stdout:4/877: rmdir d2/d11/d93/da6/dab/d5d/dae/de1/d109 39 2024-04-12T05:56:15.754 INFO:tasks.workunit.client.1.smithi194.stdout:2/707: rename d7/d22/d42/f77 to d7/d22/ff7 0 2024-04-12T05:56:15.756 INFO:tasks.workunit.client.1.smithi194.stdout:9/758: dread d1/d19/d1f/f70 [0,4194304] 0 2024-04-12T05:56:15.757 INFO:tasks.workunit.client.1.smithi194.stdout:7/613: creat d3/d16/d23/d57/d72/d81/d3a/d43/d5c/fc4 x:0 0 0 2024-04-12T05:56:15.758 INFO:tasks.workunit.client.1.smithi194.stdout:4/878: read d2/d11/f88 [319606,97813] 0 2024-04-12T05:56:15.761 INFO:tasks.workunit.client.1.smithi194.stdout:2/708: mkdir d7/d12/d25/d5d/d47/dac/db3/dd7/df8 0 2024-04-12T05:56:15.762 INFO:tasks.workunit.client.1.smithi194.stdout:3/806: dread d0/d42/f73 [0,4194304] 0 2024-04-12T05:56:15.763 INFO:tasks.workunit.client.1.smithi194.stdout:9/759: unlink d1/d19/d1f/d20/d37/d84/fdd 0 2024-04-12T05:56:15.765 INFO:tasks.workunit.client.1.smithi194.stdout:4/879: rename d2/d3/l20 to d2/d11/d1e/d2f/l132 0 2024-04-12T05:56:15.768 INFO:tasks.workunit.client.1.smithi194.stdout:9/760: creat d1/d41/fff x:0 0 0 2024-04-12T05:56:15.769 INFO:tasks.workunit.client.1.smithi194.stdout:2/709: write d7/d22/ff7 [2122471,40174] 0 2024-04-12T05:56:15.771 INFO:tasks.workunit.client.1.smithi194.stdout:4/880: rename d2/d11/d1f/c83 to d2/d85/d11d/d12d/c133 0 2024-04-12T05:56:15.771 INFO:tasks.workunit.client.1.smithi194.stdout:1/772: dread d0/f32 [0,4194304] 0 2024-04-12T05:56:15.772 INFO:tasks.workunit.client.1.smithi194.stdout:4/881: chown d2/d3/d50/l6b 10509 1 2024-04-12T05:56:15.773 INFO:tasks.workunit.client.1.smithi194.stdout:7/614: getdents d3/d16/d23/d57/d72/d81/d3a 0 2024-04-12T05:56:15.774 INFO:tasks.workunit.client.1.smithi194.stdout:2/710: unlink d7/d22/d93/d9a/la8 0 2024-04-12T05:56:15.774 INFO:tasks.workunit.client.1.smithi194.stdout:4/882: stat d2/d11/d1e/d25/c115 0 2024-04-12T05:56:15.776 INFO:tasks.workunit.client.1.smithi194.stdout:1/773: chown d0/d11/d18/l44 0 1 2024-04-12T05:56:15.777 INFO:tasks.workunit.client.1.smithi194.stdout:7/615: unlink d3/d27/f7f 0 2024-04-12T05:56:15.779 INFO:tasks.workunit.client.1.smithi194.stdout:4/883: fsync d2/d11/d46/d61/d63/da2/fa5 0 2024-04-12T05:56:15.781 INFO:tasks.workunit.client.1.smithi194.stdout:3/807: write d0/d1/d74/d39/da2/fbd [4351451,56311] 0 2024-04-12T05:56:15.782 INFO:tasks.workunit.client.1.smithi194.stdout:9/761: dwrite d1/d4e/f7e [4194304,4194304] 0 2024-04-12T05:56:15.782 INFO:tasks.workunit.client.1.smithi194.stdout:7/616: read d3/d27/f2f [436843,12589] 0 2024-04-12T05:56:15.784 INFO:tasks.workunit.client.1.smithi194.stdout:3/808: stat d0/d1/d4f/fa3 0 2024-04-12T05:56:15.788 INFO:tasks.workunit.client.1.smithi194.stdout:4/884: unlink d2/l26 0 2024-04-12T05:56:15.788 INFO:tasks.workunit.client.1.smithi194.stdout:4/885: chown d2/d11/d46/d61/d63/l12f 62 1 2024-04-12T05:56:15.788 INFO:tasks.workunit.client.1.smithi194.stdout:7/617: creat d3/d16/d23/d57/d72/fc5 x:0 0 0 2024-04-12T05:56:15.789 INFO:tasks.workunit.client.1.smithi194.stdout:9/762: mknod d1/d19/d1f/d20/d37/c100 0 2024-04-12T05:56:15.790 INFO:tasks.workunit.client.1.smithi194.stdout:4/886: fsync d2/d11/d46/d61/f92 0 2024-04-12T05:56:15.791 INFO:tasks.workunit.client.1.smithi194.stdout:7/618: unlink d3/f48 0 2024-04-12T05:56:15.795 INFO:tasks.workunit.client.1.smithi194.stdout:4/887: readlink d2/d3/ld 0 2024-04-12T05:56:15.796 INFO:tasks.workunit.client.1.smithi194.stdout:4/888: fsync d2/d3/d1b/d32/fdd 0 2024-04-12T05:56:15.796 INFO:tasks.workunit.client.1.smithi194.stdout:2/711: truncate d7/d22/d93/d9f/fb5 3515436 0 2024-04-12T05:56:15.797 INFO:tasks.workunit.client.1.smithi194.stdout:7/619: symlink d3/d16/d23/d37/d3d/d56/da4/lc6 0 2024-04-12T05:56:15.799 INFO:tasks.workunit.client.1.smithi194.stdout:4/889: fdatasync d2/d3/f74 0 2024-04-12T05:56:15.800 INFO:tasks.workunit.client.1.smithi194.stdout:2/712: chown d7/l30 12919982 1 2024-04-12T05:56:15.801 INFO:tasks.workunit.client.1.smithi194.stdout:7/620: chown d3/d16/d23/d57/d72/d81/d3a/d43/c5e 55869833 1 2024-04-12T05:56:15.803 INFO:tasks.workunit.client.1.smithi194.stdout:3/809: dread d0/d1/d74/d39/f70 [0,4194304] 0 2024-04-12T05:56:15.803 INFO:tasks.workunit.client.1.smithi194.stdout:4/890: mkdir d2/d11/d1e/d2f/d4a/d134 0 2024-04-12T05:56:15.804 INFO:tasks.workunit.client.1.smithi194.stdout:9/763: dread d1/d21/d39/d72/fb5 [0,4194304] 0 2024-04-12T05:56:15.806 INFO:tasks.workunit.client.1.smithi194.stdout:3/810: read d0/d1/d74/d39/d4d/f62 [10932,81303] 0 2024-04-12T05:56:15.806 INFO:tasks.workunit.client.1.smithi194.stdout:4/891: read d2/d11/d1e/d2f/f11e [226150,95961] 0 2024-04-12T05:56:15.807 INFO:tasks.workunit.client.1.smithi194.stdout:1/774: dread d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 [0,4194304] 0 2024-04-12T05:56:15.812 INFO:tasks.workunit.client.1.smithi194.stdout:4/892: creat d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/f135 x:0 0 0 2024-04-12T05:56:15.813 INFO:tasks.workunit.client.1.smithi194.stdout:1/775: unlink d0/d11/d18/d19/d1c/d67/cb2 0 2024-04-12T05:56:15.816 INFO:tasks.workunit.client.1.smithi194.stdout:1/776: stat d0/d11/d18/d19/d1a/d6a/dac/db5/d68 0 2024-04-12T05:56:15.816 INFO:tasks.workunit.client.1.smithi194.stdout:3/811: write d0/d1/d74/d39/fb0 [807683,83285] 0 2024-04-12T05:56:15.821 INFO:tasks.workunit.client.1.smithi194.stdout:2/713: dread d7/d22/f4d [4194304,4194304] 0 2024-04-12T05:56:15.824 INFO:tasks.workunit.client.1.smithi194.stdout:9/764: dread d1/d4e/f6d [0,4194304] 0 2024-04-12T05:56:15.824 INFO:tasks.workunit.client.1.smithi194.stdout:4/893: dwrite d2/d11/d46/d61/fe0 [0,4194304] 0 2024-04-12T05:56:15.824 INFO:tasks.workunit.client.1.smithi194.stdout:1/777: mkdir d0/d11/d18/d19/d1a/d6a/dac/db5/df8 0 2024-04-12T05:56:15.827 INFO:tasks.workunit.client.1.smithi194.stdout:2/714: mkdir d7/d12/d1a/d69/dd6/df9 0 2024-04-12T05:56:15.829 INFO:tasks.workunit.client.1.smithi194.stdout:1/778: creat d0/d11/d18/d19/d1a/d6a/dac/db5/ff9 x:0 0 0 2024-04-12T05:56:15.829 INFO:tasks.workunit.client.1.smithi194.stdout:4/894: mkdir d2/d11/d46/d61/d136 0 2024-04-12T05:56:15.830 INFO:tasks.workunit.client.1.smithi194.stdout:4/895: readlink d2/d3/d58/d82/lac 0 2024-04-12T05:56:15.831 INFO:tasks.workunit.client.1.smithi194.stdout:2/715: creat d7/d12/d25/d5d/d47/d52/d7b/db2/ffa x:0 0 0 2024-04-12T05:56:15.833 INFO:tasks.workunit.client.1.smithi194.stdout:2/716: chown d7/d22/ff7 202 1 2024-04-12T05:56:15.833 INFO:tasks.workunit.client.1.smithi194.stdout:9/765: getdents d1/d19/d1f 0 2024-04-12T05:56:15.836 INFO:tasks.workunit.client.1.smithi194.stdout:9/766: rmdir d1/d21/d77/d98/d9f 39 2024-04-12T05:56:15.836 INFO:tasks.workunit.client.1.smithi194.stdout:4/896: chown d2/d3/d51/cef 1 1 2024-04-12T05:56:15.838 INFO:tasks.workunit.client.1.smithi194.stdout:1/779: link d0/d11/c4c d0/d11/d18/d19/d1a/ded/cfa 0 2024-04-12T05:56:15.840 INFO:tasks.workunit.client.1.smithi194.stdout:4/897: write d2/d11/d93/da6/dab/d5d/f7c [543229,98697] 0 2024-04-12T05:56:15.845 INFO:tasks.workunit.client.1.smithi194.stdout:2/717: dwrite d7/d12/f3b [0,4194304] 0 2024-04-12T05:56:15.845 INFO:tasks.workunit.client.1.smithi194.stdout:3/812: dread d0/d1/d74/d39/d4d/deb/d1e/d81/d34/fa1 [4194304,4194304] 0 2024-04-12T05:56:15.847 INFO:tasks.workunit.client.1.smithi194.stdout:1/780: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/ffb x:0 0 0 2024-04-12T05:56:15.848 INFO:tasks.workunit.client.1.smithi194.stdout:3/813: chown d0/d1/d10/c72 14118796 1 2024-04-12T05:56:15.848 INFO:tasks.workunit.client.1.smithi194.stdout:1/781: chown d0/d11/d18/d19/d1c/cdc 486 1 2024-04-12T05:56:15.852 INFO:tasks.workunit.client.1.smithi194.stdout:1/782: dread - d0/d11/fc1 zero size 2024-04-12T05:56:15.853 INFO:tasks.workunit.client.1.smithi194.stdout:9/767: truncate d1/d19/d1f/d5f/f86 664792 0 2024-04-12T05:56:15.854 INFO:tasks.workunit.client.1.smithi194.stdout:9/768: chown d1/d21/d49/c90 639412477 1 2024-04-12T05:56:15.855 INFO:tasks.workunit.client.1.smithi194.stdout:9/769: fdatasync d1/fd4 0 2024-04-12T05:56:15.857 INFO:tasks.workunit.client.1.smithi194.stdout:9/770: truncate d1/d19/d1f/d20/d37/d84/fe0 425134 0 2024-04-12T05:56:15.858 INFO:tasks.workunit.client.1.smithi194.stdout:9/771: chown d1/d41/dbf/ccf 6905453 1 2024-04-12T05:56:15.858 INFO:tasks.workunit.client.1.smithi194.stdout:3/814: rename d0/d1/d74/d39/d4d/deb/d3d/dae/lb6 to d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/dc5/l112 0 2024-04-12T05:56:15.860 INFO:tasks.workunit.client.1.smithi194.stdout:9/772: read d1/d19/d1f/d5f/f85 [1637311,2520] 0 2024-04-12T05:56:15.868 INFO:tasks.workunit.client.1.smithi194.stdout:1/783: truncate d0/d11/d18/d19/d1c/d67/fb7 1741091 0 2024-04-12T05:56:15.869 INFO:tasks.workunit.client.1.smithi194.stdout:9/773: dwrite d1/d19/d1f/d20/f3d [0,4194304] 0 2024-04-12T05:56:15.870 INFO:tasks.workunit.client.1.smithi194.stdout:3/815: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d34/fa1 5857358 0 2024-04-12T05:56:15.872 INFO:tasks.workunit.client.1.smithi194.stdout:1/784: symlink d0/d11/d18/d19/d1a/d6a/d27/da8/lfc 0 2024-04-12T05:56:15.873 INFO:tasks.workunit.client.1.smithi194.stdout:1/785: stat d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/cbf 0 2024-04-12T05:56:15.873 INFO:tasks.workunit.client.1.smithi194.stdout:9/774: unlink d1/d19/d1f/d20/f8b 0 2024-04-12T05:56:15.874 INFO:tasks.workunit.client.1.smithi194.stdout:1/786: chown d0/le 1598635 1 2024-04-12T05:56:15.875 INFO:tasks.workunit.client.1.smithi194.stdout:3/816: symlink d0/d1/d74/d39/d4d/deb/db4/dd9/l113 0 2024-04-12T05:56:15.876 INFO:tasks.workunit.client.1.smithi194.stdout:9/775: readlink d1/d19/d1f/d20/l23 0 2024-04-12T05:56:15.877 INFO:tasks.workunit.client.1.smithi194.stdout:9/776: stat d1/d19/d1f/d5f/f9e 0 2024-04-12T05:56:15.881 INFO:tasks.workunit.client.1.smithi194.stdout:9/777: mknod d1/d19/d1f/d20/d37/c101 0 2024-04-12T05:56:15.881 INFO:tasks.workunit.client.1.smithi194.stdout:3/817: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/fb3 644070 0 2024-04-12T05:56:15.884 INFO:tasks.workunit.client.1.smithi194.stdout:3/818: mkdir d0/d1/d4f/dd7/d114 0 2024-04-12T05:56:15.884 INFO:tasks.workunit.client.1.smithi194.stdout:9/778: mknod d1/d19/d1f/d5f/df4/c102 0 2024-04-12T05:56:15.885 INFO:tasks.workunit.client.1.smithi194.stdout:3/819: write d0/d42/fba [950866,39861] 0 2024-04-12T05:56:15.888 INFO:tasks.workunit.client.1.smithi194.stdout:3/820: chown d0/d1/d74/d39/d4d/deb/c7a 123992 1 2024-04-12T05:56:15.890 INFO:tasks.workunit.client.1.smithi194.stdout:1/787: dread d0/d11/d18/d19/f39 [0,4194304] 0 2024-04-12T05:56:15.892 INFO:tasks.workunit.client.1.smithi194.stdout:1/788: chown d0/d11/d18/d19/d1a/d6a/d27/da8/lfc 34937958 1 2024-04-12T05:56:15.893 INFO:tasks.workunit.client.1.smithi194.stdout:3/821: mkdir d0/d1/d74/d39/d4d/deb/d8e/d115 0 2024-04-12T05:56:15.895 INFO:tasks.workunit.client.1.smithi194.stdout:3/822: creat d0/d1/d10/d8c/d8f/ddb/f116 x:0 0 0 2024-04-12T05:56:15.897 INFO:tasks.workunit.client.1.smithi194.stdout:9/779: write d1/d19/f6c [522254,63329] 0 2024-04-12T05:56:15.899 INFO:tasks.workunit.client.1.smithi194.stdout:1/789: dwrite d0/d11/d18/f76 [0,4194304] 0 2024-04-12T05:56:15.900 INFO:tasks.workunit.client.1.smithi194.stdout:9/780: stat d1/d19/d1f/d5f/f85 0 2024-04-12T05:56:15.903 INFO:tasks.workunit.client.1.smithi194.stdout:1/790: mknod d0/d11/d18/d19/d91/cfd 0 2024-04-12T05:56:15.904 INFO:tasks.workunit.client.1.smithi194.stdout:9/781: stat d1/d21/d77/d98/d9f/dde/fe7 0 2024-04-12T05:56:15.909 INFO:tasks.workunit.client.1.smithi194.stdout:1/791: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/ffe x:0 0 0 2024-04-12T05:56:15.910 INFO:tasks.workunit.client.1.smithi194.stdout:3/823: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d34/fdd [0,4194304] 0 2024-04-12T05:56:15.912 INFO:tasks.workunit.client.1.smithi194.stdout:3/824: stat d0/d1/d74/d39/d4d/deb/d1e/d33/d76 0 2024-04-12T05:56:15.912 INFO:tasks.workunit.client.1.smithi194.stdout:1/792: chown d0/d11/d18/d19/d1a/d6a/d27/d6c/f74 88765 1 2024-04-12T05:56:15.913 INFO:tasks.workunit.client.1.smithi194.stdout:9/782: link d1/l33 d1/d21/d39/d72/l103 0 2024-04-12T05:56:15.915 INFO:tasks.workunit.client.1.smithi194.stdout:3/825: write d0/d1/d74/d39/d4d/deb/db4/fed [1577374,40289] 0 2024-04-12T05:56:15.922 INFO:tasks.workunit.client.1.smithi194.stdout:3/826: symlink d0/d1/d74/d39/d4d/deb/d3d/dae/dbe/l117 0 2024-04-12T05:56:15.925 INFO:tasks.workunit.client.1.smithi194.stdout:9/783: write d1/d19/d1f/d5f/f86 [708753,39596] 0 2024-04-12T05:56:15.928 INFO:tasks.workunit.client.1.smithi194.stdout:3/827: dread d0/d1/d74/d39/d4d/deb/d3d/f94 [0,4194304] 0 2024-04-12T05:56:15.930 INFO:tasks.workunit.client.1.smithi194.stdout:9/784: creat d1/d19/d1f/d20/d37/da3/dc7/f104 x:0 0 0 2024-04-12T05:56:15.930 INFO:tasks.workunit.client.1.smithi194.stdout:1/793: dwrite d0/d11/d18/d19/d1a/d6a/d27/fe1 [0,4194304] 0 2024-04-12T05:56:15.934 INFO:tasks.workunit.client.1.smithi194.stdout:1/794: unlink d0/d11/d18/d19/d1c/f20 0 2024-04-12T05:56:15.935 INFO:tasks.workunit.client.1.smithi194.stdout:9/785: rename d1/d19/cd9 to d1/d19/d1f/d20/d37/d84/c105 0 2024-04-12T05:56:15.936 INFO:tasks.workunit.client.1.smithi194.stdout:8/468: sync 2024-04-12T05:56:15.937 INFO:tasks.workunit.client.1.smithi194.stdout:3/828: dread d0/d1/d74/d39/d4d/deb/d1e/ffb [0,4194304] 0 2024-04-12T05:56:15.938 INFO:tasks.workunit.client.1.smithi194.stdout:8/469: write d2/d8/d9/de/d10/f7c [637448,46494] 0 2024-04-12T05:56:15.939 INFO:tasks.workunit.client.1.smithi194.stdout:3/829: fsync d0/d1/d74/d39/d4d/deb/d3d/d6e/de9/f106 0 2024-04-12T05:56:15.939 INFO:tasks.workunit.client.1.smithi194.stdout:8/470: dread - d2/d8/d9/de/d38/d8f/f90 zero size 2024-04-12T05:56:15.942 INFO:tasks.workunit.client.1.smithi194.stdout:8/471: rename d2/fb to d2/d8/d15/d35/d6c/f91 0 2024-04-12T05:56:15.943 INFO:tasks.workunit.client.1.smithi194.stdout:3/830: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f118 x:0 0 0 2024-04-12T05:56:15.943 INFO:tasks.workunit.client.1.smithi194.stdout:8/472: chown d2/d8/d15/d33/d45/f55 56919588 1 2024-04-12T05:56:15.944 INFO:tasks.workunit.client.1.smithi194.stdout:3/831: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87 37196 1 2024-04-12T05:56:15.946 INFO:tasks.workunit.client.1.smithi194.stdout:8/473: symlink d2/d8/d9/de/d86/l92 0 2024-04-12T05:56:15.948 INFO:tasks.workunit.client.1.smithi194.stdout:8/474: read d2/d8/f36 [1109468,4053] 0 2024-04-12T05:56:15.950 INFO:tasks.workunit.client.1.smithi194.stdout:8/475: write d2/d8/d9/de/d10/d7d/f81 [79725,76270] 0 2024-04-12T05:56:15.950 INFO:tasks.workunit.client.1.smithi194.stdout:8/476: fsync d2/d8/d9/f2a 0 2024-04-12T05:56:15.952 INFO:tasks.workunit.client.1.smithi194.stdout:8/477: chown d2/d8/d9/de/d38/d8f/f90 307 1 2024-04-12T05:56:15.952 INFO:tasks.workunit.client.1.smithi194.stdout:8/478: fsync d2/d8/d9/d1a/f79 0 2024-04-12T05:56:15.954 INFO:tasks.workunit.client.1.smithi194.stdout:8/479: mknod d2/d8/d9/de/d38/c93 0 2024-04-12T05:56:15.955 INFO:tasks.workunit.client.1.smithi194.stdout:4/898: write d2/d11/d46/d61/d63/f84 [3241038,66616] 0 2024-04-12T05:56:15.956 INFO:tasks.workunit.client.1.smithi194.stdout:8/480: truncate d2/f2d 4749561 0 2024-04-12T05:56:15.957 INFO:tasks.workunit.client.1.smithi194.stdout:4/899: write d2/d11/d1e/d2f/f76 [144334,46937] 0 2024-04-12T05:56:15.958 INFO:tasks.workunit.client.1.smithi194.stdout:8/481: write d2/d8/d15/d35/f4c [1841406,96225] 0 2024-04-12T05:56:15.961 INFO:tasks.workunit.client.1.smithi194.stdout:8/482: fdatasync d2/d8/d66/d67/f74 0 2024-04-12T05:56:15.962 INFO:tasks.workunit.client.1.smithi194.stdout:3/832: dread d0/d1/d74/d39/d4d/deb/d3d/dae/faf [0,4194304] 0 2024-04-12T05:56:15.964 INFO:tasks.workunit.client.1.smithi194.stdout:8/483: write d2/d8/d15/f62 [691251,52719] 0 2024-04-12T05:56:15.966 INFO:tasks.workunit.client.1.smithi194.stdout:8/484: read - d2/d8/d15/d3b/f78 zero size 2024-04-12T05:56:15.966 INFO:tasks.workunit.client.1.smithi194.stdout:4/900: dwrite d2/d11/d93/da6/dab/fde [0,4194304] 0 2024-04-12T05:56:15.969 INFO:tasks.workunit.client.1.smithi194.stdout:8/485: readlink d2/d8/d9/de/d10/l64 0 2024-04-12T05:56:15.970 INFO:tasks.workunit.client.1.smithi194.stdout:3/833: getdents d0/d1/d74/d39/d4d/deb/d1e/dd3 0 2024-04-12T05:56:15.972 INFO:tasks.workunit.client.1.smithi194.stdout:3/834: mknod d0/d1/d74/d39/c119 0 2024-04-12T05:56:15.974 INFO:tasks.workunit.client.1.smithi194.stdout:4/901: unlink d2/d11/d1e/d2f/f71 0 2024-04-12T05:56:15.974 INFO:tasks.workunit.client.1.smithi194.stdout:3/835: symlink d0/d1/d74/d39/da2/l11a 0 2024-04-12T05:56:15.975 INFO:tasks.workunit.client.1.smithi194.stdout:3/836: chown d0/d1/d10/d8c/d8f/dac/cea 1585617722 1 2024-04-12T05:56:15.978 INFO:tasks.workunit.client.1.smithi194.stdout:3/837: unlink d0/d1/d74/d39/d4d/deb/l15 0 2024-04-12T05:56:15.983 INFO:tasks.workunit.client.1.smithi194.stdout:4/902: unlink d2/d3/la 0 2024-04-12T05:56:15.984 INFO:tasks.workunit.client.1.smithi194.stdout:4/903: write d2/d85/fd1 [931434,75554] 0 2024-04-12T05:56:15.990 INFO:tasks.workunit.client.1.smithi194.stdout:8/486: dread d2/d8/d9/de/d10/f1c [0,4194304] 0 2024-04-12T05:56:15.991 INFO:tasks.workunit.client.1.smithi194.stdout:8/487: chown d2/d8/d9/d1a/l4d 353 1 2024-04-12T05:56:15.991 INFO:tasks.workunit.client.1.smithi194.stdout:4/904: rename d2/d3/f91 to d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/f137 0 2024-04-12T05:56:15.994 INFO:tasks.workunit.client.1.smithi194.stdout:8/488: getdents d2/d8/d15 0 2024-04-12T05:56:15.994 INFO:tasks.workunit.client.1.smithi194.stdout:4/905: mknod d2/d11/d46/d61/d63/da2/c138 0 2024-04-12T05:56:15.996 INFO:tasks.workunit.client.1.smithi194.stdout:8/489: mkdir d2/d8/d9/de/d38/d8f/d94 0 2024-04-12T05:56:15.999 INFO:tasks.workunit.client.1.smithi194.stdout:8/490: dread d2/d8/d9/f2a [0,4194304] 0 2024-04-12T05:56:16.001 INFO:tasks.workunit.client.1.smithi194.stdout:8/491: truncate d2/d8/d9/d1a/f30 4204283 0 2024-04-12T05:56:16.002 INFO:tasks.workunit.client.1.smithi194.stdout:8/492: read - d2/d8/d9/d1a/f5a zero size 2024-04-12T05:56:16.004 INFO:tasks.workunit.client.1.smithi194.stdout:8/493: mknod d2/d8/d15/d33/d45/d52/c95 0 2024-04-12T05:56:16.004 INFO:tasks.workunit.client.1.smithi194.stdout:4/906: rmdir d2/d11/d1e/d2f/d4a/d134 0 2024-04-12T05:56:16.006 INFO:tasks.workunit.client.1.smithi194.stdout:8/494: truncate d2/d8/d9/d1a/f5a 168032 0 2024-04-12T05:56:16.008 INFO:tasks.workunit.client.1.smithi194.stdout:8/495: truncate d2/d8/d15/d33/f1b 643383 0 2024-04-12T05:56:16.009 INFO:tasks.workunit.client.1.smithi194.stdout:4/907: creat d2/d11/d1e/d2f/d4a/f139 x:0 0 0 2024-04-12T05:56:16.010 INFO:tasks.workunit.client.1.smithi194.stdout:8/496: creat d2/d8/d66/d67/f96 x:0 0 0 2024-04-12T05:56:16.012 INFO:tasks.workunit.client.1.smithi194.stdout:8/497: write d2/d8/d15/d35/d6c/f91 [2252484,29883] 0 2024-04-12T05:56:16.014 INFO:tasks.workunit.client.1.smithi194.stdout:8/498: symlink d2/d8/d9/l97 0 2024-04-12T05:56:16.017 INFO:tasks.workunit.client.1.smithi194.stdout:8/499: rename d2/d8/d9/de/d10/f1c to d2/d8/d9/de/f98 0 2024-04-12T05:56:16.018 INFO:tasks.workunit.client.1.smithi194.stdout:4/908: symlink d2/d3/d51/de9/l13a 0 2024-04-12T05:56:16.024 INFO:tasks.workunit.client.1.smithi194.stdout:4/909: stat d2/d3/d1b/d32/d3a/d3b/f75 0 2024-04-12T05:56:16.024 INFO:tasks.workunit.client.1.smithi194.stdout:4/910: chown d2/dcf/c11a 0 1 2024-04-12T05:56:16.024 INFO:tasks.workunit.client.1.smithi194.stdout:4/911: symlink d2/d11/d1e/d2f/d4a/l13b 0 2024-04-12T05:56:16.025 INFO:tasks.workunit.client.1.smithi194.stdout:4/912: write d2/d3/d1b/d32/fdd [1075780,82021] 0 2024-04-12T05:56:16.028 INFO:tasks.workunit.client.1.smithi194.stdout:4/913: mkdir d2/d3/d50/da0/d13c 0 2024-04-12T05:56:16.029 INFO:tasks.workunit.client.1.smithi194.stdout:8/500: rmdir d2/d8/d15/d33/d45/d52 39 2024-04-12T05:56:16.032 INFO:tasks.workunit.client.1.smithi194.stdout:4/914: mkdir d2/d11/d1e/d2f/d4a/d9c/dee/d13d 0 2024-04-12T05:56:16.035 INFO:tasks.workunit.client.1.smithi194.stdout:4/915: read d2/d11/d93/da6/dab/d5d/dae/db2/f34 [2719677,5188] 0 2024-04-12T05:56:16.035 INFO:tasks.workunit.client.1.smithi194.stdout:8/501: dwrite d2/d8/d9/d1a/f30 [0,4194304] 0 2024-04-12T05:56:16.038 INFO:tasks.workunit.client.1.smithi194.stdout:8/502: symlink d2/d8/d9/de/l99 0 2024-04-12T05:56:16.039 INFO:tasks.workunit.client.1.smithi194.stdout:4/916: mkdir d2/d3/d51/d13e 0 2024-04-12T05:56:16.039 INFO:tasks.workunit.client.1.smithi194.stdout:4/917: stat d2/d11/d46/d61/d63/da2/c138 0 2024-04-12T05:56:16.040 INFO:tasks.workunit.client.1.smithi194.stdout:4/918: write d2/d11/d46/d61/fe0 [2876257,17382] 0 2024-04-12T05:56:16.041 INFO:tasks.workunit.client.1.smithi194.stdout:4/919: fdatasync d2/fc 0 2024-04-12T05:56:16.048 INFO:tasks.workunit.client.1.smithi194.stdout:8/503: dwrite d2/d8/d15/d33/d45/d52/f7b [0,4194304] 0 2024-04-12T05:56:16.105 INFO:tasks.workunit.client.1.smithi194.stdout:4/920: truncate d2/d3/ff 447405 0 2024-04-12T05:56:16.107 INFO:tasks.workunit.client.1.smithi194.stdout:4/921: mkdir d2/d3/d50/da0/d13c/d13f 0 2024-04-12T05:56:16.116 INFO:tasks.workunit.client.1.smithi194.stdout:4/922: dwrite d2/d11/d1e/d2f/f11e [0,4194304] 0 2024-04-12T05:56:16.119 INFO:tasks.workunit.client.1.smithi194.stdout:4/923: symlink d2/d3/d1b/d32/db3/l140 0 2024-04-12T05:56:16.150 INFO:tasks.workunit.client.1.smithi194.stdout:0/644: sync 2024-04-12T05:56:16.185 INFO:tasks.workunit.client.1.smithi194.stdout:4/924: write d2/d3/f125 [949617,58129] 0 2024-04-12T05:56:16.186 INFO:tasks.workunit.client.1.smithi194.stdout:0/645: fsync d0/da/d1e/d1a/f31 0 2024-04-12T05:56:16.188 INFO:tasks.workunit.client.1.smithi194.stdout:0/646: write d0/da/d1e/d1a/d58/d5b/fb6 [1013920,66830] 0 2024-04-12T05:56:16.190 INFO:tasks.workunit.client.1.smithi194.stdout:4/925: rename d2/d11/d93/da6/dab/d5d/dae/db2/f2a to d2/dcf/f141 0 2024-04-12T05:56:16.190 INFO:tasks.workunit.client.1.smithi194.stdout:0/647: unlink d0/da/d1e/d1a/d24/l6e 0 2024-04-12T05:56:16.270 INFO:tasks.workunit.client.1.smithi194.stdout:4/926: creat d2/d11/d1e/d2f/d4a/d9c/de7/f142 x:0 0 0 2024-04-12T05:56:16.274 INFO:tasks.workunit.client.1.smithi194.stdout:0/648: dwrite d0/da/d1e/d27/fa0 [0,4194304] 0 2024-04-12T05:56:16.275 INFO:tasks.workunit.client.1.smithi194.stdout:4/927: read d2/d3/d1b/d32/d3a/d3b/f9f [2989528,22024] 0 2024-04-12T05:56:16.281 INFO:tasks.workunit.client.1.smithi194.stdout:4/928: creat d2/d11/d43/f143 x:0 0 0 2024-04-12T05:56:16.282 INFO:tasks.workunit.client.1.smithi194.stdout:0/649: dwrite d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:16.284 INFO:tasks.workunit.client.1.smithi194.stdout:0/650: dread - d0/d5f/fc0 zero size 2024-04-12T05:56:16.287 INFO:tasks.workunit.client.1.smithi194.stdout:0/651: symlink d0/d3b/d52/d7c/lcf 0 2024-04-12T05:56:16.314 INFO:tasks.workunit.client.1.smithi194.stdout:0/652: dwrite d0/f30 [0,4194304] 0 2024-04-12T05:56:16.317 INFO:tasks.workunit.client.1.smithi194.stdout:0/653: mknod d0/da/d1e/d1a/cd0 0 2024-04-12T05:56:16.322 INFO:tasks.workunit.client.1.smithi194.stdout:0/654: rename d0/d5f/lc9 to d0/da/d4a/d8e/ld1 0 2024-04-12T05:56:16.325 INFO:tasks.workunit.client.1.smithi194.stdout:5/735: sync 2024-04-12T05:56:16.326 INFO:tasks.workunit.client.1.smithi194.stdout:5/736: readlink d1/d5c/d95/lf4 0 2024-04-12T05:56:16.327 INFO:tasks.workunit.client.1.smithi194.stdout:5/737: dread - d1/d5c/d95/fc3 zero size 2024-04-12T05:56:16.330 INFO:tasks.workunit.client.1.smithi194.stdout:5/738: creat d1/d4/d75/ff6 x:0 0 0 2024-04-12T05:56:16.334 INFO:tasks.workunit.client.1.smithi194.stdout:5/739: mkdir d1/d4/d6/db/d15/d26/dbb/df7 0 2024-04-12T05:56:16.338 INFO:tasks.workunit.client.1.smithi194.stdout:0/655: unlink d0/da/d4a/d8e/ld1 0 2024-04-12T05:56:16.339 INFO:tasks.workunit.client.1.smithi194.stdout:5/740: symlink d1/d4/d84/lf8 0 2024-04-12T05:56:16.346 INFO:tasks.workunit.client.1.smithi194.stdout:0/656: dwrite d0/da/d4a/fca [0,4194304] 0 2024-04-12T05:56:16.353 INFO:tasks.workunit.client.1.smithi194.stdout:0/657: readlink d0/da/d1e/d1a/d24/d56/lc2 0 2024-04-12T05:56:16.356 INFO:tasks.workunit.client.1.smithi194.stdout:0/658: rmdir d0/da/d1e/d27/d54 39 2024-04-12T05:56:16.358 INFO:tasks.workunit.client.1.smithi194.stdout:5/741: dwrite d1/d4/dd7/fe2 [0,4194304] 0 2024-04-12T05:56:16.364 INFO:tasks.workunit.client.1.smithi194.stdout:0/659: rename d0/da/f91 to d0/da/d1e/d27/d87/fd2 0 2024-04-12T05:56:16.380 INFO:tasks.workunit.client.1.smithi194.stdout:5/742: dwrite d1/d4/d6/f68 [0,4194304] 0 2024-04-12T05:56:16.382 INFO:tasks.workunit.client.1.smithi194.stdout:0/660: dwrite d0/da/d1e/d1a/f1c [0,4194304] 0 2024-04-12T05:56:16.385 INFO:tasks.workunit.client.1.smithi194.stdout:5/743: symlink d1/d4/d6/db/lf9 0 2024-04-12T05:56:16.390 INFO:tasks.workunit.client.1.smithi194.stdout:5/744: getdents d1/d4/d4b 0 2024-04-12T05:56:16.390 INFO:tasks.workunit.client.1.smithi194.stdout:5/745: stat d1/d4/d6/db/la4 0 2024-04-12T05:56:16.393 INFO:tasks.workunit.client.1.smithi194.stdout:5/746: mkdir d1/d5c/d95/dfa 0 2024-04-12T05:56:16.394 INFO:tasks.workunit.client.1.smithi194.stdout:0/661: truncate d0/d3b/fc6 1015235 0 2024-04-12T05:56:16.395 INFO:tasks.workunit.client.1.smithi194.stdout:0/662: write d0/da/d4a/d8e/fb0 [303582,130129] 0 2024-04-12T05:56:16.402 INFO:tasks.workunit.client.1.smithi194.stdout:0/663: dread d0/f1d [0,4194304] 0 2024-04-12T05:56:16.402 INFO:tasks.workunit.client.1.smithi194.stdout:5/747: truncate d1/d4/d6/db/f19 2432968 0 2024-04-12T05:56:16.405 INFO:tasks.workunit.client.1.smithi194.stdout:5/748: creat d1/d4/d6/d6d/ffb x:0 0 0 2024-04-12T05:56:16.407 INFO:tasks.workunit.client.1.smithi194.stdout:0/664: rename d0/da/d1e/d1a/fa5 to d0/da/d1e/d8f/fd3 0 2024-04-12T05:56:16.409 INFO:tasks.workunit.client.1.smithi194.stdout:5/749: link d1/ld5 d1/d4/dad/lfc 0 2024-04-12T05:56:16.410 INFO:tasks.workunit.client.1.smithi194.stdout:0/665: mknod d0/d3b/d52/d7c/dbe/cd4 0 2024-04-12T05:56:16.410 INFO:tasks.workunit.client.1.smithi194.stdout:5/750: write d1/d5c/d95/f9f [5103926,73819] 0 2024-04-12T05:56:16.412 INFO:tasks.workunit.client.1.smithi194.stdout:5/751: creat d1/d4/d6/d6d/ffd x:0 0 0 2024-04-12T05:56:16.413 INFO:tasks.workunit.client.1.smithi194.stdout:5/752: readlink d1/d4/d6/d34/d52/ldc 0 2024-04-12T05:56:16.423 INFO:tasks.workunit.client.1.smithi194.stdout:5/753: write d1/d4/d6/db/d15/d26/f4e [50035,38257] 0 2024-04-12T05:56:16.425 INFO:tasks.workunit.client.1.smithi194.stdout:0/666: dread d0/da/d1e/d27/d87/fd2 [0,4194304] 0 2024-04-12T05:56:16.426 INFO:tasks.workunit.client.1.smithi194.stdout:5/754: mkdir d1/d4/dd7/dfe 0 2024-04-12T05:56:16.427 INFO:tasks.workunit.client.1.smithi194.stdout:0/667: fdatasync d0/da/da7/fba 0 2024-04-12T05:56:16.427 INFO:tasks.workunit.client.1.smithi194.stdout:5/755: chown d1/d4/d4b/fba 27 1 2024-04-12T05:56:16.428 INFO:tasks.workunit.client.1.smithi194.stdout:0/668: mknod d0/da/d1e/d1a/d24/d56/cd5 0 2024-04-12T05:56:16.433 INFO:tasks.workunit.client.1.smithi194.stdout:0/669: getdents d0/da/d1e 0 2024-04-12T05:56:16.435 INFO:tasks.workunit.client.1.smithi194.stdout:5/756: truncate d1/d4/d6/d34/d52/fcf 3569292 0 2024-04-12T05:56:16.437 INFO:tasks.workunit.client.1.smithi194.stdout:5/757: rmdir d1/d4/d6/d88 39 2024-04-12T05:56:16.440 INFO:tasks.workunit.client.1.smithi194.stdout:5/758: mkdir d1/d4/d6/db/d38/dff 0 2024-04-12T05:56:16.442 INFO:tasks.workunit.client.1.smithi194.stdout:5/759: mknod d1/d4/d6/c100 0 2024-04-12T05:56:16.443 INFO:tasks.workunit.client.1.smithi194.stdout:5/760: fsync d1/d4/d75/ff6 0 2024-04-12T05:56:16.443 INFO:tasks.workunit.client.1.smithi194.stdout:5/761: fdatasync d1/d4/d6/db/d38/fc0 0 2024-04-12T05:56:16.447 INFO:tasks.workunit.client.1.smithi194.stdout:5/762: link d1/d4/d6/c100 d1/d4/c101 0 2024-04-12T05:56:16.447 INFO:tasks.workunit.client.1.smithi194.stdout:5/763: stat d1/d4/d6/d34/d52/f77 0 2024-04-12T05:56:16.500 INFO:tasks.workunit.client.1.smithi194.stdout:6/653: sync 2024-04-12T05:56:16.516 INFO:tasks.workunit.client.1.smithi194.stdout:5/764: symlink d1/d4/d9e/ddb/l102 0 2024-04-12T05:56:16.516 INFO:tasks.workunit.client.1.smithi194.stdout:6/654: creat d3/d15/d2b/d48/d6f/d7a/fcf x:0 0 0 2024-04-12T05:56:16.517 INFO:tasks.workunit.client.1.smithi194.stdout:5/765: stat d1/d4/d6/d88/la5 0 2024-04-12T05:56:16.518 INFO:tasks.workunit.client.1.smithi194.stdout:6/655: chown d3/d2d/d33/d52/d5c/fa5 24969 1 2024-04-12T05:56:16.519 INFO:tasks.workunit.client.1.smithi194.stdout:6/656: write d3/d70/fab [1194790,54862] 0 2024-04-12T05:56:16.519 INFO:tasks.workunit.client.1.smithi194.stdout:5/766: write d1/d4/d6/d8e/f81 [4626739,130334] 0 2024-04-12T05:56:16.520 INFO:tasks.workunit.client.1.smithi194.stdout:6/657: readlink d3/d2f/d4e/d68/l73 0 2024-04-12T05:56:16.528 INFO:tasks.workunit.client.1.smithi194.stdout:5/767: dwrite d1/d4/d6/d88/f56 [0,4194304] 0 2024-04-12T05:56:16.532 INFO:tasks.workunit.client.1.smithi194.stdout:5/768: unlink d1/d4/d6/d34/fa0 0 2024-04-12T05:56:16.533 INFO:tasks.workunit.client.1.smithi194.stdout:5/769: symlink d1/d5c/d95/l103 0 2024-04-12T05:56:16.534 INFO:tasks.workunit.client.1.smithi194.stdout:5/770: chown d1/d5c/d95/df2/fa6 529031622 1 2024-04-12T05:56:16.535 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:16 smithi008 ceph-mon[39722]: pgmap v34: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 186 MiB/s wr, 562 op/s 2024-04-12T05:56:16.536 INFO:tasks.workunit.client.1.smithi194.stdout:5/771: chown d1/d4/d6/db/d15/d26/dbb/le4 3 1 2024-04-12T05:56:16.543 INFO:tasks.workunit.client.1.smithi194.stdout:5/772: dwrite d1/d5c/da7/fb1 [0,4194304] 0 2024-04-12T05:56:16.648 INFO:tasks.workunit.client.1.smithi194.stdout:7/621: sync 2024-04-12T05:56:16.648 INFO:tasks.workunit.client.1.smithi194.stdout:1/795: sync 2024-04-12T05:56:16.649 INFO:tasks.workunit.client.1.smithi194.stdout:2/718: sync 2024-04-12T05:56:16.649 INFO:tasks.workunit.client.1.smithi194.stdout:9/786: sync 2024-04-12T05:56:16.651 INFO:tasks.workunit.client.1.smithi194.stdout:7/622: chown d3/d16/d23/d57/d72/d81/d3a/d43/l80 99991 1 2024-04-12T05:56:16.651 INFO:tasks.workunit.client.1.smithi194.stdout:9/787: truncate d1/d19/d1f/d5f/f9e 5173298 0 2024-04-12T05:56:16.651 INFO:tasks.workunit.client.1.smithi194.stdout:1/796: creat d0/d11/ddd/fff x:0 0 0 2024-04-12T05:56:16.654 INFO:tasks.workunit.client.1.smithi194.stdout:7/623: chown d3/d16/d23/d57/d72/d81/d3a/d8f/cc2 4 1 2024-04-12T05:56:16.657 INFO:tasks.workunit.client.1.smithi194.stdout:2/719: dwrite d7/d12/d25/d5d/d47/d52/fa0 [0,4194304] 0 2024-04-12T05:56:16.659 INFO:tasks.workunit.client.1.smithi194.stdout:1/797: fdatasync d0/d11/d18/d19/d1a/d6a/d27/da8/fa9 0 2024-04-12T05:56:16.660 INFO:tasks.workunit.client.1.smithi194.stdout:7/624: unlink d3/d27/f97 0 2024-04-12T05:56:16.661 INFO:tasks.workunit.client.1.smithi194.stdout:1/798: truncate d0/d11/f55 4521504 0 2024-04-12T05:56:16.662 INFO:tasks.workunit.client.1.smithi194.stdout:7/625: readlink d3/d16/d23/d37/d95/lb2 0 2024-04-12T05:56:16.665 INFO:tasks.workunit.client.1.smithi194.stdout:1/799: symlink d0/d11/d18/d19/d1c/d67/l100 0 2024-04-12T05:56:16.666 INFO:tasks.workunit.client.1.smithi194.stdout:1/800: readlink d0/d11/d18/d19/d1a/d6a/d27/l94 0 2024-04-12T05:56:16.668 INFO:tasks.workunit.client.1.smithi194.stdout:7/626: getdents d3/d27/d22 0 2024-04-12T05:56:16.669 INFO:tasks.workunit.client.1.smithi194.stdout:7/627: fdatasync d3/d16/f1a 0 2024-04-12T05:56:16.671 INFO:tasks.workunit.client.1.smithi194.stdout:9/788: write d1/f69 [5180443,74507] 0 2024-04-12T05:56:16.672 INFO:tasks.workunit.client.1.smithi194.stdout:7/628: mkdir d3/d27/d22/d31/dc7 0 2024-04-12T05:56:16.678 INFO:tasks.workunit.client.1.smithi194.stdout:9/789: creat d1/d19/d1f/d20/d37/d60/f106 x:0 0 0 2024-04-12T05:56:16.678 INFO:tasks.workunit.client.1.smithi194.stdout:1/801: dwrite d0/d11/d18/d19/d1a/d6a/d27/fab [0,4194304] 0 2024-04-12T05:56:16.678 INFO:tasks.workunit.client.1.smithi194.stdout:7/629: dread d3/d16/d23/d37/f96 [0,4194304] 0 2024-04-12T05:56:16.683 INFO:tasks.workunit.client.1.smithi194.stdout:9/790: creat d1/d19/d1f/d20/d37/d60/f107 x:0 0 0 2024-04-12T05:56:16.684 INFO:tasks.workunit.client.1.smithi194.stdout:9/791: write d1/d19/d1f/d20/d37/ff3 [380100,10861] 0 2024-04-12T05:56:16.687 INFO:tasks.workunit.client.1.smithi194.stdout:7/630: creat d3/d16/d23/fc8 x:0 0 0 2024-04-12T05:56:16.689 INFO:tasks.workunit.client.1.smithi194.stdout:2/720: dread d7/d12/d25/d5d/d47/dac/db3/dd7/fb6 [0,4194304] 0 2024-04-12T05:56:16.693 INFO:tasks.workunit.client.1.smithi194.stdout:1/802: dwrite d0/d11/f7c [0,4194304] 0 2024-04-12T05:56:16.695 INFO:tasks.workunit.client.1.smithi194.stdout:7/631: creat d3/d16/d23/d37/d3d/d56/da4/fc9 x:0 0 0 2024-04-12T05:56:16.696 INFO:tasks.workunit.client.1.smithi194.stdout:9/792: write d1/d19/d1f/d20/d7d/f99 [816775,36032] 0 2024-04-12T05:56:16.700 INFO:tasks.workunit.client.1.smithi194.stdout:7/632: mknod d3/d16/d23/d57/d72/d81/d3a/cca 0 2024-04-12T05:56:16.703 INFO:tasks.workunit.client.1.smithi194.stdout:9/793: fsync d1/d19/d1f/d20/f35 0 2024-04-12T05:56:16.708 INFO:tasks.workunit.client.1.smithi194.stdout:2/721: dread d7/d12/d25/f64 [0,4194304] 0 2024-04-12T05:56:16.709 INFO:tasks.workunit.client.1.smithi194.stdout:1/803: rename d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/lbd to d0/d11/d18/d19/d1a/d6a/l101 0 2024-04-12T05:56:16.709 INFO:tasks.workunit.client.1.smithi194.stdout:7/633: dread d3/d27/d22/f2e [0,4194304] 0 2024-04-12T05:56:16.712 INFO:tasks.workunit.client.1.smithi194.stdout:1/804: read d0/d11/d18/d35/da5/fbe [1611470,71524] 0 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:3/838: sync 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:8/504: sync 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:5/773: sync 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:0/670: sync 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:6/658: sync 2024-04-12T05:56:16.715 INFO:tasks.workunit.client.1.smithi194.stdout:4/929: sync 2024-04-12T05:56:16.718 INFO:tasks.workunit.client.1.smithi194.stdout:8/505: write d2/f56 [1143424,67800] 0 2024-04-12T05:56:16.719 INFO:tasks.workunit.client.1.smithi194.stdout:2/722: dwrite d7/d12/d25/d5d/d47/f8f [0,4194304] 0 2024-04-12T05:56:16.720 INFO:tasks.workunit.client.1.smithi194.stdout:5/774: mknod d1/d4/dd7/c104 0 2024-04-12T05:56:16.723 INFO:tasks.workunit.client.1.smithi194.stdout:3/839: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/c4e 173106379 1 2024-04-12T05:56:16.725 INFO:tasks.workunit.client.1.smithi194.stdout:7/634: truncate d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f91 4217 0 2024-04-12T05:56:16.726 INFO:tasks.workunit.client.1.smithi194.stdout:8/506: creat d2/d8/d9/de/f9a x:0 0 0 2024-04-12T05:56:16.731 INFO:tasks.workunit.client.1.smithi194.stdout:5/775: dread d1/d4/d6/fc8 [0,4194304] 0 2024-04-12T05:56:16.733 INFO:tasks.workunit.client.1.smithi194.stdout:4/930: rename d2/d11/d1e/d2f/d4a/d9c/dee to d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/d144 0 2024-04-12T05:56:16.737 INFO:tasks.workunit.client.1.smithi194.stdout:1/805: dread d0/d11/f26 [0,4194304] 0 2024-04-12T05:56:16.737 INFO:tasks.workunit.client.1.smithi194.stdout:3/840: rmdir d0/d1/d74/d39/d4d/deb 39 2024-04-12T05:56:16.737 INFO:tasks.workunit.client.1.smithi194.stdout:4/931: chown d2/d11/d46/d61/d63/da2 1916071 1 2024-04-12T05:56:16.739 INFO:tasks.workunit.client.1.smithi194.stdout:7/635: write d3/d16/d5d/f84 [3352479,28866] 0 2024-04-12T05:56:16.740 INFO:tasks.workunit.client.1.smithi194.stdout:0/671: link d0/da/d1e/d27/d54/fc3 d0/da/d1e/d27/fd6 0 2024-04-12T05:56:16.741 INFO:tasks.workunit.client.1.smithi194.stdout:8/507: symlink d2/d8/d9/l9b 0 2024-04-12T05:56:16.742 INFO:tasks.workunit.client.1.smithi194.stdout:1/806: readlink d0/d11/d18/d19/d1a/l51 0 2024-04-12T05:56:16.747 INFO:tasks.workunit.client.1.smithi194.stdout:2/723: dwrite f5 [0,4194304] 0 2024-04-12T05:56:16.748 INFO:tasks.workunit.client.1.smithi194.stdout:8/508: dread d2/d8/d9/de/f61 [0,4194304] 0 2024-04-12T05:56:16.752 INFO:tasks.workunit.client.1.smithi194.stdout:3/841: dread d0/d42/f51 [0,4194304] 0 2024-04-12T05:56:16.754 INFO:tasks.workunit.client.1.smithi194.stdout:6/659: truncate d3/d2d/d33/d52/d5c/d8c/f93 1955276 0 2024-04-12T05:56:16.757 INFO:tasks.workunit.client.1.smithi194.stdout:0/672: fsync d0/da/f13 0 2024-04-12T05:56:16.757 INFO:tasks.workunit.client.1.smithi194.stdout:4/932: dwrite d2/d11/d1e/d25/fd7 [0,4194304] 0 2024-04-12T05:56:16.758 INFO:tasks.workunit.client.1.smithi194.stdout:5/776: truncate d1/d4/d6/d34/f54 1823886 0 2024-04-12T05:56:16.760 INFO:tasks.workunit.client.1.smithi194.stdout:7/636: link d3/d16/c9d d3/d16/d23/d57/ccb 0 2024-04-12T05:56:16.760 INFO:tasks.workunit.client.1.smithi194.stdout:2/724: symlink d7/d12/d25/d5d/d47/lfb 0 2024-04-12T05:56:16.762 INFO:tasks.workunit.client.1.smithi194.stdout:6/660: mknod d3/d2f/d4e/d63/d83/cd0 0 2024-04-12T05:56:16.763 INFO:tasks.workunit.client.1.smithi194.stdout:3/842: truncate d0/d1/d74/d39/fd1 627366 0 2024-04-12T05:56:16.765 INFO:tasks.workunit.client.1.smithi194.stdout:8/509: creat d2/d8/d66/d7f/f9c x:0 0 0 2024-04-12T05:56:16.765 INFO:tasks.workunit.client.1.smithi194.stdout:4/933: mkdir d2/d11/d1e/d2f/d4a/d9c/d145 0 2024-04-12T05:56:16.766 INFO:tasks.workunit.client.1.smithi194.stdout:0/673: rename d0/d5f/d73/d76/c78 to d0/d5f/cd7 0 2024-04-12T05:56:16.769 INFO:tasks.workunit.client.1.smithi194.stdout:7/637: read d3/d16/d23/d37/d3d/f61 [1869153,42867] 0 2024-04-12T05:56:16.769 INFO:tasks.workunit.client.1.smithi194.stdout:3/843: readlink d0/l3 0 2024-04-12T05:56:16.772 INFO:tasks.workunit.client.1.smithi194.stdout:7/638: read - d3/d16/d23/d57/d72/d81/f92 zero size 2024-04-12T05:56:16.772 INFO:tasks.workunit.client.1.smithi194.stdout:4/934: unlink d2/d3/d58/l60 0 2024-04-12T05:56:16.773 INFO:tasks.workunit.client.1.smithi194.stdout:0/674: readlink d0/da/l83 0 2024-04-12T05:56:16.774 INFO:tasks.workunit.client.1.smithi194.stdout:6/661: dread d3/d2f/f9e [0,4194304] 0 2024-04-12T05:56:16.779 INFO:tasks.workunit.client.1.smithi194.stdout:5/777: read d1/d4/d6/f76 [939809,72269] 0 2024-04-12T05:56:16.780 INFO:tasks.workunit.client.1.smithi194.stdout:4/935: mkdir d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/d146 0 2024-04-12T05:56:16.781 INFO:tasks.workunit.client.1.smithi194.stdout:0/675: symlink d0/d3b/d52/d7c/dbe/ld8 0 2024-04-12T05:56:16.782 INFO:tasks.workunit.client.1.smithi194.stdout:8/510: symlink d2/d8/d9/de/d38/d8f/d94/l9d 0 2024-04-12T05:56:16.783 INFO:tasks.workunit.client.1.smithi194.stdout:0/676: fsync d0/da/d4a/d8e/f72 0 2024-04-12T05:56:16.784 INFO:tasks.workunit.client.1.smithi194.stdout:4/936: fsync d2/f67 0 2024-04-12T05:56:16.784 INFO:tasks.workunit.client.1.smithi194.stdout:6/662: symlink d3/d2d/d33/d52/d62/ld1 0 2024-04-12T05:56:16.786 INFO:tasks.workunit.client.1.smithi194.stdout:5/778: symlink d1/d4/dd7/l105 0 2024-04-12T05:56:16.789 INFO:tasks.workunit.client.1.smithi194.stdout:8/511: creat d2/d8/d15/f9e x:0 0 0 2024-04-12T05:56:16.791 INFO:tasks.workunit.client.1.smithi194.stdout:0/677: unlink d0/da/d1e/d1a/d58/d5b/fa9 0 2024-04-12T05:56:16.793 INFO:tasks.workunit.client.1.smithi194.stdout:5/779: chown d1/d4/d6/db/d15/d8c/ld6 688709 1 2024-04-12T05:56:16.797 INFO:tasks.workunit.client.1.smithi194.stdout:0/678: stat d0/da/d1e/d1a/f31 0 2024-04-12T05:56:16.799 INFO:tasks.workunit.client.1.smithi194.stdout:7/639: dwrite d3/d27/f64 [0,4194304] 0 2024-04-12T05:56:16.799 INFO:tasks.workunit.client.1.smithi194.stdout:3/844: dread d0/f54 [0,4194304] 0 2024-04-12T05:56:16.800 INFO:tasks.workunit.client.1.smithi194.stdout:5/780: unlink d1/d4/d9e/faf 0 2024-04-12T05:56:16.804 INFO:tasks.workunit.client.1.smithi194.stdout:6/663: dwrite d3/f18 [4194304,4194304] 0 2024-04-12T05:56:16.806 INFO:tasks.workunit.client.1.smithi194.stdout:4/937: dwrite d2/d11/d46/d61/d63/da2/fa5 [0,4194304] 0 2024-04-12T05:56:16.808 INFO:tasks.workunit.client.1.smithi194.stdout:0/679: read d0/d3b/f74 [1090666,88451] 0 2024-04-12T05:56:16.814 INFO:tasks.workunit.client.1.smithi194.stdout:0/680: stat d0/da/d1e/d1a/l42 0 2024-04-12T05:56:16.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:16 smithi194 ceph-mon[49771]: pgmap v34: 65 pgs: 65 active+clean; 1.6 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 186 MiB/s wr, 562 op/s 2024-04-12T05:56:16.820 INFO:tasks.workunit.client.1.smithi194.stdout:7/640: rmdir d3/d16/d23/d57/d8a 0 2024-04-12T05:56:16.820 INFO:tasks.workunit.client.1.smithi194.stdout:5/781: dwrite d1/fd [0,4194304] 0 2024-04-12T05:56:16.822 INFO:tasks.workunit.client.1.smithi194.stdout:8/512: dread d2/f56 [0,4194304] 0 2024-04-12T05:56:16.823 INFO:tasks.workunit.client.1.smithi194.stdout:3/845: mknod d0/d1/d74/d39/d4d/deb/d3d/c11b 0 2024-04-12T05:56:16.823 INFO:tasks.workunit.client.1.smithi194.stdout:6/664: mknod d3/d2d/d33/d52/d5c/cd2 0 2024-04-12T05:56:16.824 INFO:tasks.workunit.client.1.smithi194.stdout:7/641: rmdir d3/d16/d23/d37 39 2024-04-12T05:56:16.825 INFO:tasks.workunit.client.1.smithi194.stdout:8/513: chown d2/f12 50417 1 2024-04-12T05:56:16.825 INFO:tasks.workunit.client.1.smithi194.stdout:6/665: chown d3/d2d/d33/fa6 3 1 2024-04-12T05:56:16.827 INFO:tasks.workunit.client.1.smithi194.stdout:4/938: rename d2/d3/d1b/d32/lc0 to d2/d11/d93/da6/dab/d5d/l147 0 2024-04-12T05:56:16.831 INFO:tasks.workunit.client.1.smithi194.stdout:5/782: truncate d1/d5c/d95/df2/f3e 86289 0 2024-04-12T05:56:16.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/514: chown d2/d8/d15/d33/d49/l6d 3750838 1 2024-04-12T05:56:16.837 INFO:tasks.workunit.client.1.smithi194.stdout:4/939: creat d2/d3/d51/d13e/f148 x:0 0 0 2024-04-12T05:56:16.837 INFO:tasks.workunit.client.1.smithi194.stdout:7/642: dwrite d3/d27/d22/f38 [0,4194304] 0 2024-04-12T05:56:16.838 INFO:tasks.workunit.client.1.smithi194.stdout:7/643: stat d3/d27/f1f 0 2024-04-12T05:56:16.840 INFO:tasks.workunit.client.1.smithi194.stdout:3/846: rename d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/dc5/l112 to d0/d1/d74/d39/d4d/deb/d3d/d6e/de9/l11c 0 2024-04-12T05:56:16.846 INFO:tasks.workunit.client.1.smithi194.stdout:6/666: write d3/d2d/d33/f4d [1217466,68404] 0 2024-04-12T05:56:16.848 INFO:tasks.workunit.client.1.smithi194.stdout:8/515: mkdir d2/d8/d9/de/d10/d7d/d84/d9f 0 2024-04-12T05:56:16.848 INFO:tasks.workunit.client.1.smithi194.stdout:6/667: chown d3/d34 44 1 2024-04-12T05:56:16.849 INFO:tasks.workunit.client.1.smithi194.stdout:0/681: dread d0/da/d1e/d1a/f6d [0,4194304] 0 2024-04-12T05:56:16.852 INFO:tasks.workunit.client.1.smithi194.stdout:3/847: mknod d0/d1/d74/d39/d4d/deb/d53/c11d 0 2024-04-12T05:56:16.852 INFO:tasks.workunit.client.1.smithi194.stdout:7/644: mkdir d3/d16/d23/d57/d72/d81/d3a/dcc 0 2024-04-12T05:56:16.852 INFO:tasks.workunit.client.1.smithi194.stdout:5/783: link d1/d4/d6/d34/c55 d1/d4/d4b/c106 0 2024-04-12T05:56:16.854 INFO:tasks.workunit.client.1.smithi194.stdout:8/516: unlink d2/d8/d9/de/d10/c19 0 2024-04-12T05:56:16.855 INFO:tasks.workunit.client.1.smithi194.stdout:6/668: fdatasync d3/d2f/d4e/d68/fa2 0 2024-04-12T05:56:16.856 INFO:tasks.workunit.client.1.smithi194.stdout:8/517: write d2/d8/d15/f62 [2529654,60242] 0 2024-04-12T05:56:16.856 INFO:tasks.workunit.client.1.smithi194.stdout:9/794: sync 2024-04-12T05:56:16.858 INFO:tasks.workunit.client.1.smithi194.stdout:3/848: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f46 3911296 1 2024-04-12T05:56:16.860 INFO:tasks.workunit.client.1.smithi194.stdout:8/518: chown d2/d8/d9/de/d10/d7d/f81 15801994 1 2024-04-12T05:56:16.860 INFO:tasks.workunit.client.1.smithi194.stdout:5/784: unlink d1/d4/d6/db/d2b/fd2 0 2024-04-12T05:56:16.861 INFO:tasks.workunit.client.1.smithi194.stdout:8/519: chown d2/d8/d9/f48 24333216 1 2024-04-12T05:56:16.862 INFO:tasks.workunit.client.1.smithi194.stdout:4/940: dread d2/d3/f74 [0,4194304] 0 2024-04-12T05:56:16.863 INFO:tasks.workunit.client.1.smithi194.stdout:8/520: stat d2/d8/d9/de/d38/f3a 0 2024-04-12T05:56:16.870 INFO:tasks.workunit.client.1.smithi194.stdout:3/849: readlink d0/d1/d10/l1b 0 2024-04-12T05:56:16.870 INFO:tasks.workunit.client.1.smithi194.stdout:6/669: dwrite d3/d15/d2b/d48/d6f/d7a/d8f/fbd [0,4194304] 0 2024-04-12T05:56:16.873 INFO:tasks.workunit.client.1.smithi194.stdout:4/941: rename d2/c8b to d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/c149 0 2024-04-12T05:56:16.874 INFO:tasks.workunit.client.1.smithi194.stdout:7/645: creat d3/fcd x:0 0 0 2024-04-12T05:56:16.874 INFO:tasks.workunit.client.1.smithi194.stdout:9/795: write d1/f18 [1015330,109265] 0 2024-04-12T05:56:16.876 INFO:tasks.workunit.client.1.smithi194.stdout:6/670: truncate d3/d2d/d33/d58/fce 1003212 0 2024-04-12T05:56:16.877 INFO:tasks.workunit.client.1.smithi194.stdout:0/682: getdents d0 0 2024-04-12T05:56:16.879 INFO:tasks.workunit.client.1.smithi194.stdout:3/850: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f10b 281106099 1 2024-04-12T05:56:16.880 INFO:tasks.workunit.client.1.smithi194.stdout:7/646: symlink d3/d27/d22/d31/lce 0 2024-04-12T05:56:16.881 INFO:tasks.workunit.client.1.smithi194.stdout:4/942: stat d2/d85/l10d 0 2024-04-12T05:56:16.881 INFO:tasks.workunit.client.1.smithi194.stdout:5/785: write d1/d4/d6/d88/dc1/fe9 [1731575,26923] 0 2024-04-12T05:56:16.882 INFO:tasks.workunit.client.1.smithi194.stdout:9/796: creat d1/d21/d49/f108 x:0 0 0 2024-04-12T05:56:16.885 INFO:tasks.workunit.client.1.smithi194.stdout:6/671: creat d3/d2f/d4e/d68/dad/dc4/fd3 x:0 0 0 2024-04-12T05:56:16.886 INFO:tasks.workunit.client.1.smithi194.stdout:9/797: creat d1/d41/d97/f109 x:0 0 0 2024-04-12T05:56:16.886 INFO:tasks.workunit.client.1.smithi194.stdout:0/683: mknod d0/da/d1e/d1a/d58/d5b/cd9 0 2024-04-12T05:56:16.888 INFO:tasks.workunit.client.1.smithi194.stdout:8/521: dwrite d2/d8/d9/de/d38/d8f/f90 [0,4194304] 0 2024-04-12T05:56:16.890 INFO:tasks.workunit.client.1.smithi194.stdout:5/786: mkdir d1/d5c/d95/df2/d107 0 2024-04-12T05:56:16.891 INFO:tasks.workunit.client.1.smithi194.stdout:3/851: stat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/da7/de6 0 2024-04-12T05:56:16.891 INFO:tasks.workunit.client.1.smithi194.stdout:7/647: chown d3/d16/d23/d37/d3d/d56/cb5 10 1 2024-04-12T05:56:16.893 INFO:tasks.workunit.client.1.smithi194.stdout:6/672: creat d3/fd4 x:0 0 0 2024-04-12T05:56:16.894 INFO:tasks.workunit.client.1.smithi194.stdout:4/943: creat d2/d11/d93/da6/dab/d5d/dae/de1/ddc/f14a x:0 0 0 2024-04-12T05:56:16.896 INFO:tasks.workunit.client.1.smithi194.stdout:5/787: fsync d1/d4/d4b/fba 0 2024-04-12T05:56:16.897 INFO:tasks.workunit.client.1.smithi194.stdout:9/798: symlink d1/d21/d77/d98/l10a 0 2024-04-12T05:56:16.898 INFO:tasks.workunit.client.1.smithi194.stdout:7/648: getdents d3/d16/d7b/dc3 0 2024-04-12T05:56:16.899 INFO:tasks.workunit.client.1.smithi194.stdout:4/944: unlink d2/d3/d58/d82/lac 0 2024-04-12T05:56:16.901 INFO:tasks.workunit.client.1.smithi194.stdout:0/684: dwrite d0/da/d1e/d1a/d58/f80 [4194304,4194304] 0 2024-04-12T05:56:16.902 INFO:tasks.workunit.client.1.smithi194.stdout:5/788: write d1/d5c/d95/df2/fec [515193,98768] 0 2024-04-12T05:56:16.903 INFO:tasks.workunit.client.1.smithi194.stdout:6/673: symlink d3/d2d/d33/db1/ld5 0 2024-04-12T05:56:16.906 INFO:tasks.workunit.client.1.smithi194.stdout:7/649: creat d3/d27/d22/d62/fcf x:0 0 0 2024-04-12T05:56:16.906 INFO:tasks.workunit.client.1.smithi194.stdout:7/650: dread - d3/d27/d6a/faa zero size 2024-04-12T05:56:16.907 INFO:tasks.workunit.client.1.smithi194.stdout:5/789: creat d1/d4/d4b/f108 x:0 0 0 2024-04-12T05:56:16.908 INFO:tasks.workunit.client.1.smithi194.stdout:0/685: creat d0/da/d1e/d27/d87/d92/fda x:0 0 0 2024-04-12T05:56:16.910 INFO:tasks.workunit.client.1.smithi194.stdout:9/799: write d1/f1d [3508650,25601] 0 2024-04-12T05:56:16.910 INFO:tasks.workunit.client.1.smithi194.stdout:6/674: truncate d3/d2f/d4e/d68/dad/dc4/fd3 1019791 0 2024-04-12T05:56:16.911 INFO:tasks.workunit.client.1.smithi194.stdout:0/686: chown d0/d3b/f74 1256046 1 2024-04-12T05:56:16.913 INFO:tasks.workunit.client.1.smithi194.stdout:0/687: chown d0/da/d1e/f41 209551 1 2024-04-12T05:56:16.917 INFO:tasks.workunit.client.1.smithi194.stdout:6/675: mknod d3/d2f/d4e/d56/cd6 0 2024-04-12T05:56:16.919 INFO:tasks.workunit.client.1.smithi194.stdout:0/688: creat d0/da/d1e/fdb x:0 0 0 2024-04-12T05:56:16.921 INFO:tasks.workunit.client.1.smithi194.stdout:3/852: dread d0/fb [4194304,4194304] 0 2024-04-12T05:56:16.922 INFO:tasks.workunit.client.1.smithi194.stdout:6/676: unlink d3/d15/f2e 0 2024-04-12T05:56:16.923 INFO:tasks.workunit.client.1.smithi194.stdout:3/853: chown d0/d1/d74/d39/d4d/c98 14116401 1 2024-04-12T05:56:16.925 INFO:tasks.workunit.client.1.smithi194.stdout:3/854: chown d0/d1/d4f/dd7/d114 242905 1 2024-04-12T05:56:16.926 INFO:tasks.workunit.client.1.smithi194.stdout:6/677: mknod d3/d2f/d4e/d56/cd7 0 2024-04-12T05:56:16.926 INFO:tasks.workunit.client.1.smithi194.stdout:7/651: truncate d3/d16/d23/d57/d72/d81/f73 1650600 0 2024-04-12T05:56:16.934 INFO:tasks.workunit.client.1.smithi194.stdout:4/945: dread d2/d3/d51/fb1 [0,4194304] 0 2024-04-12T05:56:16.936 INFO:tasks.workunit.client.1.smithi194.stdout:0/689: dwrite d0/f94 [0,4194304] 0 2024-04-12T05:56:16.936 INFO:tasks.workunit.client.1.smithi194.stdout:3/855: write d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f37 [1639829,75288] 0 2024-04-12T05:56:16.938 INFO:tasks.workunit.client.1.smithi194.stdout:5/790: dread d1/d4/d6/db/d2b/f73 [0,4194304] 0 2024-04-12T05:56:16.941 INFO:tasks.workunit.client.1.smithi194.stdout:4/946: fsync d2/d11/d46/d61/d63/da2/fa5 0 2024-04-12T05:56:16.944 INFO:tasks.workunit.client.1.smithi194.stdout:0/690: creat d0/fdc x:0 0 0 2024-04-12T05:56:16.944 INFO:tasks.workunit.client.1.smithi194.stdout:9/800: dwrite d1/d19/d1f/d5f/f91 [0,4194304] 0 2024-04-12T05:56:16.944 INFO:tasks.workunit.client.1.smithi194.stdout:5/791: fdatasync d1/d5c/d95/fc6 0 2024-04-12T05:56:16.952 INFO:tasks.workunit.client.1.smithi194.stdout:9/801: unlink d1/d19/d1f/d5f/f9e 0 2024-04-12T05:56:16.956 INFO:tasks.workunit.client.1.smithi194.stdout:5/792: mkdir d1/d109 0 2024-04-12T05:56:16.959 INFO:tasks.workunit.client.1.smithi194.stdout:3/856: rename d0/f7f to d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/dc5/f11e 0 2024-04-12T05:56:16.959 INFO:tasks.workunit.client.1.smithi194.stdout:7/652: dwrite d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f76 [0,4194304] 0 2024-04-12T05:56:16.962 INFO:tasks.workunit.client.1.smithi194.stdout:4/947: dwrite d2/d11/d46/d61/d63/da2/fa5 [0,4194304] 0 2024-04-12T05:56:16.964 INFO:tasks.workunit.client.1.smithi194.stdout:9/802: truncate d1/d19/d1f/d20/d37/da3/fb3 5186558 0 2024-04-12T05:56:16.964 INFO:tasks.workunit.client.1.smithi194.stdout:7/653: write d3/d27/fbb [68546,45935] 0 2024-04-12T05:56:16.964 INFO:tasks.workunit.client.1.smithi194.stdout:5/793: creat d1/d4/d6/d34/d52/f10a x:0 0 0 2024-04-12T05:56:16.971 INFO:tasks.workunit.client.1.smithi194.stdout:9/803: fsync d1/d41/dba/dc2/fc3 0 2024-04-12T05:56:16.972 INFO:tasks.workunit.client.1.smithi194.stdout:3/857: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/fdd 0 1 2024-04-12T05:56:16.973 INFO:tasks.workunit.client.1.smithi194.stdout:5/794: mkdir d1/d5c/da7/d10b 0 2024-04-12T05:56:16.975 INFO:tasks.workunit.client.1.smithi194.stdout:9/804: chown d1/d19/d1f/d20/d37/d60/f106 133427743 1 2024-04-12T05:56:16.975 INFO:tasks.workunit.client.1.smithi194.stdout:6/678: dread d3/d15/d2b/d48/f54 [0,4194304] 0 2024-04-12T05:56:16.977 INFO:tasks.workunit.client.1.smithi194.stdout:0/691: dread d0/da/d1e/d1a/f31 [0,4194304] 0 2024-04-12T05:56:16.978 INFO:tasks.workunit.client.1.smithi194.stdout:5/795: mknod d1/d5c/da7/c10c 0 2024-04-12T05:56:16.978 INFO:tasks.workunit.client.1.smithi194.stdout:3/858: dread - d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f103 zero size 2024-04-12T05:56:16.979 INFO:tasks.workunit.client.1.smithi194.stdout:6/679: chown d3/d2f/d4e/d63/d83/fc1 206096765 1 2024-04-12T05:56:16.979 INFO:tasks.workunit.client.1.smithi194.stdout:0/692: readlink d0/da/d4a/l88 0 2024-04-12T05:56:16.980 INFO:tasks.workunit.client.1.smithi194.stdout:7/654: rename d3/d16/d23/d37/d3d/d56/da4 to d3/d16/d23/d37/d3d/d56/dd0 0 2024-04-12T05:56:16.980 INFO:tasks.workunit.client.1.smithi194.stdout:5/796: chown d1/d4/d6/d8e/lb7 63406 1 2024-04-12T05:56:16.981 INFO:tasks.workunit.client.1.smithi194.stdout:3/859: write d0/d1/d4f/fa3 [4448523,48707] 0 2024-04-12T05:56:16.981 INFO:tasks.workunit.client.1.smithi194.stdout:6/680: chown d3/d2f/d4e/d56/cb0 32541 1 2024-04-12T05:56:16.983 INFO:tasks.workunit.client.1.smithi194.stdout:9/805: creat d1/d21/d49/dcb/f10b x:0 0 0 2024-04-12T05:56:16.983 INFO:tasks.workunit.client.1.smithi194.stdout:5/797: chown d1/d4/d6/db/la4 198422 1 2024-04-12T05:56:16.984 INFO:tasks.workunit.client.1.smithi194.stdout:0/693: symlink d0/da/d4a/d8e/ldd 0 2024-04-12T05:56:16.984 INFO:tasks.workunit.client.1.smithi194.stdout:7/655: creat d3/d16/d23/d37/d95/fd1 x:0 0 0 2024-04-12T05:56:16.986 INFO:tasks.workunit.client.1.smithi194.stdout:7/656: chown d3/d16/d23/d37/d3d 1800 1 2024-04-12T05:56:16.989 INFO:tasks.workunit.client.1.smithi194.stdout:5/798: mknod d1/d5c/d95/c10d 0 2024-04-12T05:56:16.990 INFO:tasks.workunit.client.1.smithi194.stdout:0/694: chown d0/da/d1e/d27/d54/fc3 273 1 2024-04-12T05:56:16.992 INFO:tasks.workunit.client.1.smithi194.stdout:5/799: stat d1/d5c/d95/lf4 0 2024-04-12T05:56:16.995 INFO:tasks.workunit.client.1.smithi194.stdout:7/657: symlink d3/d16/d23/d57/d72/d81/d3a/ld2 0 2024-04-12T05:56:16.997 INFO:tasks.workunit.client.1.smithi194.stdout:3/860: dwrite d0/d1/d74/d39/d4d/deb/d53/f75 [0,4194304] 0 2024-04-12T05:56:16.997 INFO:tasks.workunit.client.1.smithi194.stdout:6/681: dwrite d3/d2f/d4e/d53/f85 [0,4194304] 0 2024-04-12T05:56:16.997 INFO:tasks.workunit.client.1.smithi194.stdout:0/695: creat d0/da/da7/fde x:0 0 0 2024-04-12T05:56:16.997 INFO:tasks.workunit.client.1.smithi194.stdout:5/800: symlink d1/d4/d6/db/d38/l10e 0 2024-04-12T05:56:17.003 INFO:tasks.workunit.client.1.smithi194.stdout:7/658: write d3/d16/d7b/fb9 [1885464,42472] 0 2024-04-12T05:56:17.004 INFO:tasks.workunit.client.1.smithi194.stdout:0/696: unlink d0/da/d1e/d1a/d24/f62 0 2024-04-12T05:56:17.005 INFO:tasks.workunit.client.1.smithi194.stdout:7/659: chown d3/d16/d23/d37/d3d/c3e 6 1 2024-04-12T05:56:17.007 INFO:tasks.workunit.client.1.smithi194.stdout:6/682: rename d3/d2f/d4e/d53 to d3/d2f/d4e/d9a/dd8 0 2024-04-12T05:56:17.008 INFO:tasks.workunit.client.1.smithi194.stdout:5/801: readlink d1/ld5 0 2024-04-12T05:56:17.010 INFO:tasks.workunit.client.1.smithi194.stdout:6/683: dread - d3/d2f/d37/f81 zero size 2024-04-12T05:56:17.010 INFO:tasks.workunit.client.1.smithi194.stdout:5/802: write d1/d4/d6/db/d15/d26/f4e [536312,24284] 0 2024-04-12T05:56:17.013 INFO:tasks.workunit.client.1.smithi194.stdout:3/861: link d0/d1/d74/d39/d4d/deb/d1e/f5f d0/d1/d10/d8c/d8f/f11f 0 2024-04-12T05:56:17.013 INFO:tasks.workunit.client.1.smithi194.stdout:0/697: mknod d0/da/d1e/cdf 0 2024-04-12T05:56:17.016 INFO:tasks.workunit.client.1.smithi194.stdout:7/660: link d3/d16/d23/lad d3/d16/d23/d40/d9b/ld3 0 2024-04-12T05:56:17.018 INFO:tasks.workunit.client.1.smithi194.stdout:3/862: mkdir d0/d1/d4f/dd7/d120 0 2024-04-12T05:56:17.019 INFO:tasks.workunit.client.1.smithi194.stdout:5/803: mknod d1/d4/d6/db/c10f 0 2024-04-12T05:56:17.020 INFO:tasks.workunit.client.1.smithi194.stdout:5/804: fsync d1/d4/d6/d8e/fa8 0 2024-04-12T05:56:17.021 INFO:tasks.workunit.client.1.smithi194.stdout:7/661: creat d3/d16/d23/d57/fd4 x:0 0 0 2024-04-12T05:56:17.022 INFO:tasks.workunit.client.1.smithi194.stdout:5/805: fsync d1/d5c/d95/f9f 0 2024-04-12T05:56:17.023 INFO:tasks.workunit.client.1.smithi194.stdout:5/806: stat d1/d4/c22 0 2024-04-12T05:56:17.023 INFO:tasks.workunit.client.1.smithi194.stdout:3/863: mkdir d0/d1/d74/d39/d4d/deb/db4/d121 0 2024-04-12T05:56:17.027 INFO:tasks.workunit.client.1.smithi194.stdout:7/662: creat d3/d16/d23/d40/d49/d74/fd5 x:0 0 0 2024-04-12T05:56:17.033 INFO:tasks.workunit.client.1.smithi194.stdout:7/663: creat d3/d16/d23/d57/fd6 x:0 0 0 2024-04-12T05:56:17.038 INFO:tasks.workunit.client.1.smithi194.stdout:0/698: link d0/da/d1e/d27/d54/ca4 d0/da/d1e/d27/d87/d92/ce0 0 2024-04-12T05:56:17.040 INFO:tasks.workunit.client.1.smithi194.stdout:6/684: dwrite d3/d2f/d37/f45 [0,4194304] 0 2024-04-12T05:56:17.043 INFO:tasks.workunit.client.1.smithi194.stdout:7/664: creat d3/d16/d23/d37/d3d/d56/dd0/fd7 x:0 0 0 2024-04-12T05:56:17.045 INFO:tasks.workunit.client.1.smithi194.stdout:5/807: getdents d1/d4/d84 0 2024-04-12T05:56:17.047 INFO:tasks.workunit.client.1.smithi194.stdout:3/864: write d0/d42/f51 [149011,9735] 0 2024-04-12T05:56:17.048 INFO:tasks.workunit.client.1.smithi194.stdout:5/808: chown d1/d4/d6/db/d15/d26/dbb/cdf 58216829 1 2024-04-12T05:56:17.049 INFO:tasks.workunit.client.1.smithi194.stdout:5/809: chown d1/d4/d84/lf8 840398 1 2024-04-12T05:56:17.050 INFO:tasks.workunit.client.1.smithi194.stdout:5/810: truncate d1/d4/d6/fc8 1310074 0 2024-04-12T05:56:17.052 INFO:tasks.workunit.client.1.smithi194.stdout:7/665: dread d3/d16/d23/d40/d6f/f8b [0,4194304] 0 2024-04-12T05:56:17.053 INFO:tasks.workunit.client.1.smithi194.stdout:0/699: mkdir d0/da/d1e/d1a/d24/de1 0 2024-04-12T05:56:17.054 INFO:tasks.workunit.client.1.smithi194.stdout:0/700: chown d0/d5f/d73/d76/fbf 5054 1 2024-04-12T05:56:17.054 INFO:tasks.workunit.client.1.smithi194.stdout:3/865: creat d0/d1/d74/d39/f122 x:0 0 0 2024-04-12T05:56:17.055 INFO:tasks.workunit.client.1.smithi194.stdout:5/811: mknod d1/d4/d6/db/d38/c110 0 2024-04-12T05:56:17.057 INFO:tasks.workunit.client.1.smithi194.stdout:7/666: write d3/d27/f1d [4485038,64413] 0 2024-04-12T05:56:17.057 INFO:tasks.workunit.client.1.smithi194.stdout:5/812: write f0 [1006130,1801] 0 2024-04-12T05:56:17.061 INFO:tasks.workunit.client.1.smithi194.stdout:6/685: dwrite d3/d2f/d37/f4a [0,4194304] 0 2024-04-12T05:56:17.062 INFO:tasks.workunit.client.1.smithi194.stdout:7/667: unlink d3/d16/c9d 0 2024-04-12T05:56:17.065 INFO:tasks.workunit.client.1.smithi194.stdout:7/668: write d3/d16/d23/d37/d95/f9c [100603,25141] 0 2024-04-12T05:56:17.070 INFO:tasks.workunit.client.1.smithi194.stdout:7/669: mkdir d3/dd8 0 2024-04-12T05:56:17.071 INFO:tasks.workunit.client.1.smithi194.stdout:5/813: write d1/d4/d6/db/f7a [332915,9578] 0 2024-04-12T05:56:17.073 INFO:tasks.workunit.client.1.smithi194.stdout:6/686: dwrite d3/d2f/d4e/d63/f78 [0,4194304] 0 2024-04-12T05:56:17.076 INFO:tasks.workunit.client.1.smithi194.stdout:0/701: dread d0/da/d1e/d27/d54/f7b [4194304,4194304] 0 2024-04-12T05:56:17.078 INFO:tasks.workunit.client.1.smithi194.stdout:7/670: getdents d3/d16/d23/d40/d6f 0 2024-04-12T05:56:17.079 INFO:tasks.workunit.client.1.smithi194.stdout:0/702: mkdir d0/d3b/d52/de2 0 2024-04-12T05:56:17.081 INFO:tasks.workunit.client.1.smithi194.stdout:0/703: fsync d0/d3b/d52/fb1 0 2024-04-12T05:56:17.083 INFO:tasks.workunit.client.1.smithi194.stdout:5/814: getdents d1/d5c/d95/df2 0 2024-04-12T05:56:17.083 INFO:tasks.workunit.client.1.smithi194.stdout:0/704: fdatasync d0/da/d1e/d1a/f7d 0 2024-04-12T05:56:17.086 INFO:tasks.workunit.client.1.smithi194.stdout:0/705: fsync d0/f1d 0 2024-04-12T05:56:17.089 INFO:tasks.workunit.client.1.smithi194.stdout:5/815: dread d1/d4/d6/db/f7a [0,4194304] 0 2024-04-12T05:56:17.090 INFO:tasks.workunit.client.1.smithi194.stdout:7/671: dread d3/d16/f45 [0,4194304] 0 2024-04-12T05:56:17.095 INFO:tasks.workunit.client.1.smithi194.stdout:0/706: dwrite d0/da/d1e/d1a/f23 [0,4194304] 0 2024-04-12T05:56:17.096 INFO:tasks.workunit.client.1.smithi194.stdout:6/687: write d3/f1f [2549727,18983] 0 2024-04-12T05:56:17.098 INFO:tasks.workunit.client.1.smithi194.stdout:7/672: dread d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:17.101 INFO:tasks.workunit.client.1.smithi194.stdout:0/707: chown d0/da/d1e/d1a/cd0 1 1 2024-04-12T05:56:17.102 INFO:tasks.workunit.client.1.smithi194.stdout:0/708: write d0/d5f/fcb [229764,76586] 0 2024-04-12T05:56:17.103 INFO:tasks.workunit.client.1.smithi194.stdout:7/673: dread - d3/d16/d23/d40/d49/d74/fd5 zero size 2024-04-12T05:56:17.105 INFO:tasks.workunit.client.1.smithi194.stdout:6/688: dread d3/d2f/fa7 [0,4194304] 0 2024-04-12T05:56:17.107 INFO:tasks.workunit.client.1.smithi194.stdout:5/816: dread d1/d4/d6/db/d15/d26/f50 [0,4194304] 0 2024-04-12T05:56:17.108 INFO:tasks.workunit.client.1.smithi194.stdout:1/807: sync 2024-04-12T05:56:17.111 INFO:tasks.workunit.client.1.smithi194.stdout:7/674: rename d3/d27/d22/d31/dc7 to d3/d16/d23/d37/d3d/d56/dd0/dd9 0 2024-04-12T05:56:17.114 INFO:tasks.workunit.client.1.smithi194.stdout:6/689: mkdir d3/d2f/dd9 0 2024-04-12T05:56:17.119 INFO:tasks.workunit.client.1.smithi194.stdout:7/675: rename d3/d16/d23/d40/d49 to d3/d27/d6a/dda 0 2024-04-12T05:56:17.121 INFO:tasks.workunit.client.1.smithi194.stdout:6/690: readlink d3/d15/d2b/d48/lac 0 2024-04-12T05:56:17.122 INFO:tasks.workunit.client.1.smithi194.stdout:6/691: stat d3/d2d/d33/d58/faf 0 2024-04-12T05:56:17.122 INFO:tasks.workunit.client.1.smithi194.stdout:7/676: creat d3/d27/d22/d62/fdb x:0 0 0 2024-04-12T05:56:17.127 INFO:tasks.workunit.client.1.smithi194.stdout:5/817: write d1/d4/d6/d88/f2c [456928,39225] 0 2024-04-12T05:56:17.129 INFO:tasks.workunit.client.1.smithi194.stdout:7/677: unlink d3/d16/d23/d37/d3d/d56/lb4 0 2024-04-12T05:56:17.130 INFO:tasks.workunit.client.1.smithi194.stdout:6/692: symlink d3/d2f/d4e/d9a/dd8/lda 0 2024-04-12T05:56:17.130 INFO:tasks.workunit.client.1.smithi194.stdout:1/808: write d0/d11/d18/d19/d1a/ded/ff4 [4802019,4890] 0 2024-04-12T05:56:17.132 INFO:tasks.workunit.client.1.smithi194.stdout:5/818: mknod d1/d4/d6/d8e/c111 0 2024-04-12T05:56:17.136 INFO:tasks.workunit.client.1.smithi194.stdout:0/709: dwrite d0/da/d1e/d27/d54/fad [0,4194304] 0 2024-04-12T05:56:17.139 INFO:tasks.workunit.client.1.smithi194.stdout:0/710: chown d0/d3b/l5a 239 1 2024-04-12T05:56:17.139 INFO:tasks.workunit.client.1.smithi194.stdout:6/693: read d3/d15/f9f [1557080,67199] 0 2024-04-12T05:56:17.139 INFO:tasks.workunit.client.1.smithi194.stdout:1/809: read d0/d11/f83 [23516,120237] 0 2024-04-12T05:56:17.140 INFO:tasks.workunit.client.1.smithi194.stdout:5/819: read d1/d4/d6/db/d15/f58 [3275929,21698] 0 2024-04-12T05:56:17.140 INFO:tasks.workunit.client.1.smithi194.stdout:6/694: stat d3/d2d/d33/d52/f89 0 2024-04-12T05:56:17.142 INFO:tasks.workunit.client.1.smithi194.stdout:7/678: rmdir d3/d27 39 2024-04-12T05:56:17.145 INFO:tasks.workunit.client.1.smithi194.stdout:1/810: write d0/d11/d18/f2d [7757744,2888] 0 2024-04-12T05:56:17.147 INFO:tasks.workunit.client.1.smithi194.stdout:0/711: rename d0/da/d1e/d1a/d58 to d0/de3 0 2024-04-12T05:56:17.148 INFO:tasks.workunit.client.1.smithi194.stdout:5/820: symlink d1/d4/d6/d34/l112 0 2024-04-12T05:56:17.149 INFO:tasks.workunit.client.1.smithi194.stdout:6/695: mknod d3/d2f/d4e/d9a/db3/cdb 0 2024-04-12T05:56:17.149 INFO:tasks.workunit.client.1.smithi194.stdout:5/821: write d1/d4/d6/db/d38/ff0 [1072285,8974] 0 2024-04-12T05:56:17.151 INFO:tasks.workunit.client.1.smithi194.stdout:1/811: rmdir d0/d11/d18/d19/d1a/d6a/d27/d6c/da0 39 2024-04-12T05:56:17.151 INFO:tasks.workunit.client.1.smithi194.stdout:1/812: readlink d0/d11/d18/d35/le4 0 2024-04-12T05:56:17.152 INFO:tasks.workunit.client.1.smithi194.stdout:0/712: mkdir d0/d3b/d52/d7c/dbe/de4 0 2024-04-12T05:56:17.153 INFO:tasks.workunit.client.1.smithi194.stdout:1/813: write d0/d11/d18/d19/d1c/f46 [1033628,60497] 0 2024-04-12T05:56:17.153 INFO:tasks.workunit.client.1.smithi194.stdout:7/679: mknod d3/d27/d22/d8e/cdc 0 2024-04-12T05:56:17.156 INFO:tasks.workunit.client.1.smithi194.stdout:5/822: dread d1/d5c/d95/f72 [0,4194304] 0 2024-04-12T05:56:17.160 INFO:tasks.workunit.client.1.smithi194.stdout:7/680: truncate d3/d16/d23/d37/d3d/d56/f51 4761392 0 2024-04-12T05:56:17.160 INFO:tasks.workunit.client.1.smithi194.stdout:5/823: chown d1/d4/d6/db/d2b/f66 10521 1 2024-04-12T05:56:17.160 INFO:tasks.workunit.client.1.smithi194.stdout:1/814: dread d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f79 [0,4194304] 0 2024-04-12T05:56:17.164 INFO:tasks.workunit.client.1.smithi194.stdout:7/681: link d3/d16/d5d/fbd d3/d27/d6a/dda/fdd 0 2024-04-12T05:56:17.167 INFO:tasks.workunit.client.1.smithi194.stdout:7/682: chown d3/d16/d23/d57/fba 0 1 2024-04-12T05:56:17.167 INFO:tasks.workunit.client.1.smithi194.stdout:6/696: dread d3/d2d/d33/d52/f89 [0,4194304] 0 2024-04-12T05:56:17.169 INFO:tasks.workunit.client.1.smithi194.stdout:7/683: dread - d3/d16/d23/d57/d72/d81/f92 zero size 2024-04-12T05:56:17.169 INFO:tasks.workunit.client.1.smithi194.stdout:7/684: chown d3/fcd 634232 1 2024-04-12T05:56:17.172 INFO:tasks.workunit.client.1.smithi194.stdout:7/685: dread - d3/d16/d23/d57/d72/d81/d3a/d43/d5c/fc4 zero size 2024-04-12T05:56:17.172 INFO:tasks.workunit.client.1.smithi194.stdout:0/713: dread d0/da/d1e/d1a/f7d [0,4194304] 0 2024-04-12T05:56:17.173 INFO:tasks.workunit.client.1.smithi194.stdout:7/686: fdatasync d3/d16/d23/d37/fb3 0 2024-04-12T05:56:17.175 INFO:tasks.workunit.client.1.smithi194.stdout:0/714: creat d0/d3b/d52/fe5 x:0 0 0 2024-04-12T05:56:17.176 INFO:tasks.workunit.client.1.smithi194.stdout:7/687: mknod d3/d27/d22/d31/cde 0 2024-04-12T05:56:17.179 INFO:tasks.workunit.client.1.smithi194.stdout:7/688: rename d3/d16/d23/d40/d6f/f8b to d3/d27/d6a/fdf 0 2024-04-12T05:56:17.180 INFO:tasks.workunit.client.1.smithi194.stdout:0/715: creat d0/da/d1e/d27/fe6 x:0 0 0 2024-04-12T05:56:17.183 INFO:tasks.workunit.client.1.smithi194.stdout:0/716: creat d0/d3b/fe7 x:0 0 0 2024-04-12T05:56:17.187 INFO:tasks.workunit.client.1.smithi194.stdout:6/697: dwrite d3/d2f/f39 [0,4194304] 0 2024-04-12T05:56:17.187 INFO:tasks.workunit.client.1.smithi194.stdout:0/717: creat d0/da/d1e/d1a/d24/da8/fe8 x:0 0 0 2024-04-12T05:56:17.194 INFO:tasks.workunit.client.1.smithi194.stdout:0/718: rename d0/da/d1e/d1a/f2e to d0/da/d1e/d1a/d24/da8/fe9 0 2024-04-12T05:56:17.197 INFO:tasks.workunit.client.1.smithi194.stdout:0/719: creat d0/d3b/d52/de2/fea x:0 0 0 2024-04-12T05:56:17.200 INFO:tasks.workunit.client.1.smithi194.stdout:0/720: mkdir d0/d5f/deb 0 2024-04-12T05:56:17.201 INFO:tasks.workunit.client.1.smithi194.stdout:6/698: write d3/d2f/d37/f81 [333987,12487] 0 2024-04-12T05:56:17.203 INFO:tasks.workunit.client.1.smithi194.stdout:6/699: dread - d3/d2d/d33/d52/fcc zero size 2024-04-12T05:56:17.203 INFO:tasks.workunit.client.1.smithi194.stdout:0/721: rename d0/d5f/f6a to d0/d5f/d73/d76/fec 0 2024-04-12T05:56:17.205 INFO:tasks.workunit.client.1.smithi194.stdout:6/700: mknod d3/d15/d2b/d48/d6f/d7a/cdc 0 2024-04-12T05:56:17.214 INFO:tasks.workunit.client.1.smithi194.stdout:6/701: dwrite d3/d2f/d4e/d68/fa2 [4194304,4194304] 0 2024-04-12T05:56:17.214 INFO:tasks.workunit.client.1.smithi194.stdout:6/702: write d3/f18 [6289594,80293] 0 2024-04-12T05:56:17.217 INFO:tasks.workunit.client.1.smithi194.stdout:6/703: mkdir d3/d2d/d33/d52/d62/ddd 0 2024-04-12T05:56:17.220 INFO:tasks.workunit.client.1.smithi194.stdout:6/704: mkdir d3/d2f/d4e/d9a/dd8/d77/dde 0 2024-04-12T05:56:17.223 INFO:tasks.workunit.client.1.smithi194.stdout:6/705: truncate d3/d15/d2b/d48/d6f/d7a/d8f/fa3 1936357 0 2024-04-12T05:56:17.223 INFO:tasks.workunit.client.1.smithi194.stdout:6/706: stat d3/f18 0 2024-04-12T05:56:17.261 INFO:tasks.workunit.client.1.smithi194.stdout:6/707: dwrite d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:17.264 INFO:tasks.workunit.client.1.smithi194.stdout:2/725: sync 2024-04-12T05:56:17.269 INFO:tasks.workunit.client.1.smithi194.stdout:2/726: symlink d7/d12/d1a/d69/dd6/df9/lfc 0 2024-04-12T05:56:17.270 INFO:tasks.workunit.client.1.smithi194.stdout:2/727: chown d7/d12/d25/d5d/l7c 987596 1 2024-04-12T05:56:17.271 INFO:tasks.workunit.client.1.smithi194.stdout:2/728: write d7/d12/fe3 [788110,62961] 0 2024-04-12T05:56:17.272 INFO:tasks.workunit.client.1.smithi194.stdout:2/729: dread - d7/d22/dc9/fee zero size 2024-04-12T05:56:17.285 INFO:tasks.workunit.client.1.smithi194.stdout:2/730: rename d7/d12/d25/d5d/d47/dac/db3/dd7/c65 to d7/d12/d6d/cfd 0 2024-04-12T05:56:17.285 INFO:tasks.workunit.client.1.smithi194.stdout:2/731: read - d7/d12/d25/d5d/d47/dac/fea zero size 2024-04-12T05:56:17.289 INFO:tasks.workunit.client.1.smithi194.stdout:6/708: dread d3/d2d/d33/d52/f55 [0,4194304] 0 2024-04-12T05:56:17.289 INFO:tasks.workunit.client.1.smithi194.stdout:2/732: chown d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/lcf 1 1 2024-04-12T05:56:17.293 INFO:tasks.workunit.client.1.smithi194.stdout:6/709: write d3/d15/d2b/d48/f74 [247445,1995] 0 2024-04-12T05:56:17.295 INFO:tasks.workunit.client.1.smithi194.stdout:6/710: truncate d3/fd4 513111 0 2024-04-12T05:56:17.297 INFO:tasks.workunit.client.1.smithi194.stdout:2/733: creat d7/d12/d1a/d69/ffe x:0 0 0 2024-04-12T05:56:17.299 INFO:tasks.workunit.client.1.smithi194.stdout:2/734: chown d7/d22/d93/d9a/fc8 6865 1 2024-04-12T05:56:17.299 INFO:tasks.workunit.client.1.smithi194.stdout:6/711: fdatasync d3/d15/d2b/d48/d6f/d7a/d8f/fb5 0 2024-04-12T05:56:17.303 INFO:tasks.workunit.client.1.smithi194.stdout:2/735: creat d7/d12/d1a/d69/dd6/df9/fff x:0 0 0 2024-04-12T05:56:17.304 INFO:tasks.workunit.client.1.smithi194.stdout:6/712: write d3/d15/d2b/d48/f54 [1293733,8693] 0 2024-04-12T05:56:17.306 INFO:tasks.workunit.client.1.smithi194.stdout:2/736: write d7/d22/d93/d9f/fb0 [427387,43663] 0 2024-04-12T05:56:17.310 INFO:tasks.workunit.client.1.smithi194.stdout:6/713: link d3/d2f/fa7 d3/d15/d2b/d48/d6f/d7a/d8f/d98/fdf 0 2024-04-12T05:56:17.311 INFO:tasks.workunit.client.1.smithi194.stdout:6/714: dread - d3/d15/d2b/d48/d6f/fb7 zero size 2024-04-12T05:56:17.317 INFO:tasks.workunit.client.1.smithi194.stdout:6/715: mknod d3/d2f/dd9/ce0 0 2024-04-12T05:56:17.317 INFO:tasks.workunit.client.1.smithi194.stdout:2/737: creat d7/d22/d93/d9a/f100 x:0 0 0 2024-04-12T05:56:17.325 INFO:tasks.workunit.client.1.smithi194.stdout:2/738: truncate d7/d12/d25/d5d/d47/dac/dc2/fd4 1032494 0 2024-04-12T05:56:17.325 INFO:tasks.workunit.client.1.smithi194.stdout:2/739: fdatasync d7/d22/d93/fdf 0 2024-04-12T05:56:17.326 INFO:tasks.workunit.client.1.smithi194.stdout:6/716: getdents d3/d2f/d4e/d68/dad/dc4 0 2024-04-12T05:56:17.330 INFO:tasks.workunit.client.1.smithi194.stdout:2/740: read d7/d22/d42/f1d [1381166,42577] 0 2024-04-12T05:56:17.332 INFO:tasks.workunit.client.1.smithi194.stdout:2/741: chown d7/d22/d93/d9f/fd1 436335 1 2024-04-12T05:56:17.335 INFO:tasks.workunit.client.1.smithi194.stdout:6/717: creat d3/d2d/d33/d52/d5c/fe1 x:0 0 0 2024-04-12T05:56:17.339 INFO:tasks.workunit.client.1.smithi194.stdout:6/718: mkdir d3/d2f/d4e/d56/de2 0 2024-04-12T05:56:17.345 INFO:tasks.workunit.client.1.smithi194.stdout:2/742: rename d7/d12/d1a/d69/dd6 to d7/d12/d1a/d69/df0/d101 0 2024-04-12T05:56:17.347 INFO:tasks.workunit.client.1.smithi194.stdout:6/719: unlink d3/d2f/d4e/d9a/db3/cdb 0 2024-04-12T05:56:17.360 INFO:tasks.workunit.client.1.smithi194.stdout:2/743: getdents d7/d22/d93/d9f 0 2024-04-12T05:56:17.361 INFO:tasks.workunit.client.1.smithi194.stdout:6/720: rename d3/d15/c50 to d3/d2d/d33/d52/d62/ddd/ce3 0 2024-04-12T05:56:17.365 INFO:tasks.workunit.client.1.smithi194.stdout:6/721: truncate d3/d15/f21 4951497 0 2024-04-12T05:56:17.370 INFO:tasks.workunit.client.1.smithi194.stdout:2/744: dwrite d7/d22/ff7 [0,4194304] 0 2024-04-12T05:56:17.376 INFO:tasks.workunit.client.1.smithi194.stdout:2/745: truncate d7/d12/d1a/d69/f82 582327 0 2024-04-12T05:56:17.378 INFO:tasks.workunit.client.1.smithi194.stdout:6/722: dwrite d3/d15/d2b/d48/d6f/d7a/d8f/fbd [0,4194304] 0 2024-04-12T05:56:17.379 INFO:tasks.workunit.client.1.smithi194.stdout:2/746: chown d7/d12/d25/d5d/l7c 87 1 2024-04-12T05:56:17.382 INFO:tasks.workunit.client.1.smithi194.stdout:6/723: truncate d3/d2d/d33/d58/fce 1745911 0 2024-04-12T05:56:17.384 INFO:tasks.workunit.client.1.smithi194.stdout:2/747: creat d7/d22/d93/f102 x:0 0 0 2024-04-12T05:56:17.386 INFO:tasks.workunit.client.1.smithi194.stdout:2/748: dread - d7/d12/d25/d5d/d47/dac/ff6 zero size 2024-04-12T05:56:17.388 INFO:tasks.workunit.client.1.smithi194.stdout:6/724: truncate d3/d15/d2b/d48/d6f/d7a/d8f/fb5 907635 0 2024-04-12T05:56:17.388 INFO:tasks.workunit.client.1.smithi194.stdout:6/725: readlink d3/d2d/d33/db1/ld5 0 2024-04-12T05:56:17.409 INFO:tasks.workunit.client.1.smithi194.stdout:6/726: dwrite d3/d2d/fcb [0,4194304] 0 2024-04-12T05:56:17.412 INFO:tasks.workunit.client.1.smithi194.stdout:6/727: mkdir d3/d2d/d33/d52/d5c/de4 0 2024-04-12T05:56:17.414 INFO:tasks.workunit.client.1.smithi194.stdout:6/728: symlink d3/d2d/d33/d52/d5c/le5 0 2024-04-12T05:56:17.426 INFO:tasks.workunit.client.1.smithi194.stdout:6/729: dread d3/f3b [0,4194304] 0 2024-04-12T05:56:17.427 INFO:tasks.workunit.client.1.smithi194.stdout:6/730: write d3/d2f/d37/f81 [854327,115463] 0 2024-04-12T05:56:17.432 INFO:tasks.workunit.client.1.smithi194.stdout:6/731: write d3/f3b [4157506,100386] 0 2024-04-12T05:56:17.435 INFO:tasks.workunit.client.1.smithi194.stdout:6/732: mkdir d3/d2f/d4e/d9a/de6 0 2024-04-12T05:56:17.435 INFO:tasks.workunit.client.1.smithi194.stdout:6/733: dread - d3/d2d/d33/fa6 zero size 2024-04-12T05:56:17.442 INFO:tasks.workunit.client.1.smithi194.stdout:6/734: dwrite d3/d2d/d33/d52/d5c/f5f [0,4194304] 0 2024-04-12T05:56:17.456 INFO:tasks.workunit.client.1.smithi194.stdout:6/735: write d3/d2d/d33/d52/f55 [2558431,25197] 0 2024-04-12T05:56:17.458 INFO:tasks.workunit.client.1.smithi194.stdout:6/736: rmdir d3/d2d/d33/d58 39 2024-04-12T05:56:17.459 INFO:tasks.workunit.client.1.smithi194.stdout:6/737: read d3/d2f/d4e/d63/f78 [2269963,9470] 0 2024-04-12T05:56:17.463 INFO:tasks.workunit.client.1.smithi194.stdout:6/738: write d3/d15/d2b/d48/d6f/f82 [269950,74027] 0 2024-04-12T05:56:17.474 INFO:tasks.workunit.client.1.smithi194.stdout:6/739: dread d3/f1f [0,4194304] 0 2024-04-12T05:56:17.475 INFO:tasks.workunit.client.1.smithi194.stdout:6/740: chown d3/d2f/d37/f81 18146 1 2024-04-12T05:56:17.477 INFO:tasks.workunit.client.1.smithi194.stdout:6/741: mknod d3/d2f/d4e/d56/de2/ce7 0 2024-04-12T05:56:17.491 INFO:tasks.workunit.client.1.smithi194.stdout:6/742: dread d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:17.495 INFO:tasks.workunit.client.1.smithi194.stdout:6/743: rename d3/c44 to d3/d2f/d4e/d9a/dd8/d77/dde/ce8 0 2024-04-12T05:56:17.497 INFO:tasks.workunit.client.1.smithi194.stdout:6/744: creat d3/d2f/d4e/fe9 x:0 0 0 2024-04-12T05:56:17.500 INFO:tasks.workunit.client.1.smithi194.stdout:6/745: dread d3/d15/d2b/fc5 [0,4194304] 0 2024-04-12T05:56:17.504 INFO:tasks.workunit.client.1.smithi194.stdout:6/746: rmdir d3/d2f/d4e/d9a/de6 0 2024-04-12T05:56:17.505 INFO:tasks.workunit.client.1.smithi194.stdout:6/747: creat d3/d34/fea x:0 0 0 2024-04-12T05:56:17.510 INFO:tasks.workunit.client.1.smithi194.stdout:6/748: getdents d3/d2f/d4e/d9a/dd8/d77 0 2024-04-12T05:56:17.511 INFO:tasks.workunit.client.1.smithi194.stdout:6/749: chown d3/f84 5 1 2024-04-12T05:56:17.516 INFO:tasks.workunit.client.1.smithi194.stdout:6/750: rmdir d3 39 2024-04-12T05:56:17.518 INFO:tasks.workunit.client.1.smithi194.stdout:6/751: readlink d3/d2f/d4e/d68/l72 0 2024-04-12T05:56:17.519 INFO:tasks.workunit.client.1.smithi194.stdout:6/752: chown d3/d15/d2b/d48/lac 25800 1 2024-04-12T05:56:17.521 INFO:tasks.workunit.client.1.smithi194.stdout:6/753: getdents d3/d2f/d4e/d9a/dd8/d77 0 2024-04-12T05:56:17.527 INFO:tasks.workunit.client.1.smithi194.stdout:6/754: dwrite d3/d15/f9f [0,4194304] 0 2024-04-12T05:56:17.533 INFO:tasks.workunit.client.1.smithi194.stdout:6/755: write d3/d2d/d33/fa6 [354275,38828] 0 2024-04-12T05:56:17.534 INFO:tasks.workunit.client.1.smithi194.stdout:8/522: sync 2024-04-12T05:56:17.536 INFO:tasks.workunit.client.1.smithi194.stdout:6/756: mknod d3/d2d/d33/ceb 0 2024-04-12T05:56:17.539 INFO:tasks.workunit.client.1.smithi194.stdout:8/523: rename d2/d8/d15/d33/f1b to d2/d8/d9/de/d38/fa0 0 2024-04-12T05:56:17.540 INFO:tasks.workunit.client.1.smithi194.stdout:8/524: dread - d2/d8/d9/de/d10/d4e/f8a zero size 2024-04-12T05:56:17.540 INFO:tasks.workunit.client.1.smithi194.stdout:6/757: creat d3/d2f/d4e/d9a/dd8/d77/dde/fec x:0 0 0 2024-04-12T05:56:17.541 INFO:tasks.workunit.client.1.smithi194.stdout:6/758: chown d3/d15/d2b/fc5 15882 1 2024-04-12T05:56:17.554 INFO:tasks.workunit.client.1.smithi194.stdout:8/525: truncate d2/d8/d9/de/f1d 300836 0 2024-04-12T05:56:17.556 INFO:tasks.workunit.client.1.smithi194.stdout:8/526: mkdir d2/d8/d9/d1a/da1 0 2024-04-12T05:56:17.559 INFO:tasks.workunit.client.1.smithi194.stdout:8/527: chown d2/d8/d15/d3b/c85 20 1 2024-04-12T05:56:17.560 INFO:tasks.workunit.client.1.smithi194.stdout:8/528: write d2/d8/f6a [3230144,35687] 0 2024-04-12T05:56:17.561 INFO:tasks.workunit.client.1.smithi194.stdout:6/759: dwrite d3/d1a/f35 [0,4194304] 0 2024-04-12T05:56:17.570 INFO:tasks.workunit.client.1.smithi194.stdout:8/529: link d2/d8/d9/de/f37 d2/d8/d15/d35/fa2 0 2024-04-12T05:56:17.570 INFO:tasks.workunit.client.1.smithi194.stdout:6/760: dwrite d3/d2f/d4e/d9a/dd8/d77/dde/fec [0,4194304] 0 2024-04-12T05:56:17.582 INFO:tasks.workunit.client.1.smithi194.stdout:8/530: creat d2/d8/d15/fa3 x:0 0 0 2024-04-12T05:56:17.582 INFO:tasks.workunit.client.1.smithi194.stdout:8/531: mkdir d2/d8/d66/da4 0 2024-04-12T05:56:17.582 INFO:tasks.workunit.client.1.smithi194.stdout:6/761: dwrite d3/d2d/fcb [0,4194304] 0 2024-04-12T05:56:17.586 INFO:tasks.workunit.client.1.smithi194.stdout:6/762: stat d3/d2f/d4e/d68/dad/dc4/lca 0 2024-04-12T05:56:17.586 INFO:tasks.workunit.client.1.smithi194.stdout:8/532: fsync d2/d8/d9/de/d38/f3a 0 2024-04-12T05:56:17.587 INFO:tasks.workunit.client.1.smithi194.stdout:8/533: fsync d2/d8/d9/de/d10/d4e/f8a 0 2024-04-12T05:56:17.593 INFO:tasks.workunit.client.1.smithi194.stdout:8/534: rename d2/d8/d15/f8c to d2/d8/d9/de/fa5 0 2024-04-12T05:56:17.594 INFO:tasks.workunit.client.1.smithi194.stdout:8/535: truncate d2/d8/d66/d67/f71 660717 0 2024-04-12T05:56:17.595 INFO:tasks.workunit.client.1.smithi194.stdout:6/763: write d3/f84 [282824,49215] 0 2024-04-12T05:56:17.598 INFO:tasks.workunit.client.1.smithi194.stdout:8/536: dread d2/d8/d9/d1a/f5a [0,4194304] 0 2024-04-12T05:56:17.599 INFO:tasks.workunit.client.1.smithi194.stdout:8/537: stat d2/d8/d9/de/d38 0 2024-04-12T05:56:17.599 INFO:tasks.workunit.client.1.smithi194.stdout:6/764: getdents d3/d2f/d4e/d9a/db3 0 2024-04-12T05:56:17.600 INFO:tasks.workunit.client.1.smithi194.stdout:6/765: fsync d3/d2f/d4e/d63/faa 0 2024-04-12T05:56:17.603 INFO:tasks.workunit.client.1.smithi194.stdout:6/766: mknod d3/d15/ced 0 2024-04-12T05:56:17.606 INFO:tasks.workunit.client.1.smithi194.stdout:6/767: write d3/d2d/d33/d58/fba [36835,27654] 0 2024-04-12T05:56:17.608 INFO:tasks.workunit.client.1.smithi194.stdout:6/768: rmdir d3 39 2024-04-12T05:56:17.609 INFO:tasks.workunit.client.1.smithi194.stdout:8/538: dwrite d2/d8/d15/d3b/f78 [0,4194304] 0 2024-04-12T05:56:17.611 INFO:tasks.workunit.client.1.smithi194.stdout:6/769: chown d3/d2f/d4e/d9a/dd8/d77 11087529 1 2024-04-12T05:56:17.611 INFO:tasks.workunit.client.1.smithi194.stdout:8/539: write d2/d8/f36 [999485,99008] 0 2024-04-12T05:56:17.615 INFO:tasks.workunit.client.1.smithi194.stdout:6/770: creat d3/d2f/d4e/d9a/fee x:0 0 0 2024-04-12T05:56:17.626 INFO:tasks.workunit.client.1.smithi194.stdout:8/540: write d2/d8/d15/d33/f27 [6597227,39889] 0 2024-04-12T05:56:17.635 INFO:tasks.workunit.client.1.smithi194.stdout:8/541: dwrite d2/d8/d15/d33/f27 [4194304,4194304] 0 2024-04-12T05:56:17.643 INFO:tasks.workunit.client.1.smithi194.stdout:6/771: creat d3/d15/d2b/d48/d6f/fef x:0 0 0 2024-04-12T05:56:17.645 INFO:tasks.workunit.client.1.smithi194.stdout:8/542: write d2/d8/d15/d35/d6c/f91 [304444,74510] 0 2024-04-12T05:56:17.645 INFO:tasks.workunit.client.1.smithi194.stdout:6/772: dread d3/d2f/f9e [0,4194304] 0 2024-04-12T05:56:17.647 INFO:tasks.workunit.client.1.smithi194.stdout:8/543: rmdir d2/d8/d15/d35/d6c 39 2024-04-12T05:56:17.649 INFO:tasks.workunit.client.1.smithi194.stdout:6/773: truncate d3/d15/d2b/d48/d6f/d7a/d8f/fc0 3675 0 2024-04-12T05:56:17.651 INFO:tasks.workunit.client.1.smithi194.stdout:6/774: write d3/d2d/d33/f4d [815793,122321] 0 2024-04-12T05:56:17.654 INFO:tasks.workunit.client.1.smithi194.stdout:6/775: write d3/d2f/d4e/fe9 [251379,5786] 0 2024-04-12T05:56:17.656 INFO:tasks.workunit.client.1.smithi194.stdout:6/776: mknod d3/cf0 0 2024-04-12T05:56:17.658 INFO:tasks.workunit.client.1.smithi194.stdout:6/777: mknod d3/d2d/d33/db1/cf1 0 2024-04-12T05:56:17.660 INFO:tasks.workunit.client.1.smithi194.stdout:6/778: write d3/d2d/d33/fa6 [1269161,51558] 0 2024-04-12T05:56:17.662 INFO:tasks.workunit.client.1.smithi194.stdout:6/779: chown d3/c46 167871 1 2024-04-12T05:56:17.663 INFO:tasks.workunit.client.1.smithi194.stdout:6/780: chown d3/d2d/d33/d58/f7f 583259 1 2024-04-12T05:56:17.665 INFO:tasks.workunit.client.1.smithi194.stdout:6/781: creat d3/d2f/d4e/d56/de2/ff2 x:0 0 0 2024-04-12T05:56:17.668 INFO:tasks.workunit.client.1.smithi194.stdout:8/544: dread d2/d8/d9/de/d10/f88 [0,4194304] 0 2024-04-12T05:56:17.668 INFO:tasks.workunit.client.1.smithi194.stdout:6/782: creat d3/d2d/d33/d52/d62/ddd/ff3 x:0 0 0 2024-04-12T05:56:17.670 INFO:tasks.workunit.client.1.smithi194.stdout:6/783: symlink d3/d15/d2b/d48/d6f/d7a/d8f/lf4 0 2024-04-12T05:56:17.676 INFO:tasks.workunit.client.1.smithi194.stdout:8/545: write d2/d8/d15/d35/d6c/f6e [301967,64685] 0 2024-04-12T05:56:17.677 INFO:tasks.workunit.client.1.smithi194.stdout:8/546: dread - d2/d8/d9/de/f9a zero size 2024-04-12T05:56:17.679 INFO:tasks.workunit.client.1.smithi194.stdout:8/547: write d2/d8/d9/de/fa5 [4018848,31961] 0 2024-04-12T05:56:17.681 INFO:tasks.workunit.client.1.smithi194.stdout:8/548: fsync d2/d8/d9/d1a/f3d 0 2024-04-12T05:56:17.694 INFO:tasks.workunit.client.1.smithi194.stdout:8/549: write d2/d8/d9/de/f37 [4985485,55308] 0 2024-04-12T05:56:17.702 INFO:tasks.workunit.client.1.smithi194.stdout:8/550: truncate d2/d8/d15/d33/f27 2578145 0 2024-04-12T05:56:17.705 INFO:tasks.workunit.client.1.smithi194.stdout:8/551: fsync d2/d8/d15/d33/f18 0 2024-04-12T05:56:17.713 INFO:tasks.workunit.client.1.smithi194.stdout:8/552: rename d2/d8/d15/d33/d45/d52/l68 to d2/d8/d9/de/d10/d4e/la6 0 2024-04-12T05:56:17.716 INFO:tasks.workunit.client.1.smithi194.stdout:8/553: creat d2/d8/d66/d7f/fa7 x:0 0 0 2024-04-12T05:56:17.717 INFO:tasks.workunit.client.1.smithi194.stdout:8/554: write d2/d8/d15/d35/d6c/f6e [1028172,57905] 0 2024-04-12T05:56:17.722 INFO:tasks.workunit.client.1.smithi194.stdout:3/866: sync 2024-04-12T05:56:17.722 INFO:tasks.workunit.client.1.smithi194.stdout:4/948: sync 2024-04-12T05:56:17.722 INFO:tasks.workunit.client.1.smithi194.stdout:9/806: sync 2024-04-12T05:56:17.724 INFO:tasks.workunit.client.1.smithi194.stdout:9/807: dread - d1/d19/d1f/d20/d37/d60/f106 zero size 2024-04-12T05:56:17.735 INFO:tasks.workunit.client.1.smithi194.stdout:4/949: truncate d2/d11/d1e/d2f/d4a/f139 468982 0 2024-04-12T05:56:17.735 INFO:tasks.workunit.client.1.smithi194.stdout:3/867: dwrite d0/d1/d74/d39/fb0 [0,4194304] 0 2024-04-12T05:56:17.737 INFO:tasks.workunit.client.1.smithi194.stdout:4/950: creat d2/d11/d93/f14b x:0 0 0 2024-04-12T05:56:17.739 INFO:tasks.workunit.client.1.smithi194.stdout:3/868: chown d0/d1/d74/d39/d4d/deb/d1e/d81/d34/c4c 29640 1 2024-04-12T05:56:17.741 INFO:tasks.workunit.client.1.smithi194.stdout:8/555: truncate d2/d8/d9/de/f37 4283626 0 2024-04-12T05:56:17.742 INFO:tasks.workunit.client.1.smithi194.stdout:9/808: rename d1/d21/f4d to d1/d41/d97/f10c 0 2024-04-12T05:56:17.744 INFO:tasks.workunit.client.1.smithi194.stdout:4/951: read d2/d11/d93/da6/dab/f12a [526511,10731] 0 2024-04-12T05:56:17.751 INFO:tasks.workunit.client.1.smithi194.stdout:3/869: truncate d0/d1/d10/f2e 2704250 0 2024-04-12T05:56:17.751 INFO:tasks.workunit.client.1.smithi194.stdout:9/809: mknod d1/d19/d1f/d20/d37/da3/c10d 0 2024-04-12T05:56:17.751 INFO:tasks.workunit.client.1.smithi194.stdout:9/810: rmdir d1/d41/dba/dc2 39 2024-04-12T05:56:17.752 INFO:tasks.workunit.client.1.smithi194.stdout:9/811: stat d1/d19 0 2024-04-12T05:56:17.755 INFO:tasks.workunit.client.1.smithi194.stdout:4/952: rmdir d2/d11/d1e 39 2024-04-12T05:56:17.756 INFO:tasks.workunit.client.1.smithi194.stdout:4/953: chown d2/d11/d93/cda 3873421 1 2024-04-12T05:56:17.758 INFO:tasks.workunit.client.1.smithi194.stdout:3/870: dread d0/d1/d74/d39/d4d/deb/d1e/d81/f6b [0,4194304] 0 2024-04-12T05:56:17.759 INFO:tasks.workunit.client.1.smithi194.stdout:8/556: truncate d2/d8/f82 253652 0 2024-04-12T05:56:17.760 INFO:tasks.workunit.client.1.smithi194.stdout:4/954: creat d2/d11/d93/da6/dab/d5d/dae/db2/f14c x:0 0 0 2024-04-12T05:56:17.761 INFO:tasks.workunit.client.1.smithi194.stdout:9/812: creat d1/d19/d1f/d20/d37/d60/f10e x:0 0 0 2024-04-12T05:56:17.765 INFO:tasks.workunit.client.1.smithi194.stdout:9/813: dread - d1/d19/d1f/d20/d37/d60/f107 zero size 2024-04-12T05:56:17.766 INFO:tasks.workunit.client.1.smithi194.stdout:3/871: fsync d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/f84 0 2024-04-12T05:56:17.767 INFO:tasks.workunit.client.1.smithi194.stdout:4/955: mknod d2/d3/d50/da0/c14d 0 2024-04-12T05:56:17.767 INFO:tasks.workunit.client.1.smithi194.stdout:9/814: dread - d1/d41/fff zero size 2024-04-12T05:56:17.768 INFO:tasks.workunit.client.1.smithi194.stdout:8/557: truncate d2/d8/d15/d33/f29 357477 0 2024-04-12T05:56:17.769 INFO:tasks.workunit.client.1.smithi194.stdout:9/815: write d1/d41/fff [476721,127774] 0 2024-04-12T05:56:17.775 INFO:tasks.workunit.client.1.smithi194.stdout:3/872: creat d0/d1/d10/d89/f123 x:0 0 0 2024-04-12T05:56:17.775 INFO:tasks.workunit.client.1.smithi194.stdout:8/558: creat d2/d8/d9/de/fa8 x:0 0 0 2024-04-12T05:56:17.777 INFO:tasks.workunit.client.1.smithi194.stdout:9/816: rename d1/d19/d1f/d20/d7d to d1/da6/d10f 0 2024-04-12T05:56:17.778 INFO:tasks.workunit.client.1.smithi194.stdout:4/956: dwrite d2/d11/d93/da6/dab/d5d/dae/db2/f14c [0,4194304] 0 2024-04-12T05:56:17.780 INFO:tasks.workunit.client.1.smithi194.stdout:9/817: chown d1/d41/d97/f109 241 1 2024-04-12T05:56:17.782 INFO:tasks.workunit.client.1.smithi194.stdout:4/957: chown d2/d11/d93/da6/dab/lc4 8821 1 2024-04-12T05:56:17.782 INFO:tasks.workunit.client.1.smithi194.stdout:3/873: rename d0/d1/d74/d39/d4d/deb/d3d/d6e/de9 to d0/d1/d4f/dd7/d120/d124 0 2024-04-12T05:56:17.790 INFO:tasks.workunit.client.1.smithi194.stdout:3/874: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f37 2457368 0 2024-04-12T05:56:17.793 INFO:tasks.workunit.client.1.smithi194.stdout:4/958: dwrite d2/d3/d58/d82/dd0/f121 [0,4194304] 0 2024-04-12T05:56:17.793 INFO:tasks.workunit.client.1.smithi194.stdout:3/875: chown d0/d1/d74/ce4 30095 1 2024-04-12T05:56:17.799 INFO:tasks.workunit.client.1.smithi194.stdout:8/559: dwrite d2/d8/f54 [0,4194304] 0 2024-04-12T05:56:17.799 INFO:tasks.workunit.client.1.smithi194.stdout:9/818: dread d1/d41/dba/dc2/fc3 [0,4194304] 0 2024-04-12T05:56:17.799 INFO:tasks.workunit.client.1.smithi194.stdout:3/876: getdents d0/d1/d74/d39/d4d/deb/d8e/d115 0 2024-04-12T05:56:17.804 INFO:tasks.workunit.client.1.smithi194.stdout:3/877: creat d0/d1/d4f/dd7/de1/f125 x:0 0 0 2024-04-12T05:56:17.807 INFO:tasks.workunit.client.1.smithi194.stdout:4/959: dwrite d2/d3/d1b/d32/db3/fdb [0,4194304] 0 2024-04-12T05:56:17.812 INFO:tasks.workunit.client.1.smithi194.stdout:9/819: link d1/da6/d10f/c7f d1/d19/d1f/d20/d37/c110 0 2024-04-12T05:56:17.816 INFO:tasks.workunit.client.1.smithi194.stdout:9/820: write d1/d19/d1f/d20/fcd [1906895,63880] 0 2024-04-12T05:56:17.819 INFO:tasks.workunit.client.1.smithi194.stdout:3/878: dwrite d0/d1/d10/d8c/d8f/ddb/ff5 [0,4194304] 0 2024-04-12T05:56:17.820 INFO:tasks.workunit.client.1.smithi194.stdout:3/879: chown d0/d1/f60 1063 1 2024-04-12T05:56:17.820 INFO:tasks.workunit.client.1.smithi194.stdout:4/960: write d2/d3/ff [259019,112594] 0 2024-04-12T05:56:17.821 INFO:tasks.workunit.client.1.smithi194.stdout:9/821: mknod d1/d21/d39/df1/c111 0 2024-04-12T05:56:17.826 INFO:tasks.workunit.client.1.smithi194.stdout:3/880: mkdir d0/d1/d10/d89/d126 0 2024-04-12T05:56:17.826 INFO:tasks.workunit.client.1.smithi194.stdout:9/822: mknod d1/d21/d49/c112 0 2024-04-12T05:56:17.826 INFO:tasks.workunit.client.1.smithi194.stdout:9/823: chown d1/d21/f61 180946 1 2024-04-12T05:56:17.827 INFO:tasks.workunit.client.1.smithi194.stdout:4/961: creat d2/d11/d46/f14e x:0 0 0 2024-04-12T05:56:17.829 INFO:tasks.workunit.client.1.smithi194.stdout:3/881: creat d0/d1/d4f/dd7/de1/dee/f127 x:0 0 0 2024-04-12T05:56:17.830 INFO:tasks.workunit.client.1.smithi194.stdout:4/962: mknod d2/d3/d58/c14f 0 2024-04-12T05:56:17.832 INFO:tasks.workunit.client.1.smithi194.stdout:9/824: creat d1/d19/d1f/d20/d37/d38/f113 x:0 0 0 2024-04-12T05:56:17.834 INFO:tasks.workunit.client.1.smithi194.stdout:4/963: write d2/d3/d51/fb1 [4451991,11299] 0 2024-04-12T05:56:17.835 INFO:tasks.workunit.client.1.smithi194.stdout:9/825: creat d1/d41/dba/dc2/def/f114 x:0 0 0 2024-04-12T05:56:17.836 INFO:tasks.workunit.client.1.smithi194.stdout:9/826: fsync d1/d21/d49/f108 0 2024-04-12T05:56:17.839 INFO:tasks.workunit.client.1.smithi194.stdout:3/882: dread d0/d1/d4f/f59 [0,4194304] 0 2024-04-12T05:56:17.841 INFO:tasks.workunit.client.1.smithi194.stdout:8/560: truncate d2/d8/d9/de/fa5 1746804 0 2024-04-12T05:56:17.843 INFO:tasks.workunit.client.1.smithi194.stdout:9/827: getdents d1/d21/d77/d98/d9f 0 2024-04-12T05:56:17.844 INFO:tasks.workunit.client.1.smithi194.stdout:8/561: truncate d2/d8/d66/d67/f74 802417 0 2024-04-12T05:56:17.845 INFO:tasks.workunit.client.1.smithi194.stdout:9/828: read d1/d41/fff [160300,89555] 0 2024-04-12T05:56:17.848 INFO:tasks.workunit.client.1.smithi194.stdout:9/829: mknod d1/d21/d39/d72/c115 0 2024-04-12T05:56:17.849 INFO:tasks.workunit.client.1.smithi194.stdout:9/830: chown d1/d19/d1f/f88 3 1 2024-04-12T05:56:17.850 INFO:tasks.workunit.client.1.smithi194.stdout:8/562: dread d2/d8/d15/d3b/f78 [0,4194304] 0 2024-04-12T05:56:17.852 INFO:tasks.workunit.client.1.smithi194.stdout:4/964: rename d2/d11/d1e/f42 to d2/d11/d93/da6/f150 0 2024-04-12T05:56:17.855 INFO:tasks.workunit.client.1.smithi194.stdout:9/831: symlink d1/d19/d1f/d20/d37/d84/l116 0 2024-04-12T05:56:17.855 INFO:tasks.workunit.client.1.smithi194.stdout:4/965: rmdir d2/d3/d1b/d32/d3a/dbc/d124 39 2024-04-12T05:56:17.857 INFO:tasks.workunit.client.1.smithi194.stdout:3/883: rename d0/d1/d10/fa8 to d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d10d/f128 0 2024-04-12T05:56:17.859 INFO:tasks.workunit.client.1.smithi194.stdout:4/966: creat d2/d11/d1e/d2f/d4a/d9c/d145/f151 x:0 0 0 2024-04-12T05:56:17.860 INFO:tasks.workunit.client.1.smithi194.stdout:3/884: dread - d0/d1/d10/d8c/ff9 zero size 2024-04-12T05:56:17.862 INFO:tasks.workunit.client.1.smithi194.stdout:8/563: rename d2/d8/d9/de/d10/l13 to d2/d8/d9/de/d10/d7d/la9 0 2024-04-12T05:56:17.862 INFO:tasks.workunit.client.1.smithi194.stdout:4/967: rmdir d2/d11/d1e/d2f/d4a/d9c/de7 39 2024-04-12T05:56:17.862 INFO:tasks.workunit.client.1.smithi194.stdout:3/885: mknod d0/d1/d4f/dd7/d120/c129 0 2024-04-12T05:56:17.864 INFO:tasks.workunit.client.1.smithi194.stdout:8/564: mkdir d2/d8/daa 0 2024-04-12T05:56:17.865 INFO:tasks.workunit.client.1.smithi194.stdout:3/886: stat d0/dcf/dde 0 2024-04-12T05:56:17.865 INFO:tasks.workunit.client.1.smithi194.stdout:4/968: symlink d2/d11/d46/d61/d136/l152 0 2024-04-12T05:56:17.866 INFO:tasks.workunit.client.1.smithi194.stdout:8/565: mkdir d2/d8/d15/d33/dab 0 2024-04-12T05:56:17.867 INFO:tasks.workunit.client.1.smithi194.stdout:8/566: stat d2/d8/d15/d33/d45/d52/c95 0 2024-04-12T05:56:17.869 INFO:tasks.workunit.client.1.smithi194.stdout:4/969: creat d2/d3/d51/de9/f153 x:0 0 0 2024-04-12T05:56:17.871 INFO:tasks.workunit.client.1.smithi194.stdout:8/567: symlink d2/d8/d15/d33/d45/lac 0 2024-04-12T05:56:17.872 INFO:tasks.workunit.client.1.smithi194.stdout:8/568: write d2/d8/d15/f62 [670398,14922] 0 2024-04-12T05:56:17.872 INFO:tasks.workunit.client.1.smithi194.stdout:4/970: mkdir d2/d3/d1b/d32/d154 0 2024-04-12T05:56:17.875 INFO:tasks.workunit.client.1.smithi194.stdout:3/887: getdents d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0 0 2024-04-12T05:56:17.877 INFO:tasks.workunit.client.1.smithi194.stdout:8/569: symlink d2/d8/lad 0 2024-04-12T05:56:17.880 INFO:tasks.workunit.client.1.smithi194.stdout:3/888: creat d0/d1/d74/d39/d4d/deb/d3d/f12a x:0 0 0 2024-04-12T05:56:17.880 INFO:tasks.workunit.client.1.smithi194.stdout:3/889: stat d0/d1/f60 0 2024-04-12T05:56:17.881 INFO:tasks.workunit.client.1.smithi194.stdout:4/971: mknod d2/d11/d46/d61/d108/c155 0 2024-04-12T05:56:17.889 INFO:tasks.workunit.client.1.smithi194.stdout:3/890: link d0/d1/d10/cb9 d0/d1/d74/d39/da2/c12b 0 2024-04-12T05:56:17.889 INFO:tasks.workunit.client.1.smithi194.stdout:4/972: link d2/d3/d50/da0/cbe d2/d11/d43/c156 0 2024-04-12T05:56:17.894 INFO:tasks.workunit.client.1.smithi194.stdout:3/891: mkdir d0/d1/d74/d39/d4d/deb/d1e/d33/d76/d9a/d12c 0 2024-04-12T05:56:17.895 INFO:tasks.workunit.client.1.smithi194.stdout:3/892: stat d0/d1/f60 0 2024-04-12T05:56:17.895 INFO:tasks.workunit.client.1.smithi194.stdout:8/570: dwrite d2/d8/d9/de/fa5 [0,4194304] 0 2024-04-12T05:56:17.898 INFO:tasks.workunit.client.1.smithi194.stdout:8/571: write d2/d8/d58/f76 [9614,72340] 0 2024-04-12T05:56:17.899 INFO:tasks.workunit.client.1.smithi194.stdout:8/572: readlink d2/d8/d9/de/l99 0 2024-04-12T05:56:17.904 INFO:tasks.workunit.client.1.smithi194.stdout:4/973: getdents d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5 0 2024-04-12T05:56:17.905 INFO:tasks.workunit.client.1.smithi194.stdout:3/893: dread d0/d1/d10/d8c/d8f/ddb/ff5 [0,4194304] 0 2024-04-12T05:56:17.907 INFO:tasks.workunit.client.1.smithi194.stdout:8/573: truncate d2/d8/d15/f16 1318278 0 2024-04-12T05:56:17.910 INFO:tasks.workunit.client.1.smithi194.stdout:3/894: creat d0/d1/d74/d39/f12d x:0 0 0 2024-04-12T05:56:17.913 INFO:tasks.workunit.client.1.smithi194.stdout:4/974: getdents d2/d11/d1e/d2f 0 2024-04-12T05:56:17.914 INFO:tasks.workunit.client.1.smithi194.stdout:8/574: rename d2/d8/d66/d67/f96 to d2/d8/d15/d35/fae 0 2024-04-12T05:56:17.915 INFO:tasks.workunit.client.1.smithi194.stdout:8/575: fsync d2/d8/d15/d33/f4b 0 2024-04-12T05:56:17.917 INFO:tasks.workunit.client.1.smithi194.stdout:3/895: dread - d0/d1/d74/d39/d4d/deb/d1e/d33/fca zero size 2024-04-12T05:56:17.921 INFO:tasks.workunit.client.1.smithi194.stdout:3/896: creat d0/d1/d74/d39/f12e x:0 0 0 2024-04-12T05:56:17.921 INFO:tasks.workunit.client.1.smithi194.stdout:4/975: getdents d2/d11/d1e/d2f/d4a/d9c/de7 0 2024-04-12T05:56:17.921 INFO:tasks.workunit.client.1.smithi194.stdout:8/576: link d2/d8/d9/d1a/f25 d2/d8/d9/d1a/faf 0 2024-04-12T05:56:17.923 INFO:tasks.workunit.client.1.smithi194.stdout:4/976: mknod d2/d3/d51/d13e/c157 0 2024-04-12T05:56:17.923 INFO:tasks.workunit.client.1.smithi194.stdout:8/577: creat d2/d8/d9/de/d38/fb0 x:0 0 0 2024-04-12T05:56:17.925 INFO:tasks.workunit.client.1.smithi194.stdout:8/578: fdatasync d2/d8/d15/d33/d45/d52/f7b 0 2024-04-12T05:56:17.927 INFO:tasks.workunit.client.1.smithi194.stdout:3/897: creat d0/d1/d74/d39/d4d/deb/d3d/f12f x:0 0 0 2024-04-12T05:56:17.929 INFO:tasks.workunit.client.1.smithi194.stdout:8/579: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:17.932 INFO:tasks.workunit.client.1.smithi194.stdout:4/977: dwrite d2/d3/f105 [0,4194304] 0 2024-04-12T05:56:17.935 INFO:tasks.workunit.client.1.smithi194.stdout:8/580: mknod d2/d8/d9/de/d10/d7d/cb1 0 2024-04-12T05:56:17.940 INFO:tasks.workunit.client.1.smithi194.stdout:8/581: getdents d2/d8/d9 0 2024-04-12T05:56:17.945 INFO:tasks.workunit.client.1.smithi194.stdout:4/978: rename d2/d11/d46/d61/d63 to d2/d3/d50/d158 0 2024-04-12T05:56:17.947 INFO:tasks.workunit.client.1.smithi194.stdout:8/582: fdatasync d2/d8/d9/de/f37 0 2024-04-12T05:56:17.952 INFO:tasks.workunit.client.1.smithi194.stdout:8/583: write d2/d8/d15/d35/fa2 [893135,61137] 0 2024-04-12T05:56:17.956 INFO:tasks.workunit.client.1.smithi194.stdout:8/584: write d2/d8/d66/f70 [617487,36231] 0 2024-04-12T05:56:17.962 INFO:tasks.workunit.client.1.smithi194.stdout:4/979: write d2/d11/d93/da6/fcc [1886926,76943] 0 2024-04-12T05:56:17.967 INFO:tasks.workunit.client.1.smithi194.stdout:8/585: rename d2/d8/d9/de/d10/d7d/f81 to d2/d8/d9/de/d38/fb2 0 2024-04-12T05:56:17.968 INFO:tasks.workunit.client.1.smithi194.stdout:5/824: sync 2024-04-12T05:56:17.968 INFO:tasks.workunit.client.1.smithi194.stdout:4/980: rmdir d2/d11/d1e/d25 39 2024-04-12T05:56:17.971 INFO:tasks.workunit.client.1.smithi194.stdout:5/825: symlink d1/d5c/d95/df2/d107/l113 0 2024-04-12T05:56:17.973 INFO:tasks.workunit.client.1.smithi194.stdout:4/981: unlink d2/d11/d1f/l107 0 2024-04-12T05:56:17.983 INFO:tasks.workunit.client.1.smithi194.stdout:4/982: rmdir d2/d11/d1e/d2f/d4a/d9c/d145 39 2024-04-12T05:56:17.983 INFO:tasks.workunit.client.1.smithi194.stdout:8/586: truncate d2/d8/d9/de/d10/d4e/f60 2525322 0 2024-04-12T05:56:17.985 INFO:tasks.workunit.client.1.smithi194.stdout:8/587: mknod d2/d8/d15/d33/d49/cb3 0 2024-04-12T05:56:17.985 INFO:tasks.workunit.client.1.smithi194.stdout:4/983: fdatasync d2/d11/f88 0 2024-04-12T05:56:17.985 INFO:tasks.workunit.client.1.smithi194.stdout:5/826: dread d1/d4/d75/fb6 [0,4194304] 0 2024-04-12T05:56:17.986 INFO:tasks.workunit.client.1.smithi194.stdout:5/827: chown f0 1 1 2024-04-12T05:56:17.988 INFO:tasks.workunit.client.1.smithi194.stdout:8/588: mknod d2/d8/d15/d33/dab/cb4 0 2024-04-12T05:56:17.988 INFO:tasks.workunit.client.1.smithi194.stdout:8/589: chown d2/d8/d9 19320695 1 2024-04-12T05:56:17.988 INFO:tasks.workunit.client.1.smithi194.stdout:4/984: rename d2/d85/d11d/d12d/fca to d2/d11/d93/da6/f159 0 2024-04-12T05:56:17.989 INFO:tasks.workunit.client.1.smithi194.stdout:5/828: creat d1/d4/d6/db/d38/dff/f114 x:0 0 0 2024-04-12T05:56:17.990 INFO:tasks.workunit.client.1.smithi194.stdout:8/590: rename d2/d8 to d2/d8/d66/d67/db5 22 2024-04-12T05:56:17.990 INFO:tasks.workunit.client.1.smithi194.stdout:4/985: readlink d2/d3/d51/de9/l13a 0 2024-04-12T05:56:17.991 INFO:tasks.workunit.client.1.smithi194.stdout:5/829: chown d1/d4/d6/db/d2b/f66 890 1 2024-04-12T05:56:17.993 INFO:tasks.workunit.client.1.smithi194.stdout:5/830: chown d1/d4/d6/d88/la5 7371 1 2024-04-12T05:56:17.993 INFO:tasks.workunit.client.1.smithi194.stdout:4/986: chown d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/d144/d13d 0 1 2024-04-12T05:56:17.993 INFO:tasks.workunit.client.1.smithi194.stdout:8/591: unlink d2/d8/d9/de/d38/d8f/d94/l9d 0 2024-04-12T05:56:17.996 INFO:tasks.workunit.client.1.smithi194.stdout:5/831: mkdir d1/d4/d6/d115 0 2024-04-12T05:56:17.997 INFO:tasks.workunit.client.1.smithi194.stdout:8/592: symlink d2/d8/d9/de/lb6 0 2024-04-12T05:56:17.997 INFO:tasks.workunit.client.1.smithi194.stdout:4/987: rmdir d2/d3/d58/d82 39 2024-04-12T05:56:17.997 INFO:tasks.workunit.client.1.smithi194.stdout:5/832: write d1/d4/d6/d88/dc1/fe9 [2750469,112395] 0 2024-04-12T05:56:18.000 INFO:tasks.workunit.client.1.smithi194.stdout:8/593: truncate d2/d8/d15/d35/fae 919898 0 2024-04-12T05:56:18.003 INFO:tasks.workunit.client.1.smithi194.stdout:8/594: rename d2/d8/d9/de/d10 to d2/d8/d9/de/d10/d4e/db7 22 2024-04-12T05:56:18.004 INFO:tasks.workunit.client.1.smithi194.stdout:5/833: link d1/d4/d6/d34/d52/c7f d1/d4/d9e/c116 0 2024-04-12T05:56:18.005 INFO:tasks.workunit.client.1.smithi194.stdout:8/595: read - d2/d8/d9/d1a/f6f zero size 2024-04-12T05:56:18.006 INFO:tasks.workunit.client.1.smithi194.stdout:4/988: link d2/d11/d1e/d2f/d4a/f97 d2/d3/d1b/d32/d154/f15a 0 2024-04-12T05:56:18.011 INFO:tasks.workunit.client.1.smithi194.stdout:8/596: rename d2/d8/d15/c83 to d2/d8/d9/de/d10/d7d/cb8 0 2024-04-12T05:56:18.016 INFO:tasks.workunit.client.1.smithi194.stdout:8/597: fdatasync d2/d8/d9/d1a/f2b 0 2024-04-12T05:56:18.017 INFO:tasks.workunit.client.1.smithi194.stdout:8/598: dread - d2/d8/d15/d35/f3f zero size 2024-04-12T05:56:18.019 INFO:tasks.workunit.client.1.smithi194.stdout:5/834: write d1/d4/d6/d34/fd4 [141810,55978] 0 2024-04-12T05:56:18.023 INFO:tasks.workunit.client.1.smithi194.stdout:5/835: unlink d1/d5c/d95/df2/c62 0 2024-04-12T05:56:18.026 INFO:tasks.workunit.client.1.smithi194.stdout:5/836: creat d1/d4/d9e/f117 x:0 0 0 2024-04-12T05:56:18.035 INFO:tasks.workunit.client.1.smithi194.stdout:5/837: write d1/d5c/d95/df2/f3e [912751,92135] 0 2024-04-12T05:56:18.036 INFO:tasks.workunit.client.1.smithi194.stdout:5/838: chown d1/d5c/d95/c18 610299153 1 2024-04-12T05:56:18.038 INFO:tasks.workunit.client.1.smithi194.stdout:5/839: creat d1/d4/dd7/f118 x:0 0 0 2024-04-12T05:56:18.039 INFO:tasks.workunit.client.1.smithi194.stdout:5/840: chown d1/d4/d84/lef 1131 1 2024-04-12T05:56:18.043 INFO:tasks.workunit.client.1.smithi194.stdout:5/841: dread d1/d5c/d95/f9f [8388608,4194304] 0 2024-04-12T05:56:18.043 INFO:tasks.workunit.client.1.smithi194.stdout:5/842: readlink d1/ld5 0 2024-04-12T05:56:18.043 INFO:tasks.workunit.client.1.smithi194.stdout:5/843: stat d1/ld5 0 2024-04-12T05:56:18.044 INFO:tasks.workunit.client.1.smithi194.stdout:5/844: chown d1/d4/d6/d88/f3c 4146 1 2024-04-12T05:56:18.159 INFO:tasks.workunit.client.1.smithi194.stdout:5/845: mknod d1/d4/d6/d115/c119 0 2024-04-12T05:56:18.161 INFO:tasks.workunit.client.1.smithi194.stdout:5/846: truncate d1/d4/d6/d6d/ffb 264924 0 2024-04-12T05:56:18.166 INFO:tasks.workunit.client.1.smithi194.stdout:5/847: dwrite d1/d4/d6/d8e/f81 [0,4194304] 0 2024-04-12T05:56:18.167 INFO:tasks.workunit.client.1.smithi194.stdout:5/848: stat d1/d4/d6/db/d2b/f66 0 2024-04-12T05:56:18.169 INFO:tasks.workunit.client.1.smithi194.stdout:5/849: dread - d1/d4/d6/db/d38/dff/f114 zero size 2024-04-12T05:56:18.175 INFO:tasks.workunit.client.1.smithi194.stdout:5/850: dread d1/d4/d6/d6d/f6b [0,4194304] 0 2024-04-12T05:56:18.189 INFO:tasks.workunit.client.1.smithi194.stdout:5/851: write d1/d4/d6/db/d15/d26/f7c [4033348,117586] 0 2024-04-12T05:56:18.209 INFO:tasks.workunit.client.1.smithi194.stdout:5/852: dwrite d1/d4/d6/db/d15/d26/fd8 [0,4194304] 0 2024-04-12T05:56:18.211 INFO:tasks.workunit.client.1.smithi194.stdout:5/853: fsync d1/f3 0 2024-04-12T05:56:18.219 INFO:tasks.workunit.client.1.smithi194.stdout:1/815: sync 2024-04-12T05:56:18.219 INFO:tasks.workunit.client.1.smithi194.stdout:1/816: truncate d0/d11/fe3 659196 0 2024-04-12T05:56:18.223 INFO:tasks.workunit.client.1.smithi194.stdout:5/854: unlink d1/d4/d6/db/d15/d26/c2f 0 2024-04-12T05:56:18.225 INFO:tasks.workunit.client.1.smithi194.stdout:5/855: write d1/d4/f5 [153221,126672] 0 2024-04-12T05:56:18.233 INFO:tasks.workunit.client.1.smithi194.stdout:5/856: truncate d1/d4/d84/fb5 467972 0 2024-04-12T05:56:18.236 INFO:tasks.workunit.client.1.smithi194.stdout:5/857: dread - d1/d5c/d95/fc6 zero size 2024-04-12T05:56:18.236 INFO:tasks.workunit.client.1.smithi194.stdout:1/817: dwrite d0/d11/d18/d19/f4e [4194304,4194304] 0 2024-04-12T05:56:18.266 INFO:tasks.workunit.client.1.smithi194.stdout:5/858: dwrite d1/d4/d75/ff6 [0,4194304] 0 2024-04-12T05:56:18.280 INFO:tasks.workunit.client.1.smithi194.stdout:1/818: dread d0/d11/d18/d19/d1a/ded/ff4 [0,4194304] 0 2024-04-12T05:56:18.281 INFO:tasks.workunit.client.1.smithi194.stdout:5/859: dread d1/d4/d6/db/d15/d26/fd8 [0,4194304] 0 2024-04-12T05:56:18.292 INFO:tasks.workunit.client.1.smithi194.stdout:5/860: read d1/d4/d6/f76 [602178,1143] 0 2024-04-12T05:56:18.303 INFO:tasks.workunit.client.1.smithi194.stdout:2/749: sync 2024-04-12T05:56:18.303 INFO:tasks.workunit.client.1.smithi194.stdout:7/689: sync 2024-04-12T05:56:18.303 INFO:tasks.workunit.client.1.smithi194.stdout:6/784: sync 2024-04-12T05:56:18.303 INFO:tasks.workunit.client.1.smithi194.stdout:0/722: sync 2024-04-12T05:56:18.303 INFO:tasks.workunit.client.1.smithi194.stdout:9/832: sync 2024-04-12T05:56:18.307 INFO:tasks.workunit.client.1.smithi194.stdout:1/819: truncate d0/d11/d18/fc3 1060234 0 2024-04-12T05:56:18.317 INFO:tasks.workunit.client.1.smithi194.stdout:7/690: rename d3/d27/d22/f38 to d3/d27/d22/fe0 0 2024-04-12T05:56:18.328 INFO:tasks.workunit.client.1.smithi194.stdout:2/750: mkdir d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103 0 2024-04-12T05:56:18.330 INFO:tasks.workunit.client.1.smithi194.stdout:5/861: link d1/d5c/d95/f72 d1/d4/d9e/ddb/f11a 0 2024-04-12T05:56:18.332 INFO:tasks.workunit.client.1.smithi194.stdout:0/723: dwrite d0/da/d1e/d1a/f6d [0,4194304] 0 2024-04-12T05:56:18.344 INFO:tasks.workunit.client.1.smithi194.stdout:9/833: dread d1/d21/f61 [0,4194304] 0 2024-04-12T05:56:18.346 INFO:tasks.workunit.client.1.smithi194.stdout:8/599: sync 2024-04-12T05:56:18.346 INFO:tasks.workunit.client.1.smithi194.stdout:4/989: sync 2024-04-12T05:56:18.346 INFO:tasks.workunit.client.1.smithi194.stdout:3/898: sync 2024-04-12T05:56:18.354 INFO:tasks.workunit.client.1.smithi194.stdout:5/862: dwrite d1/d4/d6/d88/f2c [0,4194304] 0 2024-04-12T05:56:18.355 INFO:tasks.workunit.client.1.smithi194.stdout:6/785: dwrite d3/f1f [0,4194304] 0 2024-04-12T05:56:18.359 INFO:tasks.workunit.client.1.smithi194.stdout:2/751: creat d7/d22/f104 x:0 0 0 2024-04-12T05:56:18.360 INFO:tasks.workunit.client.1.smithi194.stdout:1/820: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b [0,4194304] 0 2024-04-12T05:56:18.365 INFO:tasks.workunit.client.1.smithi194.stdout:0/724: write d0/da/d1e/d1a/d24/f61 [12618645,120530] 0 2024-04-12T05:56:18.368 INFO:tasks.workunit.client.1.smithi194.stdout:5/863: write d1/d4/d6/db/d15/d26/dbb/ff1 [6224,27104] 0 2024-04-12T05:56:18.369 INFO:tasks.workunit.client.1.smithi194.stdout:0/725: chown d0/da/d1e/d27/fe6 31 1 2024-04-12T05:56:18.373 INFO:tasks.workunit.client.1.smithi194.stdout:1/821: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/c102 0 2024-04-12T05:56:18.383 INFO:tasks.workunit.client.1.smithi194.stdout:8/600: mkdir d2/d8/d15/d33/d49/db9 0 2024-04-12T05:56:18.390 INFO:tasks.workunit.client.1.smithi194.stdout:5/864: mkdir d1/d5c/da7/d10b/d11b 0 2024-04-12T05:56:18.394 INFO:tasks.workunit.client.1.smithi194.stdout:9/834: dwrite d1/d41/dba/fe8 [4194304,4194304] 0 2024-04-12T05:56:18.396 INFO:tasks.workunit.client.1.smithi194.stdout:8/601: unlink d2/d8/d15/d33/dab/cb4 0 2024-04-12T05:56:18.396 INFO:tasks.workunit.client.1.smithi194.stdout:7/691: dread d3/d16/d7b/fb9 [0,4194304] 0 2024-04-12T05:56:18.401 INFO:tasks.workunit.client.1.smithi194.stdout:3/899: dwrite d0/d1/f57 [4194304,4194304] 0 2024-04-12T05:56:18.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd='[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]': finished 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: mgrmap e27: smithi008.ixmnlp(active, starting, since 0.00912627s) 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:56:18.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:56:18.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:18.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:56:18.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:18.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:56:18.406 INFO:tasks.workunit.client.1.smithi194.stdout:2/752: dwrite d7/d22/d42/fe8 [0,4194304] 0 2024-04-12T05:56:18.407 INFO:tasks.workunit.client.1.smithi194.stdout:2/753: chown d7/d12/d25/d5d/d47/dac/ff6 140115 1 2024-04-12T05:56:18.407 INFO:tasks.workunit.client.1.smithi194.stdout:6/786: dread d3/d15/f22 [0,4194304] 0 2024-04-12T05:56:18.409 INFO:tasks.workunit.client.1.smithi194.stdout:5/865: chown d1/d4/d6/f68 345536 1 2024-04-12T05:56:18.410 INFO:tasks.workunit.client.1.smithi194.stdout:4/990: link d2/d11/d1e/d2f/c12b d2/d11/d93/da6/dab/d5d/dae/db2/d57/dc5/dfb/d146/c15b 0 2024-04-12T05:56:18.412 INFO:tasks.workunit.client.1.smithi194.stdout:3/900: readlink d0/d1/d10/lb2 0 2024-04-12T05:56:18.412 INFO:tasks.workunit.client.1.smithi194.stdout:0/726: write d0/da/f3e [6467666,25523] 0 2024-04-12T05:56:18.429 INFO:tasks.workunit.client.1.smithi194.stdout:2/754: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f105 x:0 0 0 2024-04-12T05:56:18.431 INFO:tasks.workunit.client.1.smithi194.stdout:1/822: dwrite d0/d11/f83 [0,4194304] 0 2024-04-12T05:56:18.431 INFO:tasks.workunit.client.1.smithi194.stdout:6/787: truncate d3/d2f/d4e/d9a/dd8/f6a 696692 0 2024-04-12T05:56:18.434 INFO:tasks.workunit.client.1.smithi194.stdout:9/835: link d1/da6/d10f/f9c d1/d41/f117 0 2024-04-12T05:56:18.434 INFO:tasks.workunit.client.1.smithi194.stdout:8/602: dwrite d2/d8/d9/de/d38/f3a [0,4194304] 0 2024-04-12T05:56:18.436 INFO:tasks.workunit.client.1.smithi194.stdout:5/866: rename d1/d4/c30 to d1/d4/dd7/dfe/c11c 0 2024-04-12T05:56:18.437 INFO:tasks.workunit.client.1.smithi194.stdout:4/991: write d2/d11/f52 [2881275,34330] 0 2024-04-12T05:56:18.438 INFO:tasks.workunit.client.1.smithi194.stdout:4/992: write d2/d3/d51/d13e/f148 [845152,117664] 0 2024-04-12T05:56:18.439 INFO:tasks.workunit.client.1.smithi194.stdout:8/603: chown d2/d8/lad 30268 1 2024-04-12T05:56:18.443 INFO:tasks.workunit.client.1.smithi194.stdout:8/604: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:18.444 INFO:tasks.workunit.client.1.smithi194.stdout:9/836: write d1/d19/d1f/d20/fcd [2931586,26067] 0 2024-04-12T05:56:18.450 INFO:tasks.workunit.client.1.smithi194.stdout:3/901: dwrite d0/d1/d74/d39/d4d/deb/d3d/f12f [0,4194304] 0 2024-04-12T05:56:18.454 INFO:tasks.workunit.client.1.smithi194.stdout:1/823: creat d0/d11/ddd/f103 x:0 0 0 2024-04-12T05:56:18.454 INFO:tasks.workunit.client.1.smithi194.stdout:5/867: symlink d1/d5c/da7/l11d 0 2024-04-12T05:56:18.461 INFO:tasks.workunit.client.1.smithi194.stdout:1/824: truncate d0/d11/ddd/f103 252481 0 2024-04-12T05:56:18.461 INFO:tasks.workunit.client.1.smithi194.stdout:7/692: link d3/d27/d6a/l4f d3/d16/d23/d37/le1 0 2024-04-12T05:56:18.461 INFO:tasks.workunit.client.1.smithi194.stdout:6/788: creat d3/d2d/d33/d52/ff5 x:0 0 0 2024-04-12T05:56:18.462 INFO:tasks.workunit.client.1.smithi194.stdout:3/902: mkdir d0/d1/d10/d8c/d130 0 2024-04-12T05:56:18.463 INFO:tasks.workunit.client.1.smithi194.stdout:1/825: stat d0/d11/d18/d19/d1a/d6a/dac/db5/df8 0 2024-04-12T05:56:18.463 INFO:tasks.workunit.client.1.smithi194.stdout:7/693: read - d3/d16/d23/fc8 zero size 2024-04-12T05:56:18.465 INFO:tasks.workunit.client.1.smithi194.stdout:4/993: getdents d2/d3/d1b/d32 0 2024-04-12T05:56:18.466 INFO:tasks.workunit.client.1.smithi194.stdout:6/789: symlink d3/d2f/d4e/d68/dad/dc4/lf6 0 2024-04-12T05:56:18.472 INFO:tasks.workunit.client.1.smithi194.stdout:0/727: dread d0/da/d1e/d27/fa0 [0,4194304] 0 2024-04-12T05:56:18.472 INFO:tasks.workunit.client.1.smithi194.stdout:3/903: mknod d0/d1/d74/d39/d4d/deb/db4/dd9/c131 0 2024-04-12T05:56:18.473 INFO:tasks.workunit.client.1.smithi194.stdout:3/904: readlink d0/l3 0 2024-04-12T05:56:18.477 INFO:tasks.workunit.client.1.smithi194.stdout:2/755: truncate d7/d12/d25/d5d/d47/d52/d7b/fa6 3564744 0 2024-04-12T05:56:18.480 INFO:tasks.workunit.client.1.smithi194.stdout:8/605: write d2/f43 [17232,124048] 0 2024-04-12T05:56:18.480 INFO:tasks.workunit.client.1.smithi194.stdout:5/868: write d1/d4/d4b/f7b [134146,16630] 0 2024-04-12T05:56:18.493 INFO:tasks.workunit.client.1.smithi194.stdout:4/994: dwrite d2/d11/f52 [0,4194304] 0 2024-04-12T05:56:18.496 INFO:tasks.workunit.client.1.smithi194.stdout:6/790: dwrite d3/f3b [0,4194304] 0 2024-04-12T05:56:18.498 INFO:tasks.workunit.client.1.smithi194.stdout:9/837: dwrite d1/d4e/f6d [0,4194304] 0 2024-04-12T05:56:18.501 INFO:tasks.workunit.client.1.smithi194.stdout:7/694: dwrite d3/d27/d6a/fdf [0,4194304] 0 2024-04-12T05:56:18.508 INFO:tasks.workunit.client.1.smithi194.stdout:5/869: dread - d1/d4/dd7/f118 zero size 2024-04-12T05:56:18.508 INFO:tasks.workunit.client.1.smithi194.stdout:2/756: getdents d7/d22/dc9/de5 0 2024-04-12T05:56:18.509 INFO:tasks.workunit.client.1.smithi194.stdout:0/728: rename d0/da/d1e/d1a/d24/c43 to d0/da/d1e/d1a/d24/de1/ced 0 2024-04-12T05:56:18.514 INFO:tasks.workunit.client.1.smithi194.stdout:4/995: unlink d2/d3/d50/d158/da2/cd6 0 2024-04-12T05:56:18.514 INFO:tasks.workunit.client.1.smithi194.stdout:5/870: truncate d1/d4/d9e/f117 536822 0 2024-04-12T05:56:18.515 INFO:tasks.workunit.client.1.smithi194.stdout:1/826: dwrite d0/d11/d18/d19/d1c/d67/fb7 [0,4194304] 0 2024-04-12T05:56:18.523 INFO:tasks.workunit.client.1.smithi194.stdout:7/695: mknod d3/d16/d7b/dc3/ce2 0 2024-04-12T05:56:18.523 INFO:tasks.workunit.client.1.smithi194.stdout:5/871: write d1/d4/d6/db/d2b/fcd [2861333,104038] 0 2024-04-12T05:56:18.526 INFO:tasks.workunit.client.1.smithi194.stdout:9/838: link d1/da6/d10f/f99 d1/d21/d77/d98/d9f/dde/f118 0 2024-04-12T05:56:18.528 INFO:tasks.workunit.client.1.smithi194.stdout:0/729: creat d0/da/d1e/d1a/d24/d4b/fee x:0 0 0 2024-04-12T05:56:18.530 INFO:tasks.workunit.client.1.smithi194.stdout:6/791: write d3/d2d/d33/d52/f5a [157335,97004] 0 2024-04-12T05:56:18.531 INFO:tasks.workunit.client.1.smithi194.stdout:9/839: write d1/d19/d1f/d20/d37/d60/f107 [933535,118851] 0 2024-04-12T05:56:18.531 INFO:tasks.workunit.client.1.smithi194.stdout:8/606: write d2/d8/f36 [2759577,81578] 0 2024-04-12T05:56:18.533 INFO:tasks.workunit.client.1.smithi194.stdout:1/827: rmdir d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5 39 2024-04-12T05:56:18.534 INFO:tasks.workunit.client.1.smithi194.stdout:6/792: chown d3/d2d/d33/d52/d5c 8214861 1 2024-04-12T05:56:18.536 INFO:tasks.workunit.client.1.smithi194.stdout:4/996: getdents d2/d3/d1b 0 2024-04-12T05:56:18.537 INFO:tasks.workunit.client.1.smithi194.stdout:2/757: link d7/f48 d7/d12/d25/d5d/d47/dac/db3/dd7/df8/f106 0 2024-04-12T05:56:18.543 INFO:tasks.workunit.client.1.smithi194.stdout:3/905: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f37 [0,4194304] 0 2024-04-12T05:56:18.546 INFO:tasks.workunit.client.1.smithi194.stdout:8/607: symlink d2/d8/d9/lba 0 2024-04-12T05:56:18.550 INFO:tasks.workunit.client.1.smithi194.stdout:5/872: unlink d1/d4/dd7/dfe/c11c 0 2024-04-12T05:56:18.550 INFO:tasks.workunit.client.1.smithi194.stdout:6/793: rmdir d3/d2f/d4e/d68 39 2024-04-12T05:56:18.551 INFO:tasks.workunit.client.1.smithi194.stdout:7/696: dwrite d3/d16/f45 [0,4194304] 0 2024-04-12T05:56:18.557 INFO:tasks.workunit.client.1.smithi194.stdout:0/730: dwrite d0/da/d1e/f41 [8388608,4194304] 0 2024-04-12T05:56:18.557 INFO:tasks.workunit.client.1.smithi194.stdout:5/873: creat d1/d4/d9e/f11e x:0 0 0 2024-04-12T05:56:18.558 INFO:tasks.workunit.client.1.smithi194.stdout:3/906: mkdir d0/d1/d4f/dd7/d132 0 2024-04-12T05:56:18.562 INFO:tasks.workunit.client.1.smithi194.stdout:0/731: chown d0/da/d1e/d1a/f64 123754 1 2024-04-12T05:56:18.564 INFO:tasks.workunit.client.1.smithi194.stdout:1/828: getdents d0/d11/d18/d19/d1a/d6a/d27/d9e 0 2024-04-12T05:56:18.565 INFO:tasks.workunit.client.1.smithi194.stdout:5/874: write d1/d4/d6/db/fc [3916336,17023] 0 2024-04-12T05:56:18.568 INFO:tasks.workunit.client.1.smithi194.stdout:2/758: dwrite d7/d12/d1a/d68/f95 [4194304,4194304] 0 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' 2024-04-12T05:56:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 172.21.15.194:0/1702599401' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.24385 ' entity='mgr.smithi194.xbiyaj' cmd='[{"prefix": "mgr fail", "who": "smithi194.xbiyaj"}]': finished 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: mgrmap e27: smithi008.ixmnlp(active, starting, since 0.00912627s) 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi008.ixmnlp", "id": "smithi008.ixmnlp"}]: dispatch 2024-04-12T05:56:18.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: Manager daemon smithi008.ixmnlp is now available 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:18.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/mirror_snapshot_schedule"}]: dispatch 2024-04-12T05:56:18.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:18.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi008.ixmnlp/trash_purge_schedule"}]: dispatch 2024-04-12T05:56:18.573 INFO:tasks.workunit.client.1.smithi194.stdout:3/907: write d0/d1/d74/d39/d4d/deb/d53/f75 [333906,60599] 0 2024-04-12T05:56:18.573 INFO:tasks.workunit.client.1.smithi194.stdout:8/608: write d2/d8/d9/de/d10/d4e/f60 [1123336,100160] 0 2024-04-12T05:56:18.573 INFO:tasks.workunit.client.1.smithi194.stdout:9/840: truncate d1/d19/d1f/d20/f7b 1034147 0 2024-04-12T05:56:18.585 INFO:tasks.workunit.client.1.smithi194.stdout:4/997: truncate d2/d11/d1e/d25/fd7 3310532 0 2024-04-12T05:56:18.586 INFO:tasks.workunit.client.1.smithi194.stdout:3/908: chown d0/d1/d74/d39/d4d/d63/db5/fd8 36846 1 2024-04-12T05:56:18.588 INFO:tasks.workunit.client.1.smithi194.stdout:4/998: dread - d2/d11/d93/da6/dab/d5d/dae/de1/ddc/f14a zero size 2024-04-12T05:56:18.588 INFO:tasks.workunit.client.1.smithi194.stdout:1/829: mkdir d0/d11/d18/d19/d1a/d6a/dac/d104 0 2024-04-12T05:56:18.589 INFO:tasks.workunit.client.1.smithi194.stdout:7/697: link d3/d27/l5b d3/d16/d23/d57/d72/d81/le3 0 2024-04-12T05:56:18.589 INFO:tasks.workunit.client.1.smithi194.stdout:6/794: rename d3/d2d/d33/f4d to d3/d2f/d4e/d63/ff7 0 2024-04-12T05:56:18.594 INFO:tasks.workunit.client.1.smithi194.stdout:2/759: creat d7/d12/d1a/d86/f107 x:0 0 0 2024-04-12T05:56:18.594 INFO:tasks.workunit.client.1.smithi194.stdout:0/732: getdents d0/d5f/deb 0 2024-04-12T05:56:18.595 INFO:tasks.workunit.client.1.smithi194.stdout:5/875: write d1/fa1 [202740,4561] 0 2024-04-12T05:56:18.595 INFO:tasks.workunit.client.1.smithi194.stdout:9/841: symlink d1/d21/d77/d98/dab/l119 0 2024-04-12T05:56:18.597 INFO:tasks.workunit.client.1.smithi194.stdout:4/999: mknod d2/d3/d1b/d32/d3a/c15c 0 2024-04-12T05:56:18.598 INFO:tasks.workunit.client.1.smithi194.stdout:1/830: creat d0/d11/d18/d19/d1a/d6a/d27/da8/f105 x:0 0 0 2024-04-12T05:56:18.602 INFO:tasks.workunit.client.1.smithi194.stdout:7/698: creat d3/d16/d23/d37/d3d/d56/dd0/fe4 x:0 0 0 2024-04-12T05:56:18.605 INFO:tasks.workunit.client.1.smithi194.stdout:5/876: unlink d1/d4/f5 0 2024-04-12T05:56:18.605 INFO:tasks.workunit.client.1.smithi194.stdout:6/795: creat d3/d15/d2b/d48/d6f/d7a/d8f/d98/ff8 x:0 0 0 2024-04-12T05:56:18.606 INFO:tasks.workunit.client.1.smithi194.stdout:0/733: read d0/da/d1e/d27/d54/f7b [5396483,116418] 0 2024-04-12T05:56:18.608 INFO:tasks.workunit.client.1.smithi194.stdout:1/831: dread - d0/d11/d18/d19/d1c/f9a zero size 2024-04-12T05:56:18.612 INFO:tasks.workunit.client.1.smithi194.stdout:2/760: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/f81 [542232,28222] 0 2024-04-12T05:56:18.612 INFO:tasks.workunit.client.1.smithi194.stdout:8/609: dwrite d2/d8/d9/de/fa5 [0,4194304] 0 2024-04-12T05:56:18.614 INFO:tasks.workunit.client.1.smithi194.stdout:8/610: write d2/f12 [1476251,48107] 0 2024-04-12T05:56:18.615 INFO:tasks.workunit.client.1.smithi194.stdout:5/877: rename d1/d4/d75/cc4 to d1/d4/dd7/dfe/c11f 0 2024-04-12T05:56:18.617 INFO:tasks.workunit.client.1.smithi194.stdout:7/699: dread - d3/d16/d23/d57/fd4 zero size 2024-04-12T05:56:18.618 INFO:tasks.workunit.client.1.smithi194.stdout:8/611: read - d2/d8/d9/de/d86/f8d zero size 2024-04-12T05:56:18.619 INFO:tasks.workunit.client.1.smithi194.stdout:1/832: readlink d0/d11/d18/d35/l60 0 2024-04-12T05:56:18.620 INFO:tasks.workunit.client.1.smithi194.stdout:5/878: dread - d1/d4/d6/d88/fae zero size 2024-04-12T05:56:18.622 INFO:tasks.workunit.client.1.smithi194.stdout:7/700: write d3/d27/d6a/dda/d74/fd5 [403864,74373] 0 2024-04-12T05:56:18.623 INFO:tasks.workunit.client.1.smithi194.stdout:3/909: dread d0/d1/d74/d39/d4d/deb/d53/f105 [0,4194304] 0 2024-04-12T05:56:18.627 INFO:tasks.workunit.client.1.smithi194.stdout:2/761: read d7/d12/d6d/fa5 [1763685,102691] 0 2024-04-12T05:56:18.642 INFO:tasks.workunit.client.1.smithi194.stdout:3/910: creat d0/d1/d4f/f133 x:0 0 0 2024-04-12T05:56:18.646 INFO:tasks.workunit.client.1.smithi194.stdout:2/762: write d7/d12/d25/d5d/d47/d52/f60 [842935,69526] 0 2024-04-12T05:56:18.650 INFO:tasks.workunit.client.1.smithi194.stdout:5/879: dwrite d1/d4/d6/d6d/ffb [0,4194304] 0 2024-04-12T05:56:18.651 INFO:tasks.workunit.client.1.smithi194.stdout:0/734: dwrite d0/d3b/f4c [0,4194304] 0 2024-04-12T05:56:18.662 INFO:tasks.workunit.client.1.smithi194.stdout:1/833: creat d0/d11/d18/d19/d1a/d6a/dac/db5/df8/f106 x:0 0 0 2024-04-12T05:56:18.663 INFO:tasks.workunit.client.1.smithi194.stdout:1/834: fdatasync d0/d11/d18/d19/d1a/d6a/d27/d9e/fad 0 2024-04-12T05:56:18.664 INFO:tasks.workunit.client.1.smithi194.stdout:2/763: symlink d7/d12/d1a/de9/l108 0 2024-04-12T05:56:18.665 INFO:tasks.workunit.client.1.smithi194.stdout:2/764: dread - d7/d22/f104 zero size 2024-04-12T05:56:18.666 INFO:tasks.workunit.client.1.smithi194.stdout:0/735: mkdir d0/da/d1e/d1a/d24/da8/def 0 2024-04-12T05:56:18.666 INFO:tasks.workunit.client.1.smithi194.stdout:7/701: link d3/d27/ce d3/d27/d6a/ce5 0 2024-04-12T05:56:18.670 INFO:tasks.workunit.client.1.smithi194.stdout:2/765: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/f81 [273525,127525] 0 2024-04-12T05:56:18.671 INFO:tasks.workunit.client.1.smithi194.stdout:5/880: creat d1/d5c/d95/df2/d107/f120 x:0 0 0 2024-04-12T05:56:18.672 INFO:tasks.workunit.client.1.smithi194.stdout:1/835: mknod d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/c107 0 2024-04-12T05:56:18.673 INFO:tasks.workunit.client.1.smithi194.stdout:7/702: mknod d3/d27/d6a/ce6 0 2024-04-12T05:56:18.676 INFO:tasks.workunit.client.1.smithi194.stdout:0/736: rename d0/d3b/d52/d7c to d0/de3/df0 0 2024-04-12T05:56:18.680 INFO:tasks.workunit.client.1.smithi194.stdout:5/881: mkdir d1/d4/d6/d34/d121 0 2024-04-12T05:56:18.683 INFO:tasks.workunit.client.1.smithi194.stdout:0/737: chown d0/da/d1e/d8f/ccd 15596 1 2024-04-12T05:56:18.685 INFO:tasks.workunit.client.1.smithi194.stdout:1/836: symlink d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/l108 0 2024-04-12T05:56:18.688 INFO:tasks.workunit.client.1.smithi194.stdout:0/738: mknod d0/d5f/d73/cf1 0 2024-04-12T05:56:18.689 INFO:tasks.workunit.client.1.smithi194.stdout:1/837: dread - d0/d11/d18/d19/fcc zero size 2024-04-12T05:56:18.693 INFO:tasks.workunit.client.1.smithi194.stdout:0/739: rmdir d0/da/d1e/d27/d87 39 2024-04-12T05:56:18.695 INFO:tasks.workunit.client.1.smithi194.stdout:7/703: getdents d3/d16/d23/d37 0 2024-04-12T05:56:18.700 INFO:tasks.workunit.client.1.smithi194.stdout:0/740: getdents d0/da/d1e/d8f 0 2024-04-12T05:56:18.703 INFO:tasks.workunit.client.1.smithi194.stdout:7/704: rename d3/d16/d23/d57/d72/d81/d3a/d8f/cc2 to d3/d16/d23/d57/d72/d81/d3a/dcc/ce7 0 2024-04-12T05:56:18.704 INFO:tasks.workunit.client.1.smithi194.stdout:1/838: dread d0/d11/d18/d19/d1c/f22 [0,4194304] 0 2024-04-12T05:56:18.704 INFO:tasks.workunit.client.1.smithi194.stdout:0/741: creat d0/da/da7/ff2 x:0 0 0 2024-04-12T05:56:18.705 INFO:tasks.workunit.client.1.smithi194.stdout:7/705: chown d3/d16/d23/d37/l3f 1276605833 1 2024-04-12T05:56:18.707 INFO:tasks.workunit.client.1.smithi194.stdout:0/742: symlink d0/da/lf3 0 2024-04-12T05:56:18.708 INFO:tasks.workunit.client.1.smithi194.stdout:8/612: write d2/d8/d9/de/f1d [1343552,61167] 0 2024-04-12T05:56:18.710 INFO:tasks.workunit.client.1.smithi194.stdout:1/839: unlink d0/d11/d18/d19/d1a/d6a/dac/db5/ff9 0 2024-04-12T05:56:18.711 INFO:tasks.workunit.client.1.smithi194.stdout:1/840: readlink d0/d11/d18/d19/d1c/dca/ld4 0 2024-04-12T05:56:18.713 INFO:tasks.workunit.client.1.smithi194.stdout:0/743: creat d0/ff4 x:0 0 0 2024-04-12T05:56:18.719 INFO:tasks.workunit.client.1.smithi194.stdout:7/706: link d3/d16/d23/d57/d72/d81/c6e d3/d16/d23/d57/d72/d81/d3a/d43/d5c/ce8 0 2024-04-12T05:56:18.722 INFO:tasks.workunit.client.1.smithi194.stdout:3/911: dwrite d0/d1/d74/d39/da2/fc8 [0,4194304] 0 2024-04-12T05:56:18.724 INFO:tasks.workunit.client.1.smithi194.stdout:0/744: fdatasync d0/da/d1e/f19 0 2024-04-12T05:56:18.724 INFO:tasks.workunit.client.1.smithi194.stdout:8/613: creat d2/fbb x:0 0 0 2024-04-12T05:56:18.727 INFO:tasks.workunit.client.1.smithi194.stdout:0/745: chown d0/da/l83 1767230 1 2024-04-12T05:56:18.731 INFO:tasks.workunit.client.1.smithi194.stdout:3/912: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/d134 0 2024-04-12T05:56:18.736 INFO:tasks.workunit.client.1.smithi194.stdout:7/707: creat d3/d16/d23/d57/d72/d81/fe9 x:0 0 0 2024-04-12T05:56:18.738 INFO:tasks.workunit.client.1.smithi194.stdout:2/766: dwrite d7/d12/d25/f64 [4194304,4194304] 0 2024-04-12T05:56:18.738 INFO:tasks.workunit.client.1.smithi194.stdout:0/746: mkdir d0/d5f/deb/df5 0 2024-04-12T05:56:18.741 INFO:tasks.workunit.client.1.smithi194.stdout:3/913: mkdir d0/d1/d4f/dd7/de1/d135 0 2024-04-12T05:56:18.741 INFO:tasks.workunit.client.1.smithi194.stdout:7/708: dread - d3/d27/d22/d62/fdb zero size 2024-04-12T05:56:18.742 INFO:tasks.workunit.client.1.smithi194.stdout:1/841: write d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d/f64 [4371953,75931] 0 2024-04-12T05:56:18.745 INFO:tasks.workunit.client.1.smithi194.stdout:7/709: readlink d3/d27/d6a/l98 0 2024-04-12T05:56:18.747 INFO:tasks.workunit.client.1.smithi194.stdout:5/882: dwrite d1/d4/d6/db/d2b/f66 [8388608,4194304] 0 2024-04-12T05:56:18.749 INFO:tasks.workunit.client.1.smithi194.stdout:3/914: chown d0/d1/d74/d39/d4d/deb/l20 0 1 2024-04-12T05:56:18.749 INFO:tasks.workunit.client.1.smithi194.stdout:0/747: creat d0/da/d1e/d1a/d24/de1/ff6 x:0 0 0 2024-04-12T05:56:18.754 INFO:tasks.workunit.client.1.smithi194.stdout:5/883: chown d1/d4 0 1 2024-04-12T05:56:18.756 INFO:tasks.workunit.client.1.smithi194.stdout:8/614: dread d2/d8/d15/d35/f4c [0,4194304] 0 2024-04-12T05:56:18.756 INFO:tasks.workunit.client.1.smithi194.stdout:7/710: truncate d3/d27/d22/d62/f6c 4745979 0 2024-04-12T05:56:18.758 INFO:tasks.workunit.client.1.smithi194.stdout:1/842: truncate d0/d11/d18/fa2 833565 0 2024-04-12T05:56:18.758 INFO:tasks.workunit.client.1.smithi194.stdout:0/748: creat d0/d3b/ff7 x:0 0 0 2024-04-12T05:56:18.760 INFO:tasks.workunit.client.1.smithi194.stdout:2/767: getdents d7/d12/d1a 0 2024-04-12T05:56:18.762 INFO:tasks.workunit.client.1.smithi194.stdout:1/843: chown d0/d11/d18/d19/d1c/dca 22 1 2024-04-12T05:56:18.762 INFO:tasks.workunit.client.1.smithi194.stdout:8/615: write d2/d8/d66/d67/f74 [469201,17030] 0 2024-04-12T05:56:18.763 INFO:tasks.workunit.client.1.smithi194.stdout:0/749: readlink d0/da/d1e/d1a/d24/l50 0 2024-04-12T05:56:18.764 INFO:tasks.workunit.client.1.smithi194.stdout:5/884: unlink d1/d4/d6/c94 0 2024-04-12T05:56:18.765 INFO:tasks.workunit.client.1.smithi194.stdout:7/711: creat d3/d16/d23/d37/d3d/d56/dd0/fea x:0 0 0 2024-04-12T05:56:18.767 INFO:tasks.workunit.client.1.smithi194.stdout:0/750: unlink d0/da/d1e/d1a/f64 0 2024-04-12T05:56:18.770 INFO:tasks.workunit.client.1.smithi194.stdout:5/885: unlink d1/l9b 0 2024-04-12T05:56:18.773 INFO:tasks.workunit.client.1.smithi194.stdout:0/751: creat d0/d5f/d73/ff8 x:0 0 0 2024-04-12T05:56:18.774 INFO:tasks.workunit.client.1.smithi194.stdout:7/712: mknod d3/d16/d23/d57/d72/d81/d3a/d8f/ceb 0 2024-04-12T05:56:18.776 INFO:tasks.workunit.client.1.smithi194.stdout:0/752: fdatasync d0/d3b/fb5 0 2024-04-12T05:56:18.777 INFO:tasks.workunit.client.1.smithi194.stdout:5/886: unlink d1/d4/d6/d88/la5 0 2024-04-12T05:56:18.780 INFO:tasks.workunit.client.1.smithi194.stdout:2/768: dwrite d7/d12/d1a/d86/f107 [0,4194304] 0 2024-04-12T05:56:18.784 INFO:tasks.workunit.client.1.smithi194.stdout:7/713: write d3/d16/d23/d57/fab [2012349,27281] 0 2024-04-12T05:56:18.787 INFO:tasks.workunit.client.1.smithi194.stdout:0/753: mkdir d0/d5f/df9 0 2024-04-12T05:56:18.790 INFO:tasks.workunit.client.1.smithi194.stdout:3/915: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/fd6 [0,4194304] 0 2024-04-12T05:56:18.792 INFO:tasks.workunit.client.1.smithi194.stdout:1/844: dwrite d0/d11/d18/d19/d1c/f22 [0,4194304] 0 2024-04-12T05:56:18.796 INFO:tasks.workunit.client.1.smithi194.stdout:8/616: dread d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:18.802 INFO:tasks.workunit.client.1.smithi194.stdout:7/714: creat d3/d27/d22/fec x:0 0 0 2024-04-12T05:56:18.813 INFO:tasks.workunit.client.1.smithi194.stdout:5/887: dwrite d1/d4/d6/d6d/ffb [0,4194304] 0 2024-04-12T05:56:18.813 INFO:tasks.workunit.client.1.smithi194.stdout:3/916: fsync d0/d1/d74/d39/d4d/deb/f12 0 2024-04-12T05:56:18.813 INFO:tasks.workunit.client.1.smithi194.stdout:7/715: rmdir d3/d16/d23/d57/d72/d81/d3a/d43 39 2024-04-12T05:56:18.813 INFO:tasks.workunit.client.1.smithi194.stdout:8/617: stat d2/d8/d9/d1a/f2b 0 2024-04-12T05:56:18.813 INFO:tasks.workunit.client.1.smithi194.stdout:2/769: dread d7/d12/d1a/fad [0,4194304] 0 2024-04-12T05:56:18.839 INFO:tasks.workunit.client.1.smithi194.stdout:7/716: creat d3/d16/d23/d37/d3d/d56/dd0/dd9/fed x:0 0 0 2024-04-12T05:56:18.839 INFO:tasks.workunit.client.1.smithi194.stdout:2/770: symlink d7/d12/d25/d5d/d47/d52/l109 0 2024-04-12T05:56:18.844 INFO:tasks.workunit.client.1.smithi194.stdout:2/771: dread - d7/d12/f8d zero size 2024-04-12T05:56:18.847 INFO:tasks.workunit.client.1.smithi194.stdout:8/618: getdents d2/d8/d9/de/d86 0 2024-04-12T05:56:18.847 INFO:tasks.workunit.client.1.smithi194.stdout:1/845: dwrite d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fb1 [0,4194304] 0 2024-04-12T05:56:18.848 INFO:tasks.workunit.client.1.smithi194.stdout:8/619: chown d2/d8/d66 2818331 1 2024-04-12T05:56:18.854 INFO:tasks.workunit.client.1.smithi194.stdout:0/754: dread d0/da/d1e/f19 [0,4194304] 0 2024-04-12T05:56:18.856 INFO:tasks.workunit.client.1.smithi194.stdout:7/717: write d3/d27/d22/d62/fcf [412125,65944] 0 2024-04-12T05:56:18.858 INFO:tasks.workunit.client.1.smithi194.stdout:8/620: creat d2/d8/d9/de/d38/d8f/d94/fbc x:0 0 0 2024-04-12T05:56:18.858 INFO:tasks.workunit.client.1.smithi194.stdout:1/846: fdatasync d0/d11/d18/d19/d1a/d6a/dac/db5/f7f 0 2024-04-12T05:56:18.860 INFO:tasks.workunit.client.1.smithi194.stdout:1/847: fsync d0/d11/d18/f2d 0 2024-04-12T05:56:18.864 INFO:tasks.workunit.client.1.smithi194.stdout:0/755: rename d0/da/d1e to d0/d3b/d52/dfa 0 2024-04-12T05:56:18.868 INFO:tasks.workunit.client.1.smithi194.stdout:2/772: creat d7/d12/d25/d5d/d47/f10a x:0 0 0 2024-04-12T05:56:18.875 INFO:tasks.workunit.client.1.smithi194.stdout:0/756: rename d0/d5f/d73/l95 to d0/da/d4a/lfb 0 2024-04-12T05:56:18.878 INFO:tasks.workunit.client.1.smithi194.stdout:7/718: write d3/d16/d23/d37/d3d/f61 [3043875,106735] 0 2024-04-12T05:56:18.880 INFO:tasks.workunit.client.1.smithi194.stdout:1/848: fsync d0/d11/d18/d19/d1a/d6a/d27/d48/f62 0 2024-04-12T05:56:18.885 INFO:tasks.workunit.client.1.smithi194.stdout:0/757: dwrite d0/d3b/d52/dfa/d1a/d24/de1/ff6 [0,4194304] 0 2024-04-12T05:56:18.891 INFO:tasks.workunit.client.1.smithi194.stdout:7/719: fsync d3/d16/d23/d57/d72/d81/d3a/fa7 0 2024-04-12T05:56:18.891 INFO:tasks.workunit.client.1.smithi194.stdout:2/773: write d7/d12/d25/d5d/d47/d52/d7b/fb8 [272252,58522] 0 2024-04-12T05:56:18.896 INFO:tasks.workunit.client.1.smithi194.stdout:7/720: rmdir d3/d27/d22/d62 39 2024-04-12T05:56:18.896 INFO:tasks.workunit.client.1.smithi194.stdout:0/758: symlink d0/de3/d5b/lfc 0 2024-04-12T05:56:18.897 INFO:tasks.workunit.client.1.smithi194.stdout:1/849: symlink d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ddf/l109 0 2024-04-12T05:56:18.907 INFO:tasks.workunit.client.1.smithi194.stdout:2/774: dwrite d7/d12/d1a/d86/fe1 [0,4194304] 0 2024-04-12T05:56:18.912 INFO:tasks.workunit.client.1.smithi194.stdout:7/721: dwrite d3/d16/d23/d37/d3d/d56/dd0/fc9 [0,4194304] 0 2024-04-12T05:56:18.912 INFO:tasks.workunit.client.1.smithi194.stdout:2/775: dread - d7/d12/d1a/d69/df0/d101/df9/fff zero size 2024-04-12T05:56:18.914 INFO:tasks.workunit.client.1.smithi194.stdout:2/776: fsync d7/d22/d93/d9f/fb0 0 2024-04-12T05:56:18.938 INFO:tasks.workunit.client.1.smithi194.stdout:1/850: mknod d0/d11/c10a 0 2024-04-12T05:56:18.941 INFO:tasks.workunit.client.1.smithi194.stdout:0/759: write d0/d3b/d52/dfa/d27/f2a [1460949,57737] 0 2024-04-12T05:56:18.942 INFO:tasks.workunit.client.1.smithi194.stdout:0/760: chown d0/d5f/fcb 90127 1 2024-04-12T05:56:18.954 INFO:tasks.workunit.client.1.smithi194.stdout:0/761: mknod d0/de3/df0/cfd 0 2024-04-12T05:56:18.955 INFO:tasks.workunit.client.1.smithi194.stdout:2/777: getdents d7/d12/d1a/de9 0 2024-04-12T05:56:18.968 INFO:tasks.workunit.client.1.smithi194.stdout:0/762: dwrite d0/d3b/d52/dfa/d1a/d24/da8/fe8 [0,4194304] 0 2024-04-12T05:56:18.969 INFO:tasks.workunit.client.1.smithi194.stdout:1/851: write d0/d11/f9f [2649286,30866] 0 2024-04-12T05:56:18.990 INFO:tasks.workunit.client.1.smithi194.stdout:5/888: write d1/d4/d6/f76 [2068286,95021] 0 2024-04-12T05:56:19.001 INFO:tasks.workunit.client.1.smithi194.stdout:5/889: dwrite d1/d4/d6/db/d15/d8c/f8d [0,4194304] 0 2024-04-12T05:56:19.005 INFO:tasks.workunit.client.1.smithi194.stdout:2/778: write d7/f11 [8954601,89764] 0 2024-04-12T05:56:19.014 INFO:tasks.workunit.client.1.smithi194.stdout:0/763: chown d0/d3b/d52/dfa/d27/d87/d92/fda 372 1 2024-04-12T05:56:19.015 INFO:tasks.workunit.client.1.smithi194.stdout:1/852: getdents d0/d11/d18/d19/d1a/d6a/dac/dba 0 2024-04-12T05:56:19.015 INFO:tasks.workunit.client.1.smithi194.stdout:1/853: stat d0/d11/ddd/fff 0 2024-04-12T05:56:19.026 INFO:tasks.workunit.client.1.smithi194.stdout:9/842: sync 2024-04-12T05:56:19.026 INFO:tasks.workunit.client.1.smithi194.stdout:6/796: sync 2024-04-12T05:56:19.027 INFO:tasks.workunit.client.1.smithi194.stdout:0/764: dwrite d0/da/da7/fbc [0,4194304] 0 2024-04-12T05:56:19.040 INFO:tasks.workunit.client.1.smithi194.stdout:6/797: read d3/d2d/fcb [759476,34836] 0 2024-04-12T05:56:19.042 INFO:tasks.workunit.client.1.smithi194.stdout:5/890: dwrite d1/d4/d4b/fba [0,4194304] 0 2024-04-12T05:56:19.043 INFO:tasks.workunit.client.1.smithi194.stdout:6/798: fsync d3/d2f/f39 0 2024-04-12T05:56:19.043 INFO:tasks.workunit.client.1.smithi194.stdout:1/854: dread d0/d11/d18/d19/f1f [0,4194304] 0 2024-04-12T05:56:19.044 INFO:tasks.workunit.client.1.smithi194.stdout:2/779: dread d7/d12/d25/d5d/d47/d52/d7b/fa6 [0,4194304] 0 2024-04-12T05:56:19.049 INFO:tasks.workunit.client.1.smithi194.stdout:1/855: stat d0/d11/d18/d19/d1a/l37 0 2024-04-12T05:56:19.050 INFO:tasks.workunit.client.1.smithi194.stdout:2/780: write d7/d12/d25/d5d/f51 [9200260,82238] 0 2024-04-12T05:56:19.050 INFO:tasks.workunit.client.1.smithi194.stdout:5/891: write d1/d5c/da7/ff3 [4779935,34739] 0 2024-04-12T05:56:19.059 INFO:tasks.workunit.client.1.smithi194.stdout:3/917: dwrite d0/d1/d74/d39/d4d/deb/d3d/fc0 [0,4194304] 0 2024-04-12T05:56:19.063 INFO:tasks.workunit.client.1.smithi194.stdout:9/843: truncate d1/d19/d1f/d20/f35 531907 0 2024-04-12T05:56:19.065 INFO:tasks.workunit.client.1.smithi194.stdout:2/781: creat d7/d12/d1a/dbf/f10b x:0 0 0 2024-04-12T05:56:19.068 INFO:tasks.workunit.client.1.smithi194.stdout:2/782: fdatasync d7/d12/d1a/fad 0 2024-04-12T05:56:19.073 INFO:tasks.workunit.client.1.smithi194.stdout:5/892: dread d1/d4/d6/f76 [0,4194304] 0 2024-04-12T05:56:19.076 INFO:tasks.workunit.client.1.smithi194.stdout:2/783: creat d7/d12/d25/d5d/d47/dac/db3/dd7/df8/f10c x:0 0 0 2024-04-12T05:56:19.077 INFO:tasks.workunit.client.1.smithi194.stdout:9/844: write f0 [5478168,108606] 0 2024-04-12T05:56:19.078 INFO:tasks.workunit.client.1.smithi194.stdout:6/799: truncate d3/d2f/d4e/d68/fa2 1662648 0 2024-04-12T05:56:19.078 INFO:tasks.workunit.client.1.smithi194.stdout:2/784: stat d7/d22/f5c 0 2024-04-12T05:56:19.080 INFO:tasks.workunit.client.1.smithi194.stdout:9/845: mkdir d1/d4e/d11a 0 2024-04-12T05:56:19.083 INFO:tasks.workunit.client.1.smithi194.stdout:2/785: dread d7/d12/d1a/d69/d7e/fe0 [0,4194304] 0 2024-04-12T05:56:19.085 INFO:tasks.workunit.client.1.smithi194.stdout:1/856: dread d0/d11/d18/d19/d1a/d6a/dac/fce [0,4194304] 0 2024-04-12T05:56:19.086 INFO:tasks.workunit.client.1.smithi194.stdout:5/893: creat d1/d4/d6/d6d/f122 x:0 0 0 2024-04-12T05:56:19.086 INFO:tasks.workunit.client.1.smithi194.stdout:0/765: dread d0/d3b/d52/dfa/f17 [0,4194304] 0 2024-04-12T05:56:19.088 INFO:tasks.workunit.client.1.smithi194.stdout:3/918: fsync d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f41 0 2024-04-12T05:56:19.093 INFO:tasks.workunit.client.1.smithi194.stdout:9/846: getdents d1/d41/d97/d9d 0 2024-04-12T05:56:19.094 INFO:tasks.workunit.client.1.smithi194.stdout:5/894: chown d1/d4/d6/c20 55917896 1 2024-04-12T05:56:19.096 INFO:tasks.workunit.client.1.smithi194.stdout:1/857: chown d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/l8e 5997349 1 2024-04-12T05:56:19.097 INFO:tasks.workunit.client.1.smithi194.stdout:7/722: rename d3/d27/l5b to d3/d27/d22/d31/lee 0 2024-04-12T05:56:19.100 INFO:tasks.workunit.client.1.smithi194.stdout:9/847: symlink d1/d21/d77/d98/d9f/l11b 0 2024-04-12T05:56:19.100 INFO:tasks.workunit.client.1.smithi194.stdout:3/919: dread d0/d1/d74/d39/d4d/f62 [0,4194304] 0 2024-04-12T05:56:19.102 INFO:tasks.workunit.client.1.smithi194.stdout:6/800: dwrite d3/d15/d2b/d48/d6f/d7a/d8f/fb5 [0,4194304] 0 2024-04-12T05:56:19.103 INFO:tasks.workunit.client.1.smithi194.stdout:5/895: truncate d1/d4/d6/fc8 2179426 0 2024-04-12T05:56:19.108 INFO:tasks.workunit.client.1.smithi194.stdout:9/848: write d1/d41/fe5 [455910,100806] 0 2024-04-12T05:56:19.114 INFO:tasks.workunit.client.1.smithi194.stdout:2/786: dwrite d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/faf [4194304,4194304] 0 2024-04-12T05:56:19.115 INFO:tasks.workunit.client.1.smithi194.stdout:6/801: dread d3/d15/d2b/d48/f74 [0,4194304] 0 2024-04-12T05:56:19.119 INFO:tasks.workunit.client.1.smithi194.stdout:6/802: chown d3/d34/cbb 3318 1 2024-04-12T05:56:19.121 INFO:tasks.workunit.client.1.smithi194.stdout:0/766: dread d0/d3b/d52/dfa/d27/d54/f7b [0,4194304] 0 2024-04-12T05:56:19.122 INFO:tasks.workunit.client.1.smithi194.stdout:9/849: symlink d1/d4e/d11a/l11c 0 2024-04-12T05:56:19.122 INFO:tasks.workunit.client.1.smithi194.stdout:0/767: readlink d0/d3b/d52/dfa/d1a/d24/l50 0 2024-04-12T05:56:19.123 INFO:tasks.workunit.client.1.smithi194.stdout:3/920: mkdir d0/d1/d74/d39/d4d/deb/d3d/d136 0 2024-04-12T05:56:19.126 INFO:tasks.workunit.client.1.smithi194.stdout:0/768: rmdir d0 39 2024-04-12T05:56:19.126 INFO:tasks.workunit.client.1.smithi194.stdout:5/896: dread d1/f82 [0,4194304] 0 2024-04-12T05:56:19.130 INFO:tasks.workunit.client.1.smithi194.stdout:3/921: truncate d0/d1/d74/d39/fb8 984543 0 2024-04-12T05:56:19.133 INFO:tasks.workunit.client.1.smithi194.stdout:2/787: dwrite d7/d12/f3d [0,4194304] 0 2024-04-12T05:56:19.136 INFO:tasks.workunit.client.1.smithi194.stdout:3/922: write d0/d1/d10/d89/fd4 [1483164,93618] 0 2024-04-12T05:56:19.140 INFO:tasks.workunit.client.1.smithi194.stdout:5/897: mknod d1/d5c/d95/df2/c123 0 2024-04-12T05:56:19.141 INFO:tasks.workunit.client.1.smithi194.stdout:9/850: truncate d1/d19/d1f/d5f/f85 2763763 0 2024-04-12T05:56:19.145 INFO:tasks.workunit.client.1.smithi194.stdout:0/769: write d0/d3b/d52/dfa/d27/d54/fc3 [648774,22103] 0 2024-04-12T05:56:19.147 INFO:tasks.workunit.client.1.smithi194.stdout:8/621: sync 2024-04-12T05:56:19.150 INFO:tasks.workunit.client.1.smithi194.stdout:3/923: getdents d0/d1/d10/d8c/d8f/dac/de3/d107 0 2024-04-12T05:56:19.150 INFO:tasks.workunit.client.1.smithi194.stdout:9/851: chown d1/d19/d1f/d20/d37/d84/fe0 30698 1 2024-04-12T05:56:19.151 INFO:tasks.workunit.client.1.smithi194.stdout:0/770: dread - d0/fdc zero size 2024-04-12T05:56:19.155 INFO:tasks.workunit.client.1.smithi194.stdout:6/803: dwrite d3/d2f/d4e/d9a/dd8/d77/fb6 [0,4194304] 0 2024-04-12T05:56:19.158 INFO:tasks.workunit.client.1.smithi194.stdout:1/858: dread d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ddf/f86 [0,4194304] 0 2024-04-12T05:56:19.158 INFO:tasks.workunit.client.1.smithi194.stdout:7/723: dwrite d3/d27/f1b [4194304,4194304] 0 2024-04-12T05:56:19.161 INFO:tasks.workunit.client.1.smithi194.stdout:8/622: mknod d2/d8/d15/cbd 0 2024-04-12T05:56:19.166 INFO:tasks.workunit.client.1.smithi194.stdout:1/859: fsync d0/d11/d18/d19/fda 0 2024-04-12T05:56:19.174 INFO:tasks.workunit.client.1.smithi194.stdout:5/898: dread d1/d4/d6/db/d15/d26/f50 [0,4194304] 0 2024-04-12T05:56:19.175 INFO:tasks.workunit.client.1.smithi194.stdout:2/788: dwrite d7/d12/d25/d70/fa1 [0,4194304] 0 2024-04-12T05:56:19.176 INFO:tasks.workunit.client.1.smithi194.stdout:6/804: truncate d3/d1a/f23 196905 0 2024-04-12T05:56:19.182 INFO:tasks.workunit.client.1.smithi194.stdout:6/805: truncate d3/d70/fab 1645646 0 2024-04-12T05:56:19.184 INFO:tasks.workunit.client.1.smithi194.stdout:0/771: dread d0/d3b/d52/dfa/d27/fd6 [0,4194304] 0 2024-04-12T05:56:19.189 INFO:tasks.workunit.client.1.smithi194.stdout:3/924: rmdir d0/d1/d4f/dd7 39 2024-04-12T05:56:19.191 INFO:tasks.workunit.client.1.smithi194.stdout:1/860: link d0/d11/d18/d19/d1a/d6a/d27/d48/f62 d0/d11/d18/d19/d1c/dca/f10b 0 2024-04-12T05:56:19.194 INFO:tasks.workunit.client.1.smithi194.stdout:9/852: dwrite d1/d21/d39/d72/fb5 [0,4194304] 0 2024-04-12T05:56:19.195 INFO:tasks.workunit.client.1.smithi194.stdout:8/623: dwrite d2/d8/d15/f62 [0,4194304] 0 2024-04-12T05:56:19.206 INFO:tasks.workunit.client.1.smithi194.stdout:8/624: readlink d2/d8/d9/de/d38/l51 0 2024-04-12T05:56:19.207 INFO:tasks.workunit.client.1.smithi194.stdout:8/625: chown d2/d8/d15/d33/d45/lac 255052 1 2024-04-12T05:56:19.207 INFO:tasks.workunit.client.1.smithi194.stdout:7/724: dwrite d3/d16/d23/d37/d3d/d56/f51 [0,4194304] 0 2024-04-12T05:56:19.207 INFO:tasks.workunit.client.1.smithi194.stdout:6/806: symlink d3/d2d/d33/d52/d5c/d8c/lf9 0 2024-04-12T05:56:19.207 INFO:tasks.workunit.client.1.smithi194.stdout:0/772: dread - d0/d3b/d52/dfa/d27/fe6 zero size 2024-04-12T05:56:19.215 INFO:tasks.workunit.client.1.smithi194.stdout:6/807: fsync d3/d70/f8b 0 2024-04-12T05:56:19.215 INFO:tasks.workunit.client.1.smithi194.stdout:8/626: write d2/d8/d9/de/d10/d4e/f60 [1391475,16332] 0 2024-04-12T05:56:19.215 INFO:tasks.workunit.client.1.smithi194.stdout:9/853: creat d1/d21/d77/d98/d9f/dde/f11d x:0 0 0 2024-04-12T05:56:19.219 INFO:tasks.workunit.client.1.smithi194.stdout:7/725: truncate d3/d16/d23/fc8 808016 0 2024-04-12T05:56:19.222 INFO:tasks.workunit.client.1.smithi194.stdout:1/861: write d0/d11/d18/d19/d1a/d6a/d27/d9e/fad [2031432,102462] 0 2024-04-12T05:56:19.223 INFO:tasks.workunit.client.1.smithi194.stdout:0/773: dread d0/d3b/d52/dfa/d1a/d24/f61 [4194304,4194304] 0 2024-04-12T05:56:19.227 INFO:tasks.workunit.client.1.smithi194.stdout:9/854: creat d1/d21/d39/f11e x:0 0 0 2024-04-12T05:56:19.228 INFO:tasks.workunit.client.1.smithi194.stdout:2/789: dread d7/d12/d1a/de9/fed [0,4194304] 0 2024-04-12T05:56:19.228 INFO:tasks.workunit.client.1.smithi194.stdout:3/925: rename d0/d1/d74/d39/d4d/deb/d1e/dc4/f10a to d0/d1/d74/d39/d4d/deb/db4/f137 0 2024-04-12T05:56:19.229 INFO:tasks.workunit.client.1.smithi194.stdout:6/808: write d3/d2f/d37/f3f [990302,71261] 0 2024-04-12T05:56:19.234 INFO:tasks.workunit.client.1.smithi194.stdout:8/627: rename d2/d8/d66/d67 to d2/d8/d9/de/d86/dbe 0 2024-04-12T05:56:19.236 INFO:tasks.workunit.client.1.smithi194.stdout:7/726: truncate d3/d16/d23/d57/d72/d81/d3a/fa7 3493437 0 2024-04-12T05:56:19.237 INFO:tasks.workunit.client.1.smithi194.stdout:0/774: chown d0/d3b/d52/dfa/d27/c70 240643 1 2024-04-12T05:56:19.239 INFO:tasks.workunit.client.1.smithi194.stdout:7/727: chown d3/d27/d22/d31/cde 431 1 2024-04-12T05:56:19.241 INFO:tasks.workunit.client.1.smithi194.stdout:6/809: creat d3/d2f/d4e/d9a/dd8/d77/ffa x:0 0 0 2024-04-12T05:56:19.241 INFO:tasks.workunit.client.1.smithi194.stdout:1/862: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/c10c 0 2024-04-12T05:56:19.243 INFO:tasks.workunit.client.1.smithi194.stdout:8/628: creat d2/d8/d9/de/d38/d8f/d94/fbf x:0 0 0 2024-04-12T05:56:19.245 INFO:tasks.workunit.client.1.smithi194.stdout:0/775: truncate d0/d3b/d52/f84 841780 0 2024-04-12T05:56:19.245 INFO:tasks.workunit.client.1.smithi194.stdout:3/926: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f138 x:0 0 0 2024-04-12T05:56:19.249 INFO:tasks.workunit.client.1.smithi194.stdout:9/855: dwrite d1/d19/d1f/d20/d37/d84/fe0 [0,4194304] 0 2024-04-12T05:56:19.249 INFO:tasks.workunit.client.1.smithi194.stdout:7/728: truncate d3/d27/d22/d8e/fb0 1023454 0 2024-04-12T05:56:19.249 INFO:tasks.workunit.client.1.smithi194.stdout:6/810: unlink d3/d2d/d33/d52/f89 0 2024-04-12T05:56:19.249 INFO:tasks.workunit.client.1.smithi194.stdout:5/899: dread d1/d4/d6/f13 [0,4194304] 0 2024-04-12T05:56:19.250 INFO:tasks.workunit.client.1.smithi194.stdout:9/856: write d1/f69 [3441407,1178] 0 2024-04-12T05:56:19.251 INFO:tasks.workunit.client.1.smithi194.stdout:2/790: write d7/d12/d25/d5d/d47/d52/d7b/fa6 [303957,8452] 0 2024-04-12T05:56:19.253 INFO:tasks.workunit.client.1.smithi194.stdout:1/863: read - d0/d11/d18/d19/d1c/dca/fd0 zero size 2024-04-12T05:56:19.253 INFO:tasks.workunit.client.1.smithi194.stdout:6/811: fdatasync d3/f3b 0 2024-04-12T05:56:19.255 INFO:tasks.workunit.client.1.smithi194.stdout:0/776: truncate d0/d3b/ff7 425885 0 2024-04-12T05:56:19.255 INFO:tasks.workunit.client.1.smithi194.stdout:7/729: dread - d3/d16/d23/d37/d3d/d56/dd0/dd9/fed zero size 2024-04-12T05:56:19.257 INFO:tasks.workunit.client.1.smithi194.stdout:8/629: creat d2/d8/d15/d33/d45/d52/fc0 x:0 0 0 2024-04-12T05:56:19.257 INFO:tasks.workunit.client.1.smithi194.stdout:3/927: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f139 x:0 0 0 2024-04-12T05:56:19.262 INFO:tasks.workunit.client.1.smithi194.stdout:3/928: read d0/d1/f60 [211077,82364] 0 2024-04-12T05:56:19.265 INFO:tasks.workunit.client.1.smithi194.stdout:9/857: chown d1/l74 2279 1 2024-04-12T05:56:19.265 INFO:tasks.workunit.client.1.smithi194.stdout:2/791: creat d7/d12/d1a/d69/df0/d101/f10d x:0 0 0 2024-04-12T05:56:19.267 INFO:tasks.workunit.client.1.smithi194.stdout:1/864: creat d0/d11/d18/d19/d1a/ded/f10d x:0 0 0 2024-04-12T05:56:19.271 INFO:tasks.workunit.client.1.smithi194.stdout:6/812: truncate d3/d15/f24 328080 0 2024-04-12T05:56:19.272 INFO:tasks.workunit.client.1.smithi194.stdout:0/777: dwrite d0/d3b/d52/dfa/fdb [0,4194304] 0 2024-04-12T05:56:19.273 INFO:tasks.workunit.client.1.smithi194.stdout:8/630: mkdir d2/d8/d66/dc1 0 2024-04-12T05:56:19.273 INFO:tasks.workunit.client.1.smithi194.stdout:3/929: creat d0/d1/d74/d39/da2/f13a x:0 0 0 2024-04-12T05:56:19.279 INFO:tasks.workunit.client.1.smithi194.stdout:8/631: rename d2/d8/d9/d1a/da1 to d2/d8/d66/dc1/dc2 0 2024-04-12T05:56:19.280 INFO:tasks.workunit.client.1.smithi194.stdout:2/792: dread d7/d12/d1a/d69/f82 [0,4194304] 0 2024-04-12T05:56:19.281 INFO:tasks.workunit.client.1.smithi194.stdout:3/930: mkdir d0/d1/d74/d39/d4d/deb/d8e/d13b 0 2024-04-12T05:56:19.293 INFO:tasks.workunit.client.1.smithi194.stdout:6/813: write d3/d2f/fa7 [908012,86010] 0 2024-04-12T05:56:19.296 INFO:tasks.workunit.client.1.smithi194.stdout:9/858: link d1/d19/d1f/d20/d37/d84/l116 d1/d19/d1f/l11f 0 2024-04-12T05:56:19.297 INFO:tasks.workunit.client.1.smithi194.stdout:8/632: unlink d2/d8/f36 0 2024-04-12T05:56:19.297 INFO:tasks.workunit.client.1.smithi194.stdout:0/778: dwrite d0/d3b/d52/dfa/d1a/d24/d56/f8d [4194304,4194304] 0 2024-04-12T05:56:19.303 INFO:tasks.workunit.client.1.smithi194.stdout:2/793: write d7/d12/d1a/f24 [461133,93935] 0 2024-04-12T05:56:19.305 INFO:tasks.workunit.client.1.smithi194.stdout:1/865: rename d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/l8e to d0/d11/d18/d35/l10e 0 2024-04-12T05:56:19.313 INFO:tasks.workunit.client.1.smithi194.stdout:1/866: write d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ff1 [193831,84806] 0 2024-04-12T05:56:19.314 INFO:tasks.workunit.client.1.smithi194.stdout:9/859: symlink d1/d19/d1f/d20/l120 0 2024-04-12T05:56:19.316 INFO:tasks.workunit.client.1.smithi194.stdout:8/633: dwrite d2/d8/d15/d33/d45/d52/fc0 [0,4194304] 0 2024-04-12T05:56:19.317 INFO:tasks.workunit.client.1.smithi194.stdout:1/867: chown d0/d11/d18/d19/d1a/laf 0 1 2024-04-12T05:56:19.319 INFO:tasks.workunit.client.1.smithi194.stdout:8/634: write d2/d8/d15/fa3 [965046,15082] 0 2024-04-12T05:56:19.326 INFO:tasks.workunit.client.1.smithi194.stdout:3/931: rename d0/d1/d74/d39/d4d/deb/d53/f105 to d0/d1/d74/d39/d4d/f13c 0 2024-04-12T05:56:19.329 INFO:tasks.workunit.client.1.smithi194.stdout:8/635: symlink d2/d8/d15/d33/lc3 0 2024-04-12T05:56:19.329 INFO:tasks.workunit.client.1.smithi194.stdout:1/868: fdatasync d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 0 2024-04-12T05:56:19.331 INFO:tasks.workunit.client.1.smithi194.stdout:6/814: rename d3/d2f/d4e/d68/dad/dc4/lf6 to d3/d2f/d4e/d56/de2/lfb 0 2024-04-12T05:56:19.333 INFO:tasks.workunit.client.1.smithi194.stdout:3/932: creat d0/d1/d74/d39/d4d/deb/f13d x:0 0 0 2024-04-12T05:56:19.336 INFO:tasks.workunit.client.1.smithi194.stdout:8/636: write d2/d8/d9/de/f37 [585399,78204] 0 2024-04-12T05:56:19.342 INFO:tasks.workunit.client.1.smithi194.stdout:0/779: getdents d0/da/d4a/d8e 0 2024-04-12T05:56:19.343 INFO:tasks.workunit.client.1.smithi194.stdout:5/900: dwrite d1/d4/d6/f3d [0,4194304] 0 2024-04-12T05:56:19.345 INFO:tasks.workunit.client.1.smithi194.stdout:1/869: mknod d0/d11/d18/d19/d1a/d6a/d27/da8/c10f 0 2024-04-12T05:56:19.346 INFO:tasks.workunit.client.1.smithi194.stdout:3/933: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/da7/f13e x:0 0 0 2024-04-12T05:56:19.355 INFO:tasks.workunit.client.1.smithi194.stdout:7/730: dwrite d3/d16/d23/f66 [0,4194304] 0 2024-04-12T05:56:19.357 INFO:tasks.workunit.client.1.smithi194.stdout:6/815: dwrite d3/d15/d2b/d48/d6f/d7a/d8f/fbd [0,4194304] 0 2024-04-12T05:56:19.357 INFO:tasks.workunit.client.1.smithi194.stdout:0/780: fsync d0/f94 0 2024-04-12T05:56:19.358 INFO:tasks.workunit.client.1.smithi194.stdout:1/870: symlink d0/l110 0 2024-04-12T05:56:19.360 INFO:tasks.workunit.client.1.smithi194.stdout:5/901: mknod d1/d4/d6/db/d15/d8c/c124 0 2024-04-12T05:56:19.361 INFO:tasks.workunit.client.1.smithi194.stdout:0/781: write d0/de3/d5b/fb6 [1792141,84069] 0 2024-04-12T05:56:19.364 INFO:tasks.workunit.client.1.smithi194.stdout:7/731: stat d3/d27/d6a/dda/d74/c88 0 2024-04-12T05:56:19.366 INFO:tasks.workunit.client.1.smithi194.stdout:7/732: fdatasync d3/d27/d6a/f82 0 2024-04-12T05:56:19.368 INFO:tasks.workunit.client.1.smithi194.stdout:0/782: mknod d0/d3b/d9c/cfe 0 2024-04-12T05:56:19.368 INFO:tasks.workunit.client.1.smithi194.stdout:3/934: rmdir d0/d1/d74/d39/d4d/deb/d3d/d136 0 2024-04-12T05:56:19.371 INFO:tasks.workunit.client.1.smithi194.stdout:6/816: creat d3/d15/d2b/d48/ffc x:0 0 0 2024-04-12T05:56:19.379 INFO:tasks.workunit.client.1.smithi194.stdout:1/871: dwrite d0/d11/d18/d19/d1a/ded/fee [0,4194304] 0 2024-04-12T05:56:19.381 INFO:tasks.workunit.client.1.smithi194.stdout:9/860: dread d1/d19/d1f/d5f/f85 [0,4194304] 0 2024-04-12T05:56:19.382 INFO:tasks.workunit.client.1.smithi194.stdout:3/935: mkdir d0/d1/d74/d39/d4d/deb/db4/d13f 0 2024-04-12T05:56:19.387 INFO:tasks.workunit.client.1.smithi194.stdout:0/783: mknod d0/d3b/d52/dfa/d1a/cff 0 2024-04-12T05:56:19.391 INFO:tasks.workunit.client.1.smithi194.stdout:9/861: mkdir d1/d19/d1f/d5f/df4/d121 0 2024-04-12T05:56:19.392 INFO:tasks.workunit.client.1.smithi194.stdout:7/733: dwrite d3/d16/f45 [0,4194304] 0 2024-04-12T05:56:19.392 INFO:tasks.workunit.client.1.smithi194.stdout:6/817: mknod d3/d2d/d33/d52/d62/ddd/cfd 0 2024-04-12T05:56:19.394 INFO:tasks.workunit.client.1.smithi194.stdout:6/818: write d3/d2d/d33/d58/f7f [1608556,123725] 0 2024-04-12T05:56:19.398 INFO:tasks.workunit.client.1.smithi194.stdout:1/872: truncate d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/ffe 128181 0 2024-04-12T05:56:19.398 INFO:tasks.workunit.client.1.smithi194.stdout:0/784: write d0/d3b/d52/dfa/d27/d87/d92/fda [538229,56945] 0 2024-04-12T05:56:19.400 INFO:tasks.workunit.client.1.smithi194.stdout:0/785: stat d0/d3b/d52/fb1 0 2024-04-12T05:56:19.402 INFO:tasks.workunit.client.1.smithi194.stdout:3/936: unlink d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/da7/f13e 0 2024-04-12T05:56:19.406 INFO:tasks.workunit.client.1.smithi194.stdout:0/786: chown d0/d5f/d73/l82 1929 1 2024-04-12T05:56:19.410 INFO:tasks.workunit.client.1.smithi194.stdout:3/937: symlink d0/d1/d74/d39/d4d/deb/d1e/d81/d34/l140 0 2024-04-12T05:56:19.410 INFO:tasks.workunit.client.1.smithi194.stdout:0/787: symlink d0/de3/l100 0 2024-04-12T05:56:19.412 INFO:tasks.workunit.client.1.smithi194.stdout:6/819: dread d3/d2d/d33/fa6 [0,4194304] 0 2024-04-12T05:56:19.415 INFO:tasks.workunit.client.1.smithi194.stdout:6/820: fsync d3/d2d/d33/d52/d5c/fe1 0 2024-04-12T05:56:19.415 INFO:tasks.workunit.client.1.smithi194.stdout:5/902: dread d1/d4/d6/db/d15/d26/fd8 [0,4194304] 0 2024-04-12T05:56:19.494 INFO:tasks.workunit.client.1.smithi194.stdout:7/734: mkdir d3/def 0 2024-04-12T05:56:19.497 INFO:tasks.workunit.client.1.smithi194.stdout:3/938: mkdir d0/d1/d4f/dd7/d141 0 2024-04-12T05:56:19.501 INFO:tasks.workunit.client.1.smithi194.stdout:5/903: truncate d1/d4/d6/db/d38/dff/f114 465711 0 2024-04-12T05:56:19.502 INFO:tasks.workunit.client.1.smithi194.stdout:3/939: stat d0/l7 0 2024-04-12T05:56:19.504 INFO:tasks.workunit.client.1.smithi194.stdout:6/821: rename d3/d2f/d4e/d56/cd6 to d3/d2f/d4e/d9a/db3/cfe 0 2024-04-12T05:56:19.505 INFO:tasks.workunit.client.1.smithi194.stdout:1/873: dwrite d0/d11/d18/d19/d1c/d67/fd1 [0,4194304] 0 2024-04-12T05:56:19.513 INFO:tasks.workunit.client.1.smithi194.stdout:5/904: symlink d1/d4/dd7/dfe/l125 0 2024-04-12T05:56:19.514 INFO:tasks.workunit.client.1.smithi194.stdout:1/874: fsync d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 0 2024-04-12T05:56:19.517 INFO:tasks.workunit.client.1.smithi194.stdout:0/788: dwrite d0/da/d4a/f5e [0,4194304] 0 2024-04-12T05:56:19.520 INFO:tasks.workunit.client.1.smithi194.stdout:5/905: chown d1/d5c/d95/df2/d107 7288739 1 2024-04-12T05:56:19.521 INFO:tasks.workunit.client.1.smithi194.stdout:5/906: readlink d1/d4/dd7/dfe/l125 0 2024-04-12T05:56:19.522 INFO:tasks.workunit.client.1.smithi194.stdout:5/907: chown d1/fa3 76 1 2024-04-12T05:56:19.522 INFO:tasks.workunit.client.1.smithi194.stdout:7/735: dread - d3/d27/d22/d62/fdb zero size 2024-04-12T05:56:19.525 INFO:tasks.workunit.client.1.smithi194.stdout:0/789: write d0/de3/d5b/fce [277019,9627] 0 2024-04-12T05:56:19.528 INFO:tasks.workunit.client.1.smithi194.stdout:5/908: symlink d1/d4/d6/db/d38/dff/l126 0 2024-04-12T05:56:19.530 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: [12/Apr/2024:05:56:17] ENGINE Bus STARTING 2024-04-12T05:56:19.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: [12/Apr/2024:05:56:18] ENGINE Serving on http://172.21.15.8:8765 2024-04-12T05:56:19.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: [12/Apr/2024:05:56:18] ENGINE Serving on https://172.21.15.8:7150 2024-04-12T05:56:19.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: [12/Apr/2024:05:56:18] ENGINE Bus STARTED 2024-04-12T05:56:19.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: mgrmap e28: smithi008.ixmnlp(active, since 1.02321s) 2024-04-12T05:56:19.531 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:19 smithi194 ceph-mon[49771]: pgmap v3: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:19.531 INFO:tasks.workunit.client.1.smithi194.stdout:3/940: read d0/d1/d74/d39/d4d/deb/d3d/f12f [4102297,81116] 0 2024-04-12T05:56:19.536 INFO:tasks.workunit.client.1.smithi194.stdout:6/822: getdents d3/d15/d2b 0 2024-04-12T05:56:19.540 INFO:tasks.workunit.client.1.smithi194.stdout:6/823: write d3/d15/d2b/d48/d6f/d7a/fcf [168020,41317] 0 2024-04-12T05:56:19.541 INFO:tasks.workunit.client.1.smithi194.stdout:7/736: chown d3/d27/d6a/ce5 48584 1 2024-04-12T05:56:19.543 INFO:tasks.workunit.client.1.smithi194.stdout:6/824: creat d3/d2f/d4e/d68/fff x:0 0 0 2024-04-12T05:56:19.544 INFO:tasks.workunit.client.1.smithi194.stdout:3/941: mknod d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/da7/de6/c142 0 2024-04-12T05:56:19.549 INFO:tasks.workunit.client.1.smithi194.stdout:5/909: write d1/d4/d84/fe5 [11398,112784] 0 2024-04-12T05:56:19.553 INFO:tasks.workunit.client.1.smithi194.stdout:6/825: write d3/d2d/d33/fa1 [1513874,5582] 0 2024-04-12T05:56:19.555 INFO:tasks.workunit.client.1.smithi194.stdout:7/737: truncate d3/d16/d23/d37/d95/fd1 807653 0 2024-04-12T05:56:19.555 INFO:tasks.workunit.client.1.smithi194.stdout:1/875: dwrite d0/d11/d18/d19/d1a/fa6 [0,4194304] 0 2024-04-12T05:56:19.555 INFO:tasks.workunit.client.1.smithi194.stdout:6/826: stat d3/d15 0 2024-04-12T05:56:19.556 INFO:tasks.workunit.client.1.smithi194.stdout:3/942: getdents d0/d1/d74/d39/d4d/deb/d1e/d33/d76 0 2024-04-12T05:56:19.560 INFO:tasks.workunit.client.1.smithi194.stdout:1/876: creat d0/d11/f111 x:0 0 0 2024-04-12T05:56:19.560 INFO:tasks.workunit.client.1.smithi194.stdout:3/943: chown d0/d1/d74/d39/d4d/deb/d1e/d33/d76/cab 19583 1 2024-04-12T05:56:19.563 INFO:tasks.workunit.client.1.smithi194.stdout:6/827: symlink d3/d2d/d33/db1/l100 0 2024-04-12T05:56:19.564 INFO:tasks.workunit.client.1.smithi194.stdout:6/828: chown d3/d15/d2b/d48/d6f/fb7 48766136 1 2024-04-12T05:56:19.566 INFO:tasks.workunit.client.1.smithi194.stdout:5/910: rmdir d1/d5c/d95/dfa 0 2024-04-12T05:56:19.574 INFO:tasks.workunit.client.1.smithi194.stdout:7/738: dwrite d3/d16/d23/d37/d3d/d56/dd0/fc9 [0,4194304] 0 2024-04-12T05:56:19.581 INFO:tasks.workunit.client.1.smithi194.stdout:6/829: dwrite d3/d2f/d4e/d63/faa [0,4194304] 0 2024-04-12T05:56:19.584 INFO:tasks.workunit.client.1.smithi194.stdout:1/877: dwrite d0/d11/d18/fc3 [0,4194304] 0 2024-04-12T05:56:19.584 INFO:tasks.workunit.client.1.smithi194.stdout:6/830: symlink d3/d70/l101 0 2024-04-12T05:56:19.586 INFO:tasks.workunit.client.1.smithi194.stdout:5/911: write d1/d4/f1c [359826,77563] 0 2024-04-12T05:56:19.594 INFO:tasks.workunit.client.1.smithi194.stdout:5/912: mknod d1/d4/d75/c127 0 2024-04-12T05:56:19.595 INFO:tasks.workunit.client.1.smithi194.stdout:5/913: readlink d1/d4/l1b 0 2024-04-12T05:56:19.598 INFO:tasks.workunit.client.1.smithi194.stdout:1/878: creat d0/d11/d18/d19/d1a/d6a/d27/d6c/f112 x:0 0 0 2024-04-12T05:56:19.598 INFO:tasks.workunit.client.1.smithi194.stdout:7/739: dwrite d3/d16/d23/d57/fd6 [0,4194304] 0 2024-04-12T05:56:19.601 INFO:tasks.workunit.client.1.smithi194.stdout:7/740: chown d3/d27/d6a/dda/d74/lac 519409 1 2024-04-12T05:56:19.602 INFO:tasks.workunit.client.1.smithi194.stdout:5/914: creat d1/d4/d6/d6d/f128 x:0 0 0 2024-04-12T05:56:19.609 INFO:tasks.workunit.client.1.smithi194.stdout:1/879: dwrite d0/d11/fae [0,4194304] 0 2024-04-12T05:56:19.613 INFO:tasks.workunit.client.1.smithi194.stdout:6/831: dread d3/d2f/d4e/d9a/dd8/f6a [0,4194304] 0 2024-04-12T05:56:19.618 INFO:tasks.workunit.client.1.smithi194.stdout:6/832: mknod d3/d2f/dd9/c102 0 2024-04-12T05:56:19.622 INFO:tasks.workunit.client.1.smithi194.stdout:6/833: rename d3/d2d/d33/db1/ld5 to d3/l103 0 2024-04-12T05:56:19.622 INFO:tasks.workunit.client.1.smithi194.stdout:7/741: creat d3/d27/d22/ff0 x:0 0 0 2024-04-12T05:56:19.624 INFO:tasks.workunit.client.1.smithi194.stdout:7/742: fdatasync d3/d27/f64 0 2024-04-12T05:56:19.625 INFO:tasks.workunit.client.1.smithi194.stdout:6/834: mknod d3/d2f/d4e/d68/c104 0 2024-04-12T05:56:19.627 INFO:tasks.workunit.client.1.smithi194.stdout:5/915: creat d1/d4/f129 x:0 0 0 2024-04-12T05:56:19.628 INFO:tasks.workunit.client.1.smithi194.stdout:7/743: symlink d3/d16/d5d/lf1 0 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: [12/Apr/2024:05:56:17] ENGINE Bus STARTING 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: [12/Apr/2024:05:56:18] ENGINE Serving on http://172.21.15.8:8765 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: [12/Apr/2024:05:56:18] ENGINE Serving on https://172.21.15.8:7150 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: [12/Apr/2024:05:56:18] ENGINE Bus STARTED 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: mgrmap e28: smithi008.ixmnlp(active, since 1.02321s) 2024-04-12T05:56:19.629 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:19 smithi008 ceph-mon[39722]: pgmap v3: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:19.631 INFO:tasks.workunit.client.1.smithi194.stdout:7/744: stat d3/d16/d23/d57/d72/d81/d3a/cca 0 2024-04-12T05:56:19.632 INFO:tasks.workunit.client.1.smithi194.stdout:5/916: write d1/d4/d6/d34/f54 [651031,76250] 0 2024-04-12T05:56:19.639 INFO:tasks.workunit.client.1.smithi194.stdout:6/835: dwrite d3/d2d/d33/fa1 [0,4194304] 0 2024-04-12T05:56:19.655 INFO:tasks.workunit.client.1.smithi194.stdout:7/745: rmdir d3/d16/d23/d37/d3d/d56/dd0/dd9 39 2024-04-12T05:56:19.656 INFO:tasks.workunit.client.1.smithi194.stdout:6/836: stat d3/d2f/d4e/d68/dad/lc6 0 2024-04-12T05:56:19.656 INFO:tasks.workunit.client.1.smithi194.stdout:5/917: dwrite d1/d4/d6/db/d15/d26/f7c [4194304,4194304] 0 2024-04-12T05:56:19.663 INFO:tasks.workunit.client.1.smithi194.stdout:7/746: symlink d3/d16/d7b/lf2 0 2024-04-12T05:56:19.665 INFO:tasks.workunit.client.1.smithi194.stdout:7/747: readlink d3/d16/d23/d37/d3d/l42 0 2024-04-12T05:56:19.669 INFO:tasks.workunit.client.1.smithi194.stdout:6/837: dread d3/d70/fab [0,4194304] 0 2024-04-12T05:56:19.673 INFO:tasks.workunit.client.1.smithi194.stdout:7/748: rename d3/d27/f39 to d3/def/ff3 0 2024-04-12T05:56:19.673 INFO:tasks.workunit.client.1.smithi194.stdout:5/918: link d1/d4/dd7/c104 d1/d4/d6/db/d15/d26/dbb/c12a 0 2024-04-12T05:56:19.677 INFO:tasks.workunit.client.1.smithi194.stdout:7/749: write d3/d27/d6a/dda/d74/fd5 [398404,49274] 0 2024-04-12T05:56:19.678 INFO:tasks.workunit.client.1.smithi194.stdout:6/838: dread d3/d15/f24 [0,4194304] 0 2024-04-12T05:56:19.679 INFO:tasks.workunit.client.1.smithi194.stdout:7/750: write d3/d16/d23/d37/fb3 [3841273,42164] 0 2024-04-12T05:56:19.688 INFO:tasks.workunit.client.1.smithi194.stdout:6/839: rename d3/d2d/d33/cae to d3/d2f/d4e/d63/d83/c105 0 2024-04-12T05:56:19.690 INFO:tasks.workunit.client.1.smithi194.stdout:7/751: dwrite d3/d27/fbb [0,4194304] 0 2024-04-12T05:56:19.693 INFO:tasks.workunit.client.1.smithi194.stdout:7/752: chown d3/d27/f46 709990962 1 2024-04-12T05:56:19.698 INFO:tasks.workunit.client.1.smithi194.stdout:6/840: link d3/d2d/d33/c65 d3/d15/d2b/d48/d6f/d7a/c106 0 2024-04-12T05:56:19.703 INFO:tasks.workunit.client.1.smithi194.stdout:7/753: chown d3/d16/d23/d57/d72/d81/d3a/d43/d5c/ce8 0 1 2024-04-12T05:56:19.706 INFO:tasks.workunit.client.1.smithi194.stdout:6/841: dwrite d3/d2f/f39 [4194304,4194304] 0 2024-04-12T05:56:19.709 INFO:tasks.workunit.client.1.smithi194.stdout:7/754: unlink d3/d27/f2c 0 2024-04-12T05:56:19.710 INFO:tasks.workunit.client.1.smithi194.stdout:6/842: mkdir d3/d2d/d33/d107 0 2024-04-12T05:56:19.712 INFO:tasks.workunit.client.1.smithi194.stdout:6/843: chown d3/d2f/d4e/d63/faa 0 1 2024-04-12T05:56:19.715 INFO:tasks.workunit.client.1.smithi194.stdout:7/755: fsync d3/d27/d22/fe0 0 2024-04-12T05:56:19.722 INFO:tasks.workunit.client.1.smithi194.stdout:7/756: creat d3/d16/d23/d57/d72/ff4 x:0 0 0 2024-04-12T05:56:19.725 INFO:tasks.workunit.client.1.smithi194.stdout:6/844: symlink d3/d15/dc8/l108 0 2024-04-12T05:56:19.727 INFO:tasks.workunit.client.1.smithi194.stdout:6/845: chown d3/fd4 58 1 2024-04-12T05:56:19.727 INFO:tasks.workunit.client.1.smithi194.stdout:5/919: dwrite d1/d4/d6/d34/d52/fcf [0,4194304] 0 2024-04-12T05:56:19.729 INFO:tasks.workunit.client.1.smithi194.stdout:5/920: chown d1/d4/d6/c20 8041 1 2024-04-12T05:56:19.730 INFO:tasks.workunit.client.1.smithi194.stdout:7/757: unlink d3/d16/d23/d40/c59 0 2024-04-12T05:56:19.734 INFO:tasks.workunit.client.1.smithi194.stdout:5/921: fdatasync d1/d4/d6/f3d 0 2024-04-12T05:56:19.735 INFO:tasks.workunit.client.1.smithi194.stdout:7/758: symlink d3/d16/d23/d37/lf5 0 2024-04-12T05:56:19.739 INFO:tasks.workunit.client.1.smithi194.stdout:5/922: symlink d1/d4/d6/db/l12b 0 2024-04-12T05:56:19.739 INFO:tasks.workunit.client.1.smithi194.stdout:7/759: stat l1 0 2024-04-12T05:56:19.746 INFO:tasks.workunit.client.1.smithi194.stdout:6/846: dwrite d3/d2f/d37/f81 [0,4194304] 0 2024-04-12T05:56:19.746 INFO:tasks.workunit.client.1.smithi194.stdout:7/760: mkdir d3/d16/d23/d57/df6 0 2024-04-12T05:56:19.749 INFO:tasks.workunit.client.1.smithi194.stdout:6/847: dread - d3/d2f/d4e/d9a/dd8/fa4 zero size 2024-04-12T05:56:19.753 INFO:tasks.workunit.client.1.smithi194.stdout:5/923: read d1/d4/d6/db/f7a [301285,99605] 0 2024-04-12T05:56:19.756 INFO:tasks.workunit.client.1.smithi194.stdout:5/924: symlink d1/d4/d6/d34/d52/l12c 0 2024-04-12T05:56:19.770 INFO:tasks.workunit.client.1.smithi194.stdout:5/925: dwrite d1/d4/f25 [8388608,4194304] 0 2024-04-12T05:56:19.771 INFO:tasks.workunit.client.1.smithi194.stdout:5/926: stat d1/d4/d6/d34/d52/fb8 0 2024-04-12T05:56:19.779 INFO:tasks.workunit.client.1.smithi194.stdout:6/848: dread d3/d15/d2b/d48/d6f/f82 [0,4194304] 0 2024-04-12T05:56:19.782 INFO:tasks.workunit.client.1.smithi194.stdout:5/927: unlink d1/d4/d6/d34/d52/c7f 0 2024-04-12T05:56:19.784 INFO:tasks.workunit.client.1.smithi194.stdout:5/928: write d1/d5c/d95/fc6 [230014,106347] 0 2024-04-12T05:56:19.785 INFO:tasks.workunit.client.1.smithi194.stdout:6/849: dread d3/d2d/d33/d58/fce [0,4194304] 0 2024-04-12T05:56:19.823 INFO:tasks.workunit.client.1.smithi194.stdout:5/929: mknod d1/d4/c12d 0 2024-04-12T05:56:19.824 INFO:tasks.workunit.client.1.smithi194.stdout:6/850: dread d3/d70/fab [0,4194304] 0 2024-04-12T05:56:19.827 INFO:tasks.workunit.client.1.smithi194.stdout:5/930: chown d1/d4/d6/db/d15/d26/c47 94 1 2024-04-12T05:56:19.829 INFO:tasks.workunit.client.1.smithi194.stdout:5/931: dread - d1/d4/d6/d88/fae zero size 2024-04-12T05:56:19.829 INFO:tasks.workunit.client.1.smithi194.stdout:6/851: creat d3/d2f/d4e/d68/dad/dc4/f109 x:0 0 0 2024-04-12T05:56:19.830 INFO:tasks.workunit.client.1.smithi194.stdout:5/932: chown d1/d4/f25 0 1 2024-04-12T05:56:19.834 INFO:tasks.workunit.client.1.smithi194.stdout:5/933: mknod d1/d5c/da7/c12e 0 2024-04-12T05:56:19.837 INFO:tasks.workunit.client.1.smithi194.stdout:6/852: rename d3/d2f/d4e/d56 to d3/d2d/d33/d52/d62/d10a 0 2024-04-12T05:56:19.837 INFO:tasks.workunit.client.1.smithi194.stdout:5/934: creat d1/d5c/da7/f12f x:0 0 0 2024-04-12T05:56:19.840 INFO:tasks.workunit.client.1.smithi194.stdout:6/853: rename d3/d2d/dc3 to d3/d70/d10b 0 2024-04-12T05:56:19.844 INFO:tasks.workunit.client.1.smithi194.stdout:5/935: symlink d1/d4/d6/d34/d52/l130 0 2024-04-12T05:56:19.854 INFO:tasks.workunit.client.1.smithi194.stdout:6/854: dwrite d3/d2d/d33/fa1 [0,4194304] 0 2024-04-12T05:56:19.856 INFO:tasks.workunit.client.1.smithi194.stdout:6/855: read d3/d2d/d33/fa1 [4101412,102067] 0 2024-04-12T05:56:19.859 INFO:tasks.workunit.client.1.smithi194.stdout:6/856: symlink d3/d2f/d4e/d63/d83/l10c 0 2024-04-12T05:56:19.860 INFO:tasks.workunit.client.1.smithi194.stdout:6/857: stat d3/d2d/d33/d52/d5c/c88 0 2024-04-12T05:56:19.863 INFO:tasks.workunit.client.1.smithi194.stdout:5/936: link d1/d5c/d95/df2/f3e d1/d4/d6/db/d15/f131 0 2024-04-12T05:56:19.865 INFO:tasks.workunit.client.1.smithi194.stdout:6/858: read d3/d2f/d4e/d63/f78 [969532,124851] 0 2024-04-12T05:56:19.867 INFO:tasks.workunit.client.1.smithi194.stdout:6/859: mkdir d3/d15/dc8/d10d 0 2024-04-12T05:56:19.868 INFO:tasks.workunit.client.1.smithi194.stdout:5/937: dread d1/d4/d6/d34/d52/fcf [0,4194304] 0 2024-04-12T05:56:19.877 INFO:tasks.workunit.client.1.smithi194.stdout:5/938: dread d1/d4/d6/db/d2b/f33 [0,4194304] 0 2024-04-12T05:56:19.881 INFO:tasks.workunit.client.1.smithi194.stdout:5/939: mknod d1/d4/d6/db/d15/d26/c132 0 2024-04-12T05:56:19.887 INFO:tasks.workunit.client.1.smithi194.stdout:5/940: dwrite d1/d4/d6/db/d38/ff0 [0,4194304] 0 2024-04-12T05:56:19.890 INFO:tasks.workunit.client.1.smithi194.stdout:5/941: unlink d1/d5c/d95/f2e 0 2024-04-12T05:56:19.896 INFO:tasks.workunit.client.1.smithi194.stdout:5/942: creat d1/d4/d6/db/d15/d26/f133 x:0 0 0 2024-04-12T05:56:19.897 INFO:tasks.workunit.client.1.smithi194.stdout:5/943: chown d1/d4/dd7/dfe/l125 0 1 2024-04-12T05:56:19.899 INFO:tasks.workunit.client.1.smithi194.stdout:5/944: mknod d1/d4/d6/db/d15/d26/dbb/df7/c134 0 2024-04-12T05:56:19.905 INFO:tasks.workunit.client.1.smithi194.stdout:5/945: link d1/d5c/d95/f69 d1/d4/dd7/f135 0 2024-04-12T05:56:19.907 INFO:tasks.workunit.client.1.smithi194.stdout:5/946: creat d1/d4/d6/d88/dc1/f136 x:0 0 0 2024-04-12T05:56:19.908 INFO:tasks.workunit.client.1.smithi194.stdout:5/947: write d1/d4/d6/d88/dc1/fe9 [1589413,120457] 0 2024-04-12T05:56:19.911 INFO:tasks.workunit.client.1.smithi194.stdout:5/948: mkdir d1/d4/dad/d137 0 2024-04-12T05:56:19.914 INFO:tasks.workunit.client.1.smithi194.stdout:5/949: mkdir d1/d4/d6/d115/d138 0 2024-04-12T05:56:19.915 INFO:tasks.workunit.client.1.smithi194.stdout:5/950: write d1/d4/d6/d88/dc1/f136 [1036948,6287] 0 2024-04-12T05:56:19.917 INFO:tasks.workunit.client.1.smithi194.stdout:5/951: mkdir d1/d5c/d139 0 2024-04-12T05:56:19.919 INFO:tasks.workunit.client.1.smithi194.stdout:5/952: symlink d1/d4/d6/db/d15/d26/dbb/df7/l13a 0 2024-04-12T05:56:19.920 INFO:tasks.workunit.client.1.smithi194.stdout:5/953: chown d1/d4/d6/d34/d52/lb2 80604148 1 2024-04-12T05:56:19.928 INFO:tasks.workunit.client.1.smithi194.stdout:5/954: dwrite d1/d5c/da7/fb1 [4194304,4194304] 0 2024-04-12T05:56:19.931 INFO:tasks.workunit.client.1.smithi194.stdout:5/955: chown d1/d4/d6/db/d15/d26/fcc 2437347 1 2024-04-12T05:56:19.940 INFO:tasks.workunit.client.1.smithi194.stdout:5/956: link d1/d4/d6/d8e/f81 d1/d4/d6/db/d15/f13b 0 2024-04-12T05:56:19.942 INFO:tasks.workunit.client.1.smithi194.stdout:5/957: dread d1/d5c/d95/fc6 [0,4194304] 0 2024-04-12T05:56:19.945 INFO:tasks.workunit.client.1.smithi194.stdout:5/958: creat d1/d4/d6/db/d38/dff/f13c x:0 0 0 2024-04-12T05:56:19.950 INFO:tasks.workunit.client.1.smithi194.stdout:5/959: getdents d1/d4/dd7 0 2024-04-12T05:56:19.954 INFO:tasks.workunit.client.1.smithi194.stdout:5/960: rename d1/d4/d6/d88/dc1 to d1/d4/d6/d6d/d13d 0 2024-04-12T05:56:19.955 INFO:tasks.workunit.client.1.smithi194.stdout:5/961: fsync d1/d4/d6/db/f53 0 2024-04-12T05:56:19.969 INFO:tasks.workunit.client.1.smithi194.stdout:5/962: dread d1/d4/d6/d88/f2c [0,4194304] 0 2024-04-12T05:56:19.973 INFO:tasks.workunit.client.1.smithi194.stdout:5/963: dread d1/d4/d6/db/d15/d26/f7c [4194304,4194304] 0 2024-04-12T05:56:19.973 INFO:tasks.workunit.client.1.smithi194.stdout:5/964: chown d1/d4/d6/d34/d52/f77 54774 1 2024-04-12T05:56:19.978 INFO:tasks.workunit.client.1.smithi194.stdout:5/965: rename d1/d4/d6/d115/d138 to d1/d4/d6/d115/d13e 0 2024-04-12T05:56:19.985 INFO:tasks.workunit.client.1.smithi194.stdout:5/966: mknod d1/d4/d6/db/d15/d26/c13f 0 2024-04-12T05:56:19.990 INFO:tasks.workunit.client.1.smithi194.stdout:5/967: creat d1/d4/d6/db/d38/f140 x:0 0 0 2024-04-12T05:56:19.998 INFO:tasks.workunit.client.1.smithi194.stdout:5/968: link d1/d4/d6/db/d15/f58 d1/d4/f141 0 2024-04-12T05:56:19.999 INFO:tasks.workunit.client.1.smithi194.stdout:5/969: write d1/d4/d6/db/d38/fc0 [74641,116674] 0 2024-04-12T05:56:20.003 INFO:tasks.workunit.client.1.smithi194.stdout:5/970: truncate d1/d4/d6/f13 4851709 0 2024-04-12T05:56:20.009 INFO:tasks.workunit.client.1.smithi194.stdout:2/794: sync 2024-04-12T05:56:20.009 INFO:tasks.workunit.client.1.smithi194.stdout:8/637: sync 2024-04-12T05:56:20.009 INFO:tasks.workunit.client.1.smithi194.stdout:9/862: sync 2024-04-12T05:56:20.009 INFO:tasks.workunit.client.1.smithi194.stdout:0/790: sync 2024-04-12T05:56:20.010 INFO:tasks.workunit.client.1.smithi194.stdout:5/971: chown d1/d4/d6/db/d15/f58 247398 1 2024-04-12T05:56:20.011 INFO:tasks.workunit.client.1.smithi194.stdout:0/791: symlink d0/d5f/d73/l101 0 2024-04-12T05:56:20.013 INFO:tasks.workunit.client.1.smithi194.stdout:5/972: symlink d1/d4/d6/db/d2b/l142 0 2024-04-12T05:56:20.014 INFO:tasks.workunit.client.1.smithi194.stdout:9/863: mknod d1/d41/dba/dc2/c122 0 2024-04-12T05:56:20.014 INFO:tasks.workunit.client.1.smithi194.stdout:5/973: write d1/d4/d75/fe8 [5034360,30453] 0 2024-04-12T05:56:20.015 INFO:tasks.workunit.client.1.smithi194.stdout:2/795: creat d7/d12/d1a/dbf/f10e x:0 0 0 2024-04-12T05:56:20.016 INFO:tasks.workunit.client.1.smithi194.stdout:8/638: fdatasync d2/d8/d9/de/d10/d4e/f8a 0 2024-04-12T05:56:20.022 INFO:tasks.workunit.client.1.smithi194.stdout:2/796: write d7/d12/d1a/d69/df0/d101/df9/fff [795511,93230] 0 2024-04-12T05:56:20.025 INFO:tasks.workunit.client.1.smithi194.stdout:0/792: dwrite d0/d5f/fcb [0,4194304] 0 2024-04-12T05:56:20.025 INFO:tasks.workunit.client.1.smithi194.stdout:8/639: dread d2/d8/f26 [0,4194304] 0 2024-04-12T05:56:20.030 INFO:tasks.workunit.client.1.smithi194.stdout:2/797: creat d7/d12/d1a/d69/df0/f10f x:0 0 0 2024-04-12T05:56:20.031 INFO:tasks.workunit.client.1.smithi194.stdout:9/864: dwrite d1/d4e/f6d [0,4194304] 0 2024-04-12T05:56:20.031 INFO:tasks.workunit.client.1.smithi194.stdout:8/640: fsync d2/d8/d9/de/d38/fb2 0 2024-04-12T05:56:20.032 INFO:tasks.workunit.client.1.smithi194.stdout:5/974: dwrite d1/d4/d6/f68 [0,4194304] 0 2024-04-12T05:56:20.035 INFO:tasks.workunit.client.1.smithi194.stdout:2/798: chown d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/ld0 44403370 1 2024-04-12T05:56:20.035 INFO:tasks.workunit.client.1.smithi194.stdout:0/793: dread d0/d3b/d52/dfa/d27/d54/f7b [0,4194304] 0 2024-04-12T05:56:20.036 INFO:tasks.workunit.client.1.smithi194.stdout:9/865: creat d1/d4e/dea/dfc/f123 x:0 0 0 2024-04-12T05:56:20.041 INFO:tasks.workunit.client.1.smithi194.stdout:0/794: rename d0/de3/df0/dbe/de4 to d0/d5f/d73/d102 0 2024-04-12T05:56:20.042 INFO:tasks.workunit.client.1.smithi194.stdout:9/866: read d1/d41/d97/f10c [1068285,96632] 0 2024-04-12T05:56:20.042 INFO:tasks.workunit.client.1.smithi194.stdout:8/641: link d2/d8/d15/d33/c4f d2/d8/d15/d33/dab/cc4 0 2024-04-12T05:56:20.044 INFO:tasks.workunit.client.1.smithi194.stdout:0/795: fsync d0/d3b/f74 0 2024-04-12T05:56:20.044 INFO:tasks.workunit.client.1.smithi194.stdout:9/867: chown d1/d41/f6f 23948674 1 2024-04-12T05:56:20.045 INFO:tasks.workunit.client.1.smithi194.stdout:2/799: dwrite d7/d22/d42/fb7 [0,4194304] 0 2024-04-12T05:56:20.048 INFO:tasks.workunit.client.1.smithi194.stdout:5/975: creat d1/d4/d84/f143 x:0 0 0 2024-04-12T05:56:20.049 INFO:tasks.workunit.client.1.smithi194.stdout:8/642: fsync d2/d8/d9/d1a/faf 0 2024-04-12T05:56:20.054 INFO:tasks.workunit.client.1.smithi194.stdout:0/796: rename d0/d5f/fc0 to d0/d3b/d52/dfa/d8f/f103 0 2024-04-12T05:56:20.057 INFO:tasks.workunit.client.1.smithi194.stdout:2/800: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/f110 x:0 0 0 2024-04-12T05:56:20.058 INFO:tasks.workunit.client.1.smithi194.stdout:8/643: mkdir d2/d8/dc5 0 2024-04-12T05:56:20.059 INFO:tasks.workunit.client.1.smithi194.stdout:9/868: dread d1/d19/d1f/f4b [0,4194304] 0 2024-04-12T05:56:20.061 INFO:tasks.workunit.client.1.smithi194.stdout:2/801: write d7/d12/d1a/d68/f95 [4567135,70349] 0 2024-04-12T05:56:20.061 INFO:tasks.workunit.client.1.smithi194.stdout:0/797: read d0/de3/df0/f9f [1074658,127939] 0 2024-04-12T05:56:20.067 INFO:tasks.workunit.client.1.smithi194.stdout:9/869: chown d1/d19/d1f/d20/d37/d84/l116 3581745 1 2024-04-12T05:56:20.069 INFO:tasks.workunit.client.1.smithi194.stdout:2/802: write d7/d22/d93/d9a/fc8 [3217953,28238] 0 2024-04-12T05:56:20.069 INFO:tasks.workunit.client.1.smithi194.stdout:5/976: dwrite d1/d4/d6/db/f7a [0,4194304] 0 2024-04-12T05:56:20.073 INFO:tasks.workunit.client.1.smithi194.stdout:9/870: chown d1/d19/d1f/d20/d37/d38/l50 4020489 1 2024-04-12T05:56:20.073 INFO:tasks.workunit.client.1.smithi194.stdout:5/977: chown d1/d5c/d95/df2/lbe 55 1 2024-04-12T05:56:20.075 INFO:tasks.workunit.client.1.smithi194.stdout:2/803: mkdir d7/d12/d25/d5d/d47/d52/d7b/db2/d111 0 2024-04-12T05:56:20.076 INFO:tasks.workunit.client.1.smithi194.stdout:8/644: dwrite d2/d8/d9/de/f37 [0,4194304] 0 2024-04-12T05:56:20.080 INFO:tasks.workunit.client.1.smithi194.stdout:0/798: truncate d0/da/da7/fbc 301056 0 2024-04-12T05:56:20.086 INFO:tasks.workunit.client.1.smithi194.stdout:2/804: truncate d7/d22/fc1 340043 0 2024-04-12T05:56:20.086 INFO:tasks.workunit.client.1.smithi194.stdout:9/871: dwrite d1/d19/d1f/d5f/f91 [0,4194304] 0 2024-04-12T05:56:20.087 INFO:tasks.workunit.client.1.smithi194.stdout:5/978: getdents d1/d4/dad 0 2024-04-12T05:56:20.087 INFO:tasks.workunit.client.1.smithi194.stdout:2/805: chown d7/d22/f91 29 1 2024-04-12T05:56:20.088 INFO:tasks.workunit.client.1.smithi194.stdout:8/645: rename d2/d8/d15/fa3 to d2/d8/d9/de/d38/d8f/fc6 0 2024-04-12T05:56:20.089 INFO:tasks.workunit.client.1.smithi194.stdout:9/872: fsync d1/d21/d39/f11e 0 2024-04-12T05:56:20.093 INFO:tasks.workunit.client.1.smithi194.stdout:5/979: mknod d1/d4/d6/db/d2b/c144 0 2024-04-12T05:56:20.095 INFO:tasks.workunit.client.1.smithi194.stdout:0/799: read d0/d3b/d52/dfa/f41 [2980424,125378] 0 2024-04-12T05:56:20.096 INFO:tasks.workunit.client.1.smithi194.stdout:2/806: truncate d7/d12/d25/d5d/d47/dac/fea 241720 0 2024-04-12T05:56:20.100 INFO:tasks.workunit.client.1.smithi194.stdout:5/980: mknod d1/d4/d9e/ddb/c145 0 2024-04-12T05:56:20.101 INFO:tasks.workunit.client.1.smithi194.stdout:0/800: mkdir d0/d5f/d73/d76/d104 0 2024-04-12T05:56:20.101 INFO:tasks.workunit.client.1.smithi194.stdout:2/807: unlink d7/d12/d1a/dbf/fda 0 2024-04-12T05:56:20.106 INFO:tasks.workunit.client.1.smithi194.stdout:8/646: dwrite d2/d8/d9/de/d38/d8f/d94/fbc [0,4194304] 0 2024-04-12T05:56:20.107 INFO:tasks.workunit.client.1.smithi194.stdout:3/944: sync 2024-04-12T05:56:20.107 INFO:tasks.workunit.client.1.smithi194.stdout:7/761: sync 2024-04-12T05:56:20.107 INFO:tasks.workunit.client.1.smithi194.stdout:1/880: sync 2024-04-12T05:56:20.108 INFO:tasks.workunit.client.1.smithi194.stdout:0/801: mknod d0/d5f/d73/d76/c105 0 2024-04-12T05:56:20.110 INFO:tasks.workunit.client.1.smithi194.stdout:5/981: getdents d1/d4/d4b 0 2024-04-12T05:56:20.115 INFO:tasks.workunit.client.1.smithi194.stdout:8/647: creat d2/d8/d9/de/d10/d7d/d84/d9f/fc7 x:0 0 0 2024-04-12T05:56:20.128 INFO:tasks.workunit.client.1.smithi194.stdout:3/945: write d0/d1/d10/d8c/d8f/f11f [529930,67925] 0 2024-04-12T05:56:20.132 INFO:tasks.workunit.client.1.smithi194.stdout:9/873: dwrite d1/d21/d77/d98/d9f/dde/fe7 [0,4194304] 0 2024-04-12T05:56:20.133 INFO:tasks.workunit.client.1.smithi194.stdout:7/762: truncate d3/d16/d23/d37/d3d/d56/f51 3629557 0 2024-04-12T05:56:20.133 INFO:tasks.workunit.client.1.smithi194.stdout:5/982: dread d1/d5c/d95/df2/fec [0,4194304] 0 2024-04-12T05:56:20.134 INFO:tasks.workunit.client.1.smithi194.stdout:1/881: dwrite d0/d11/d18/d19/f4e [0,4194304] 0 2024-04-12T05:56:20.134 INFO:tasks.workunit.client.1.smithi194.stdout:8/648: dread d2/d8/d9/de/d86/dbe/f71 [0,4194304] 0 2024-04-12T05:56:20.137 INFO:tasks.workunit.client.1.smithi194.stdout:9/874: chown d1/d41/dbf/lce 0 1 2024-04-12T05:56:20.137 INFO:tasks.workunit.client.1.smithi194.stdout:2/808: dread d7/d22/d93/d9f/fb5 [0,4194304] 0 2024-04-12T05:56:20.139 INFO:tasks.workunit.client.1.smithi194.stdout:3/946: creat d0/d1/d10/d89/d126/f143 x:0 0 0 2024-04-12T05:56:20.144 INFO:tasks.workunit.client.1.smithi194.stdout:2/809: chown d7/d22/f104 5002379 1 2024-04-12T05:56:20.149 INFO:tasks.workunit.client.1.smithi194.stdout:8/649: mkdir d2/d8/dc8 0 2024-04-12T05:56:20.149 INFO:tasks.workunit.client.1.smithi194.stdout:2/810: fsync d7/d12/d25/d5d/d47/d52/d7b/fb8 0 2024-04-12T05:56:20.151 INFO:tasks.workunit.client.1.smithi194.stdout:1/882: creat d0/d11/d18/d19/d1a/d6a/d27/d48/da7/f113 x:0 0 0 2024-04-12T05:56:20.153 INFO:tasks.workunit.client.1.smithi194.stdout:0/802: dread d0/d3b/d52/dfa/d1a/d24/da8/fe8 [0,4194304] 0 2024-04-12T05:56:20.154 INFO:tasks.workunit.client.1.smithi194.stdout:6/860: sync 2024-04-12T05:56:20.155 INFO:tasks.workunit.client.1.smithi194.stdout:7/763: rename d3/d16/d23/d37/d3d/d56/dd0/fe4 to d3/d16/d23/d57/d72/d81/ff7 0 2024-04-12T05:56:20.158 INFO:tasks.workunit.client.1.smithi194.stdout:2/811: creat d7/d12/d25/d5d/d47/d52/d7b/db2/f112 x:0 0 0 2024-04-12T05:56:20.161 INFO:tasks.workunit.client.1.smithi194.stdout:9/875: dwrite d1/d41/d97/f109 [0,4194304] 0 2024-04-12T05:56:20.162 INFO:tasks.workunit.client.1.smithi194.stdout:1/883: fsync d0/d11/d18/d19/d1a/d6a/d27/d9e/fad 0 2024-04-12T05:56:20.166 INFO:tasks.workunit.client.1.smithi194.stdout:1/884: chown d0/d11/c7a 282685 1 2024-04-12T05:56:20.167 INFO:tasks.workunit.client.1.smithi194.stdout:7/764: dread - d3/d16/d23/d57/d72/ff4 zero size 2024-04-12T05:56:20.168 INFO:tasks.workunit.client.1.smithi194.stdout:2/812: chown d7/d12/d1a/d69/d7e/lbd 10864893 1 2024-04-12T05:56:20.171 INFO:tasks.workunit.client.1.smithi194.stdout:3/947: dread d0/d42/f73 [0,4194304] 0 2024-04-12T05:56:20.173 INFO:tasks.workunit.client.1.smithi194.stdout:2/813: chown d7/d12/d1a/de9 504 1 2024-04-12T05:56:20.174 INFO:tasks.workunit.client.1.smithi194.stdout:9/876: chown d1/d19/d1f/d20/d37/d60/c8c 4161286 1 2024-04-12T05:56:20.179 INFO:tasks.workunit.client.1.smithi194.stdout:8/650: dwrite d2/d8/d9/de/d10/f88 [4194304,4194304] 0 2024-04-12T05:56:20.182 INFO:tasks.workunit.client.1.smithi194.stdout:1/885: creat d0/d11/d18/d19/d1c/f114 x:0 0 0 2024-04-12T05:56:20.186 INFO:tasks.workunit.client.1.smithi194.stdout:0/803: dwrite d0/de3/df0/f9f [0,4194304] 0 2024-04-12T05:56:20.190 INFO:tasks.workunit.client.1.smithi194.stdout:5/983: dread d1/d4/d6/db/d2b/fcd [0,4194304] 0 2024-04-12T05:56:20.197 INFO:tasks.workunit.client.1.smithi194.stdout:2/814: write d7/d12/d25/d5d/d47/d52/f55 [2285381,34820] 0 2024-04-12T05:56:20.197 INFO:tasks.workunit.client.1.smithi194.stdout:0/804: write d0/da/d4a/fca [964625,102100] 0 2024-04-12T05:56:20.199 INFO:tasks.workunit.client.1.smithi194.stdout:9/877: rename d1/d41/dba/fd2 to d1/d41/d97/d9d/f124 0 2024-04-12T05:56:20.199 INFO:tasks.workunit.client.1.smithi194.stdout:2/815: read - d7/d22/d93/f102 zero size 2024-04-12T05:56:20.201 INFO:tasks.workunit.client.1.smithi194.stdout:5/984: symlink d1/d4/d6/d6d/l146 0 2024-04-12T05:56:20.202 INFO:tasks.workunit.client.1.smithi194.stdout:6/861: dwrite d3/d1a/f23 [0,4194304] 0 2024-04-12T05:56:20.203 INFO:tasks.workunit.client.1.smithi194.stdout:1/886: stat d0/d11/d18/d19/d1a/d6a/d27/d9e 0 2024-04-12T05:56:20.212 INFO:tasks.workunit.client.1.smithi194.stdout:5/985: readlink d1/d4/d6/db/d2b/lce 0 2024-04-12T05:56:20.221 INFO:tasks.workunit.client.1.smithi194.stdout:8/651: dwrite d2/d8/d9/de/d38/f3a [0,4194304] 0 2024-04-12T05:56:20.221 INFO:tasks.workunit.client.1.smithi194.stdout:2/816: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/fc4 [436459,126911] 0 2024-04-12T05:56:20.221 INFO:tasks.workunit.client.1.smithi194.stdout:0/805: dwrite d0/de3/df0/f9f [0,4194304] 0 2024-04-12T05:56:20.222 INFO:tasks.workunit.client.1.smithi194.stdout:5/986: chown d1/d4/dad 370384366 1 2024-04-12T05:56:20.228 INFO:tasks.workunit.client.1.smithi194.stdout:3/948: dwrite d0/d1/d74/d39/d4d/deb/d3d/f69 [4194304,4194304] 0 2024-04-12T05:56:20.228 INFO:tasks.workunit.client.1.smithi194.stdout:7/765: dread d3/d16/d23/d57/d72/d81/d3a/fa7 [0,4194304] 0 2024-04-12T05:56:20.237 INFO:tasks.workunit.client.1.smithi194.stdout:0/806: dwrite d0/da/f3e [4194304,4194304] 0 2024-04-12T05:56:20.240 INFO:tasks.workunit.client.1.smithi194.stdout:9/878: link d1/d19/d1f/d20/d37/l93 d1/d19/l125 0 2024-04-12T05:56:20.241 INFO:tasks.workunit.client.1.smithi194.stdout:1/887: write d0/d11/d18/d19/d1c/d67/fde [340457,22648] 0 2024-04-12T05:56:20.245 INFO:tasks.workunit.client.1.smithi194.stdout:9/879: truncate d1/d41/dba/dc2/fc3 1084738 0 2024-04-12T05:56:20.249 INFO:tasks.workunit.client.1.smithi194.stdout:3/949: truncate d0/d1/d74/d39/d4d/deb/d1e/d33/fca 604864 0 2024-04-12T05:56:20.250 INFO:tasks.workunit.client.1.smithi194.stdout:0/807: creat d0/d3b/d52/dfa/d1a/d3d/f106 x:0 0 0 2024-04-12T05:56:20.251 INFO:tasks.workunit.client.1.smithi194.stdout:8/652: creat d2/d8/d9/de/d10/d7d/d84/fc9 x:0 0 0 2024-04-12T05:56:20.254 INFO:tasks.workunit.client.1.smithi194.stdout:7/766: creat d3/d16/d23/d37/d3d/d56/dd0/dd9/ff8 x:0 0 0 2024-04-12T05:56:20.255 INFO:tasks.workunit.client.1.smithi194.stdout:5/987: dwrite d1/d4/d6/db/d2b/f66 [0,4194304] 0 2024-04-12T05:56:20.260 INFO:tasks.workunit.client.1.smithi194.stdout:9/880: rename d1/d41/fff to d1/d21/d77/d98/dab/f126 0 2024-04-12T05:56:20.263 INFO:tasks.workunit.client.1.smithi194.stdout:6/862: dwrite d3/d15/f24 [0,4194304] 0 2024-04-12T05:56:20.265 INFO:tasks.workunit.client.1.smithi194.stdout:9/881: rmdir d1/da6 39 2024-04-12T05:56:20.265 INFO:tasks.workunit.client.1.smithi194.stdout:5/988: rmdir d1/d4/d6/db/d15/d26 39 2024-04-12T05:56:20.274 INFO:tasks.workunit.client.1.smithi194.stdout:0/808: rename d0/d3b/d52/dfa/d1a/f7d to d0/da/d4a/d8e/f107 0 2024-04-12T05:56:20.275 INFO:tasks.workunit.client.1.smithi194.stdout:5/989: unlink d1/d4/c12d 0 2024-04-12T05:56:20.278 INFO:tasks.workunit.client.1.smithi194.stdout:6/863: symlink d3/d2d/d33/d52/d62/d10a/l10e 0 2024-04-12T05:56:20.278 INFO:tasks.workunit.client.1.smithi194.stdout:3/950: truncate d0/d1/d74/d39/d4d/deb/d1e/d33/fca 219737 0 2024-04-12T05:56:20.279 INFO:tasks.workunit.client.1.smithi194.stdout:7/767: link d3/d16/d23/d37/d3d/l42 d3/d16/d23/d40/d6f/lf9 0 2024-04-12T05:56:20.283 INFO:tasks.workunit.client.1.smithi194.stdout:3/951: read d0/d1/d74/d39/da2/fbd [2815169,1653] 0 2024-04-12T05:56:20.286 INFO:tasks.workunit.client.1.smithi194.stdout:5/990: link d1/d4/d6/d34/d52/fcf d1/d4/d6/db/d38/f147 0 2024-04-12T05:56:20.292 INFO:tasks.workunit.client.1.smithi194.stdout:3/952: link d0/d1/d74/c3c d0/d1/d74/d39/d4d/c144 0 2024-04-12T05:56:20.298 INFO:tasks.workunit.client.1.smithi194.stdout:3/953: creat d0/d1/d74/d39/d4d/d63/db5/f145 x:0 0 0 2024-04-12T05:56:20.298 INFO:tasks.workunit.client.1.smithi194.stdout:3/954: stat d0/d1 0 2024-04-12T05:56:20.301 INFO:tasks.workunit.client.1.smithi194.stdout:5/991: rmdir d1/d4/dad/d137 0 2024-04-12T05:56:20.303 INFO:tasks.workunit.client.1.smithi194.stdout:5/992: chown d1/d4/d6/db/d38/f140 81 1 2024-04-12T05:56:20.304 INFO:tasks.workunit.client.1.smithi194.stdout:3/955: mkdir d0/d1/d74/d39/d4d/deb/d8e/d13b/d146 0 2024-04-12T05:56:20.309 INFO:tasks.workunit.client.1.smithi194.stdout:7/768: dread d3/d27/d6a/f55 [0,4194304] 0 2024-04-12T05:56:20.309 INFO:tasks.workunit.client.1.smithi194.stdout:0/809: fdatasync d0/d3b/d52/dfa/d27/fd6 0 2024-04-12T05:56:20.310 INFO:tasks.workunit.client.1.smithi194.stdout:2/817: truncate d7/d12/d1a/f24 460008 0 2024-04-12T05:56:20.310 INFO:tasks.workunit.client.1.smithi194.stdout:3/956: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/f147 x:0 0 0 2024-04-12T05:56:20.313 INFO:tasks.workunit.client.1.smithi194.stdout:2/818: read d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/faf [3865105,72399] 0 2024-04-12T05:56:20.324 INFO:tasks.workunit.client.1.smithi194.stdout:8/653: write d2/d8/d9/de/d10/f24 [747022,94690] 0 2024-04-12T05:56:20.327 INFO:tasks.workunit.client.1.smithi194.stdout:5/993: fdatasync f0 0 2024-04-12T05:56:20.331 INFO:tasks.workunit.client.1.smithi194.stdout:3/957: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f103 [0,4194304] 0 2024-04-12T05:56:20.331 INFO:tasks.workunit.client.1.smithi194.stdout:8/654: chown d2/d8/d9/de/l1f 3851 1 2024-04-12T05:56:20.333 INFO:tasks.workunit.client.1.smithi194.stdout:7/769: dwrite d3/d16/d23/f83 [0,4194304] 0 2024-04-12T05:56:20.334 INFO:tasks.workunit.client.1.smithi194.stdout:2/819: rename d7 to d7/d22/d93/d113 22 2024-04-12T05:56:20.339 INFO:tasks.workunit.client.1.smithi194.stdout:7/770: stat d3/d16/d7b/lf2 0 2024-04-12T05:56:20.339 INFO:tasks.workunit.client.1.smithi194.stdout:5/994: mknod d1/d4/d6/d6d/c148 0 2024-04-12T05:56:20.339 INFO:tasks.workunit.client.1.smithi194.stdout:2/820: truncate d7/d22/dc9/fee 664337 0 2024-04-12T05:56:20.342 INFO:tasks.workunit.client.1.smithi194.stdout:8/655: mkdir d2/d8/d66/da4/dca 0 2024-04-12T05:56:20.348 INFO:tasks.workunit.client.1.smithi194.stdout:5/995: dwrite d1/d4/d6/db/fdd [0,4194304] 0 2024-04-12T05:56:20.349 INFO:tasks.workunit.client.1.smithi194.stdout:3/958: creat d0/d1/d4f/dd7/d141/f148 x:0 0 0 2024-04-12T05:56:20.350 INFO:tasks.workunit.client.1.smithi194.stdout:7/771: symlink d3/d16/d23/d57/df6/lfa 0 2024-04-12T05:56:20.351 INFO:tasks.workunit.client.1.smithi194.stdout:5/996: write d1/d5c/da7/f12f [187764,130152] 0 2024-04-12T05:56:20.351 INFO:tasks.workunit.client.1.smithi194.stdout:2/821: symlink d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103/l114 0 2024-04-12T05:56:20.363 INFO:tasks.workunit.client.1.smithi194.stdout:9/882: mknod d1/d41/dba/c127 0 2024-04-12T05:56:20.363 INFO:tasks.workunit.client.1.smithi194.stdout:8/656: getdents d2/d8/d9/de/d38 0 2024-04-12T05:56:20.365 INFO:tasks.workunit.client.1.smithi194.stdout:3/959: fsync d0/d1/d74/d39/d4d/deb/d1e/d81/d34/f46 0 2024-04-12T05:56:20.372 INFO:tasks.workunit.client.1.smithi194.stdout:5/997: dwrite d1/d4/d4b/f108 [0,4194304] 0 2024-04-12T05:56:20.372 INFO:tasks.workunit.client.1.smithi194.stdout:0/810: mknod d0/d3b/d52/dfa/d1a/c108 0 2024-04-12T05:56:20.374 INFO:tasks.workunit.client.1.smithi194.stdout:9/883: mkdir d1/d21/d77/d98/dab/d128 0 2024-04-12T05:56:20.374 INFO:tasks.workunit.client.1.smithi194.stdout:7/772: dwrite d3/d27/d6a/f82 [0,4194304] 0 2024-04-12T05:56:20.376 INFO:tasks.workunit.client.1.smithi194.stdout:8/657: creat d2/d8/d15/d33/d49/fcb x:0 0 0 2024-04-12T05:56:20.378 INFO:tasks.workunit.client.1.smithi194.stdout:6/864: truncate d3/d2d/d33/d58/fce 763253 0 2024-04-12T05:56:20.379 INFO:tasks.workunit.client.1.smithi194.stdout:3/960: symlink d0/d1/d74/d39/d4d/deb/d1e/d33/d76/d9a/d12c/l149 0 2024-04-12T05:56:20.379 INFO:tasks.workunit.client.1.smithi194.stdout:5/998: creat d1/d4/d6/d115/d13e/f149 x:0 0 0 2024-04-12T05:56:20.381 INFO:tasks.workunit.client.1.smithi194.stdout:0/811: link d0/f30 d0/d5f/d73/d102/f109 0 2024-04-12T05:56:20.383 INFO:tasks.workunit.client.1.smithi194.stdout:9/884: chown d1/d19/d1f/d20/d37/d84/c105 27271355 1 2024-04-12T05:56:20.386 INFO:tasks.workunit.client.1.smithi194.stdout:0/812: rmdir d0/d3b/d52/dfa/d1a/d24 39 2024-04-12T05:56:20.388 INFO:tasks.workunit.client.1.smithi194.stdout:9/885: fsync d1/d41/dbf/ff6 0 2024-04-12T05:56:20.394 INFO:tasks.workunit.client.1.smithi194.stdout:9/886: stat d1/da6/d10f/fad 0 2024-04-12T05:56:20.395 INFO:tasks.workunit.client.1.smithi194.stdout:0/813: write d0/d3b/d52/dfa/d1a/d24/d56/f8d [9039059,6471] 0 2024-04-12T05:56:20.396 INFO:tasks.workunit.client.1.smithi194.stdout:6/865: dwrite d3/d1a/f35 [0,4194304] 0 2024-04-12T05:56:20.397 INFO:tasks.workunit.client.1.smithi194.stdout:6/866: write d3/d15/f24 [411036,12719] 0 2024-04-12T05:56:20.400 INFO:tasks.workunit.client.1.smithi194.stdout:2/822: rename d7/d12/d1a/c40 to d7/d12/d1a/d69/df0/d101/df9/c115 0 2024-04-12T05:56:20.401 INFO:tasks.workunit.client.1.smithi194.stdout:6/867: fsync d3/d1a/f23 0 2024-04-12T05:56:20.409 INFO:tasks.workunit.client.1.smithi194.stdout:3/961: dwrite d0/d1/d4f/f133 [0,4194304] 0 2024-04-12T05:56:20.411 INFO:tasks.workunit.client.1.smithi194.stdout:5/999: dread d1/d4/d75/fbf [0,4194304] 0 2024-04-12T05:56:20.412 INFO:tasks.workunit.client.1.smithi194.stdout:2/823: rmdir d7/d12/d25/d5d/d47/d52/d7b/db2 39 2024-04-12T05:56:20.412 INFO:tasks.workunit.client.1.smithi194.stdout:7/773: rename d3/d27/d6a/dda/d74/c88 to d3/d27/d22/cfb 0 2024-04-12T05:56:20.413 INFO:tasks.workunit.client.1.smithi194.stdout:2/824: fdatasync d7/f11 0 2024-04-12T05:56:20.416 INFO:tasks.workunit.client.1.smithi194.stdout:3/962: creat d0/d1/d74/d39/d4d/deb/d1e/d81/d134/f14a x:0 0 0 2024-04-12T05:56:20.416 INFO:tasks.workunit.client.1.smithi194.stdout:6/868: symlink d3/d2f/d4e/d63/d83/l10f 0 2024-04-12T05:56:20.417 INFO:tasks.workunit.client.1.smithi194.stdout:3/963: readlink d0/d1/d74/d39/da2/l11a 0 2024-04-12T05:56:20.417 INFO:tasks.workunit.client.1.smithi194.stdout:7/774: unlink d3/d16/d7b/dc3/ce2 0 2024-04-12T05:56:20.421 INFO:tasks.workunit.client.1.smithi194.stdout:2/825: mknod d7/d12/d25/d5d/d47/d52/d7b/db2/c116 0 2024-04-12T05:56:20.422 INFO:tasks.workunit.client.1.smithi194.stdout:8/658: truncate d2/d8/d15/d33/d45/f55 729670 0 2024-04-12T05:56:20.424 INFO:tasks.workunit.client.1.smithi194.stdout:8/659: chown d2/c63 54814 1 2024-04-12T05:56:20.424 INFO:tasks.workunit.client.1.smithi194.stdout:2/826: truncate d7/d12/d1a/f3e 5010174 0 2024-04-12T05:56:20.426 INFO:tasks.workunit.client.1.smithi194.stdout:7/775: creat d3/def/ffc x:0 0 0 2024-04-12T05:56:20.430 INFO:tasks.workunit.client.1.smithi194.stdout:8/660: unlink d2/d8/d9/de/d10/d4e/la6 0 2024-04-12T05:56:20.430 INFO:tasks.workunit.client.1.smithi194.stdout:7/776: unlink d3/d27/d22/d31/lce 0 2024-04-12T05:56:20.431 INFO:tasks.workunit.client.1.smithi194.stdout:9/887: write d1/d19/d1f/d20/f35 [281087,121249] 0 2024-04-12T05:56:20.433 INFO:tasks.workunit.client.1.smithi194.stdout:7/777: chown d3/d27/d22/d8e/cdc 108143 1 2024-04-12T05:56:20.433 INFO:tasks.workunit.client.1.smithi194.stdout:2/827: mkdir d7/d12/d25/d5d/d47/dac/db3/dd7/d117 0 2024-04-12T05:56:20.434 INFO:tasks.workunit.client.1.smithi194.stdout:8/661: mkdir d2/d8/d9/de/d10/dcc 0 2024-04-12T05:56:20.436 INFO:tasks.workunit.client.1.smithi194.stdout:6/869: write d3/d70/fab [785540,20557] 0 2024-04-12T05:56:20.438 INFO:tasks.workunit.client.1.smithi194.stdout:8/662: write d2/d8/d9/de/d10/f88 [3887691,72430] 0 2024-04-12T05:56:20.439 INFO:tasks.workunit.client.1.smithi194.stdout:7/778: link d3/d27/f1b d3/d16/d23/d57/d72/d81/d3a/ffd 0 2024-04-12T05:56:20.442 INFO:tasks.workunit.client.1.smithi194.stdout:9/888: creat d1/d21/d77/d98/f129 x:0 0 0 2024-04-12T05:56:20.442 INFO:tasks.workunit.client.1.smithi194.stdout:6/870: mknod d3/d2d/d33/d52/d62/ddd/c110 0 2024-04-12T05:56:20.446 INFO:tasks.workunit.client.1.smithi194.stdout:8/663: rename d2/d8/d9/de/d10/d7d/d84/fc9 to d2/d8/d9/de/d38/d8f/fcd 0 2024-04-12T05:56:20.449 INFO:tasks.workunit.client.1.smithi194.stdout:9/889: readlink d1/d21/lc1 0 2024-04-12T05:56:20.449 INFO:tasks.workunit.client.1.smithi194.stdout:7/779: link d3/d16/d23/d37/d3d/d56/f87 d3/d27/ffe 0 2024-04-12T05:56:20.449 INFO:tasks.workunit.client.1.smithi194.stdout:6/871: rmdir d3/d2d/d33 39 2024-04-12T05:56:20.452 INFO:tasks.workunit.client.1.smithi194.stdout:8/664: read d2/d8/d9/de/f61 [18096,88152] 0 2024-04-12T05:56:20.452 INFO:tasks.workunit.client.1.smithi194.stdout:3/964: dwrite d0/d1/d74/d39/d4d/deb/f88 [4194304,4194304] 0 2024-04-12T05:56:20.452 INFO:tasks.workunit.client.1.smithi194.stdout:7/780: write d3/d27/d22/ff0 [245595,124081] 0 2024-04-12T05:56:20.452 INFO:tasks.workunit.client.1.smithi194.stdout:2/828: write d7/f48 [600274,65116] 0 2024-04-12T05:56:20.456 INFO:tasks.workunit.client.1.smithi194.stdout:3/965: fsync d0/d1/d74/d39/f7e 0 2024-04-12T05:56:20.456 INFO:tasks.workunit.client.1.smithi194.stdout:2/829: write d7/d12/f3d [3050635,89921] 0 2024-04-12T05:56:20.459 INFO:tasks.workunit.client.1.smithi194.stdout:6/872: truncate d3/d2d/fcb 3476846 0 2024-04-12T05:56:20.462 INFO:tasks.workunit.client.1.smithi194.stdout:3/966: fsync d0/d1/d74/d39/d4d/deb/d3d/f12f 0 2024-04-12T05:56:20.463 INFO:tasks.workunit.client.1.smithi194.stdout:9/890: creat d1/d19/d1f/d20/d37/f12a x:0 0 0 2024-04-12T05:56:20.464 INFO:tasks.workunit.client.1.smithi194.stdout:2/830: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f118 x:0 0 0 2024-04-12T05:56:20.466 INFO:tasks.workunit.client.1.smithi194.stdout:6/873: creat d3/d2d/d33/d52/d62/d10a/de2/f111 x:0 0 0 2024-04-12T05:56:20.467 INFO:tasks.workunit.client.1.smithi194.stdout:8/665: getdents d2/d8/d66/da4/dca 0 2024-04-12T05:56:20.468 INFO:tasks.workunit.client.1.smithi194.stdout:2/831: dread d7/d12/d1a/d69/f82 [0,4194304] 0 2024-04-12T05:56:20.469 INFO:tasks.workunit.client.1.smithi194.stdout:7/781: getdents d3/d16/d23/d57/d72/d81/d3a 0 2024-04-12T05:56:20.470 INFO:tasks.workunit.client.1.smithi194.stdout:2/832: write d7/d12/d25/d5d/f51 [6543553,106822] 0 2024-04-12T05:56:20.472 INFO:tasks.workunit.client.1.smithi194.stdout:7/782: creat d3/d27/d22/d8e/fff x:0 0 0 2024-04-12T05:56:20.474 INFO:tasks.workunit.client.1.smithi194.stdout:8/666: dread d2/d8/d9/d1a/f25 [0,4194304] 0 2024-04-12T05:56:20.474 INFO:tasks.workunit.client.1.smithi194.stdout:2/833: truncate d7/d12/d25/d5d/d47/d52/d7b/db2/f112 351174 0 2024-04-12T05:56:20.477 INFO:tasks.workunit.client.1.smithi194.stdout:7/783: rename d3/d27/d6a/ce6 to d3/d16/d23/d37/c100 0 2024-04-12T05:56:20.480 INFO:tasks.workunit.client.1.smithi194.stdout:2/834: dwrite d7/d22/d93/d9a/fc8 [0,4194304] 0 2024-04-12T05:56:20.485 INFO:tasks.workunit.client.1.smithi194.stdout:7/784: dwrite d3/d16/d23/d37/d3d/f61 [0,4194304] 0 2024-04-12T05:56:20.490 INFO:tasks.workunit.client.1.smithi194.stdout:7/785: creat d3/d16/f101 x:0 0 0 2024-04-12T05:56:20.491 INFO:tasks.workunit.client.1.smithi194.stdout:7/786: stat d3/d16/d23/d37/f7c 0 2024-04-12T05:56:20.492 INFO:tasks.workunit.client.1.smithi194.stdout:7/787: stat d3/d16/d23/d37/c7a 0 2024-04-12T05:56:20.497 INFO:tasks.workunit.client.1.smithi194.stdout:7/788: rename d3/d16/d7b/lf2 to d3/d16/d23/d57/d72/d81/d3a/d8f/l102 0 2024-04-12T05:56:20.499 INFO:tasks.workunit.client.1.smithi194.stdout:7/789: unlink d3/d16/d5d/fbd 0 2024-04-12T05:56:20.513 INFO:tasks.workunit.client.1.smithi194.stdout:7/790: truncate d3/d16/d7b/fb9 1625300 0 2024-04-12T05:56:20.514 INFO:tasks.workunit.client.1.smithi194.stdout:7/791: chown d3/d16/d23/d37/d3d/d56/dd0/dd9/fed 15 1 2024-04-12T05:56:20.517 INFO:tasks.workunit.client.1.smithi194.stdout:7/792: mkdir d3/d27/d22/d31/d103 0 2024-04-12T05:56:20.521 INFO:tasks.workunit.client.1.smithi194.stdout:1/888: sync 2024-04-12T05:56:20.521 INFO:tasks.workunit.client.1.smithi194.stdout:7/793: rename d3/d27/d22/d8e/cb1 to d3/d16/d23/d40/c104 0 2024-04-12T05:56:20.522 INFO:tasks.workunit.client.1.smithi194.stdout:7/794: chown d3/d27/f69 78 1 2024-04-12T05:56:20.527 INFO:tasks.workunit.client.1.smithi194.stdout:7/795: dread d3/d27/fbb [0,4194304] 0 2024-04-12T05:56:20.529 INFO:tasks.workunit.client.1.smithi194.stdout:9/891: link d1/d21/d77/f92 d1/d41/dba/f12b 0 2024-04-12T05:56:20.531 INFO:tasks.workunit.client.1.smithi194.stdout:2/835: getdents d7/d22/dc9 0 2024-04-12T05:56:20.532 INFO:tasks.workunit.client.1.smithi194.stdout:1/889: write d0/d11/d18/d19/d1c/dca/fd0 [534792,73983] 0 2024-04-12T05:56:20.534 INFO:tasks.workunit.client.1.smithi194.stdout:9/892: truncate d1/d4e/dea/dfc/f123 476949 0 2024-04-12T05:56:20.536 INFO:tasks.workunit.client.1.smithi194.stdout:1/890: creat d0/f115 x:0 0 0 2024-04-12T05:56:20.537 INFO:tasks.workunit.client.1.smithi194.stdout:2/836: truncate d7/d12/f41 3284841 0 2024-04-12T05:56:20.538 INFO:tasks.workunit.client.1.smithi194.stdout:7/796: dwrite d3/d27/d6a/dda/d74/fd5 [0,4194304] 0 2024-04-12T05:56:20.539 INFO:tasks.workunit.client.1.smithi194.stdout:9/893: creat d1/d21/d39/df1/f12c x:0 0 0 2024-04-12T05:56:20.542 INFO:tasks.workunit.client.1.smithi194.stdout:7/797: mknod d3/def/c105 0 2024-04-12T05:56:20.545 INFO:tasks.workunit.client.1.smithi194.stdout:7/798: chown d3/d16/d23/d37/d95/lb2 636710 1 2024-04-12T05:56:20.546 INFO:tasks.workunit.client.1.smithi194.stdout:2/837: rename d7/d12/d25/d5d/d47/d52/l109 to d7/d12/d1a/d69/l119 0 2024-04-12T05:56:20.548 INFO:tasks.workunit.client.1.smithi194.stdout:1/891: dwrite d0/d11/d18/d19/f39 [0,4194304] 0 2024-04-12T05:56:20.550 INFO:tasks.workunit.client.1.smithi194.stdout:2/838: write d7/d12/d1a/d69/d7e/fe4 [782473,80127] 0 2024-04-12T05:56:20.551 INFO:tasks.workunit.client.1.smithi194.stdout:7/799: creat d3/d16/d23/d57/d72/d81/d3a/f106 x:0 0 0 2024-04-12T05:56:20.553 INFO:tasks.workunit.client.1.smithi194.stdout:1/892: mknod d0/d11/d18/d19/d1a/d6a/d27/d48/da7/c116 0 2024-04-12T05:56:20.555 INFO:tasks.workunit.client.1.smithi194.stdout:7/800: stat d3/d16/d23/d57/d72/d81/d3a/d43/d5c/ce8 0 2024-04-12T05:56:20.561 INFO:tasks.workunit.client.1.smithi194.stdout:7/801: creat d3/d16/d23/d57/d72/d81/d3a/dcc/f107 x:0 0 0 2024-04-12T05:56:20.565 INFO:tasks.workunit.client.1.smithi194.stdout:1/893: write d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fdb [271269,42642] 0 2024-04-12T05:56:20.568 INFO:tasks.workunit.client.1.smithi194.stdout:2/839: dread d7/d12/d25/d5d/f2f [0,4194304] 0 2024-04-12T05:56:20.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:20 smithi194 ceph-mon[49771]: pgmap v4: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:20.570 INFO:tasks.workunit.client.1.smithi194.stdout:1/894: mkdir d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/d117 0 2024-04-12T05:56:20.572 INFO:tasks.workunit.client.1.smithi194.stdout:7/802: getdents d3/d16/d23/d37/d3d/d56/dd0 0 2024-04-12T05:56:20.573 INFO:tasks.workunit.client.1.smithi194.stdout:1/895: creat d0/d11/f118 x:0 0 0 2024-04-12T05:56:20.573 INFO:tasks.workunit.client.1.smithi194.stdout:7/803: readlink d3/d16/d23/d40/d9b/la6 0 2024-04-12T05:56:20.574 INFO:tasks.workunit.client.1.smithi194.stdout:2/840: creat d7/d12/d25/d5d/d47/d52/d7b/db2/d111/f11a x:0 0 0 2024-04-12T05:56:20.576 INFO:tasks.workunit.client.1.smithi194.stdout:9/894: dread d1/d19/d1f/f70 [0,4194304] 0 2024-04-12T05:56:20.577 INFO:tasks.workunit.client.1.smithi194.stdout:1/896: mknod d0/d11/d18/d19/d1a/ded/c119 0 2024-04-12T05:56:20.578 INFO:tasks.workunit.client.1.smithi194.stdout:7/804: mknod d3/d27/d22/d31/c108 0 2024-04-12T05:56:20.579 INFO:tasks.workunit.client.1.smithi194.stdout:9/895: mkdir d1/d21/d77/d12d 0 2024-04-12T05:56:20.583 INFO:tasks.workunit.client.1.smithi194.stdout:2/841: dwrite d7/d12/f3b [0,4194304] 0 2024-04-12T05:56:20.583 INFO:tasks.workunit.client.1.smithi194.stdout:1/897: mknod d0/d11/d18/d19/d1c/dca/c11a 0 2024-04-12T05:56:20.593 INFO:tasks.workunit.client.1.smithi194.stdout:7/805: write d3/d16/d23/d57/fab [3230715,125544] 0 2024-04-12T05:56:20.603 INFO:tasks.workunit.client.1.smithi194.stdout:1/898: dwrite d0/d11/d18/d19/d1a/d6a/d27/d48/f62 [0,4194304] 0 2024-04-12T05:56:20.607 INFO:tasks.workunit.client.1.smithi194.stdout:2/842: dread d7/d12/d1a/d68/f95 [0,4194304] 0 2024-04-12T05:56:20.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/896: dread d1/d41/d97/d9d/f124 [0,4194304] 0 2024-04-12T05:56:20.612 INFO:tasks.workunit.client.1.smithi194.stdout:1/899: stat d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fdb 0 2024-04-12T05:56:20.619 INFO:tasks.workunit.client.1.smithi194.stdout:7/806: dwrite d3/d16/d23/d57/d72/d81/d3a/f60 [4194304,4194304] 0 2024-04-12T05:56:20.619 INFO:tasks.workunit.client.1.smithi194.stdout:9/897: rename d1/d21/d39/f89 to d1/da6/f12e 0 2024-04-12T05:56:20.619 INFO:tasks.workunit.client.1.smithi194.stdout:1/900: dread - d0/d11/d18/d19/d1a/d6a/dac/db5/df8/f106 zero size 2024-04-12T05:56:20.619 INFO:tasks.workunit.client.1.smithi194.stdout:7/807: chown d3/d27/d6a/dda/l4b 125531392 1 2024-04-12T05:56:20.623 INFO:tasks.workunit.client.1.smithi194.stdout:1/901: symlink d0/d11/d18/d19/d1a/d6a/dac/db5/d68/l11b 0 2024-04-12T05:56:20.624 INFO:tasks.workunit.client.1.smithi194.stdout:1/902: stat d0/d11/d18/d19/d1a/d6a/d27/da8/c10f 0 2024-04-12T05:56:20.629 INFO:tasks.workunit.client.1.smithi194.stdout:7/808: dwrite d3/d16/f45 [0,4194304] 0 2024-04-12T05:56:20.631 INFO:tasks.workunit.client.1.smithi194.stdout:1/903: mknod d0/d11/d18/d19/d1a/d6a/d27/d6c/da0/c11c 0 2024-04-12T05:56:20.633 INFO:tasks.workunit.client.1.smithi194.stdout:1/904: write d0/d11/d18/d19/d1a/d6a/d27/d2e/fb3 [313633,20932] 0 2024-04-12T05:56:20.635 INFO:tasks.workunit.client.1.smithi194.stdout:7/809: creat d3/dd8/f109 x:0 0 0 2024-04-12T05:56:20.635 INFO:tasks.workunit.client.1.smithi194.stdout:2/843: dread d7/d12/f3d [0,4194304] 0 2024-04-12T05:56:20.637 INFO:tasks.workunit.client.1.smithi194.stdout:7/810: write d3/d27/d22/fe0 [1004140,79872] 0 2024-04-12T05:56:20.641 INFO:tasks.workunit.client.1.smithi194.stdout:1/905: unlink d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ca3 0 2024-04-12T05:56:20.643 INFO:tasks.workunit.client.1.smithi194.stdout:1/906: fsync d0/d11/d18/d19/d1a/fa6 0 2024-04-12T05:56:20.643 INFO:tasks.workunit.client.1.smithi194.stdout:7/811: creat d3/d16/d23/d57/d72/d81/d3a/dcc/f10a x:0 0 0 2024-04-12T05:56:20.647 INFO:tasks.workunit.client.1.smithi194.stdout:1/907: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/d117/f11d x:0 0 0 2024-04-12T05:56:20.648 INFO:tasks.workunit.client.1.smithi194.stdout:7/812: dread d3/d16/d23/d37/d95/f9c [0,4194304] 0 2024-04-12T05:56:20.653 INFO:tasks.workunit.client.1.smithi194.stdout:7/813: rename d3/def/ff3 to d3/d27/d22/d31/f10b 0 2024-04-12T05:56:20.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:20 smithi008 ceph-mon[39722]: pgmap v4: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:20.668 INFO:tasks.workunit.client.1.smithi194.stdout:1/908: dwrite d0/d11/f111 [0,4194304] 0 2024-04-12T05:56:20.671 INFO:tasks.workunit.client.1.smithi194.stdout:2/844: dwrite d7/d12/d1a/d69/d7e/fe0 [0,4194304] 0 2024-04-12T05:56:20.676 INFO:tasks.workunit.client.1.smithi194.stdout:2/845: symlink d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103/l11b 0 2024-04-12T05:56:20.680 INFO:tasks.workunit.client.1.smithi194.stdout:2/846: creat d7/d12/d1a/d69/f11c x:0 0 0 2024-04-12T05:56:20.688 INFO:tasks.workunit.client.1.smithi194.stdout:0/814: sync 2024-04-12T05:56:20.690 INFO:tasks.workunit.client.1.smithi194.stdout:0/815: mknod d0/de3/df0/dbe/c10a 0 2024-04-12T05:56:20.692 INFO:tasks.workunit.client.1.smithi194.stdout:0/816: symlink d0/d3b/d52/dfa/d8f/l10b 0 2024-04-12T05:56:20.693 INFO:tasks.workunit.client.1.smithi194.stdout:7/814: rename d3/d27/fbb to d3/d27/d22/d8e/f10c 0 2024-04-12T05:56:20.694 INFO:tasks.workunit.client.1.smithi194.stdout:0/817: truncate d0/d3b/d52/dfa/d1a/d24/d4b/fee 1031548 0 2024-04-12T05:56:20.696 INFO:tasks.workunit.client.1.smithi194.stdout:0/818: read d0/de3/d5b/fce [262405,81825] 0 2024-04-12T05:56:20.696 INFO:tasks.workunit.client.1.smithi194.stdout:7/815: write d3/d16/d23/d37/f7c [147401,53790] 0 2024-04-12T05:56:20.697 INFO:tasks.workunit.client.1.smithi194.stdout:0/819: chown d0/d3b/d52/dfa/d8f 25018015 1 2024-04-12T05:56:20.719 INFO:tasks.workunit.client.1.smithi194.stdout:7/816: read d3/d27/d22/ff0 [293194,118344] 0 2024-04-12T05:56:20.721 INFO:tasks.workunit.client.1.smithi194.stdout:0/820: stat d0/da/d4a/d8e/laf 0 2024-04-12T05:56:20.722 INFO:tasks.workunit.client.1.smithi194.stdout:7/817: write d3/d27/d22/f2e [795492,21923] 0 2024-04-12T05:56:20.724 INFO:tasks.workunit.client.1.smithi194.stdout:7/818: write d3/d27/d6a/f55 [174538,27084] 0 2024-04-12T05:56:20.724 INFO:tasks.workunit.client.1.smithi194.stdout:7/819: symlink d3/d27/d6a/dda/l10d 2 2024-04-12T05:56:20.726 INFO:tasks.workunit.client.1.smithi194.stdout:7/820: chown d3/d16/d23/d57/d72/d81/d3a/d43/c7e 62 1 2024-04-12T05:56:20.727 INFO:tasks.workunit.client.1.smithi194.stdout:7/821: chown d3/d16/f19 0 1 2024-04-12T05:56:20.728 INFO:tasks.workunit.client.1.smithi194.stdout:7/822: chown d3/d16/d23/la0 4 1 2024-04-12T05:56:20.728 INFO:tasks.workunit.client.1.smithi194.stdout:0/821: write d0/d3b/d52/dfa/f32 [1686407,79936] 0 2024-04-12T05:56:20.729 INFO:tasks.workunit.client.1.smithi194.stdout:7/823: stat d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f91 0 2024-04-12T05:56:20.731 INFO:tasks.workunit.client.1.smithi194.stdout:7/824: fsync d3/fcd 0 2024-04-12T05:56:20.731 INFO:tasks.workunit.client.1.smithi194.stdout:0/822: mkdir d0/d3b/d52/dfa/d1a/d24/de1/d10c 0 2024-04-12T05:56:20.733 INFO:tasks.workunit.client.1.smithi194.stdout:0/823: mknod d0/d5f/d73/d76/c10d 0 2024-04-12T05:56:20.734 INFO:tasks.workunit.client.1.smithi194.stdout:7/825: creat d3/d16/d23/d40/d6f/f10e x:0 0 0 2024-04-12T05:56:20.734 INFO:tasks.workunit.client.1.smithi194.stdout:0/824: readlink d0/l86 0 2024-04-12T05:56:20.735 INFO:tasks.workunit.client.1.smithi194.stdout:0/825: chown d0/d3b/d52/dfa/d27/c70 116 1 2024-04-12T05:56:20.736 INFO:tasks.workunit.client.1.smithi194.stdout:0/826: write d0/d3b/d52/dfa/d1a/f6d [2675083,128671] 0 2024-04-12T05:56:20.738 INFO:tasks.workunit.client.1.smithi194.stdout:7/826: dread d3/d27/d22/d62/f6c [0,4194304] 0 2024-04-12T05:56:20.739 INFO:tasks.workunit.client.1.smithi194.stdout:0/827: chown d0/d3b/d52/dfa/d1a/d3d/f7a 886096169 1 2024-04-12T05:56:20.741 INFO:tasks.workunit.client.1.smithi194.stdout:0/828: symlink d0/d5f/deb/l10e 0 2024-04-12T05:56:20.747 INFO:tasks.workunit.client.1.smithi194.stdout:0/829: dwrite d0/da/d4a/fca [4194304,4194304] 0 2024-04-12T05:56:20.748 INFO:tasks.workunit.client.1.smithi194.stdout:0/830: chown d0/d3b/d52/dfa/d1a/d24/d56 228493848 1 2024-04-12T05:56:20.753 INFO:tasks.workunit.client.1.smithi194.stdout:7/827: dread d3/d27/f1d [0,4194304] 0 2024-04-12T05:56:20.755 INFO:tasks.workunit.client.1.smithi194.stdout:7/828: chown d3/d16/d23/d57/d72/d81/d3a/d43 55 1 2024-04-12T05:56:20.760 INFO:tasks.workunit.client.1.smithi194.stdout:0/831: dwrite d0/d3b/d52/dfa/d27/f3f [0,4194304] 0 2024-04-12T05:56:20.763 INFO:tasks.workunit.client.1.smithi194.stdout:0/832: chown d0/da/d4a/d8e/ldd 126204783 1 2024-04-12T05:56:20.767 INFO:tasks.workunit.client.1.smithi194.stdout:0/833: creat d0/da/d4a/d8e/f10f x:0 0 0 2024-04-12T05:56:20.773 INFO:tasks.workunit.client.1.smithi194.stdout:7/829: getdents d3/d27/d22 0 2024-04-12T05:56:20.773 INFO:tasks.workunit.client.1.smithi194.stdout:0/834: dread d0/d3b/d52/dfa/d27/d54/fc3 [0,4194304] 0 2024-04-12T05:56:20.777 INFO:tasks.workunit.client.1.smithi194.stdout:7/830: unlink d3/d16/d23/d57/df6/lfa 0 2024-04-12T05:56:20.778 INFO:tasks.workunit.client.1.smithi194.stdout:0/835: read d0/d3b/d52/dfa/d1a/f1c [1195196,86851] 0 2024-04-12T05:56:20.779 INFO:tasks.workunit.client.1.smithi194.stdout:7/831: readlink d3/d16/d23/d57/d72/d81/d3a/d43/l4e 0 2024-04-12T05:56:20.780 INFO:tasks.workunit.client.1.smithi194.stdout:0/836: chown d0/d3b/d52/dfa/d1a/f23 52770044 1 2024-04-12T05:56:20.781 INFO:tasks.workunit.client.1.smithi194.stdout:0/837: chown d0/d3b/d52/dfa/d27/c70 57 1 2024-04-12T05:56:20.784 INFO:tasks.workunit.client.1.smithi194.stdout:7/832: getdents d3/d16/d7b 0 2024-04-12T05:56:20.786 INFO:tasks.workunit.client.1.smithi194.stdout:0/838: write d0/d3b/d52/dfa/f15 [2281939,80007] 0 2024-04-12T05:56:20.787 INFO:tasks.workunit.client.1.smithi194.stdout:7/833: rename d3/d16/d23/c32 to d3/dd8/c10f 0 2024-04-12T05:56:20.787 INFO:tasks.workunit.client.1.smithi194.stdout:0/839: stat d0/d5f/d73/d76/fbf 0 2024-04-12T05:56:20.790 INFO:tasks.workunit.client.1.smithi194.stdout:7/834: fdatasync d3/d27/f46 0 2024-04-12T05:56:20.791 INFO:tasks.workunit.client.1.smithi194.stdout:0/840: getdents d0/d5f 0 2024-04-12T05:56:20.794 INFO:tasks.workunit.client.1.smithi194.stdout:0/841: symlink d0/d3b/d52/dfa/d1a/d24/de1/l110 0 2024-04-12T05:56:20.797 INFO:tasks.workunit.client.1.smithi194.stdout:0/842: symlink d0/de3/df0/dbe/l111 0 2024-04-12T05:56:20.797 INFO:tasks.workunit.client.1.smithi194.stdout:7/835: dwrite d3/d16/d23/fc8 [0,4194304] 0 2024-04-12T05:56:20.800 INFO:tasks.workunit.client.1.smithi194.stdout:0/843: dread - d0/d3b/d52/dfa/d1a/d24/d4b/f65 zero size 2024-04-12T05:56:20.806 INFO:tasks.workunit.client.1.smithi194.stdout:7/836: creat d3/d16/d23/f110 x:0 0 0 2024-04-12T05:56:20.813 INFO:tasks.workunit.client.1.smithi194.stdout:7/837: dwrite d3/d27/d22/d62/fdb [0,4194304] 0 2024-04-12T05:56:20.816 INFO:tasks.workunit.client.1.smithi194.stdout:7/838: rmdir d3/d27/d6a/dda 39 2024-04-12T05:56:20.818 INFO:tasks.workunit.client.1.smithi194.stdout:0/844: dwrite d0/d3b/d52/dfa/d1a/d3d/f7a [4194304,4194304] 0 2024-04-12T05:56:20.821 INFO:tasks.workunit.client.1.smithi194.stdout:7/839: truncate d3/d16/d23/d57/d72/ff4 898865 0 2024-04-12T05:56:20.830 INFO:tasks.workunit.client.1.smithi194.stdout:7/840: mkdir d3/d16/d23/d57/d72/d81/d3a/d111 0 2024-04-12T05:56:20.835 INFO:tasks.workunit.client.1.smithi194.stdout:7/841: creat d3/d16/d23/d57/f112 x:0 0 0 2024-04-12T05:56:20.837 INFO:tasks.workunit.client.1.smithi194.stdout:7/842: chown d3/d27/f7 30450 1 2024-04-12T05:56:20.839 INFO:tasks.workunit.client.1.smithi194.stdout:7/843: fdatasync d3/d27/d22/f77 0 2024-04-12T05:56:20.841 INFO:tasks.workunit.client.1.smithi194.stdout:0/845: dwrite d0/da/f13 [0,4194304] 0 2024-04-12T05:56:20.847 INFO:tasks.workunit.client.1.smithi194.stdout:7/844: dwrite d3/d16/d23/d37/f7c [0,4194304] 0 2024-04-12T05:56:20.852 INFO:tasks.workunit.client.1.smithi194.stdout:7/845: rename d3/d27/ce to d3/d16/d7b/dc3/c113 0 2024-04-12T05:56:20.855 INFO:tasks.workunit.client.1.smithi194.stdout:7/846: mkdir d3/d16/d23/d40/d114 0 2024-04-12T05:56:20.861 INFO:tasks.workunit.client.1.smithi194.stdout:7/847: rename d3/d16/d23/d37/d3d/d56/f51 to d3/d27/d6a/f115 0 2024-04-12T05:56:20.864 INFO:tasks.workunit.client.1.smithi194.stdout:7/848: rmdir d3/d27/d6a/dda 39 2024-04-12T05:56:20.866 INFO:tasks.workunit.client.1.smithi194.stdout:7/849: symlink d3/d16/d23/d37/d3d/d56/l116 0 2024-04-12T05:56:20.869 INFO:tasks.workunit.client.1.smithi194.stdout:7/850: creat d3/d27/d22/d31/f117 x:0 0 0 2024-04-12T05:56:20.873 INFO:tasks.workunit.client.1.smithi194.stdout:7/851: creat d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f118 x:0 0 0 2024-04-12T05:56:20.877 INFO:tasks.workunit.client.1.smithi194.stdout:7/852: link d3/d27/c2a d3/d16/d7b/dc3/c119 0 2024-04-12T05:56:20.879 INFO:tasks.workunit.client.1.smithi194.stdout:7/853: mkdir d3/d16/d23/d40/d114/d11a 0 2024-04-12T05:56:20.879 INFO:tasks.workunit.client.1.smithi194.stdout:0/846: creat d0/d3b/d52/de2/f112 x:0 0 0 2024-04-12T05:56:20.880 INFO:tasks.workunit.client.1.smithi194.stdout:7/854: write d3/d16/d5d/f84 [2652778,113205] 0 2024-04-12T05:56:20.882 INFO:tasks.workunit.client.1.smithi194.stdout:0/847: mkdir d0/d3b/d52/dfa/d27/d87/d92/d113 0 2024-04-12T05:56:20.883 INFO:tasks.workunit.client.1.smithi194.stdout:7/855: read d3/d27/d6a/dda/d74/fd5 [4189436,49626] 0 2024-04-12T05:56:20.883 INFO:tasks.workunit.client.1.smithi194.stdout:0/848: chown d0/de3/d5b/lfc 6 1 2024-04-12T05:56:20.885 INFO:tasks.workunit.client.1.smithi194.stdout:0/849: truncate d0/de3/df0/f9f 5002280 0 2024-04-12T05:56:20.885 INFO:tasks.workunit.client.1.smithi194.stdout:0/850: readlink d0/d5f/d73/l101 0 2024-04-12T05:56:20.887 INFO:tasks.workunit.client.1.smithi194.stdout:7/856: mkdir d3/d16/d11b 0 2024-04-12T05:56:20.891 INFO:tasks.workunit.client.1.smithi194.stdout:0/851: dwrite d0/da/d4a/f5e [0,4194304] 0 2024-04-12T05:56:20.892 INFO:tasks.workunit.client.1.smithi194.stdout:0/852: write d0/da/d4a/fca [5802026,59985] 0 2024-04-12T05:56:20.897 INFO:tasks.workunit.client.1.smithi194.stdout:7/857: rmdir d3/d16/d23/d57/d72/d81/d3a 39 2024-04-12T05:56:20.905 INFO:tasks.workunit.client.1.smithi194.stdout:7/858: rename d3/d16/d23/d57/d72/d81/d3a/f106 to d3/d16/d23/d40/f11c 0 2024-04-12T05:56:20.910 INFO:tasks.workunit.client.1.smithi194.stdout:7/859: creat d3/d16/d23/d57/d72/d81/d3a/f11d x:0 0 0 2024-04-12T05:56:20.912 INFO:tasks.workunit.client.1.smithi194.stdout:0/853: dread d0/d3b/d52/dfa/f35 [8388608,4194304] 0 2024-04-12T05:56:20.913 INFO:tasks.workunit.client.1.smithi194.stdout:0/854: chown d0/de3/cae 1529 1 2024-04-12T05:56:20.913 INFO:tasks.workunit.client.1.smithi194.stdout:7/860: creat d3/d27/d6a/f11e x:0 0 0 2024-04-12T05:56:20.913 INFO:tasks.workunit.client.1.smithi194.stdout:0/855: stat d0/d3b/d52/dfa/d1a/d3d/c47 0 2024-04-12T05:56:20.916 INFO:tasks.workunit.client.1.smithi194.stdout:0/856: creat d0/da/da7/f114 x:0 0 0 2024-04-12T05:56:20.916 INFO:tasks.workunit.client.1.smithi194.stdout:7/861: symlink d3/d16/d23/d57/d72/d81/d3a/d111/l11f 0 2024-04-12T05:56:20.918 INFO:tasks.workunit.client.1.smithi194.stdout:7/862: truncate d3/d16/d23/d40/f11c 887730 0 2024-04-12T05:56:20.918 INFO:tasks.workunit.client.1.smithi194.stdout:0/857: symlink d0/da/dc4/l115 0 2024-04-12T05:56:20.920 INFO:tasks.workunit.client.1.smithi194.stdout:0/858: creat d0/de3/d5b/f116 x:0 0 0 2024-04-12T05:56:20.925 INFO:tasks.workunit.client.1.smithi194.stdout:7/863: dwrite d3/d27/d6a/f82 [0,4194304] 0 2024-04-12T05:56:20.930 INFO:tasks.workunit.client.1.smithi194.stdout:7/864: truncate d3/d27/d6a/f115 2655542 0 2024-04-12T05:56:20.935 INFO:tasks.workunit.client.1.smithi194.stdout:7/865: write d3/d16/d23/d57/d72/d81/d3a/d43/fb8 [493578,97109] 0 2024-04-12T05:56:20.936 INFO:tasks.workunit.client.1.smithi194.stdout:7/866: readlink d3/d27/d6a/dda/l4b 0 2024-04-12T05:56:20.936 INFO:tasks.workunit.client.1.smithi194.stdout:0/859: dread d0/d3b/d52/dfa/d1a/f6d [0,4194304] 0 2024-04-12T05:56:20.938 INFO:tasks.workunit.client.1.smithi194.stdout:7/867: dread - d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f118 zero size 2024-04-12T05:56:20.941 INFO:tasks.workunit.client.1.smithi194.stdout:7/868: symlink d3/d27/d6a/dda/d74/l120 0 2024-04-12T05:56:20.942 INFO:tasks.workunit.client.1.smithi194.stdout:7/869: chown d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f76 1826083 1 2024-04-12T05:56:20.943 INFO:tasks.workunit.client.1.smithi194.stdout:7/870: dread - d3/d27/d22/d8e/fff zero size 2024-04-12T05:56:20.951 INFO:tasks.workunit.client.1.smithi194.stdout:0/860: dwrite d0/d3b/d52/dfa/d1a/d24/d4b/f93 [0,4194304] 0 2024-04-12T05:56:20.952 INFO:tasks.workunit.client.1.smithi194.stdout:7/871: dwrite d3/d27/d22/d8e/f10c [0,4194304] 0 2024-04-12T05:56:20.955 INFO:tasks.workunit.client.1.smithi194.stdout:7/872: read d3/d27/d6a/f82 [748845,91186] 0 2024-04-12T05:56:20.959 INFO:tasks.workunit.client.1.smithi194.stdout:7/873: chown d3/d27/d22/d31/lee 101 1 2024-04-12T05:56:20.966 INFO:tasks.workunit.client.1.smithi194.stdout:7/874: truncate d3/d27/f1b 7129899 0 2024-04-12T05:56:20.983 INFO:tasks.workunit.client.1.smithi194.stdout:3/967: sync 2024-04-12T05:56:20.983 INFO:tasks.workunit.client.1.smithi194.stdout:6/874: sync 2024-04-12T05:56:20.983 INFO:tasks.workunit.client.1.smithi194.stdout:8/667: sync 2024-04-12T05:56:20.985 INFO:tasks.workunit.client.1.smithi194.stdout:6/875: chown d3/d2f/d4e/d68/c104 846 1 2024-04-12T05:56:20.985 INFO:tasks.workunit.client.1.smithi194.stdout:8/668: write d2/d8/d66/d7f/fa7 [778868,57553] 0 2024-04-12T05:56:20.986 INFO:tasks.workunit.client.1.smithi194.stdout:3/968: creat d0/d1/d4f/dd7/d141/f14b x:0 0 0 2024-04-12T05:56:20.987 INFO:tasks.workunit.client.1.smithi194.stdout:0/861: dread d0/d3b/d52/dfa/d27/d54/fad [0,4194304] 0 2024-04-12T05:56:20.988 INFO:tasks.workunit.client.1.smithi194.stdout:6/876: creat d3/d2f/d4e/d9a/f112 x:0 0 0 2024-04-12T05:56:20.988 INFO:tasks.workunit.client.1.smithi194.stdout:8/669: mknod d2/d8/d15/d3b/cce 0 2024-04-12T05:56:20.990 INFO:tasks.workunit.client.1.smithi194.stdout:7/875: truncate d3/d16/d23/f66 3516575 0 2024-04-12T05:56:20.991 INFO:tasks.workunit.client.1.smithi194.stdout:7/876: dread - d3/d16/d23/d40/d6f/f10e zero size 2024-04-12T05:56:20.992 INFO:tasks.workunit.client.1.smithi194.stdout:6/877: creat d3/d2d/d33/db1/f113 x:0 0 0 2024-04-12T05:56:20.993 INFO:tasks.workunit.client.1.smithi194.stdout:8/670: symlink d2/d8/d9/de/d10/lcf 0 2024-04-12T05:56:20.993 INFO:tasks.workunit.client.1.smithi194.stdout:0/862: rmdir d0/d3b/d52/dfa/d1a/d24/d4b 39 2024-04-12T05:56:20.995 INFO:tasks.workunit.client.1.smithi194.stdout:7/877: read d3/d16/d5d/f65 [101999,52193] 0 2024-04-12T05:56:20.995 INFO:tasks.workunit.client.1.smithi194.stdout:8/671: truncate d2/fbb 619996 0 2024-04-12T05:56:20.998 INFO:tasks.workunit.client.1.smithi194.stdout:6/878: creat d3/d15/d2b/d48/d6f/d7a/d8f/d98/f114 x:0 0 0 2024-04-12T05:56:20.999 INFO:tasks.workunit.client.1.smithi194.stdout:3/969: dwrite d0/d1/d10/d8c/d8f/ddb/ff5 [0,4194304] 0 2024-04-12T05:56:21.001 INFO:tasks.workunit.client.1.smithi194.stdout:0/863: getdents d0/da/d4a/d8e 0 2024-04-12T05:56:21.008 INFO:tasks.workunit.client.1.smithi194.stdout:8/672: mknod d2/d8/d9/de/d10/d7d/d84/d9f/cd0 0 2024-04-12T05:56:21.008 INFO:tasks.workunit.client.1.smithi194.stdout:6/879: mkdir d3/d2f/d4e/d68/dad/dc4/d115 0 2024-04-12T05:56:21.010 INFO:tasks.workunit.client.1.smithi194.stdout:0/864: dread - d0/de3/df0/fab zero size 2024-04-12T05:56:21.011 INFO:tasks.workunit.client.1.smithi194.stdout:7/878: dwrite d3/d16/d23/d37/d3d/d56/dd0/fc9 [4194304,4194304] 0 2024-04-12T05:56:21.015 INFO:tasks.workunit.client.1.smithi194.stdout:6/880: creat d3/d2f/d4e/d63/d83/f116 x:0 0 0 2024-04-12T05:56:21.020 INFO:tasks.workunit.client.1.smithi194.stdout:8/673: creat d2/d8/d15/d33/d45/fd1 x:0 0 0 2024-04-12T05:56:21.022 INFO:tasks.workunit.client.1.smithi194.stdout:7/879: truncate d3/d27/d6a/faa 510793 0 2024-04-12T05:56:21.022 INFO:tasks.workunit.client.1.smithi194.stdout:3/970: dwrite d0/d1/d10/d8c/d8f/ddb/f116 [0,4194304] 0 2024-04-12T05:56:21.025 INFO:tasks.workunit.client.1.smithi194.stdout:8/674: read d2/d8/d9/de/f61 [37152,911] 0 2024-04-12T05:56:21.025 INFO:tasks.workunit.client.1.smithi194.stdout:1/909: sync 2024-04-12T05:56:21.025 INFO:tasks.workunit.client.1.smithi194.stdout:9/898: sync 2024-04-12T05:56:21.025 INFO:tasks.workunit.client.1.smithi194.stdout:2/847: sync 2024-04-12T05:56:21.028 INFO:tasks.workunit.client.1.smithi194.stdout:6/881: dread d3/d15/d2b/d48/d6f/d7a/fcf [0,4194304] 0 2024-04-12T05:56:21.028 INFO:tasks.workunit.client.1.smithi194.stdout:0/865: dwrite d0/f1d [0,4194304] 0 2024-04-12T05:56:21.038 INFO:tasks.workunit.client.1.smithi194.stdout:9/899: stat d1/d41/dbf/lce 0 2024-04-12T05:56:21.039 INFO:tasks.workunit.client.1.smithi194.stdout:8/675: write d2/d8/d9/de/f98 [926486,36099] 0 2024-04-12T05:56:21.040 INFO:tasks.workunit.client.1.smithi194.stdout:6/882: mkdir d3/d2d/d33/d107/d117 0 2024-04-12T05:56:21.042 INFO:tasks.workunit.client.1.smithi194.stdout:1/910: mknod d0/d11/d18/d19/d1a/d6a/dac/db5/d68/c11e 0 2024-04-12T05:56:21.043 INFO:tasks.workunit.client.1.smithi194.stdout:8/676: rmdir d2/d8/d58 39 2024-04-12T05:56:21.045 INFO:tasks.workunit.client.1.smithi194.stdout:9/900: chown d1/d19/d1f/d5f/l8d 0 1 2024-04-12T05:56:21.047 INFO:tasks.workunit.client.1.smithi194.stdout:1/911: write d0/f85 [4229940,91779] 0 2024-04-12T05:56:21.048 INFO:tasks.workunit.client.1.smithi194.stdout:7/880: dwrite d3/d27/f1f [0,4194304] 0 2024-04-12T05:56:21.050 INFO:tasks.workunit.client.1.smithi194.stdout:9/901: chown d1/d21/d39/df1 0 1 2024-04-12T05:56:21.052 INFO:tasks.workunit.client.1.smithi194.stdout:9/902: dread - d1/d21/d77/d98/fc8 zero size 2024-04-12T05:56:21.053 INFO:tasks.workunit.client.1.smithi194.stdout:3/971: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/fc7 351897 0 2024-04-12T05:56:21.054 INFO:tasks.workunit.client.1.smithi194.stdout:0/866: dwrite d0/d3b/d52/dfa/f41 [0,4194304] 0 2024-04-12T05:56:21.055 INFO:tasks.workunit.client.1.smithi194.stdout:1/912: mkdir d0/d11/d18/d19/d1c/d67/d11f 0 2024-04-12T05:56:21.055 INFO:tasks.workunit.client.1.smithi194.stdout:2/848: write d7/d12/d25/d5d/d47/dac/db3/dd7/fc5 [656240,65334] 0 2024-04-12T05:56:21.058 INFO:tasks.workunit.client.1.smithi194.stdout:6/883: link d3/d2d/d33/d52/d5c/l76 d3/d2d/d33/d107/l118 0 2024-04-12T05:56:21.060 INFO:tasks.workunit.client.1.smithi194.stdout:0/867: chown d0/d3b/d52/fe5 295 1 2024-04-12T05:56:21.061 INFO:tasks.workunit.client.1.smithi194.stdout:6/884: chown d3/d2f/d4e/d63/d83/fc1 908267 1 2024-04-12T05:56:21.061 INFO:tasks.workunit.client.1.smithi194.stdout:7/881: getdents d3/d16/d23/d40/d6f 0 2024-04-12T05:56:21.066 INFO:tasks.workunit.client.1.smithi194.stdout:9/903: rename d1/d19/d1f to d1/d4e/dea/dfc/d12f 0 2024-04-12T05:56:21.066 INFO:tasks.workunit.client.1.smithi194.stdout:7/882: truncate d3/d27/f64 2011963 0 2024-04-12T05:56:21.070 INFO:tasks.workunit.client.1.smithi194.stdout:3/972: rename d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/faa to d0/d1/d4f/dd7/f14c 0 2024-04-12T05:56:21.073 INFO:tasks.workunit.client.1.smithi194.stdout:1/913: getdents d0/d11/d18/d19/d1a/d6a 0 2024-04-12T05:56:21.075 INFO:tasks.workunit.client.1.smithi194.stdout:9/904: symlink d1/d4e/dea/dfc/d12f/d20/d37/d84/df2/l130 0 2024-04-12T05:56:21.076 INFO:tasks.workunit.client.1.smithi194.stdout:0/868: getdents d0 0 2024-04-12T05:56:21.077 INFO:tasks.workunit.client.1.smithi194.stdout:6/885: link d3/d34/l92 d3/d2f/d4e/d63/l119 0 2024-04-12T05:56:21.077 INFO:tasks.workunit.client.1.smithi194.stdout:2/849: dread d7/d12/d25/f31 [0,4194304] 0 2024-04-12T05:56:21.078 INFO:tasks.workunit.client.1.smithi194.stdout:9/905: dread - d1/d21/d77/d98/f129 zero size 2024-04-12T05:56:21.078 INFO:tasks.workunit.client.1.smithi194.stdout:1/914: symlink d0/d11/ddd/de6/l120 0 2024-04-12T05:56:21.080 INFO:tasks.workunit.client.1.smithi194.stdout:3/973: stat d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/l109 0 2024-04-12T05:56:21.081 INFO:tasks.workunit.client.1.smithi194.stdout:6/886: mkdir d3/d2d/d33/d11a 0 2024-04-12T05:56:21.082 INFO:tasks.workunit.client.1.smithi194.stdout:9/906: mkdir d1/d41/d97/d131 0 2024-04-12T05:56:21.086 INFO:tasks.workunit.client.1.smithi194.stdout:3/974: stat d0/d1/d74/d39/d4d/deb/d1e/d81/d34/d10d 0 2024-04-12T05:56:21.086 INFO:tasks.workunit.client.1.smithi194.stdout:9/907: symlink d1/d41/dba/l132 0 2024-04-12T05:56:21.087 INFO:tasks.workunit.client.1.smithi194.stdout:7/883: dread d3/d16/d23/d37/f54 [0,4194304] 0 2024-04-12T05:56:21.087 INFO:tasks.workunit.client.1.smithi194.stdout:9/908: stat d1/d21 0 2024-04-12T05:56:21.088 INFO:tasks.workunit.client.1.smithi194.stdout:2/850: rename d7/d22/f4d to d7/d12/d25/d5d/d47/d52/d7b/f11d 0 2024-04-12T05:56:21.093 INFO:tasks.workunit.client.1.smithi194.stdout:9/909: creat d1/d21/d77/d98/d9f/dde/f133 x:0 0 0 2024-04-12T05:56:21.093 INFO:tasks.workunit.client.1.smithi194.stdout:1/915: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/f121 x:0 0 0 2024-04-12T05:56:21.094 INFO:tasks.workunit.client.1.smithi194.stdout:3/975: creat d0/d1/d74/d39/d4d/deb/d1e/d33/d76/d9a/f14d x:0 0 0 2024-04-12T05:56:21.095 INFO:tasks.workunit.client.1.smithi194.stdout:1/916: readlink d0/d11/d18/d19/d1c/l49 0 2024-04-12T05:56:21.097 INFO:tasks.workunit.client.1.smithi194.stdout:0/869: link d0/d3b/d52/dfa/d27/d54/fc3 d0/d3b/d52/dfa/d1a/d24/d56/f117 0 2024-04-12T05:56:21.099 INFO:tasks.workunit.client.1.smithi194.stdout:7/884: readlink d3/d16/d23/d57/d72/d81/d3a/d8f/l102 0 2024-04-12T05:56:21.100 INFO:tasks.workunit.client.1.smithi194.stdout:2/851: mkdir d7/d12/d25/d5d/d11e 0 2024-04-12T05:56:21.102 INFO:tasks.workunit.client.1.smithi194.stdout:7/885: fsync d3/dd8/f109 0 2024-04-12T05:56:21.102 INFO:tasks.workunit.client.1.smithi194.stdout:0/870: creat d0/d3b/d52/dfa/d1a/d24/de1/f118 x:0 0 0 2024-04-12T05:56:21.104 INFO:tasks.workunit.client.1.smithi194.stdout:9/910: mknod d1/d4e/dea/dfc/d12f/d20/d37/c134 0 2024-04-12T05:56:21.104 INFO:tasks.workunit.client.1.smithi194.stdout:6/887: dread d3/d2f/d4e/d9a/dd8/d77/fb6 [0,4194304] 0 2024-04-12T05:56:21.105 INFO:tasks.workunit.client.1.smithi194.stdout:3/976: dwrite d0/d1/d74/d39/f70 [0,4194304] 0 2024-04-12T05:56:21.109 INFO:tasks.workunit.client.1.smithi194.stdout:1/917: creat d0/d11/d18/d19/f122 x:0 0 0 2024-04-12T05:56:21.112 INFO:tasks.workunit.client.1.smithi194.stdout:9/911: chown d1/d4e/dea/dfc/d12f/d20/d37/d84/fb8 4081 1 2024-04-12T05:56:21.112 INFO:tasks.workunit.client.1.smithi194.stdout:7/886: rename d3/d27/d6a/faa to d3/d16/d5d/f121 0 2024-04-12T05:56:21.113 INFO:tasks.workunit.client.1.smithi194.stdout:2/852: unlink d7/d12/d25/d5d/d47/dac/fcb 0 2024-04-12T05:56:21.113 INFO:tasks.workunit.client.1.smithi194.stdout:6/888: mknod d3/d2f/d4e/d9a/db3/c11b 0 2024-04-12T05:56:21.114 INFO:tasks.workunit.client.1.smithi194.stdout:7/887: fsync d3/d27/d22/d8e/fff 0 2024-04-12T05:56:21.116 INFO:tasks.workunit.client.1.smithi194.stdout:0/871: mknod d0/d3b/d52/dfa/d27/d87/d92/d113/c119 0 2024-04-12T05:56:21.117 INFO:tasks.workunit.client.1.smithi194.stdout:1/918: creat d0/d11/d18/d19/d1c/df0/f123 x:0 0 0 2024-04-12T05:56:21.117 INFO:tasks.workunit.client.1.smithi194.stdout:3/977: symlink d0/d1/d74/d39/d4d/deb/d3d/dae/dbe/l14e 0 2024-04-12T05:56:21.118 INFO:tasks.workunit.client.1.smithi194.stdout:9/912: rename d1/d41/dbf/de9 to d1/da6/d135 0 2024-04-12T05:56:21.119 INFO:tasks.workunit.client.1.smithi194.stdout:6/889: read d3/d15/d2b/d48/d6f/d7a/d8f/fc0 [10,56765] 0 2024-04-12T05:56:21.120 INFO:tasks.workunit.client.1.smithi194.stdout:2/853: creat d7/d12/d1a/d69/df0/d101/df9/f11f x:0 0 0 2024-04-12T05:56:21.120 INFO:tasks.workunit.client.1.smithi194.stdout:7/888: dread - d3/d16/d23/d57/d72/d81/d3a/f11d zero size 2024-04-12T05:56:21.120 INFO:tasks.workunit.client.1.smithi194.stdout:9/913: readlink d1/d19/l62 0 2024-04-12T05:56:21.122 INFO:tasks.workunit.client.1.smithi194.stdout:3/978: creat d0/d1/f14f x:0 0 0 2024-04-12T05:56:21.122 INFO:tasks.workunit.client.1.smithi194.stdout:0/872: fdatasync d0/d3b/fc6 0 2024-04-12T05:56:21.125 INFO:tasks.workunit.client.1.smithi194.stdout:1/919: symlink d0/d11/d18/d19/d1a/d6a/dac/db5/df8/l124 0 2024-04-12T05:56:21.126 INFO:tasks.workunit.client.1.smithi194.stdout:0/873: readlink d0/de3/df0/dbe/ld8 0 2024-04-12T05:56:21.126 INFO:tasks.workunit.client.1.smithi194.stdout:2/854: dread d7/d22/dc9/fee [0,4194304] 0 2024-04-12T05:56:21.128 INFO:tasks.workunit.client.1.smithi194.stdout:9/914: creat d1/d21/d39/df1/f136 x:0 0 0 2024-04-12T05:56:21.129 INFO:tasks.workunit.client.1.smithi194.stdout:2/855: chown d7/d12/f8d 928 1 2024-04-12T05:56:21.130 INFO:tasks.workunit.client.1.smithi194.stdout:1/920: readlink d0/d11/d18/d19/d1a/d6a/dac/db5/d68/l11b 0 2024-04-12T05:56:21.130 INFO:tasks.workunit.client.1.smithi194.stdout:3/979: mknod d0/d1/d10/c150 0 2024-04-12T05:56:21.133 INFO:tasks.workunit.client.1.smithi194.stdout:9/915: readlink d1/d4e/dea/dfc/d12f/l11f 0 2024-04-12T05:56:21.135 INFO:tasks.workunit.client.1.smithi194.stdout:7/889: creat d3/d16/d23/d57/d72/d81/f122 x:0 0 0 2024-04-12T05:56:21.135 INFO:tasks.workunit.client.1.smithi194.stdout:0/874: symlink d0/d3b/d52/dfa/d27/d54/l11a 0 2024-04-12T05:56:21.141 INFO:tasks.workunit.client.1.smithi194.stdout:9/916: unlink d1/d21/d39/c3b 0 2024-04-12T05:56:21.143 INFO:tasks.workunit.client.1.smithi194.stdout:2/856: dread d7/d12/d25/d5d/d47/dac/db3/dd7/fc5 [0,4194304] 0 2024-04-12T05:56:21.144 INFO:tasks.workunit.client.1.smithi194.stdout:3/980: dwrite d0/d1/f14f [0,4194304] 0 2024-04-12T05:56:21.145 INFO:tasks.workunit.client.1.smithi194.stdout:7/890: write d3/d16/d23/d57/d72/d81/d3a/d43/d5c/fc4 [804869,85218] 0 2024-04-12T05:56:21.146 INFO:tasks.workunit.client.1.smithi194.stdout:6/890: dread d3/d2f/d4e/d9a/db3/fbc [0,4194304] 0 2024-04-12T05:56:21.147 INFO:tasks.workunit.client.1.smithi194.stdout:0/875: creat d0/de3/df0/dbe/f11b x:0 0 0 2024-04-12T05:56:21.150 INFO:tasks.workunit.client.1.smithi194.stdout:0/876: fsync d0/da/d4a/d8e/f10f 0 2024-04-12T05:56:21.153 INFO:tasks.workunit.client.1.smithi194.stdout:3/981: symlink d0/d1/d74/l151 0 2024-04-12T05:56:21.153 INFO:tasks.workunit.client.1.smithi194.stdout:2/857: chown d7/d12/f41 1554690823 1 2024-04-12T05:56:21.155 INFO:tasks.workunit.client.1.smithi194.stdout:7/891: rename d3/d16/d23/d57/d72/d81/d3a/d43/l4e to d3/d16/d23/d57/d72/l123 0 2024-04-12T05:56:21.156 INFO:tasks.workunit.client.1.smithi194.stdout:6/891: dread d3/d2f/f9e [0,4194304] 0 2024-04-12T05:56:21.156 INFO:tasks.workunit.client.1.smithi194.stdout:3/982: read d0/d42/f51 [200638,3280] 0 2024-04-12T05:56:21.161 INFO:tasks.workunit.client.1.smithi194.stdout:9/917: write d1/d4e/dea/dfc/d12f/f52 [298949,68297] 0 2024-04-12T05:56:21.165 INFO:tasks.workunit.client.1.smithi194.stdout:3/983: mknod d0/c152 0 2024-04-12T05:56:21.165 INFO:tasks.workunit.client.1.smithi194.stdout:7/892: creat d3/d16/d7b/f124 x:0 0 0 2024-04-12T05:56:21.168 INFO:tasks.workunit.client.1.smithi194.stdout:6/892: dread d3/d2d/f3d [0,4194304] 0 2024-04-12T05:56:21.169 INFO:tasks.workunit.client.1.smithi194.stdout:2/858: chown d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f118 10128920 1 2024-04-12T05:56:21.170 INFO:tasks.workunit.client.1.smithi194.stdout:0/877: dwrite d0/d3b/d52/dfa/d8f/fd3 [0,4194304] 0 2024-04-12T05:56:21.173 INFO:tasks.workunit.client.1.smithi194.stdout:1/921: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/fd5 [0,4194304] 0 2024-04-12T05:56:21.175 INFO:tasks.workunit.client.1.smithi194.stdout:3/984: creat d0/d1/d4f/f153 x:0 0 0 2024-04-12T05:56:21.175 INFO:tasks.workunit.client.1.smithi194.stdout:2/859: readlink d7/d12/d1a/de9/l108 0 2024-04-12T05:56:21.176 INFO:tasks.workunit.client.1.smithi194.stdout:9/918: mkdir d1/d4e/dea/dfc/d12f/d5f/df4/d137 0 2024-04-12T05:56:21.178 INFO:tasks.workunit.client.1.smithi194.stdout:2/860: chown d7/fa3 463726 1 2024-04-12T05:56:21.178 INFO:tasks.workunit.client.1.smithi194.stdout:0/878: creat d0/d3b/d52/dfa/d1a/d24/de1/f11c x:0 0 0 2024-04-12T05:56:21.180 INFO:tasks.workunit.client.1.smithi194.stdout:7/893: write d3/d27/f1b [6439313,80700] 0 2024-04-12T05:56:21.184 INFO:tasks.workunit.client.1.smithi194.stdout:0/879: mknod d0/d5f/c11d 0 2024-04-12T05:56:21.184 INFO:tasks.workunit.client.1.smithi194.stdout:2/861: write d7/d12/d25/d70/fa1 [4327817,61107] 0 2024-04-12T05:56:21.191 INFO:tasks.workunit.client.1.smithi194.stdout:6/893: dwrite d3/d2d/d33/d52/d5c/fa5 [0,4194304] 0 2024-04-12T05:56:21.192 INFO:tasks.workunit.client.1.smithi194.stdout:8/677: sync 2024-04-12T05:56:21.193 INFO:tasks.workunit.client.1.smithi194.stdout:7/894: fdatasync d3/d16/d23/d57/d72/d81/d3a/f60 0 2024-04-12T05:56:21.195 INFO:tasks.workunit.client.1.smithi194.stdout:1/922: rename d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/c5d to d0/d11/d18/d19/d1a/ded/c125 0 2024-04-12T05:56:21.197 INFO:tasks.workunit.client.1.smithi194.stdout:2/862: dread d7/d22/d93/d9f/fb0 [0,4194304] 0 2024-04-12T05:56:21.198 INFO:tasks.workunit.client.1.smithi194.stdout:3/985: mkdir d0/d1/d74/d39/d4d/deb/d3d/d154 0 2024-04-12T05:56:21.200 INFO:tasks.workunit.client.1.smithi194.stdout:8/678: creat d2/d8/daa/fd2 x:0 0 0 2024-04-12T05:56:21.201 INFO:tasks.workunit.client.1.smithi194.stdout:6/894: symlink d3/d2f/dd9/l11c 0 2024-04-12T05:56:21.201 INFO:tasks.workunit.client.1.smithi194.stdout:7/895: dread - d3/d16/d23/d57/d72/d81/d3a/dcc/f107 zero size 2024-04-12T05:56:21.203 INFO:tasks.workunit.client.1.smithi194.stdout:3/986: stat d0/d1/d74/d39/d4d/deb/d3d/f69 0 2024-04-12T05:56:21.208 INFO:tasks.workunit.client.1.smithi194.stdout:6/895: chown d3/d15/d2b/d48/d6f/d7a/d8f/lf4 5 1 2024-04-12T05:56:21.208 INFO:tasks.workunit.client.1.smithi194.stdout:2/863: fsync d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f105 0 2024-04-12T05:56:21.209 INFO:tasks.workunit.client.1.smithi194.stdout:7/896: write d3/d27/d22/d8e/fff [675995,16760] 0 2024-04-12T05:56:21.212 INFO:tasks.workunit.client.1.smithi194.stdout:9/919: dread d1/d21/d39/d72/fb5 [0,4194304] 0 2024-04-12T05:56:21.214 INFO:tasks.workunit.client.1.smithi194.stdout:0/880: dread d0/d3b/d52/dfa/fdb [0,4194304] 0 2024-04-12T05:56:21.216 INFO:tasks.workunit.client.1.smithi194.stdout:1/923: dwrite d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/fd5 [0,4194304] 0 2024-04-12T05:56:21.216 INFO:tasks.workunit.client.1.smithi194.stdout:2/864: symlink d7/d12/d25/d70/df4/l120 0 2024-04-12T05:56:21.217 INFO:tasks.workunit.client.1.smithi194.stdout:9/920: creat d1/da6/d135/f138 x:0 0 0 2024-04-12T05:56:21.227 INFO:tasks.workunit.client.1.smithi194.stdout:8/679: dread d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:56:21.228 INFO:tasks.workunit.client.1.smithi194.stdout:7/897: mknod d3/d16/d7b/dc3/c125 0 2024-04-12T05:56:21.228 INFO:tasks.workunit.client.1.smithi194.stdout:0/881: write d0/d3b/d52/dfa/d8f/f103 [295621,32833] 0 2024-04-12T05:56:21.230 INFO:tasks.workunit.client.1.smithi194.stdout:6/896: link d3/d2f/d4e/d9a/dd8/fa4 d3/d2f/d4e/d9a/dd8/f11d 0 2024-04-12T05:56:21.230 INFO:tasks.workunit.client.1.smithi194.stdout:7/898: mknod d3/d27/d6a/dda/d74/c126 0 2024-04-12T05:56:21.237 INFO:tasks.workunit.client.1.smithi194.stdout:9/921: write d1/d41/d97/f10c [234943,49247] 0 2024-04-12T05:56:21.241 INFO:tasks.workunit.client.1.smithi194.stdout:2/865: rename d7/d12/d1a/d69/df0/d101/f10d to d7/d12/d1a/d69/df0/d101/f121 0 2024-04-12T05:56:21.244 INFO:tasks.workunit.client.1.smithi194.stdout:9/922: stat d1/d41/dc9 0 2024-04-12T05:56:21.250 INFO:tasks.workunit.client.1.smithi194.stdout:8/680: dread d2/d8/d9/de/d38/fb2 [0,4194304] 0 2024-04-12T05:56:21.253 INFO:tasks.workunit.client.1.smithi194.stdout:2/866: rmdir d7/d12/d1a/d86 39 2024-04-12T05:56:21.253 INFO:tasks.workunit.client.1.smithi194.stdout:3/987: dwrite d0/d1/d74/d39/f3b [8388608,4194304] 0 2024-04-12T05:56:21.256 INFO:tasks.workunit.client.1.smithi194.stdout:0/882: dwrite d0/f1d [0,4194304] 0 2024-04-12T05:56:21.256 INFO:tasks.workunit.client.1.smithi194.stdout:6/897: write d3/d2f/d4e/d68/fa2 [1890598,672] 0 2024-04-12T05:56:21.262 INFO:tasks.workunit.client.1.smithi194.stdout:1/924: dread d0/d11/f83 [0,4194304] 0 2024-04-12T05:56:21.265 INFO:tasks.workunit.client.1.smithi194.stdout:7/899: dwrite d3/d16/d23/d37/d3d/f61 [0,4194304] 0 2024-04-12T05:56:21.267 INFO:tasks.workunit.client.1.smithi194.stdout:2/867: write d7/d22/d93/fdf [88756,72450] 0 2024-04-12T05:56:21.277 INFO:tasks.workunit.client.1.smithi194.stdout:2/868: rmdir d7/d12/d1a 39 2024-04-12T05:56:21.277 INFO:tasks.workunit.client.1.smithi194.stdout:7/900: fsync d3/d27/d22/d31/f10b 0 2024-04-12T05:56:21.280 INFO:tasks.workunit.client.1.smithi194.stdout:0/883: fdatasync d0/d3b/d52/dfa/d1a/d24/d4b/f65 0 2024-04-12T05:56:21.283 INFO:tasks.workunit.client.1.smithi194.stdout:8/681: dwrite d2/d8/d66/d7f/fa7 [0,4194304] 0 2024-04-12T05:56:21.284 INFO:tasks.workunit.client.1.smithi194.stdout:7/901: dread - d3/d16/d23/d57/d72/d81/d3a/dcc/f10a zero size 2024-04-12T05:56:21.285 INFO:tasks.workunit.client.1.smithi194.stdout:2/869: truncate d7/d12/d25/d5d/d47/dac/fea 450917 0 2024-04-12T05:56:21.287 INFO:tasks.workunit.client.1.smithi194.stdout:1/925: mknod d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/c126 0 2024-04-12T05:56:21.287 INFO:tasks.workunit.client.1.smithi194.stdout:0/884: write d0/d3b/d52/dfa/d1a/d3d/f7a [5149672,47834] 0 2024-04-12T05:56:21.290 INFO:tasks.workunit.client.1.smithi194.stdout:6/898: creat d3/f11e x:0 0 0 2024-04-12T05:56:21.290 INFO:tasks.workunit.client.1.smithi194.stdout:3/988: mkdir d0/d1/d74/d39/d4d/deb/d1e/d81/d155 0 2024-04-12T05:56:21.291 INFO:tasks.workunit.client.1.smithi194.stdout:7/902: symlink d3/d27/d6a/dda/l127 0 2024-04-12T05:56:21.292 INFO:tasks.workunit.client.1.smithi194.stdout:1/926: mkdir d0/d11/d18/d35/da5/d127 0 2024-04-12T05:56:21.295 INFO:tasks.workunit.client.1.smithi194.stdout:0/885: rename d0/d3b/f4c to d0/da/d4a/d8e/f11e 0 2024-04-12T05:56:21.296 INFO:tasks.workunit.client.1.smithi194.stdout:7/903: mkdir d3/d16/d23/d40/d9b/d128 0 2024-04-12T05:56:21.298 INFO:tasks.workunit.client.1.smithi194.stdout:6/899: symlink d3/d2d/d33/d52/d62/d10a/de2/l11f 0 2024-04-12T05:56:21.298 INFO:tasks.workunit.client.1.smithi194.stdout:0/886: write d0/da/da7/f114 [892941,25831] 0 2024-04-12T05:56:21.303 INFO:tasks.workunit.client.1.smithi194.stdout:3/989: truncate d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/d87/df0/f138 429578 0 2024-04-12T05:56:21.305 INFO:tasks.workunit.client.1.smithi194.stdout:0/887: mkdir d0/de3/d5b/d11f 0 2024-04-12T05:56:21.305 INFO:tasks.workunit.client.1.smithi194.stdout:1/927: rename d0/d11/d18/fa2 to d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/f128 0 2024-04-12T05:56:21.306 INFO:tasks.workunit.client.1.smithi194.stdout:1/928: dread - d0/f115 zero size 2024-04-12T05:56:21.308 INFO:tasks.workunit.client.1.smithi194.stdout:0/888: creat d0/de3/d5b/f120 x:0 0 0 2024-04-12T05:56:21.309 INFO:tasks.workunit.client.1.smithi194.stdout:7/904: truncate d3/d16/d23/d57/d72/d81/f73 1575565 0 2024-04-12T05:56:21.319 INFO:tasks.workunit.client.1.smithi194.stdout:7/905: fsync d3/d16/d23/d37/d95/fd1 0 2024-04-12T05:56:21.321 INFO:tasks.workunit.client.1.smithi194.stdout:0/889: getdents d0/d5f/df9 0 2024-04-12T05:56:21.323 INFO:tasks.workunit.client.1.smithi194.stdout:0/890: chown d0/d3b/d52/dfa/d1a/d24/de1/f118 1838 1 2024-04-12T05:56:21.326 INFO:tasks.workunit.client.1.smithi194.stdout:0/891: write d0/de3/d5b/f120 [503353,15122] 0 2024-04-12T05:56:21.328 INFO:tasks.workunit.client.1.smithi194.stdout:1/929: fdatasync d0/d11/d18/d35/da5/f97 0 2024-04-12T05:56:21.329 INFO:tasks.workunit.client.1.smithi194.stdout:7/906: truncate d3/d16/d23/d57/fba 871941 0 2024-04-12T05:56:21.331 INFO:tasks.workunit.client.1.smithi194.stdout:7/907: chown d3/d16/d23/d57/d72/d81/d3a/dcc 495484 1 2024-04-12T05:56:21.332 INFO:tasks.workunit.client.1.smithi194.stdout:7/908: readlink d3/d27/d6a/dda/d74/lac 0 2024-04-12T05:56:21.335 INFO:tasks.workunit.client.1.smithi194.stdout:0/892: fsync d0/da/d4a/d8e/f72 0 2024-04-12T05:56:21.335 INFO:tasks.workunit.client.1.smithi194.stdout:1/930: write d0/d11/d18/d19/f1f [2933968,13626] 0 2024-04-12T05:56:21.336 INFO:tasks.workunit.client.1.smithi194.stdout:7/909: creat d3/d27/d22/d31/d103/f129 x:0 0 0 2024-04-12T05:56:21.342 INFO:tasks.workunit.client.1.smithi194.stdout:6/900: dread d3/d2f/d37/f45 [0,4194304] 0 2024-04-12T05:56:21.344 INFO:tasks.workunit.client.1.smithi194.stdout:1/931: chown d0/d11/d18/d19/d1a/d6a/d27/d6c/c88 7 1 2024-04-12T05:56:21.345 INFO:tasks.workunit.client.1.smithi194.stdout:6/901: write d3/d2f/d4e/d9a/dd8/f85 [1389191,109577] 0 2024-04-12T05:56:21.347 INFO:tasks.workunit.client.1.smithi194.stdout:7/910: creat d3/d27/d6a/dda/f12a x:0 0 0 2024-04-12T05:56:21.347 INFO:tasks.workunit.client.1.smithi194.stdout:3/990: dwrite d0/d1/d4f/f78 [0,4194304] 0 2024-04-12T05:56:21.356 INFO:tasks.workunit.client.1.smithi194.stdout:6/902: symlink d3/d15/d2b/d48/d6f/l120 0 2024-04-12T05:56:21.356 INFO:tasks.workunit.client.1.smithi194.stdout:2/870: dread d7/d12/d25/d70/fa1 [0,4194304] 0 2024-04-12T05:56:21.357 INFO:tasks.workunit.client.1.smithi194.stdout:7/911: write d3/d16/d23/f83 [115737,81550] 0 2024-04-12T05:56:21.366 INFO:tasks.workunit.client.1.smithi194.stdout:6/903: read d3/d2f/d37/f4a [2776257,100496] 0 2024-04-12T05:56:21.366 INFO:tasks.workunit.client.1.smithi194.stdout:7/912: creat d3/d16/d23/d37/d95/f12b x:0 0 0 2024-04-12T05:56:21.367 INFO:tasks.workunit.client.1.smithi194.stdout:1/932: dwrite d0/d11/d18/d19/d1a/d6a/d27/d6c/f112 [0,4194304] 0 2024-04-12T05:56:21.374 INFO:tasks.workunit.client.1.smithi194.stdout:2/871: dwrite d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f105 [0,4194304] 0 2024-04-12T05:56:21.377 INFO:tasks.workunit.client.1.smithi194.stdout:0/893: dread d0/f53 [0,4194304] 0 2024-04-12T05:56:21.377 INFO:tasks.workunit.client.1.smithi194.stdout:3/991: write d0/d1/d74/d39/d4d/deb/d1e/d33/f68 [2091269,101785] 0 2024-04-12T05:56:21.378 INFO:tasks.workunit.client.1.smithi194.stdout:6/904: fsync d3/d2f/d4e/d9a/dd8/fa8 0 2024-04-12T05:56:21.379 INFO:tasks.workunit.client.1.smithi194.stdout:2/872: mkdir d7/d22/d93/d9a/d122 0 2024-04-12T05:56:21.381 INFO:tasks.workunit.client.1.smithi194.stdout:2/873: chown d7/d12/d25/d5d/d11e 29225 1 2024-04-12T05:56:21.386 INFO:tasks.workunit.client.1.smithi194.stdout:6/905: rmdir d3/d2f/d4e/d68/dad/dc4 39 2024-04-12T05:56:21.389 INFO:tasks.workunit.client.1.smithi194.stdout:7/913: mkdir d3/d27/d22/d62/d12c 0 2024-04-12T05:56:21.391 INFO:tasks.workunit.client.1.smithi194.stdout:1/933: creat d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/f129 x:0 0 0 2024-04-12T05:56:21.392 INFO:tasks.workunit.client.1.smithi194.stdout:0/894: truncate d0/d3b/d52/dfa/d27/d54/f71 4427697 0 2024-04-12T05:56:21.392 INFO:tasks.workunit.client.1.smithi194.stdout:3/992: creat d0/d1/d10/f156 x:0 0 0 2024-04-12T05:56:21.393 INFO:tasks.workunit.client.1.smithi194.stdout:2/874: mkdir d7/d12/d25/d5d/d11e/d123 0 2024-04-12T05:56:21.393 INFO:tasks.workunit.client.1.smithi194.stdout:6/906: mkdir d3/d2d/d33/d121 0 2024-04-12T05:56:21.396 INFO:tasks.workunit.client.1.smithi194.stdout:1/934: mknod d0/c12a 0 2024-04-12T05:56:21.397 INFO:tasks.workunit.client.1.smithi194.stdout:6/907: symlink d3/d2d/d33/db1/l122 0 2024-04-12T05:56:21.398 INFO:tasks.workunit.client.1.smithi194.stdout:1/935: stat d0/d11/f9f 0 2024-04-12T05:56:21.398 INFO:tasks.workunit.client.1.smithi194.stdout:2/875: creat d7/d12/d25/d5d/d11e/f124 x:0 0 0 2024-04-12T05:56:21.399 INFO:tasks.workunit.client.1.smithi194.stdout:0/895: unlink d0/d3b/d52/dfa/d1a/c108 0 2024-04-12T05:56:21.399 INFO:tasks.workunit.client.1.smithi194.stdout:1/936: readlink d0/d11/d18/d19/d1a/d6a/d27/l94 0 2024-04-12T05:56:21.400 INFO:tasks.workunit.client.1.smithi194.stdout:6/908: creat d3/d2f/d4e/d9a/dd8/d77/f123 x:0 0 0 2024-04-12T05:56:21.402 INFO:tasks.workunit.client.1.smithi194.stdout:6/909: dread - d3/d2d/d33/d52/d5c/fe1 zero size 2024-04-12T05:56:21.402 INFO:tasks.workunit.client.1.smithi194.stdout:1/937: rmdir d0/d11/d18/d19/d1c 39 2024-04-12T05:56:21.403 INFO:tasks.workunit.client.1.smithi194.stdout:6/910: fdatasync d3/d2f/d4e/d9a/dd8/f85 0 2024-04-12T05:56:21.404 INFO:tasks.workunit.client.1.smithi194.stdout:0/896: rename d0/d3b/d52/dfa/d1a/d3d/f7a to d0/d5f/f121 0 2024-04-12T05:56:21.406 INFO:tasks.workunit.client.1.smithi194.stdout:0/897: write d0/d3b/f74 [447809,72028] 0 2024-04-12T05:56:21.406 INFO:tasks.workunit.client.1.smithi194.stdout:6/911: fdatasync d3/d2d/d33/d52/f5a 0 2024-04-12T05:56:21.407 INFO:tasks.workunit.client.1.smithi194.stdout:3/993: symlink d0/d1/d74/d39/d4d/deb/d3d/l157 0 2024-04-12T05:56:21.409 INFO:tasks.workunit.client.1.smithi194.stdout:1/938: write d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fdb [202877,48358] 0 2024-04-12T05:56:21.410 INFO:tasks.workunit.client.1.smithi194.stdout:7/914: getdents d3/d16/d23/d37/d3d/d56/dd0/dd9 0 2024-04-12T05:56:21.410 INFO:tasks.workunit.client.1.smithi194.stdout:0/898: dread - d0/de3/df0/fab zero size 2024-04-12T05:56:21.413 INFO:tasks.workunit.client.1.smithi194.stdout:0/899: symlink d0/da/d4a/l122 0 2024-04-12T05:56:21.415 INFO:tasks.workunit.client.1.smithi194.stdout:6/912: link d3/d2d/d33/d52/d62/d10a/cb0 d3/d2d/d33/d107/d117/c124 0 2024-04-12T05:56:21.418 INFO:tasks.workunit.client.1.smithi194.stdout:1/939: mkdir d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/d12b 0 2024-04-12T05:56:21.420 INFO:tasks.workunit.client.1.smithi194.stdout:3/994: dread d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/d79/f84 [0,4194304] 0 2024-04-12T05:56:21.421 INFO:tasks.workunit.client.1.smithi194.stdout:7/915: rename d3/d16/d7b/l7d to d3/d16/d23/d57/d72/d81/l12d 0 2024-04-12T05:56:21.425 INFO:tasks.workunit.client.1.smithi194.stdout:0/900: rename d0/da/d4a/f5e to d0/de3/df0/dbe/f123 0 2024-04-12T05:56:21.427 INFO:tasks.workunit.client.1.smithi194.stdout:1/940: truncate d0/d11/d18/d19/d1c/f24 825149 0 2024-04-12T05:56:21.428 INFO:tasks.workunit.client.1.smithi194.stdout:7/916: chown d3/d16/d23/d57/d72/l123 1 1 2024-04-12T05:56:21.429 INFO:tasks.workunit.client.1.smithi194.stdout:6/913: dread d3/d2d/d33/d52/d5c/d8c/f93 [0,4194304] 0 2024-04-12T05:56:21.430 INFO:tasks.workunit.client.1.smithi194.stdout:1/941: truncate d0/d11/f118 835223 0 2024-04-12T05:56:21.431 INFO:tasks.workunit.client.1.smithi194.stdout:1/942: fsync d0/d11/d18/d19/f4e 0 2024-04-12T05:56:21.432 INFO:tasks.workunit.client.1.smithi194.stdout:6/914: mknod d3/d2d/c125 0 2024-04-12T05:56:21.434 INFO:tasks.workunit.client.1.smithi194.stdout:7/917: mkdir d3/def/d12e 0 2024-04-12T05:56:21.436 INFO:tasks.workunit.client.1.smithi194.stdout:6/915: mkdir d3/d2f/d4e/d63/d83/d126 0 2024-04-12T05:56:21.436 INFO:tasks.workunit.client.1.smithi194.stdout:1/943: read d0/d11/d18/d19/d1a/d6a/d27/da8/d5b/f9b [4626581,88741] 0 2024-04-12T05:56:21.437 INFO:tasks.workunit.client.1.smithi194.stdout:6/916: chown d3/d2f 39312 1 2024-04-12T05:56:21.438 INFO:tasks.workunit.client.1.smithi194.stdout:3/995: dwrite d0/d1/d74/d39/d4d/deb/d1e/d81/f48 [0,4194304] 0 2024-04-12T05:56:21.440 INFO:tasks.workunit.client.1.smithi194.stdout:3/996: write d0/d1/d4f/dd7/de1/f125 [338678,123597] 0 2024-04-12T05:56:21.445 INFO:tasks.workunit.client.1.smithi194.stdout:0/901: dread d0/d3b/d52/dfa/d27/fa0 [0,4194304] 0 2024-04-12T05:56:21.445 INFO:tasks.workunit.client.1.smithi194.stdout:1/944: mknod d0/d11/d18/d19/d1a/d6a/d27/c12c 0 2024-04-12T05:56:21.448 INFO:tasks.workunit.client.1.smithi194.stdout:6/917: truncate d3/d2f/d4e/d68/fff 693995 0 2024-04-12T05:56:21.449 INFO:tasks.workunit.client.1.smithi194.stdout:6/918: dread - d3/d2f/d4e/d9a/f112 zero size 2024-04-12T05:56:21.450 INFO:tasks.workunit.client.1.smithi194.stdout:6/919: write d3/d15/f9f [4410251,52194] 0 2024-04-12T05:56:21.451 INFO:tasks.workunit.client.1.smithi194.stdout:3/997: mknod d0/d1/d74/d39/d4d/deb/d1e/d81/c158 0 2024-04-12T05:56:21.451 INFO:tasks.workunit.client.1.smithi194.stdout:1/945: rmdir d0/d11/d18 39 2024-04-12T05:56:21.452 INFO:tasks.workunit.client.1.smithi194.stdout:6/920: stat d3/d2f/d4e/d9a/db3/fbc 0 2024-04-12T05:56:21.456 INFO:tasks.workunit.client.1.smithi194.stdout:0/902: rename d0/de3/df0/lcf to d0/da/d4a/d8e/l124 0 2024-04-12T05:56:21.456 INFO:tasks.workunit.client.1.smithi194.stdout:3/998: symlink d0/d1/d4f/dd7/d132/l159 0 2024-04-12T05:56:21.456 INFO:tasks.workunit.client.1.smithi194.stdout:1/946: mknod d0/c12d 0 2024-04-12T05:56:21.458 INFO:tasks.workunit.client.1.smithi194.stdout:6/921: write d3/d15/f21 [2626078,113176] 0 2024-04-12T05:56:21.458 INFO:tasks.workunit.client.1.smithi194.stdout:0/903: mknod d0/d5f/c125 0 2024-04-12T05:56:21.459 INFO:tasks.workunit.client.1.smithi194.stdout:6/922: read - d3/d34/fea zero size 2024-04-12T05:56:21.460 INFO:tasks.workunit.client.1.smithi194.stdout:3/999: read d0/d1/d74/d39/d4d/deb/d1e/d81/d5b/d64/fad [2264436,127569] 0 2024-04-12T05:56:21.462 INFO:tasks.workunit.client.1.smithi194.stdout:6/923: chown d3/d2d/d33/c38 51373239 1 2024-04-12T05:56:21.464 INFO:tasks.workunit.client.1.smithi194.stdout:0/904: dread d0/de3/df0/dbe/f123 [0,4194304] 0 2024-04-12T05:56:21.466 INFO:tasks.workunit.client.1.smithi194.stdout:1/947: mkdir d0/d11/d18/d19/d1a/d6a/dac/d104/d12e 0 2024-04-12T05:56:21.472 INFO:tasks.workunit.client.1.smithi194.stdout:0/905: dwrite d0/d3b/fb5 [0,4194304] 0 2024-04-12T05:56:21.479 INFO:tasks.workunit.client.1.smithi194.stdout:0/906: rename d0/d3b/fb5 to d0/d5f/df9/f126 0 2024-04-12T05:56:21.482 INFO:tasks.workunit.client.1.smithi194.stdout:1/948: dread d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 [0,4194304] 0 2024-04-12T05:56:21.484 INFO:tasks.workunit.client.1.smithi194.stdout:0/907: rename d0/da/da7/ff2 to d0/de3/df0/f127 0 2024-04-12T05:56:21.493 INFO:tasks.workunit.client.1.smithi194.stdout:1/949: creat d0/d11/d18/d19/d1a/d6a/dac/d104/f12f x:0 0 0 2024-04-12T05:56:21.495 INFO:tasks.workunit.client.1.smithi194.stdout:0/908: unlink d0/d3b/d52/dfa/d1a/d24/de1/ced 0 2024-04-12T05:56:21.502 INFO:tasks.workunit.client.1.smithi194.stdout:0/909: mkdir d0/d3b/d52/dfa/d27/d87/d128 0 2024-04-12T05:56:21.504 INFO:tasks.workunit.client.1.smithi194.stdout:1/950: dwrite d0/d11/f118 [0,4194304] 0 2024-04-12T05:56:21.506 INFO:tasks.workunit.client.1.smithi194.stdout:6/924: dwrite d3/d2d/d33/d58/faf [0,4194304] 0 2024-04-12T05:56:21.510 INFO:tasks.workunit.client.1.smithi194.stdout:6/925: dread - d3/d2f/d4e/d9a/dd8/d77/ffa zero size 2024-04-12T05:56:21.513 INFO:tasks.workunit.client.1.smithi194.stdout:6/926: chown d3/d34/fea 309400 1 2024-04-12T05:56:21.514 INFO:tasks.workunit.client.1.smithi194.stdout:6/927: write d3/d15/d2b/d48/f54 [2460826,13446] 0 2024-04-12T05:56:21.515 INFO:tasks.workunit.client.1.smithi194.stdout:6/928: chown d3/d2d/d33/db1/cf1 656612 1 2024-04-12T05:56:21.517 INFO:tasks.workunit.client.1.smithi194.stdout:1/951: dwrite d0/d11/d18/d19/d1c/dca/f10b [0,4194304] 0 2024-04-12T05:56:21.526 INFO:tasks.workunit.client.1.smithi194.stdout:6/929: mkdir d3/d2f/d127 0 2024-04-12T05:56:21.532 INFO:tasks.workunit.client.1.smithi194.stdout:6/930: mkdir d3/d15/d2b/d48/d6f/d7a/d8f/d128 0 2024-04-12T05:56:21.532 INFO:tasks.workunit.client.1.smithi194.stdout:6/931: read - d3/d15/d2b/d48/d6f/d7a/d8f/d98/ff8 zero size 2024-04-12T05:56:21.536 INFO:tasks.workunit.client.1.smithi194.stdout:6/932: fsync d3/d15/d2b/d48/d6f/d7a/d8f/fc0 0 2024-04-12T05:56:21.540 INFO:tasks.workunit.client.1.smithi194.stdout:6/933: creat d3/d15/d2b/d48/f129 x:0 0 0 2024-04-12T05:56:21.540 INFO:tasks.workunit.client.1.smithi194.stdout:0/910: dread d0/d3b/f7e [0,4194304] 0 2024-04-12T05:56:21.546 INFO:tasks.workunit.client.1.smithi194.stdout:0/911: write d0/d3b/d52/dfa/d27/d54/fad [139788,65961] 0 2024-04-12T05:56:21.551 INFO:tasks.workunit.client.1.smithi194.stdout:0/912: write d0/d3b/d52/dfa/d27/f3f [1392782,130046] 0 2024-04-12T05:56:21.551 INFO:tasks.workunit.client.1.smithi194.stdout:1/952: dread d0/d11/d18/d19/d1c/f6b [0,4194304] 0 2024-04-12T05:56:21.554 INFO:tasks.workunit.client.1.smithi194.stdout:1/953: write d0/d11/d18/d19/d1a/d6a/dac/db5/df8/f106 [688920,45115] 0 2024-04-12T05:56:21.554 INFO:tasks.workunit.client.1.smithi194.stdout:6/934: dread d3/d2f/d37/f4a [4194304,4194304] 0 2024-04-12T05:56:21.556 INFO:tasks.workunit.client.1.smithi194.stdout:0/913: creat d0/da/d4a/d8e/f129 x:0 0 0 2024-04-12T05:56:21.557 INFO:tasks.workunit.client.1.smithi194.stdout:1/954: write d0/d11/d18/d19/d1a/d6a/dac/db5/df8/f106 [1249841,17889] 0 2024-04-12T05:56:21.558 INFO:tasks.workunit.client.1.smithi194.stdout:6/935: write d3/d2f/d4e/d9a/dd8/d77/dde/fec [2137265,89745] 0 2024-04-12T05:56:21.563 INFO:tasks.workunit.client.1.smithi194.stdout:1/955: mknod d0/d11/d18/d19/d1a/d6a/d27/d48/c130 0 2024-04-12T05:56:21.567 INFO:tasks.workunit.client.1.smithi194.stdout:1/956: chown d0/d11/d18/d19/d1c/l5f 7854 1 2024-04-12T05:56:21.568 INFO:tasks.workunit.client.1.smithi194.stdout:9/923: sync 2024-04-12T05:56:21.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:21 smithi194 ceph-mon[49771]: mgrmap e29: smithi008.ixmnlp(active, since 3s) 2024-04-12T05:56:21.570 INFO:tasks.workunit.client.1.smithi194.stdout:0/914: dwrite d0/d3b/fe7 [0,4194304] 0 2024-04-12T05:56:21.576 INFO:tasks.workunit.client.1.smithi194.stdout:6/936: dwrite d3/d2d/d33/d52/d5c/fa5 [0,4194304] 0 2024-04-12T05:56:21.580 INFO:tasks.workunit.client.1.smithi194.stdout:0/915: mkdir d0/d12a 0 2024-04-12T05:56:21.581 INFO:tasks.workunit.client.1.smithi194.stdout:1/957: rename d0/d11/d18/d35/lcb to d0/d11/d18/d19/d1a/d6a/d27/l131 0 2024-04-12T05:56:21.582 INFO:tasks.workunit.client.1.smithi194.stdout:9/924: mknod d1/d21/d77/d98/d9f/dde/c139 0 2024-04-12T05:56:21.582 INFO:tasks.workunit.client.1.smithi194.stdout:0/916: rename d0/d3b/d52/dfa to d0/d3b/d52/dfa/d27/d54/d12b 22 2024-04-12T05:56:21.584 INFO:tasks.workunit.client.1.smithi194.stdout:6/937: mknod d3/d2f/c12a 0 2024-04-12T05:56:21.587 INFO:tasks.workunit.client.1.smithi194.stdout:0/917: mknod d0/de3/df0/c12c 0 2024-04-12T05:56:21.595 INFO:tasks.workunit.client.1.smithi194.stdout:1/958: write d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 [929924,85003] 0 2024-04-12T05:56:21.599 INFO:tasks.workunit.client.1.smithi194.stdout:0/918: dwrite d0/d3b/d52/dfa/d1a/d24/de1/f11c [0,4194304] 0 2024-04-12T05:56:21.601 INFO:tasks.workunit.client.1.smithi194.stdout:1/959: write d0/d11/d18/d19/d1a/ded/ff4 [3300854,11921] 0 2024-04-12T05:56:21.607 INFO:tasks.workunit.client.1.smithi194.stdout:1/960: mkdir d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/d132 0 2024-04-12T05:56:21.608 INFO:tasks.workunit.client.1.smithi194.stdout:0/919: getdents d0/d3b 0 2024-04-12T05:56:21.610 INFO:tasks.workunit.client.1.smithi194.stdout:6/938: dread d3/d15/d2b/d48/f4f [0,4194304] 0 2024-04-12T05:56:21.610 INFO:tasks.workunit.client.1.smithi194.stdout:9/925: dread d1/d4e/f6d [0,4194304] 0 2024-04-12T05:56:21.612 INFO:tasks.workunit.client.1.smithi194.stdout:9/926: fdatasync d1/d21/d39/df1/f12c 0 2024-04-12T05:56:21.613 INFO:tasks.workunit.client.1.smithi194.stdout:0/920: write d0/d3b/d52/dfa/d1a/f6d [1779427,65437] 0 2024-04-12T05:56:21.614 INFO:tasks.workunit.client.1.smithi194.stdout:6/939: fdatasync d3/d2d/d33/d52/d5c/f5f 0 2024-04-12T05:56:21.618 INFO:tasks.workunit.client.1.smithi194.stdout:1/961: dread d0/d11/f118 [0,4194304] 0 2024-04-12T05:56:21.619 INFO:tasks.workunit.client.1.smithi194.stdout:9/927: symlink d1/d21/d77/d98/dab/d128/l13a 0 2024-04-12T05:56:21.623 INFO:tasks.workunit.client.1.smithi194.stdout:9/928: write d1/d41/dba/fe8 [8236324,112895] 0 2024-04-12T05:56:21.626 INFO:tasks.workunit.client.1.smithi194.stdout:6/940: dwrite d3/f3b [0,4194304] 0 2024-04-12T05:56:21.627 INFO:tasks.workunit.client.1.smithi194.stdout:0/921: dwrite d0/da/d4a/fca [4194304,4194304] 0 2024-04-12T05:56:21.634 INFO:tasks.workunit.client.1.smithi194.stdout:1/962: dwrite d0/f85 [4194304,4194304] 0 2024-04-12T05:56:21.641 INFO:tasks.workunit.client.1.smithi194.stdout:6/941: mknod d3/d2d/d33/d11a/c12b 0 2024-04-12T05:56:21.641 INFO:tasks.workunit.client.1.smithi194.stdout:9/929: write d1/d21/d77/d98/d9f/dde/f118 [1360882,35000] 0 2024-04-12T05:56:21.644 INFO:tasks.workunit.client.1.smithi194.stdout:2/876: sync 2024-04-12T05:56:21.644 INFO:tasks.workunit.client.1.smithi194.stdout:8/682: sync 2024-04-12T05:56:21.646 INFO:tasks.workunit.client.1.smithi194.stdout:2/877: stat d7/d12/d25/d5d/d47/dac 0 2024-04-12T05:56:21.646 INFO:tasks.workunit.client.1.smithi194.stdout:0/922: dread d0/d3b/d52/dfa/d1a/f6d [0,4194304] 0 2024-04-12T05:56:21.650 INFO:tasks.workunit.client.1.smithi194.stdout:8/683: fdatasync d2/d8/d9/d1a/f5a 0 2024-04-12T05:56:21.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:21 smithi008 ceph-mon[39722]: mgrmap e29: smithi008.ixmnlp(active, since 3s) 2024-04-12T05:56:21.653 INFO:tasks.workunit.client.1.smithi194.stdout:2/878: chown d7/d12/d25/d5d/d47/d52/d7b/db2/f112 8425622 1 2024-04-12T05:56:21.653 INFO:tasks.workunit.client.1.smithi194.stdout:0/923: symlink d0/da/da7/l12d 0 2024-04-12T05:56:21.654 INFO:tasks.workunit.client.1.smithi194.stdout:0/924: stat d0/d5f/cd7 0 2024-04-12T05:56:21.655 INFO:tasks.workunit.client.1.smithi194.stdout:1/963: link d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9/f129 d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/f133 0 2024-04-12T05:56:21.659 INFO:tasks.workunit.client.1.smithi194.stdout:1/964: symlink d0/d11/d18/d19/d91/l134 0 2024-04-12T05:56:21.661 INFO:tasks.workunit.client.1.smithi194.stdout:2/879: fdatasync d7/d12/d1a/d86/fe1 0 2024-04-12T05:56:21.662 INFO:tasks.workunit.client.1.smithi194.stdout:6/942: dread d3/d15/f21 [0,4194304] 0 2024-04-12T05:56:21.663 INFO:tasks.workunit.client.1.smithi194.stdout:2/880: chown d7/d12/d1a/d69/d7e/lbd 4 1 2024-04-12T05:56:21.675 INFO:tasks.workunit.client.1.smithi194.stdout:1/965: creat d0/d11/d18/d19/d1a/d6a/dac/dba/f135 x:0 0 0 2024-04-12T05:56:21.677 INFO:tasks.workunit.client.1.smithi194.stdout:8/684: write d2/d8/d9/de/f2e [2320053,66710] 0 2024-04-12T05:56:21.683 INFO:tasks.workunit.client.1.smithi194.stdout:9/930: dread d1/fb [0,4194304] 0 2024-04-12T05:56:21.684 INFO:tasks.workunit.client.1.smithi194.stdout:9/931: chown d1/d4e/dea/dfc/d12f/d20/f57 3113367 1 2024-04-12T05:56:21.685 INFO:tasks.workunit.client.1.smithi194.stdout:9/932: stat d1/d4e/dea/dfc/d12f/d20/d37/d38/l59 0 2024-04-12T05:56:21.687 INFO:tasks.workunit.client.1.smithi194.stdout:2/881: mknod d7/d12/d1a/d86/c125 0 2024-04-12T05:56:21.690 INFO:tasks.workunit.client.1.smithi194.stdout:9/933: creat d1/d21/d39/dc4/f13b x:0 0 0 2024-04-12T05:56:21.693 INFO:tasks.workunit.client.1.smithi194.stdout:1/966: rename d0/d11/d18/d19/d1a/d6a/dac/dba to d0/d11/d18/d19/d1a/d6a/d27/d48/da7/d136 0 2024-04-12T05:56:21.695 INFO:tasks.workunit.client.1.smithi194.stdout:2/882: fdatasync d7/d22/f5c 0 2024-04-12T05:56:21.701 INFO:tasks.workunit.client.1.smithi194.stdout:2/883: mkdir d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d126 0 2024-04-12T05:56:21.701 INFO:tasks.workunit.client.1.smithi194.stdout:1/967: dwrite d0/d11/d18/d19/d1c/f46 [0,4194304] 0 2024-04-12T05:56:21.703 INFO:tasks.workunit.client.1.smithi194.stdout:9/934: getdents d1/d21/d49/dcb 0 2024-04-12T05:56:21.704 INFO:tasks.workunit.client.1.smithi194.stdout:2/884: creat d7/d12/d1a/dbf/f127 x:0 0 0 2024-04-12T05:56:21.713 INFO:tasks.workunit.client.1.smithi194.stdout:1/968: dread d0/d11/d18/d19/d1c/dca/f10b [0,4194304] 0 2024-04-12T05:56:21.715 INFO:tasks.workunit.client.1.smithi194.stdout:2/885: mknod d7/d12/d25/d5d/d11e/c128 0 2024-04-12T05:56:21.719 INFO:tasks.workunit.client.1.smithi194.stdout:2/886: chown d7/d12/d1a/d69/f82 2416 1 2024-04-12T05:56:21.719 INFO:tasks.workunit.client.1.smithi194.stdout:1/969: chown d0/d11/d18/d19/d1a/d6a/d27/d5a/d9d 0 1 2024-04-12T05:56:21.726 INFO:tasks.workunit.client.1.smithi194.stdout:8/685: dwrite d2/d8/d9/f48 [0,4194304] 0 2024-04-12T05:56:21.727 INFO:tasks.workunit.client.1.smithi194.stdout:6/943: dwrite d3/d70/f86 [0,4194304] 0 2024-04-12T05:56:21.731 INFO:tasks.workunit.client.1.smithi194.stdout:8/686: dread - d2/d8/d66/d7f/f9c zero size 2024-04-12T05:56:21.732 INFO:tasks.workunit.client.1.smithi194.stdout:2/887: rmdir d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a 39 2024-04-12T05:56:21.735 INFO:tasks.workunit.client.1.smithi194.stdout:1/970: mknod d0/d11/d18/d19/d1a/d6a/d27/d48/da7/df3/c137 0 2024-04-12T05:56:21.737 INFO:tasks.workunit.client.1.smithi194.stdout:6/944: rename d3/d34/c66 to d3/d15/dc8/d10d/c12c 0 2024-04-12T05:56:21.738 INFO:tasks.workunit.client.1.smithi194.stdout:2/888: write d7/d12/d25/d5d/d47/f7f [779552,94052] 0 2024-04-12T05:56:21.738 INFO:tasks.workunit.client.1.smithi194.stdout:8/687: symlink d2/d8/d9/de/d10/d4e/ld3 0 2024-04-12T05:56:21.741 INFO:tasks.workunit.client.1.smithi194.stdout:9/935: dwrite d1/d4e/dea/dfc/d12f/d20/d37/d38/f5c [0,4194304] 0 2024-04-12T05:56:21.746 INFO:tasks.workunit.client.1.smithi194.stdout:1/971: rename d0/d11/d18/d19/d1a/d6a/d27/da8/c10f to d0/d11/d18/d19/d1a/d6a/dac/db5/df8/c138 0 2024-04-12T05:56:21.751 INFO:tasks.workunit.client.1.smithi194.stdout:9/936: dread d1/d41/f6f [0,4194304] 0 2024-04-12T05:56:21.751 INFO:tasks.workunit.client.1.smithi194.stdout:1/972: mknod d0/d11/d18/d19/d1a/d6a/d27/d48/da7/c139 0 2024-04-12T05:56:21.754 INFO:tasks.workunit.client.1.smithi194.stdout:2/889: write d7/d12/d25/f4f [1507651,126890] 0 2024-04-12T05:56:21.755 INFO:tasks.workunit.client.1.smithi194.stdout:9/937: mknod d1/d4e/dea/dfc/c13c 0 2024-04-12T05:56:21.757 INFO:tasks.workunit.client.1.smithi194.stdout:6/945: link d3/d2d/d33/d107/l118 d3/d1a/l12d 0 2024-04-12T05:56:21.757 INFO:tasks.workunit.client.1.smithi194.stdout:1/973: dread d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/f61 [0,4194304] 0 2024-04-12T05:56:21.759 INFO:tasks.workunit.client.1.smithi194.stdout:6/946: getdents d3/d2d/d33/d121 0 2024-04-12T05:56:21.761 INFO:tasks.workunit.client.1.smithi194.stdout:6/947: symlink d3/d2d/d33/l12e 0 2024-04-12T05:56:21.764 INFO:tasks.workunit.client.1.smithi194.stdout:6/948: creat d3/d2f/d4e/d9a/dd8/d77/f12f x:0 0 0 2024-04-12T05:56:21.766 INFO:tasks.workunit.client.1.smithi194.stdout:2/890: write d7/d22/d42/f2a [881616,24053] 0 2024-04-12T05:56:21.766 INFO:tasks.workunit.client.1.smithi194.stdout:6/949: getdents d3/d2d/d33/d121 0 2024-04-12T05:56:21.767 INFO:tasks.workunit.client.1.smithi194.stdout:6/950: stat d3/d15/d2b/d48/f129 0 2024-04-12T05:56:21.767 INFO:tasks.workunit.client.1.smithi194.stdout:2/891: dread - d7/d12/d1a/d69/ffe zero size 2024-04-12T05:56:21.769 INFO:tasks.workunit.client.1.smithi194.stdout:2/892: chown d7/d12/d1a/de9/l108 84415730 1 2024-04-12T05:56:21.772 INFO:tasks.workunit.client.1.smithi194.stdout:2/893: unlink d7/d12/d6d/cfd 0 2024-04-12T05:56:21.774 INFO:tasks.workunit.client.1.smithi194.stdout:6/951: truncate d3/d15/d2b/fc5 1153607 0 2024-04-12T05:56:21.774 INFO:tasks.workunit.client.1.smithi194.stdout:6/952: stat d3/d15/d2b/d48/lac 0 2024-04-12T05:56:21.775 INFO:tasks.workunit.client.1.smithi194.stdout:2/894: rename d7/d12/d25/d5d/d47/d52/d7b/f11d to d7/d12/d25/f129 0 2024-04-12T05:56:21.778 INFO:tasks.workunit.client.1.smithi194.stdout:2/895: creat d7/d12/d25/d5d/d47/d52/d7b/db2/f12a x:0 0 0 2024-04-12T05:56:21.779 INFO:tasks.workunit.client.1.smithi194.stdout:6/953: rename d3/d15/d2b/d48/d6f/d7a/d8f/d98/lc9 to d3/d15/d2b/d48/d6f/d7a/d8f/l130 0 2024-04-12T05:56:21.780 INFO:tasks.workunit.client.1.smithi194.stdout:6/954: chown d3/d2f/dd9 5987839 1 2024-04-12T05:56:21.780 INFO:tasks.workunit.client.1.smithi194.stdout:2/896: symlink d7/d12/d1a/d68/l12b 0 2024-04-12T05:56:21.782 INFO:tasks.workunit.client.1.smithi194.stdout:6/955: mkdir d3/d2d/d33/d52/d131 0 2024-04-12T05:56:21.783 INFO:tasks.workunit.client.1.smithi194.stdout:2/897: fdatasync d7/d12/d25/d5d/d47/d52/d7b/db2/ffa 0 2024-04-12T05:56:21.785 INFO:tasks.workunit.client.1.smithi194.stdout:6/956: creat d3/d15/d2b/d48/d6f/d7a/d8f/d128/f132 x:0 0 0 2024-04-12T05:56:21.786 INFO:tasks.workunit.client.1.smithi194.stdout:2/898: symlink d7/d12/d25/d5d/d47/d52/l12c 0 2024-04-12T05:56:21.793 INFO:tasks.workunit.client.1.smithi194.stdout:2/899: dwrite f5 [0,4194304] 0 2024-04-12T05:56:21.794 INFO:tasks.workunit.client.1.smithi194.stdout:2/900: chown d7/d22/d93/d9f/fd5 104 1 2024-04-12T05:56:21.796 INFO:tasks.workunit.client.1.smithi194.stdout:6/957: write d3/d15/d2b/d48/d6f/d7a/d8f/fc0 [143846,56569] 0 2024-04-12T05:56:21.796 INFO:tasks.workunit.client.1.smithi194.stdout:2/901: fsync d7/d12/d25/d5d/d47/f7f 0 2024-04-12T05:56:21.850 INFO:tasks.workunit.client.1.smithi194.stdout:6/958: unlink d3/d2d/d33/d52/d62/d10a/de2/ce7 0 2024-04-12T05:56:21.850 INFO:tasks.workunit.client.1.smithi194.stdout:7/918: sync 2024-04-12T05:56:21.851 INFO:tasks.workunit.client.1.smithi194.stdout:2/902: symlink d7/d22/d93/l12d 0 2024-04-12T05:56:21.853 INFO:tasks.workunit.client.1.smithi194.stdout:7/919: readlink d3/d16/d23/d57/d72/d81/d3a/d43/l6b 0 2024-04-12T05:56:21.854 INFO:tasks.workunit.client.1.smithi194.stdout:6/959: creat d3/d1a/f133 x:0 0 0 2024-04-12T05:56:21.855 INFO:tasks.workunit.client.1.smithi194.stdout:6/960: rmdir d3/d34 39 2024-04-12T05:56:21.856 INFO:tasks.workunit.client.1.smithi194.stdout:7/920: symlink d3/d16/d23/d57/d72/d81/d3a/d43/l12f 0 2024-04-12T05:56:21.858 INFO:tasks.workunit.client.1.smithi194.stdout:2/903: link d7/d12/d25/d5d/d47/dac/db3/dd7/f97 d7/d22/dc9/de5/f12e 0 2024-04-12T05:56:21.859 INFO:tasks.workunit.client.1.smithi194.stdout:7/921: unlink d3/d27/d6a/dda/l4b 0 2024-04-12T05:56:21.859 INFO:tasks.workunit.client.1.smithi194.stdout:2/904: write d7/d22/d93/d9a/fc8 [1767737,123835] 0 2024-04-12T05:56:21.864 INFO:tasks.workunit.client.1.smithi194.stdout:2/905: link d7/f48 d7/d12/d6d/f12f 0 2024-04-12T05:56:21.864 INFO:tasks.workunit.client.1.smithi194.stdout:7/922: dread d3/d16/d23/d57/d72/d81/d3a/d43/fb8 [0,4194304] 0 2024-04-12T05:56:21.866 INFO:tasks.workunit.client.1.smithi194.stdout:7/923: write d3/d16/d23/f83 [4668908,91149] 0 2024-04-12T05:56:21.867 INFO:tasks.workunit.client.1.smithi194.stdout:7/924: chown d3/d16/d23/d57/f112 23162783 1 2024-04-12T05:56:21.876 INFO:tasks.workunit.client.1.smithi194.stdout:2/906: dwrite d7/fa3 [0,4194304] 0 2024-04-12T05:56:21.913 INFO:tasks.workunit.client.1.smithi194.stdout:7/925: write d3/d16/d23/d40/d9b/fa5 [787342,114804] 0 2024-04-12T05:56:21.964 INFO:tasks.workunit.client.1.smithi194.stdout:7/926: dwrite d3/d27/f46 [0,4194304] 0 2024-04-12T05:56:21.969 INFO:tasks.workunit.client.1.smithi194.stdout:7/927: rename d3/d16/d23/d57/d72/d81/d3a/d43/l12f to d3/def/d12e/l130 0 2024-04-12T05:56:21.977 INFO:tasks.workunit.client.1.smithi194.stdout:7/928: unlink d3/d16/d23/d57/d72/d81/d3a/dcc/f107 0 2024-04-12T05:56:21.984 INFO:tasks.workunit.client.1.smithi194.stdout:7/929: rename d3/d16/d5d/f121 to d3/def/d12e/f131 0 2024-04-12T05:56:21.985 INFO:tasks.workunit.client.1.smithi194.stdout:7/930: write d3/d16/d23/f110 [616204,49602] 0 2024-04-12T05:56:21.986 INFO:tasks.workunit.client.1.smithi194.stdout:7/931: readlink d3/d27/d6a/dda/d74/lac 0 2024-04-12T05:56:21.987 INFO:tasks.workunit.client.1.smithi194.stdout:7/932: write d3/d27/d22/f2e [1328898,80165] 0 2024-04-12T05:56:22.014 INFO:tasks.workunit.client.1.smithi194.stdout:7/933: write d3/d16/d23/d57/d72/fc5 [569052,78655] 0 2024-04-12T05:56:22.039 INFO:tasks.workunit.client.1.smithi194.stdout:7/934: truncate d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f76 1887823 0 2024-04-12T05:56:22.046 INFO:tasks.workunit.client.1.smithi194.stdout:0/925: sync 2024-04-12T05:56:22.048 INFO:tasks.workunit.client.1.smithi194.stdout:0/926: mknod d0/de3/d5b/c12e 0 2024-04-12T05:56:22.050 INFO:tasks.workunit.client.1.smithi194.stdout:0/927: mknod d0/da/d4a/c12f 0 2024-04-12T05:56:22.051 INFO:tasks.workunit.client.1.smithi194.stdout:0/928: read d0/f30 [1441269,85734] 0 2024-04-12T05:56:22.052 INFO:tasks.workunit.client.1.smithi194.stdout:7/935: dread d3/d16/d23/d37/d3d/d56/f87 [0,4194304] 0 2024-04-12T05:56:22.062 INFO:tasks.workunit.client.1.smithi194.stdout:7/936: getdents d3/d27/d6a/dda 0 2024-04-12T05:56:22.064 INFO:tasks.workunit.client.1.smithi194.stdout:0/929: dread d0/da/f3e [0,4194304] 0 2024-04-12T05:56:22.080 INFO:tasks.workunit.client.1.smithi194.stdout:0/930: dread d0/d3b/d52/dfa/d1a/d24/d4b/f93 [0,4194304] 0 2024-04-12T05:56:22.080 INFO:tasks.workunit.client.1.smithi194.stdout:0/931: chown d0 1279962035 1 2024-04-12T05:56:22.081 INFO:tasks.workunit.client.1.smithi194.stdout:0/932: dread - d0/da/da7/fba zero size 2024-04-12T05:56:22.083 INFO:tasks.workunit.client.1.smithi194.stdout:0/933: write d0/d3b/d52/dfa/d1a/d3d/f106 [186476,24978] 0 2024-04-12T05:56:22.085 INFO:tasks.workunit.client.1.smithi194.stdout:0/934: truncate d0/f14 3722407 0 2024-04-12T05:56:22.093 INFO:tasks.workunit.client.1.smithi194.stdout:9/938: sync 2024-04-12T05:56:22.093 INFO:tasks.workunit.client.1.smithi194.stdout:8/688: sync 2024-04-12T05:56:22.093 INFO:tasks.workunit.client.1.smithi194.stdout:1/974: sync 2024-04-12T05:56:22.093 INFO:tasks.workunit.client.1.smithi194.stdout:6/961: sync 2024-04-12T05:56:22.093 INFO:tasks.workunit.client.1.smithi194.stdout:2/907: sync 2024-04-12T05:56:22.096 INFO:tasks.workunit.client.1.smithi194.stdout:0/935: dwrite d0/da/d4a/d8e/f68 [0,4194304] 0 2024-04-12T05:56:22.097 INFO:tasks.workunit.client.1.smithi194.stdout:2/908: fsync d7/d12/d25/d5d/d47/d52/f55 0 2024-04-12T05:56:22.099 INFO:tasks.workunit.client.1.smithi194.stdout:9/939: rmdir d1/d4e/dea/dfc/d12f/d20/d37 39 2024-04-12T05:56:22.103 INFO:tasks.workunit.client.1.smithi194.stdout:6/962: unlink d3/d2d/d33/d52/d62/d10a/de2/f111 0 2024-04-12T05:56:22.105 INFO:tasks.workunit.client.1.smithi194.stdout:7/937: truncate d3/d16/d23/d57/d72/d81/d3a/d43/d5c/fc4 98713 0 2024-04-12T05:56:22.111 INFO:tasks.workunit.client.1.smithi194.stdout:8/689: dwrite d2/d8/d15/f16 [0,4194304] 0 2024-04-12T05:56:22.112 INFO:tasks.workunit.client.1.smithi194.stdout:7/938: mknod d3/d16/d23/c132 0 2024-04-12T05:56:22.112 INFO:tasks.workunit.client.1.smithi194.stdout:0/936: write d0/da/f39 [1218964,119554] 0 2024-04-12T05:56:22.113 INFO:tasks.workunit.client.1.smithi194.stdout:9/940: creat d1/f13d x:0 0 0 2024-04-12T05:56:22.113 INFO:tasks.workunit.client.1.smithi194.stdout:8/690: write d2/d8/d9/de/d10/d4e/f8a [406597,95947] 0 2024-04-12T05:56:22.118 INFO:tasks.workunit.client.1.smithi194.stdout:9/941: read d1/d4e/dea/dfc/d12f/f4b [553810,123987] 0 2024-04-12T05:56:22.118 INFO:tasks.workunit.client.1.smithi194.stdout:2/909: link d7/d12/cf2 d7/d22/d93/d9a/d122/c130 0 2024-04-12T05:56:22.119 INFO:tasks.workunit.client.1.smithi194.stdout:6/963: write d3/d2f/d4e/d63/d83/fc7 [10192,110624] 0 2024-04-12T05:56:22.120 INFO:tasks.workunit.client.1.smithi194.stdout:0/937: symlink d0/d3b/l130 0 2024-04-12T05:56:22.125 INFO:tasks.workunit.client.1.smithi194.stdout:2/910: dread - d7/d12/d25/d5d/d47/d52/d7b/db2/d111/f11a zero size 2024-04-12T05:56:22.126 INFO:tasks.workunit.client.1.smithi194.stdout:6/964: mkdir d3/d34/d134 0 2024-04-12T05:56:22.128 INFO:tasks.workunit.client.1.smithi194.stdout:0/938: read d0/d3b/d52/dfa/d1a/d24/da8/fe8 [285140,103841] 0 2024-04-12T05:56:22.129 INFO:tasks.workunit.client.1.smithi194.stdout:1/975: dwrite d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/f128 [0,4194304] 0 2024-04-12T05:56:22.129 INFO:tasks.workunit.client.1.smithi194.stdout:7/939: rmdir d3/d16 39 2024-04-12T05:56:22.131 INFO:tasks.workunit.client.1.smithi194.stdout:8/691: write d2/d8/d15/d33/f27 [2974836,63028] 0 2024-04-12T05:56:22.133 INFO:tasks.workunit.client.1.smithi194.stdout:8/692: read d2/f56 [168252,88001] 0 2024-04-12T05:56:22.135 INFO:tasks.workunit.client.1.smithi194.stdout:9/942: link d1/d21/d77/c81 d1/d21/d39/c13e 0 2024-04-12T05:56:22.138 INFO:tasks.workunit.client.1.smithi194.stdout:2/911: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f131 x:0 0 0 2024-04-12T05:56:22.139 INFO:tasks.workunit.client.1.smithi194.stdout:9/943: write d1/d21/d49/dcb/f10b [872253,82935] 0 2024-04-12T05:56:22.140 INFO:tasks.workunit.client.1.smithi194.stdout:6/965: symlink d3/d2d/d33/d52/d5c/l135 0 2024-04-12T05:56:22.140 INFO:tasks.workunit.client.1.smithi194.stdout:8/693: write d2/d8/d9/de/d38/d8f/fc6 [827297,72258] 0 2024-04-12T05:56:22.141 INFO:tasks.workunit.client.1.smithi194.stdout:1/976: creat d0/f13a x:0 0 0 2024-04-12T05:56:22.148 INFO:tasks.workunit.client.1.smithi194.stdout:2/912: dread - d7/d22/d93/d9f/fd1 zero size 2024-04-12T05:56:22.148 INFO:tasks.workunit.client.1.smithi194.stdout:0/939: getdents d0/d3b/d52/dfa/d1a 0 2024-04-12T05:56:22.148 INFO:tasks.workunit.client.1.smithi194.stdout:7/940: rename d3/d16/d7b/dc3/c113 to d3/d16/d23/d57/d72/d81/d3a/d43/d5c/c133 0 2024-04-12T05:56:22.150 INFO:tasks.workunit.client.1.smithi194.stdout:6/966: mkdir d3/d2f/d4e/d9a/dd8/d77/d136 0 2024-04-12T05:56:22.150 INFO:tasks.workunit.client.1.smithi194.stdout:7/941: fdatasync d3/d27/f2f 0 2024-04-12T05:56:22.151 INFO:tasks.workunit.client.1.smithi194.stdout:9/944: symlink d1/d21/d77/d98/dab/d128/l13f 0 2024-04-12T05:56:22.153 INFO:tasks.workunit.client.1.smithi194.stdout:1/977: dread d0/d11/d18/d35/da5/f97 [0,4194304] 0 2024-04-12T05:56:22.153 INFO:tasks.workunit.client.1.smithi194.stdout:8/694: rename d2/d8/d9/de/f1d to d2/d8/d9/de/d10/d7d/fd4 0 2024-04-12T05:56:22.155 INFO:tasks.workunit.client.1.smithi194.stdout:7/942: dread - d3/d16/d23/d37/d3d/d56/dd0/fea zero size 2024-04-12T05:56:22.155 INFO:tasks.workunit.client.1.smithi194.stdout:9/945: mkdir d1/d41/d140 0 2024-04-12T05:56:22.156 INFO:tasks.workunit.client.1.smithi194.stdout:6/967: creat d3/d2d/d33/d52/d5c/d8c/f137 x:0 0 0 2024-04-12T05:56:22.156 INFO:tasks.workunit.client.1.smithi194.stdout:8/695: chown d2/d8/d66/d7f 119981 1 2024-04-12T05:56:22.156 INFO:tasks.workunit.client.1.smithi194.stdout:0/940: write d0/de3/d5b/fce [1290535,107174] 0 2024-04-12T05:56:22.157 INFO:tasks.workunit.client.1.smithi194.stdout:7/943: dread - d3/d16/f101 zero size 2024-04-12T05:56:22.161 INFO:tasks.workunit.client.1.smithi194.stdout:6/968: unlink d3/d15/f9f 0 2024-04-12T05:56:22.161 INFO:tasks.workunit.client.1.smithi194.stdout:1/978: write d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/ff1 [68376,105689] 0 2024-04-12T05:56:22.162 INFO:tasks.workunit.client.1.smithi194.stdout:8/696: readlink d2/d8/d9/l9b 0 2024-04-12T05:56:22.164 INFO:tasks.workunit.client.1.smithi194.stdout:1/979: read d0/d11/d18/d19/d1a/d6a/d27/da8/fa9 [565425,71837] 0 2024-04-12T05:56:22.165 INFO:tasks.workunit.client.1.smithi194.stdout:7/944: creat d3/f134 x:0 0 0 2024-04-12T05:56:22.165 INFO:tasks.workunit.client.1.smithi194.stdout:8/697: chown d2/d8/d9/de/d10/d7d/d84/d9f/fc7 132 1 2024-04-12T05:56:22.167 INFO:tasks.workunit.client.1.smithi194.stdout:6/969: mknod d3/d15/d2b/d48/d6f/c138 0 2024-04-12T05:56:22.169 INFO:tasks.workunit.client.1.smithi194.stdout:2/913: dread d7/d12/d25/d5d/f2f [0,4194304] 0 2024-04-12T05:56:22.171 INFO:tasks.workunit.client.1.smithi194.stdout:8/698: symlink d2/d8/d15/d33/dab/ld5 0 2024-04-12T05:56:22.171 INFO:tasks.workunit.client.1.smithi194.stdout:9/946: link d1/d19/f2e d1/d4e/dea/dfc/d12f/d20/f141 0 2024-04-12T05:56:22.172 INFO:tasks.workunit.client.1.smithi194.stdout:0/941: link d0/d3b/d52/dfa/d8f/ccd d0/d3b/d52/dfa/d1a/d24/c131 0 2024-04-12T05:56:22.174 INFO:tasks.workunit.client.1.smithi194.stdout:0/942: readlink d0/da/d4a/l122 0 2024-04-12T05:56:22.174 INFO:tasks.workunit.client.1.smithi194.stdout:6/970: creat d3/d2f/d4e/d9a/dd8/d77/d136/f139 x:0 0 0 2024-04-12T05:56:22.174 INFO:tasks.workunit.client.1.smithi194.stdout:2/914: chown d7/d12/d25/d5d/d47/d52/d7b/fb8 815374 1 2024-04-12T05:56:22.175 INFO:tasks.workunit.client.1.smithi194.stdout:8/699: stat d2/d8/f82 0 2024-04-12T05:56:22.175 INFO:tasks.workunit.client.1.smithi194.stdout:0/943: dread - d0/fdc zero size 2024-04-12T05:56:22.175 INFO:tasks.workunit.client.1.smithi194.stdout:7/945: rmdir d3/d27/d6a/dda/d74 39 2024-04-12T05:56:22.180 INFO:tasks.workunit.client.1.smithi194.stdout:7/946: symlink d3/d16/d23/d40/d114/l135 0 2024-04-12T05:56:22.180 INFO:tasks.workunit.client.1.smithi194.stdout:8/700: symlink d2/d8/d9/de/d86/dbe/ld6 0 2024-04-12T05:56:22.183 INFO:tasks.workunit.client.1.smithi194.stdout:1/980: dread d0/d11/d18/d19/f36 [0,4194304] 0 2024-04-12T05:56:22.184 INFO:tasks.workunit.client.1.smithi194.stdout:2/915: mknod d7/d12/d25/d5d/d47/dac/db3/dd7/df8/c132 0 2024-04-12T05:56:22.185 INFO:tasks.workunit.client.1.smithi194.stdout:7/947: chown d3/d16/d23/d57/c63 105 1 2024-04-12T05:56:22.186 INFO:tasks.workunit.client.1.smithi194.stdout:0/944: write d0/da/d4a/d8e/f11e [4783630,87371] 0 2024-04-12T05:56:22.186 INFO:tasks.workunit.client.1.smithi194.stdout:1/981: stat d0/d11/ddd/de6/l120 0 2024-04-12T05:56:22.189 INFO:tasks.workunit.client.1.smithi194.stdout:2/916: truncate d7/d12/fe3 1007096 0 2024-04-12T05:56:22.189 INFO:tasks.workunit.client.1.smithi194.stdout:7/948: readlink d3/d16/d23/d37/d3d/d56/dd0/lc6 0 2024-04-12T05:56:22.191 INFO:tasks.workunit.client.1.smithi194.stdout:0/945: read - d0/de3/df0/dbe/f11b zero size 2024-04-12T05:56:22.193 INFO:tasks.workunit.client.1.smithi194.stdout:2/917: symlink d7/d12/d1a/d68/l133 0 2024-04-12T05:56:22.193 INFO:tasks.workunit.client.1.smithi194.stdout:7/949: stat d3/d16/d23/d37/d95/lb2 0 2024-04-12T05:56:22.193 INFO:tasks.workunit.client.1.smithi194.stdout:8/701: dwrite d2/d8/d9/de/d10/d4e/f8a [0,4194304] 0 2024-04-12T05:56:22.199 INFO:tasks.workunit.client.1.smithi194.stdout:0/946: unlink d0/d3b/d52/fb1 0 2024-04-12T05:56:22.202 INFO:tasks.workunit.client.1.smithi194.stdout:8/702: mkdir d2/d8/d15/d33/d45/dd7 0 2024-04-12T05:56:22.204 INFO:tasks.workunit.client.1.smithi194.stdout:1/982: read d0/d11/d18/d19/d1a/d6a/d27/d2e/d42/fdb [164819,113552] 0 2024-04-12T05:56:22.204 INFO:tasks.workunit.client.1.smithi194.stdout:0/947: fsync d0/d5f/d73/d76/fec 0 2024-04-12T05:56:22.207 INFO:tasks.workunit.client.1.smithi194.stdout:7/950: creat d3/d16/d23/d40/d9b/d128/f136 x:0 0 0 2024-04-12T05:56:22.208 INFO:tasks.workunit.client.1.smithi194.stdout:2/918: dwrite d7/fa3 [0,4194304] 0 2024-04-12T05:56:22.211 INFO:tasks.workunit.client.1.smithi194.stdout:9/947: dwrite d1/d4e/dea/dfc/d12f/d20/f141 [0,4194304] 0 2024-04-12T05:56:22.211 INFO:tasks.workunit.client.1.smithi194.stdout:1/983: chown d0/d11/d18/d19/d1a/d6a/d27/d48/f62 0 1 2024-04-12T05:56:22.212 INFO:tasks.workunit.client.1.smithi194.stdout:8/703: creat d2/d8/d9/de/d10/d4e/fd8 x:0 0 0 2024-04-12T05:56:22.215 INFO:tasks.workunit.client.1.smithi194.stdout:8/704: chown d2/d8/d15/d33/d45/d52/f7b 625 1 2024-04-12T05:56:22.217 INFO:tasks.workunit.client.1.smithi194.stdout:9/948: dread d1/fb [0,4194304] 0 2024-04-12T05:56:22.217 INFO:tasks.workunit.client.1.smithi194.stdout:1/984: truncate d0/d11/d18/d35/da5/f72 250928 0 2024-04-12T05:56:22.218 INFO:tasks.workunit.client.1.smithi194.stdout:2/919: getdents d7/d12/d1a/d69/d7e 0 2024-04-12T05:56:22.218 INFO:tasks.workunit.client.1.smithi194.stdout:0/948: dread d0/d3b/d52/dfa/d27/d87/d92/faa [0,4194304] 0 2024-04-12T05:56:22.220 INFO:tasks.workunit.client.1.smithi194.stdout:8/705: mkdir d2/d8/d15/d33/d49/db9/dd9 0 2024-04-12T05:56:22.226 INFO:tasks.workunit.client.1.smithi194.stdout:7/951: rename d3/d16/d23/d57/fab to d3/d27/d22/d62/d12c/f137 0 2024-04-12T05:56:22.227 INFO:tasks.workunit.client.1.smithi194.stdout:8/706: mknod d2/d8/daa/cda 0 2024-04-12T05:56:22.228 INFO:tasks.workunit.client.1.smithi194.stdout:0/949: truncate d0/d3b/d52/dfa/d8f/f103 1169454 0 2024-04-12T05:56:22.238 INFO:tasks.workunit.client.1.smithi194.stdout:7/952: dwrite d3/d16/d23/d37/d3d/d56/dd0/fd7 [0,4194304] 0 2024-04-12T05:56:22.241 INFO:tasks.workunit.client.1.smithi194.stdout:0/950: rmdir d0/de3/df0/dbe 39 2024-04-12T05:56:22.242 INFO:tasks.workunit.client.1.smithi194.stdout:7/953: mknod d3/d27/d22/d31/d103/c138 0 2024-04-12T05:56:22.242 INFO:tasks.workunit.client.1.smithi194.stdout:1/985: link d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/c126 d0/d11/d18/d19/d1c/c13b 0 2024-04-12T05:56:22.243 INFO:tasks.workunit.client.1.smithi194.stdout:7/954: fdatasync d3/d27/d22/d8e/fb0 0 2024-04-12T05:56:22.244 INFO:tasks.workunit.client.1.smithi194.stdout:7/955: stat d3/d16/d23/d37 0 2024-04-12T05:56:22.245 INFO:tasks.workunit.client.1.smithi194.stdout:9/949: rename d1/d21/d49/c68 to d1/d41/d97/d9d/c142 0 2024-04-12T05:56:22.247 INFO:tasks.workunit.client.1.smithi194.stdout:0/951: write d0/de3/df0/f127 [489719,60686] 0 2024-04-12T05:56:22.247 INFO:tasks.workunit.client.1.smithi194.stdout:7/956: read d3/d16/f45 [372398,5993] 0 2024-04-12T05:56:22.247 INFO:tasks.workunit.client.1.smithi194.stdout:9/950: chown d1/l74 61271956 1 2024-04-12T05:56:22.247 INFO:tasks.workunit.client.1.smithi194.stdout:1/986: mkdir d0/d11/d18/d19/d1a/d6a/d27/da8/d13c 0 2024-04-12T05:56:22.252 INFO:tasks.workunit.client.1.smithi194.stdout:1/987: creat d0/d11/d18/d35/f13d x:0 0 0 2024-04-12T05:56:22.257 INFO:tasks.workunit.client.1.smithi194.stdout:8/707: rmdir d2/d8/daa 39 2024-04-12T05:56:22.257 INFO:tasks.workunit.client.1.smithi194.stdout:7/957: dwrite d3/d16/d23/d37/d3d/d56/dd0/fc9 [4194304,4194304] 0 2024-04-12T05:56:22.260 INFO:tasks.workunit.client.1.smithi194.stdout:1/988: mkdir d0/d11/d18/d19/d1a/ded/d13e 0 2024-04-12T05:56:22.260 INFO:tasks.workunit.client.1.smithi194.stdout:2/920: truncate d7/d12/d25/f4f 848897 0 2024-04-12T05:56:22.263 INFO:tasks.workunit.client.1.smithi194.stdout:0/952: truncate d0/da/f89 398029 0 2024-04-12T05:56:22.263 INFO:tasks.workunit.client.1.smithi194.stdout:2/921: fsync d7/d22/d93/d9a/fc8 0 2024-04-12T05:56:22.264 INFO:tasks.workunit.client.1.smithi194.stdout:7/958: readlink d3/d27/d6a/dda/d74/l120 0 2024-04-12T05:56:22.266 INFO:tasks.workunit.client.1.smithi194.stdout:1/989: mknod d0/d11/d18/d19/d1a/ded/c13f 0 2024-04-12T05:56:22.267 INFO:tasks.workunit.client.1.smithi194.stdout:7/959: read d3/d27/d22/d62/f6c [651693,18864] 0 2024-04-12T05:56:22.269 INFO:tasks.workunit.client.1.smithi194.stdout:1/990: stat d0/d11/d18/d19/d1a/d6a/d27/l94 0 2024-04-12T05:56:22.269 INFO:tasks.workunit.client.1.smithi194.stdout:0/953: creat d0/d3b/d9c/f132 x:0 0 0 2024-04-12T05:56:22.272 INFO:tasks.workunit.client.1.smithi194.stdout:9/951: getdents d1/d4e/dea/dfc/d12f/d20/d37 0 2024-04-12T05:56:22.272 INFO:tasks.workunit.client.1.smithi194.stdout:7/960: dread d3/d16/d23/d37/d95/fd1 [0,4194304] 0 2024-04-12T05:56:22.272 INFO:tasks.workunit.client.1.smithi194.stdout:2/922: mkdir d7/d22/d93/d134 0 2024-04-12T05:56:22.274 INFO:tasks.workunit.client.1.smithi194.stdout:8/708: creat d2/d8/d9/de/d10/d4e/fdb x:0 0 0 2024-04-12T05:56:22.277 INFO:tasks.workunit.client.1.smithi194.stdout:7/961: mknod d3/d27/d22/d62/c139 0 2024-04-12T05:56:22.277 INFO:tasks.workunit.client.1.smithi194.stdout:1/991: dread d0/f85 [4194304,4194304] 0 2024-04-12T05:56:22.278 INFO:tasks.workunit.client.1.smithi194.stdout:2/923: chown d7/d12/d1a/d69/df0/d101/df9/c115 108359 1 2024-04-12T05:56:22.281 INFO:tasks.workunit.client.1.smithi194.stdout:7/962: read d3/d16/d23/d37/f7c [2531745,121073] 0 2024-04-12T05:56:22.282 INFO:tasks.workunit.client.1.smithi194.stdout:1/992: read d0/d11/d18/d19/d1a/d6a/d27/fab [1768013,99720] 0 2024-04-12T05:56:22.285 INFO:tasks.workunit.client.1.smithi194.stdout:2/924: mkdir d7/d12/d25/d70/df1/d135 0 2024-04-12T05:56:22.285 INFO:tasks.workunit.client.1.smithi194.stdout:8/709: link d2/d8/d9/de/d38/f4a d2/d8/d15/d35/fdc 0 2024-04-12T05:56:22.288 INFO:tasks.workunit.client.1.smithi194.stdout:7/963: rmdir d3/d16/d5d 39 2024-04-12T05:56:22.289 INFO:tasks.workunit.client.1.smithi194.stdout:1/993: rmdir d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9 39 2024-04-12T05:56:22.294 INFO:tasks.workunit.client.1.smithi194.stdout:8/710: readlink d2/d8/d58/l75 0 2024-04-12T05:56:22.294 INFO:tasks.workunit.client.1.smithi194.stdout:0/954: dread d0/da/d4a/d8e/f107 [0,4194304] 0 2024-04-12T05:56:22.294 INFO:tasks.workunit.client.1.smithi194.stdout:7/964: mknod d3/d16/d23/d57/d72/d81/d3a/d8f/c13a 0 2024-04-12T05:56:22.295 INFO:tasks.workunit.client.1.smithi194.stdout:2/925: rename d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/faf to d7/d12/d25/d5d/d47/d52/d7b/db2/d111/f136 0 2024-04-12T05:56:22.297 INFO:tasks.workunit.client.1.smithi194.stdout:0/955: fdatasync d0/d3b/d9c/f132 0 2024-04-12T05:56:22.298 INFO:tasks.workunit.client.1.smithi194.stdout:9/952: dread d1/d4e/dea/dfc/d12f/f70 [0,4194304] 0 2024-04-12T05:56:22.299 INFO:tasks.workunit.client.1.smithi194.stdout:8/711: mkdir d2/d8/d15/d33/ddd 0 2024-04-12T05:56:22.300 INFO:tasks.workunit.client.1.smithi194.stdout:7/965: mkdir d3/d16/d23/d37/d13b 0 2024-04-12T05:56:22.300 INFO:tasks.workunit.client.1.smithi194.stdout:2/926: mkdir d7/d12/d1a/dbf/d137 0 2024-04-12T05:56:22.301 INFO:tasks.workunit.client.1.smithi194.stdout:2/927: stat d7/d12/d1a/dbf 0 2024-04-12T05:56:22.303 INFO:tasks.workunit.client.1.smithi194.stdout:1/994: rename d0/d11/d18/d19/d1a/d6a/d27/d6c/da0/c11c to d0/d11/d18/d19/d1a/d6a/d27/d48/da7/df3/c140 0 2024-04-12T05:56:22.303 INFO:tasks.workunit.client.1.smithi194.stdout:2/928: stat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/lcf 0 2024-04-12T05:56:22.305 INFO:tasks.workunit.client.1.smithi194.stdout:1/995: readlink d0/d11/d18/d19/d1a/l69 0 2024-04-12T05:56:22.314 INFO:tasks.workunit.client.1.smithi194.stdout:0/956: mkdir d0/d3b/d52/dfa/d1a/d24/d133 0 2024-04-12T05:56:22.315 INFO:tasks.workunit.client.1.smithi194.stdout:2/929: mknod d7/d12/d1a/d68/c138 0 2024-04-12T05:56:22.315 INFO:tasks.workunit.client.1.smithi194.stdout:7/966: dwrite d3/d16/d23/d37/d95/fd1 [0,4194304] 0 2024-04-12T05:56:22.317 INFO:tasks.workunit.client.1.smithi194.stdout:1/996: creat d0/d11/d18/f141 x:0 0 0 2024-04-12T05:56:22.321 INFO:tasks.workunit.client.1.smithi194.stdout:9/953: getdents d1/da6 0 2024-04-12T05:56:22.325 INFO:tasks.workunit.client.1.smithi194.stdout:8/712: dwrite d2/d8/f26 [0,4194304] 0 2024-04-12T05:56:22.327 INFO:tasks.workunit.client.1.smithi194.stdout:0/957: creat d0/d3b/d52/dfa/d1a/f134 x:0 0 0 2024-04-12T05:56:22.328 INFO:tasks.workunit.client.1.smithi194.stdout:1/997: symlink d0/l142 0 2024-04-12T05:56:22.335 INFO:tasks.workunit.client.1.smithi194.stdout:2/930: creat d7/d12/d25/d5d/d47/f139 x:0 0 0 2024-04-12T05:56:22.336 INFO:tasks.workunit.client.1.smithi194.stdout:8/713: symlink d2/d8/d15/d33/d45/dd7/lde 0 2024-04-12T05:56:22.336 INFO:tasks.workunit.client.1.smithi194.stdout:7/967: mkdir d3/d16/d23/d57/df6/d13c 0 2024-04-12T05:56:22.336 INFO:tasks.workunit.client.1.smithi194.stdout:0/958: mknod d0/da/da7/c135 0 2024-04-12T05:56:22.339 INFO:tasks.workunit.client.1.smithi194.stdout:7/968: read - d3/d16/d23/d40/d9b/d128/f136 zero size 2024-04-12T05:56:22.341 INFO:tasks.workunit.client.1.smithi194.stdout:8/714: chown d2/d8/d9/de/f2e 254505 1 2024-04-12T05:56:22.341 INFO:tasks.workunit.client.1.smithi194.stdout:7/969: dread - d3/d16/d23/d57/d72/d81/f122 zero size 2024-04-12T05:56:22.343 INFO:tasks.workunit.client.1.smithi194.stdout:1/998: dread d0/d11/d18/d19/d1a/d6a/d27/d5a/f73 [0,4194304] 0 2024-04-12T05:56:22.344 INFO:tasks.workunit.client.1.smithi194.stdout:9/954: rename d1/d4e/dea/dfc/d12f/d20/d37/l93 to d1/d4e/dea/dfc/d12f/d20/d37/d84/l143 0 2024-04-12T05:56:22.345 INFO:tasks.workunit.client.1.smithi194.stdout:2/931: readlink d7/d12/d25/d5d/d47/dac/ldc 0 2024-04-12T05:56:22.345 INFO:tasks.workunit.client.1.smithi194.stdout:8/715: fdatasync d2/d8/d9/de/d10/d7d/d84/d9f/fc7 0 2024-04-12T05:56:22.345 INFO:tasks.workunit.client.1.smithi194.stdout:0/959: symlink d0/d12a/l136 0 2024-04-12T05:56:22.353 INFO:tasks.workunit.client.1.smithi194.stdout:2/932: truncate d7/d12/d1a/d69/d7e/fe0 5209993 0 2024-04-12T05:56:22.354 INFO:tasks.workunit.client.1.smithi194.stdout:1/999: rmdir d0/d11/d18/d19/d1a/d6a/d27/d5a/de7/dc5/db9 39 2024-04-12T05:56:22.356 INFO:tasks.workunit.client.1.smithi194.stdout:9/955: read - d1/d4e/dea/dfc/d12f/d20/d37/f12a zero size 2024-04-12T05:56:22.358 INFO:tasks.workunit.client.1.smithi194.stdout:9/956: fdatasync d1/d4e/dea/dfc/d12f/d20/d37/d38/f113 0 2024-04-12T05:56:22.358 INFO:tasks.workunit.client.1.smithi194.stdout:0/960: write d0/d3b/d52/dfa/d1a/f134 [650559,26197] 0 2024-04-12T05:56:22.360 INFO:tasks.workunit.client.1.smithi194.stdout:2/933: getdents d7/d12/d1a/d69/d7e 0 2024-04-12T05:56:22.364 INFO:tasks.workunit.client.1.smithi194.stdout:9/957: chown d1/d21/d39/c13e 1698 1 2024-04-12T05:56:22.370 INFO:tasks.workunit.client.1.smithi194.stdout:2/934: creat d7/d12/d6d/f13a x:0 0 0 2024-04-12T05:56:22.371 INFO:tasks.workunit.client.1.smithi194.stdout:8/716: dwrite d2/d8/d9/d1a/f79 [0,4194304] 0 2024-04-12T05:56:22.376 INFO:tasks.workunit.client.1.smithi194.stdout:0/961: dread d0/da/d4a/d8e/f68 [0,4194304] 0 2024-04-12T05:56:22.378 INFO:tasks.workunit.client.1.smithi194.stdout:2/935: write d7/d12/d25/d5d/d47/f139 [850733,34321] 0 2024-04-12T05:56:22.380 INFO:tasks.workunit.client.1.smithi194.stdout:0/962: mkdir d0/da/d137 0 2024-04-12T05:56:22.380 INFO:tasks.workunit.client.1.smithi194.stdout:7/970: dread d3/d27/fa [0,4194304] 0 2024-04-12T05:56:22.384 INFO:tasks.workunit.client.1.smithi194.stdout:8/717: rename d2/d8/d9/de/d86 to d2/d8/d15/d35/d6c/ddf 0 2024-04-12T05:56:22.384 INFO:tasks.workunit.client.1.smithi194.stdout:9/958: truncate d1/d4e/dea/dfc/d12f/d20/f7b 802355 0 2024-04-12T05:56:22.385 INFO:tasks.workunit.client.1.smithi194.stdout:2/936: write d7/d12/d25/d5d/d47/d52/d7b/db2/f12a [228659,5337] 0 2024-04-12T05:56:22.386 INFO:tasks.workunit.client.1.smithi194.stdout:0/963: creat d0/d5f/d73/f138 x:0 0 0 2024-04-12T05:56:22.388 INFO:tasks.workunit.client.1.smithi194.stdout:9/959: truncate d1/da6/fa8 3608169 0 2024-04-12T05:56:22.388 INFO:tasks.workunit.client.1.smithi194.stdout:8/718: symlink d2/d8/d15/le0 0 2024-04-12T05:56:22.393 INFO:tasks.workunit.client.1.smithi194.stdout:2/937: creat d7/d12/d1a/d69/df0/f13b x:0 0 0 2024-04-12T05:56:22.395 INFO:tasks.workunit.client.1.smithi194.stdout:6/971: sync 2024-04-12T05:56:22.397 INFO:tasks.workunit.client.1.smithi194.stdout:2/938: write d7/d12/d1a/de9/fed [589363,96323] 0 2024-04-12T05:56:22.404 INFO:tasks.workunit.client.1.smithi194.stdout:8/719: dwrite d2/d8/d9/de/f2e [0,4194304] 0 2024-04-12T05:56:22.410 INFO:tasks.workunit.client.1.smithi194.stdout:2/939: symlink d7/d22/dc9/l13c 0 2024-04-12T05:56:22.412 INFO:tasks.workunit.client.1.smithi194.stdout:8/720: write d2/d8/d15/d33/d49/fcb [949715,47381] 0 2024-04-12T05:56:22.419 INFO:tasks.workunit.client.1.smithi194.stdout:7/971: dwrite d3/d16/d23/d40/d6f/f9a [0,4194304] 0 2024-04-12T05:56:22.424 INFO:tasks.workunit.client.1.smithi194.stdout:6/972: dwrite d3/d2d/d33/d52/fcc [0,4194304] 0 2024-04-12T05:56:22.428 INFO:tasks.workunit.client.1.smithi194.stdout:9/960: dread d1/d4e/dea/dfc/d12f/d5f/f91 [0,4194304] 0 2024-04-12T05:56:22.431 INFO:tasks.workunit.client.1.smithi194.stdout:0/964: dread d0/f53 [4194304,4194304] 0 2024-04-12T05:56:22.431 INFO:tasks.workunit.client.1.smithi194.stdout:2/940: creat d7/d12/d25/d5d/d47/dac/dc2/f13d x:0 0 0 2024-04-12T05:56:22.432 INFO:tasks.workunit.client.1.smithi194.stdout:2/941: readlink d7/l4e 0 2024-04-12T05:56:22.434 INFO:tasks.workunit.client.1.smithi194.stdout:6/973: fsync d3/d15/d2b/d48/d6f/d7a/d8f/d98/ff8 0 2024-04-12T05:56:22.435 INFO:tasks.workunit.client.1.smithi194.stdout:0/965: unlink d0/d3b/d52/dfa/d27/d87/c8a 0 2024-04-12T05:56:22.436 INFO:tasks.workunit.client.1.smithi194.stdout:9/961: stat d1/d4e/dea/dfc/d12f/d5f/f85 0 2024-04-12T05:56:22.437 INFO:tasks.workunit.client.1.smithi194.stdout:0/966: mkdir d0/d5f/df9/d139 0 2024-04-12T05:56:22.440 INFO:tasks.workunit.client.1.smithi194.stdout:9/962: write d1/d41/dbf/ff6 [1018977,12413] 0 2024-04-12T05:56:22.441 INFO:tasks.workunit.client.1.smithi194.stdout:2/942: mknod d7/d12/d25/d5d/d47/dac/dc2/c13e 0 2024-04-12T05:56:22.444 INFO:tasks.workunit.client.1.smithi194.stdout:0/967: mknod d0/d3b/d52/c13a 0 2024-04-12T05:56:22.444 INFO:tasks.workunit.client.1.smithi194.stdout:8/721: dwrite d2/d8/d9/de/d38/d8f/fcd [0,4194304] 0 2024-04-12T05:56:22.446 INFO:tasks.workunit.client.1.smithi194.stdout:0/968: chown d0/d3b/d52/dfa/d27/d87/d128 319 1 2024-04-12T05:56:22.447 INFO:tasks.workunit.client.1.smithi194.stdout:7/972: mkdir d3/d13d 0 2024-04-12T05:56:22.450 INFO:tasks.workunit.client.1.smithi194.stdout:7/973: write d3/d16/d23/d57/d72/d81/d3a/d43/d5c/f118 [771988,71450] 0 2024-04-12T05:56:22.456 INFO:tasks.workunit.client.1.smithi194.stdout:9/963: truncate d1/d4e/dea/dfc/d12f/d20/fed 529829 0 2024-04-12T05:56:22.459 INFO:tasks.workunit.client.1.smithi194.stdout:2/943: rename d7/d12/d1a/d69/c6f to d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d126/c13f 0 2024-04-12T05:56:22.462 INFO:tasks.workunit.client.1.smithi194.stdout:9/964: creat d1/d21/d49/dcb/f144 x:0 0 0 2024-04-12T05:56:22.463 INFO:tasks.workunit.client.1.smithi194.stdout:8/722: rename d2/d8/lad to d2/d8/d15/d35/d6c/ddf/dbe/le1 0 2024-04-12T05:56:22.464 INFO:tasks.workunit.client.1.smithi194.stdout:7/974: dwrite d3/d27/d6a/f11e [0,4194304] 0 2024-04-12T05:56:22.469 INFO:tasks.workunit.client.1.smithi194.stdout:8/723: chown d2/d8/d15/d35/d6c/ddf/l92 8 1 2024-04-12T05:56:22.470 INFO:tasks.workunit.client.1.smithi194.stdout:8/724: write d2/f43 [943415,48809] 0 2024-04-12T05:56:22.471 INFO:tasks.workunit.client.1.smithi194.stdout:7/975: dread d3/d16/d23/d37/f96 [0,4194304] 0 2024-04-12T05:56:22.473 INFO:tasks.workunit.client.1.smithi194.stdout:0/969: rename d0/d3b/d52/dfa/d1a/d24/lb9 to d0/d5f/df9/d139/l13b 0 2024-04-12T05:56:22.474 INFO:tasks.workunit.client.1.smithi194.stdout:2/944: link d7/c37 d7/d22/d93/d134/c140 0 2024-04-12T05:56:22.474 INFO:tasks.workunit.client.1.smithi194.stdout:7/976: chown d3/d16/d23/d40/d9b/laf 70973432 1 2024-04-12T05:56:22.476 INFO:tasks.workunit.client.1.smithi194.stdout:2/945: chown d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f105 1 1 2024-04-12T05:56:22.477 INFO:tasks.workunit.client.1.smithi194.stdout:2/946: fsync d7/d12/d1a/d69/f11c 0 2024-04-12T05:56:22.478 INFO:tasks.workunit.client.1.smithi194.stdout:6/974: truncate d3/d15/d2b/d48/d6f/d7a/d8f/fc0 19899 0 2024-04-12T05:56:22.479 INFO:tasks.workunit.client.1.smithi194.stdout:8/725: mkdir d2/d8/d15/d35/d6c/ddf/dbe/de2 0 2024-04-12T05:56:22.479 INFO:tasks.workunit.client.1.smithi194.stdout:7/977: mknod d3/d16/d7b/dc3/c13e 0 2024-04-12T05:56:22.480 INFO:tasks.workunit.client.1.smithi194.stdout:6/975: write d3/d15/d2b/d48/ffc [380152,44545] 0 2024-04-12T05:56:22.483 INFO:tasks.workunit.client.1.smithi194.stdout:2/947: rmdir d7/d22/d93/d134 39 2024-04-12T05:56:22.484 INFO:tasks.workunit.client.1.smithi194.stdout:0/970: write d0/de3/df0/dbe/f123 [2689252,16090] 0 2024-04-12T05:56:22.486 INFO:tasks.workunit.client.1.smithi194.stdout:7/978: write d3/d27/d22/d31/f10b [6834914,62915] 0 2024-04-12T05:56:22.487 INFO:tasks.workunit.client.1.smithi194.stdout:2/948: fdatasync d7/d22/d93/fbe 0 2024-04-12T05:56:22.490 INFO:tasks.workunit.client.1.smithi194.stdout:2/949: read - d7/d12/d1a/dbf/f10b zero size 2024-04-12T05:56:22.491 INFO:tasks.workunit.client.1.smithi194.stdout:6/976: rename d3/d2d/d33/d58 to d3/d2f/dd9/d13a 0 2024-04-12T05:56:22.493 INFO:tasks.workunit.client.1.smithi194.stdout:7/979: mkdir d3/d16/d23/d57/d72/d81/d3a/d8f/d13f 0 2024-04-12T05:56:22.493 INFO:tasks.workunit.client.1.smithi194.stdout:8/726: mkdir d2/d8/d9/de/d10/dcc/de3 0 2024-04-12T05:56:22.494 INFO:tasks.workunit.client.1.smithi194.stdout:0/971: dwrite d0/d3b/d52/dfa/d8f/f103 [0,4194304] 0 2024-04-12T05:56:22.494 INFO:tasks.workunit.client.1.smithi194.stdout:6/977: write d3/d70/fab [2473632,98347] 0 2024-04-12T05:56:22.496 INFO:tasks.workunit.client.1.smithi194.stdout:9/965: write d1/d4e/dea/dfc/d12f/d20/d37/da3/fb3 [3619186,85011] 0 2024-04-12T05:56:22.503 INFO:tasks.workunit.client.1.smithi194.stdout:8/727: creat d2/d8/dc5/fe4 x:0 0 0 2024-04-12T05:56:22.504 INFO:tasks.workunit.client.1.smithi194.stdout:7/980: dread d3/d16/d23/f110 [0,4194304] 0 2024-04-12T05:56:22.504 INFO:tasks.workunit.client.1.smithi194.stdout:2/950: symlink d7/d12/d25/d5d/d11e/d123/l141 0 2024-04-12T05:56:22.505 INFO:tasks.workunit.client.1.smithi194.stdout:0/972: symlink d0/d3b/d52/l13c 0 2024-04-12T05:56:22.506 INFO:tasks.workunit.client.1.smithi194.stdout:9/966: truncate d1/d4e/dea/dfc/d12f/d20/fb4 280502 0 2024-04-12T05:56:22.509 INFO:tasks.workunit.client.1.smithi194.stdout:8/728: chown d2/d8/d9/de/d38/d8f/d94/fbc 0 1 2024-04-12T05:56:22.509 INFO:tasks.workunit.client.1.smithi194.stdout:8/729: chown d2/d8/d58/c6b 2327 1 2024-04-12T05:56:22.509 INFO:tasks.workunit.client.1.smithi194.stdout:7/981: fdatasync d3/d16/d23/d37/f96 0 2024-04-12T05:56:22.509 INFO:tasks.workunit.client.1.smithi194.stdout:8/730: dread - d2/d8/d9/de/fa8 zero size 2024-04-12T05:56:22.512 INFO:tasks.workunit.client.1.smithi194.stdout:2/951: mkdir d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103/d142 0 2024-04-12T05:56:22.512 INFO:tasks.workunit.client.1.smithi194.stdout:7/982: mknod d3/d16/d23/d37/d3d/d56/dd0/dd9/c140 0 2024-04-12T05:56:22.515 INFO:tasks.workunit.client.1.smithi194.stdout:2/952: dread - d7/d12/d25/d5d/d47/dac/ff6 zero size 2024-04-12T05:56:22.516 INFO:tasks.workunit.client.1.smithi194.stdout:8/731: symlink d2/d8/d15/le5 0 2024-04-12T05:56:22.516 INFO:tasks.workunit.client.1.smithi194.stdout:7/983: chown d3/d16/d23/d57/d72/d81/d3a/d8f/ceb 1952 1 2024-04-12T05:56:22.518 INFO:tasks.workunit.client.1.smithi194.stdout:2/953: fsync d7/d12/d1a/d69/f11c 0 2024-04-12T05:56:22.519 INFO:tasks.workunit.client.1.smithi194.stdout:0/973: write d0/d3b/d52/dfa/d27/d54/f71 [1685456,24894] 0 2024-04-12T05:56:22.521 INFO:tasks.workunit.client.1.smithi194.stdout:8/732: fsync d2/d8/f82 0 2024-04-12T05:56:22.524 INFO:tasks.workunit.client.1.smithi194.stdout:7/984: creat d3/d27/d22/d62/d12c/f141 x:0 0 0 2024-04-12T05:56:22.524 INFO:tasks.workunit.client.1.smithi194.stdout:6/978: dwrite d3/d2f/d4e/d9a/dd8/f6a [0,4194304] 0 2024-04-12T05:56:22.525 INFO:tasks.workunit.client.1.smithi194.stdout:2/954: symlink d7/d12/d1a/dbf/l143 0 2024-04-12T05:56:22.530 INFO:tasks.workunit.client.1.smithi194.stdout:6/979: readlink d3/d2f/dd9/d13a/l9c 0 2024-04-12T05:56:22.530 INFO:tasks.workunit.client.1.smithi194.stdout:2/955: write d7/d12/d1a/d69/df0/d101/df9/fff [916250,49369] 0 2024-04-12T05:56:22.532 INFO:tasks.workunit.client.1.smithi194.stdout:6/980: mknod d3/d2f/d4e/d9a/dd8/d77/dde/c13b 0 2024-04-12T05:56:22.537 INFO:tasks.workunit.client.1.smithi194.stdout:8/733: dread d2/d8/d9/de/d10/f7c [0,4194304] 0 2024-04-12T05:56:22.537 INFO:tasks.workunit.client.1.smithi194.stdout:7/985: rename d3/d16/d23/d37/d95/c99 to d3/d16/d23/c142 0 2024-04-12T05:56:22.542 INFO:tasks.workunit.client.1.smithi194.stdout:8/734: symlink d2/d8/d9/de/d10/le6 0 2024-04-12T05:56:22.544 INFO:tasks.workunit.client.1.smithi194.stdout:6/981: rename d3/d15/c11 to d3/d15/dc8/c13c 0 2024-04-12T05:56:22.544 INFO:tasks.workunit.client.1.smithi194.stdout:7/986: getdents d3/def 0 2024-04-12T05:56:22.547 INFO:tasks.workunit.client.1.smithi194.stdout:6/982: truncate d3/d15/d2b/d48/ffc 1239831 0 2024-04-12T05:56:22.549 INFO:tasks.workunit.client.1.smithi194.stdout:0/974: dread d0/d3b/fe7 [0,4194304] 0 2024-04-12T05:56:22.551 INFO:tasks.workunit.client.1.smithi194.stdout:7/987: dread d3/d27/f46 [0,4194304] 0 2024-04-12T05:56:22.553 INFO:tasks.workunit.client.1.smithi194.stdout:6/983: link d3/d2f/dd9/ce0 d3/d2d/d33/c13d 0 2024-04-12T05:56:22.554 INFO:tasks.workunit.client.1.smithi194.stdout:2/956: dread d7/d12/d1a/d69/fb1 [0,4194304] 0 2024-04-12T05:56:22.557 INFO:tasks.workunit.client.1.smithi194.stdout:6/984: chown d3/d2d/d33/d52/d62/d10a/de2/lfb 7199 1 2024-04-12T05:56:22.557 INFO:tasks.workunit.client.1.smithi194.stdout:6/985: chown d3/d15/f24 58700 1 2024-04-12T05:56:22.558 INFO:tasks.workunit.client.1.smithi194.stdout:6/986: dread - d3/d2f/d4e/d9a/dd8/fa8 zero size 2024-04-12T05:56:22.559 INFO:tasks.workunit.client.1.smithi194.stdout:7/988: creat d3/d27/d6a/dda/f143 x:0 0 0 2024-04-12T05:56:22.559 INFO:tasks.workunit.client.1.smithi194.stdout:2/957: link d7/d12/d25/leb d7/d12/d25/d70/df1/l144 0 2024-04-12T05:56:22.561 INFO:tasks.workunit.client.1.smithi194.stdout:7/989: read d3/d27/fa [2970922,19937] 0 2024-04-12T05:56:22.562 INFO:tasks.workunit.client.1.smithi194.stdout:6/987: mknod d3/d2f/d4e/c13e 0 2024-04-12T05:56:22.563 INFO:tasks.workunit.client.1.smithi194.stdout:2/958: creat d7/d22/d93/d9f/f145 x:0 0 0 2024-04-12T05:56:22.566 INFO:tasks.workunit.client.1.smithi194.stdout:7/990: rmdir d3/d27/d22/d62 39 2024-04-12T05:56:22.567 INFO:tasks.workunit.client.1.smithi194.stdout:0/975: truncate d0/d3b/d52/dfa/d27/d54/fad 669686 0 2024-04-12T05:56:22.567 INFO:tasks.workunit.client.1.smithi194.stdout:2/959: truncate d7/d22/d42/fd3 468197 0 2024-04-12T05:56:22.568 INFO:tasks.workunit.client.1.smithi194.stdout:0/976: fdatasync d0/da/d4a/d8e/f11e 0 2024-04-12T05:56:22.570 INFO:tasks.workunit.client.1.smithi194.stdout:6/988: rename d3/d2d/d33/d52/d62/ddd/ff3 to d3/d2f/d4e/d68/dad/dc4/d115/f13f 0 2024-04-12T05:56:22.572 INFO:tasks.workunit.client.1.smithi194.stdout:7/991: rename d3/d16/d23/d37/le1 to d3/d16/d23/d37/l144 0 2024-04-12T05:56:22.572 INFO:tasks.workunit.client.1.smithi194.stdout:0/977: truncate d0/d3b/d52/dfa/d27/d87/d92/fda 1414353 0 2024-04-12T05:56:22.572 INFO:tasks.workunit.client.1.smithi194.stdout:2/960: creat d7/d22/dc9/de5/f146 x:0 0 0 2024-04-12T05:56:22.573 INFO:tasks.workunit.client.1.smithi194.stdout:0/978: chown d0/da/f39 31981644 1 2024-04-12T05:56:22.573 INFO:tasks.workunit.client.1.smithi194.stdout:6/989: symlink d3/d2d/d33/d52/d5c/d8c/l140 0 2024-04-12T05:56:22.574 INFO:tasks.workunit.client.1.smithi194.stdout:2/961: chown d7/d12/d25/d5d/d47/d52/d7b 209834302 1 2024-04-12T05:56:22.575 INFO:tasks.workunit.client.1.smithi194.stdout:0/979: symlink d0/d3b/d52/de2/l13d 0 2024-04-12T05:56:22.577 INFO:tasks.workunit.client.1.smithi194.stdout:7/992: chown d3/d27/d22/ff0 0 1 2024-04-12T05:56:22.585 INFO:tasks.workunit.client.1.smithi194.stdout:0/980: dwrite d0/d3b/d52/de2/fea [0,4194304] 0 2024-04-12T05:56:22.589 INFO:tasks.workunit.client.1.smithi194.stdout:0/981: rename d0/d3b/d52/dfa/d8f/l10b to d0/l13e 0 2024-04-12T05:56:22.589 INFO:tasks.workunit.client.1.smithi194.stdout:7/993: link d3/d16/d23/d37/d95/f9c d3/d27/d6a/f145 0 2024-04-12T05:56:22.590 INFO:tasks.workunit.client.1.smithi194.stdout:6/990: truncate d3/d15/d2b/d48/f4f 993810 0 2024-04-12T05:56:22.594 INFO:tasks.workunit.client.1.smithi194.stdout:6/991: truncate d3/d2f/d4e/d63/d83/f116 705838 0 2024-04-12T05:56:22.595 INFO:tasks.workunit.client.1.smithi194.stdout:6/992: chown d3/d15/f24 57978 1 2024-04-12T05:56:22.597 INFO:tasks.workunit.client.1.smithi194.stdout:0/982: rename d0/da/d4a/d8e/f10f to d0/de3/df0/f13f 0 2024-04-12T05:56:22.600 INFO:tasks.workunit.client.1.smithi194.stdout:6/993: truncate d3/d2f/d4e/d9a/fee 137597 0 2024-04-12T05:56:22.600 INFO:tasks.workunit.client.1.smithi194.stdout:0/983: mkdir d0/da/d4a/d140 0 2024-04-12T05:56:22.606 INFO:tasks.workunit.client.1.smithi194.stdout:7/994: dwrite d3/def/d12e/f131 [0,4194304] 0 2024-04-12T05:56:22.608 INFO:tasks.workunit.client.1.smithi194.stdout:7/995: chown d3/d16/d23/d40/f11c 87926937 1 2024-04-12T05:56:22.609 INFO:tasks.workunit.client.1.smithi194.stdout:6/994: dwrite d3/fd4 [0,4194304] 0 2024-04-12T05:56:22.618 INFO:tasks.workunit.client.1.smithi194.stdout:0/984: write d0/d3b/d52/dfa/d1a/d24/d56/f117 [132721,597] 0 2024-04-12T05:56:22.620 INFO:tasks.workunit.client.1.smithi194.stdout:0/985: truncate d0/da/da7/f114 986698 0 2024-04-12T05:56:22.621 INFO:tasks.workunit.client.1.smithi194.stdout:6/995: link d3/d2d/d33/d52/f5a d3/d15/d2b/d48/d6f/d7a/d8f/d128/f141 0 2024-04-12T05:56:22.623 INFO:tasks.workunit.client.1.smithi194.stdout:0/986: write d0/d3b/d52/de2/fea [2448025,38475] 0 2024-04-12T05:56:22.625 INFO:tasks.workunit.client.1.smithi194.stdout:6/996: symlink d3/d15/l142 0 2024-04-12T05:56:22.633 INFO:tasks.workunit.client.1.smithi194.stdout:7/996: rmdir d3/d27/d6a 39 2024-04-12T05:56:22.634 INFO:tasks.workunit.client.1.smithi194.stdout:0/987: dwrite d0/d3b/d52/dfa/d1a/d24/d4b/f93 [4194304,4194304] 0 2024-04-12T05:56:22.640 INFO:tasks.workunit.client.1.smithi194.stdout:7/997: symlink d3/d16/d23/d57/d72/d81/d3a/d43/d5c/l146 0 2024-04-12T05:56:22.640 INFO:tasks.workunit.client.1.smithi194.stdout:7/998: chown d3/d16/d11b 182 1 2024-04-12T05:56:22.641 INFO:tasks.workunit.client.1.smithi194.stdout:7/999: dread - d3/def/ffc zero size 2024-04-12T05:56:22.643 INFO:tasks.workunit.client.1.smithi194.stdout:0/988: getdents d0/d3b/d52/dfa/d1a/d24/d56 0 2024-04-12T05:56:22.643 INFO:tasks.workunit.client.1.smithi194.stdout:6/997: write d3/d2f/d4e/d9a/dd8/f11d [725798,28596] 0 2024-04-12T05:56:22.646 INFO:tasks.workunit.client.1.smithi194.stdout:0/989: chown d0/da/d4a/d8e/ldd 94772209 1 2024-04-12T05:56:22.655 INFO:tasks.workunit.client.1.smithi194.stdout:0/990: write d0/f30 [1033511,53674] 0 2024-04-12T05:56:22.656 INFO:tasks.workunit.client.1.smithi194.stdout:6/998: dread d3/d15/d2b/d48/d6f/d7a/d8f/fa3 [0,4194304] 0 2024-04-12T05:56:22.658 INFO:tasks.workunit.client.1.smithi194.stdout:6/999: dread - d3/d15/d2b/d48/d6f/d7a/d8f/d98/ff8 zero size 2024-04-12T05:56:22.661 INFO:tasks.workunit.client.1.smithi194.stdout:0/991: truncate d0/d3b/d52/dfa/f41 10121676 0 2024-04-12T05:56:22.661 INFO:tasks.workunit.client.1.smithi194.stdout:0/992: dread - d0/da/da7/fba zero size 2024-04-12T05:56:22.663 INFO:tasks.workunit.client.1.smithi194.stdout:0/993: unlink d0/d3b/d52/dfa/d27/d87/c8c 0 2024-04-12T05:56:22.682 INFO:tasks.workunit.client.1.smithi194.stdout:0/994: dread d0/d3b/d52/dfa/d27/d54/f7b [4194304,4194304] 0 2024-04-12T05:56:22.684 INFO:tasks.workunit.client.1.smithi194.stdout:0/995: write d0/d3b/d9c/f132 [354210,110923] 0 2024-04-12T05:56:22.695 INFO:tasks.workunit.client.1.smithi194.stdout:0/996: write d0/da/f3e [3016099,81681] 0 2024-04-12T05:56:22.697 INFO:tasks.workunit.client.1.smithi194.stdout:0/997: creat d0/da/d4a/f141 x:0 0 0 2024-04-12T05:56:22.712 INFO:tasks.workunit.client.1.smithi194.stdout:0/998: write d0/da/f89 [1128039,22410] 0 2024-04-12T05:56:22.724 INFO:tasks.workunit.client.1.smithi194.stdout:0/999: write d0/d3b/d52/dfa/f19 [1535860,49563] 0 2024-04-12T05:56:22.744 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:22 smithi194 ceph-mon[49771]: pgmap v5: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:22.801 INFO:tasks.workunit.client.1.smithi194.stdout:8/735: sync 2024-04-12T05:56:22.802 INFO:tasks.workunit.client.1.smithi194.stdout:8/736: readlink d2/d8/d15/d35/d6c/ddf/l92 0 2024-04-12T05:56:22.805 INFO:tasks.workunit.client.1.smithi194.stdout:8/737: creat d2/d8/d15/d33/dab/fe7 x:0 0 0 2024-04-12T05:56:22.805 INFO:tasks.workunit.client.1.smithi194.stdout:8/738: stat d2/d8/d9/de/d10/le6 0 2024-04-12T05:56:22.808 INFO:tasks.workunit.client.1.smithi194.stdout:8/739: mkdir d2/d8/d15/d33/d45/de8 0 2024-04-12T05:56:22.814 INFO:tasks.workunit.client.1.smithi194.stdout:8/740: dwrite d2/d8/d15/d33/d45/d52/f7b [4194304,4194304] 0 2024-04-12T05:56:22.815 INFO:tasks.workunit.client.1.smithi194.stdout:8/741: fsync d2/d8/d9/de/f37 0 2024-04-12T05:56:22.828 INFO:tasks.workunit.client.1.smithi194.stdout:8/742: truncate d2/d8/d15/d33/d45/d52/f7b 4360261 0 2024-04-12T05:56:22.829 INFO:tasks.workunit.client.1.smithi194.stdout:8/743: read - d2/d8/d9/de/f9a zero size 2024-04-12T05:56:22.834 INFO:tasks.workunit.client.1.smithi194.stdout:8/744: dwrite d2/d8/d9/de/f37 [0,4194304] 0 2024-04-12T05:56:22.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/745: chown d2/d8/d15/d33/d49/f8e 71 1 2024-04-12T05:56:22.841 INFO:tasks.workunit.client.1.smithi194.stdout:8/746: dwrite d2/d8/d66/d7f/f9c [0,4194304] 0 2024-04-12T05:56:22.845 INFO:tasks.workunit.client.1.smithi194.stdout:8/747: unlink d2/d8/d9/de/lb6 0 2024-04-12T05:56:22.851 INFO:tasks.workunit.client.1.smithi194.stdout:8/748: symlink d2/d8/le9 0 2024-04-12T05:56:22.853 INFO:tasks.workunit.client.1.smithi194.stdout:8/749: rmdir d2/d8/d9/de/d10/dcc 39 2024-04-12T05:56:22.854 INFO:tasks.workunit.client.1.smithi194.stdout:8/750: dread - d2/d8/d9/de/d10/d4e/fd8 zero size 2024-04-12T05:56:22.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:22 smithi008 ceph-mon[39722]: pgmap v5: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:22.964 INFO:tasks.workunit.client.1.smithi194.stdout:9/967: sync 2024-04-12T05:56:22.964 INFO:tasks.workunit.client.1.smithi194.stdout:2/962: sync 2024-04-12T05:56:22.966 INFO:tasks.workunit.client.1.smithi194.stdout:9/968: chown d1/d21/d49/dcb/f144 49 1 2024-04-12T05:56:22.968 INFO:tasks.workunit.client.1.smithi194.stdout:2/963: mkdir d7/d12/d1a/d69/df0/d101/d147 0 2024-04-12T05:56:22.971 INFO:tasks.workunit.client.1.smithi194.stdout:2/964: mknod d7/d22/dc9/c148 0 2024-04-12T05:56:22.974 INFO:tasks.workunit.client.1.smithi194.stdout:2/965: creat d7/d12/d25/d5d/d47/dac/db3/dd7/df8/f149 x:0 0 0 2024-04-12T05:56:22.976 INFO:tasks.workunit.client.1.smithi194.stdout:2/966: write d7/d12/d1a/dbf/f127 [513388,34166] 0 2024-04-12T05:56:22.976 INFO:tasks.workunit.client.1.smithi194.stdout:9/969: truncate d1/f69 3284486 0 2024-04-12T05:56:22.980 INFO:tasks.workunit.client.1.smithi194.stdout:9/970: creat d1/d4e/dea/dfc/d12f/d5f/df4/d137/f145 x:0 0 0 2024-04-12T05:56:22.982 INFO:tasks.workunit.client.1.smithi194.stdout:2/967: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f14a x:0 0 0 2024-04-12T05:56:22.984 INFO:tasks.workunit.client.1.smithi194.stdout:9/971: rename d1/d21/d77/d98/dab/d128 to d1/d4e/dea/dfc/d146 0 2024-04-12T05:56:22.986 INFO:tasks.workunit.client.1.smithi194.stdout:2/968: symlink d7/d12/l14b 0 2024-04-12T05:56:22.988 INFO:tasks.workunit.client.1.smithi194.stdout:2/969: write d7/d12/d25/f64 [2717522,59010] 0 2024-04-12T05:56:22.989 INFO:tasks.workunit.client.1.smithi194.stdout:9/972: rename d1/d21/d39/c13e to d1/d41/dba/c147 0 2024-04-12T05:56:22.994 INFO:tasks.workunit.client.1.smithi194.stdout:2/970: dread d7/d12/d6d/f12f [0,4194304] 0 2024-04-12T05:56:22.999 INFO:tasks.workunit.client.1.smithi194.stdout:2/971: creat d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f14c x:0 0 0 2024-04-12T05:56:23.000 INFO:tasks.workunit.client.1.smithi194.stdout:9/973: write d1/d21/f61 [1604422,54420] 0 2024-04-12T05:56:23.005 INFO:tasks.workunit.client.1.smithi194.stdout:2/972: write d7/d12/f41 [4028151,107501] 0 2024-04-12T05:56:23.007 INFO:tasks.workunit.client.1.smithi194.stdout:2/973: read d7/d22/d93/fdf [133596,127850] 0 2024-04-12T05:56:23.009 INFO:tasks.workunit.client.1.smithi194.stdout:2/974: creat d7/d12/d25/d70/df4/f14d x:0 0 0 2024-04-12T05:56:23.013 INFO:tasks.workunit.client.1.smithi194.stdout:2/975: unlink d7/d12/c17 0 2024-04-12T05:56:23.015 INFO:tasks.workunit.client.1.smithi194.stdout:9/974: dread d1/d4e/dea/dfc/d12f/d20/d37/da3/fb3 [0,4194304] 0 2024-04-12T05:56:23.015 INFO:tasks.workunit.client.1.smithi194.stdout:2/976: unlink d7/d12/d25/d70/l8e 0 2024-04-12T05:56:23.016 INFO:tasks.workunit.client.1.smithi194.stdout:9/975: chown d1/d21/d77/d98/d9f/dde/c139 200495 1 2024-04-12T05:56:23.017 INFO:tasks.workunit.client.1.smithi194.stdout:2/977: mknod d7/d12/d1a/dbf/c14e 0 2024-04-12T05:56:23.018 INFO:tasks.workunit.client.1.smithi194.stdout:9/976: fdatasync d1/d4e/dea/dfc/d12f/d5f/f86 0 2024-04-12T05:56:23.022 INFO:tasks.workunit.client.1.smithi194.stdout:9/977: unlink d1/d21/d39/f11e 0 2024-04-12T05:56:23.024 INFO:tasks.workunit.client.1.smithi194.stdout:9/978: write d1/d4e/dea/dfc/d12f/d20/d37/d60/f107 [7106,111633] 0 2024-04-12T05:56:23.031 INFO:tasks.workunit.client.1.smithi194.stdout:2/978: dread d7/d12/d25/d5d/d47/d52/d7b/fa6 [0,4194304] 0 2024-04-12T05:56:23.032 INFO:tasks.workunit.client.1.smithi194.stdout:9/979: dwrite d1/d19/f2e [0,4194304] 0 2024-04-12T05:56:23.035 INFO:tasks.workunit.client.1.smithi194.stdout:9/980: symlink d1/d41/dbf/l148 0 2024-04-12T05:56:23.038 INFO:tasks.workunit.client.1.smithi194.stdout:8/751: sync 2024-04-12T05:56:23.038 INFO:tasks.workunit.client.1.smithi194.stdout:9/981: fsync d1/d4e/dea/dfc/d12f/d5f/f85 0 2024-04-12T05:56:23.039 INFO:tasks.workunit.client.1.smithi194.stdout:2/979: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f131 [547390,106690] 0 2024-04-12T05:56:23.040 INFO:tasks.workunit.client.1.smithi194.stdout:9/982: creat d1/d41/d97/f149 x:0 0 0 2024-04-12T05:56:23.044 INFO:tasks.workunit.client.1.smithi194.stdout:9/983: chown d1/d4e/dea/dfc/d12f/d20/fed 59135 1 2024-04-12T05:56:23.044 INFO:tasks.workunit.client.1.smithi194.stdout:8/752: rmdir d2/d8/d15/d35/d6c/ddf/dbe/de2 0 2024-04-12T05:56:23.045 INFO:tasks.workunit.client.1.smithi194.stdout:2/980: creat d7/d12/d1a/d69/df0/d101/df9/f14f x:0 0 0 2024-04-12T05:56:23.046 INFO:tasks.workunit.client.1.smithi194.stdout:9/984: mkdir d1/da6/d10f/d14a 0 2024-04-12T05:56:23.047 INFO:tasks.workunit.client.1.smithi194.stdout:8/753: mkdir d2/d8/d9/de/d38/d8f/d94/dea 0 2024-04-12T05:56:23.048 INFO:tasks.workunit.client.1.smithi194.stdout:2/981: creat d7/d12/d1a/dbf/f150 x:0 0 0 2024-04-12T05:56:23.049 INFO:tasks.workunit.client.1.smithi194.stdout:8/754: chown d2/d8/d9/de/d38/c93 34335878 1 2024-04-12T05:56:23.050 INFO:tasks.workunit.client.1.smithi194.stdout:9/985: symlink d1/d4e/dea/dfc/d12f/d20/d37/d84/df2/l14b 0 2024-04-12T05:56:23.057 INFO:tasks.workunit.client.1.smithi194.stdout:8/755: dread d2/d8/d9/de/d10/d4e/f73 [0,4194304] 0 2024-04-12T05:56:23.058 INFO:tasks.workunit.client.1.smithi194.stdout:2/982: dwrite d7/d12/d1a/d69/df0/d101/df9/f14f [0,4194304] 0 2024-04-12T05:56:23.066 INFO:tasks.workunit.client.1.smithi194.stdout:8/756: unlink d2/d8/d9/de/f9a 0 2024-04-12T05:56:23.066 INFO:tasks.workunit.client.1.smithi194.stdout:2/983: rename d7/d22/d93/fdf to d7/d12/d6d/f151 0 2024-04-12T05:56:23.068 INFO:tasks.workunit.client.1.smithi194.stdout:8/757: creat d2/d8/d9/de/d38/feb x:0 0 0 2024-04-12T05:56:23.070 INFO:tasks.workunit.client.1.smithi194.stdout:9/986: dwrite d1/d41/f6f [0,4194304] 0 2024-04-12T05:56:23.075 INFO:tasks.workunit.client.1.smithi194.stdout:8/758: dread d2/d8/f26 [0,4194304] 0 2024-04-12T05:56:23.078 INFO:tasks.workunit.client.1.smithi194.stdout:9/987: creat d1/d41/d140/f14c x:0 0 0 2024-04-12T05:56:23.078 INFO:tasks.workunit.client.1.smithi194.stdout:2/984: write d7/d22/dc9/de5/f12e [721286,91018] 0 2024-04-12T05:56:23.080 INFO:tasks.workunit.client.1.smithi194.stdout:8/759: creat d2/d8/fec x:0 0 0 2024-04-12T05:56:23.081 INFO:tasks.workunit.client.1.smithi194.stdout:2/985: truncate d7/d22/dc9/de5/f12e 1118592 0 2024-04-12T05:56:23.081 INFO:tasks.workunit.client.1.smithi194.stdout:2/986: chown d7/d22/d93/d9f 505 1 2024-04-12T05:56:23.083 INFO:tasks.workunit.client.1.smithi194.stdout:2/987: write d7/d12/d25/d5d/d47/f7f [1499130,84291] 0 2024-04-12T05:56:23.084 INFO:tasks.workunit.client.1.smithi194.stdout:2/988: write d7/d12/d1a/d69/df0/f13b [750955,75373] 0 2024-04-12T05:56:23.085 INFO:tasks.workunit.client.1.smithi194.stdout:9/988: mkdir d1/d21/d49/dcb/d14d 0 2024-04-12T05:56:23.087 INFO:tasks.workunit.client.1.smithi194.stdout:9/989: chown d1/d4e/dea/dfc/d12f/d20/d37/d60/ffb 65675152 1 2024-04-12T05:56:23.091 INFO:tasks.workunit.client.1.smithi194.stdout:9/990: creat d1/d4e/dea/dfc/d12f/f14e x:0 0 0 2024-04-12T05:56:23.094 INFO:tasks.workunit.client.1.smithi194.stdout:2/989: dwrite d7/d12/d25/f64 [4194304,4194304] 0 2024-04-12T05:56:23.098 INFO:tasks.workunit.client.1.smithi194.stdout:8/760: dwrite d2/d8/f6a [0,4194304] 0 2024-04-12T05:56:23.100 INFO:tasks.workunit.client.1.smithi194.stdout:2/990: getdents d7/d12/d1a/de9 0 2024-04-12T05:56:23.102 INFO:tasks.workunit.client.1.smithi194.stdout:8/761: symlink d2/d8/d15/d33/ddd/led 0 2024-04-12T05:56:23.104 INFO:tasks.workunit.client.1.smithi194.stdout:2/991: creat d7/d12/d1a/d69/df0/f152 x:0 0 0 2024-04-12T05:56:23.107 INFO:tasks.workunit.client.1.smithi194.stdout:2/992: stat d7/d12/d1a/d69/f82 0 2024-04-12T05:56:23.108 INFO:tasks.workunit.client.1.smithi194.stdout:9/991: dwrite d1/d21/d77/d98/d9f/dde/fec [0,4194304] 0 2024-04-12T05:56:23.110 INFO:tasks.workunit.client.1.smithi194.stdout:8/762: link d2/d8/d15/d33/d49/cb3 d2/d8/dc8/cee 0 2024-04-12T05:56:23.114 INFO:tasks.workunit.client.1.smithi194.stdout:9/992: link d1/d19/l62 d1/d21/d49/dcb/l14f 0 2024-04-12T05:56:23.115 INFO:tasks.workunit.client.1.smithi194.stdout:2/993: write d7/d12/d25/d5d/d47/dac/db3/dd7/d4a/d8a/f14a [104414,127854] 0 2024-04-12T05:56:23.121 INFO:tasks.workunit.client.1.smithi194.stdout:2/994: creat d7/d12/d25/d5d/f153 x:0 0 0 2024-04-12T05:56:23.126 INFO:tasks.workunit.client.1.smithi194.stdout:2/995: rename d7/d12/d25/d70/df4 to d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103/d142/d154 0 2024-04-12T05:56:23.129 INFO:tasks.workunit.client.1.smithi194.stdout:9/993: truncate d1/d41/dba/fe8 1304918 0 2024-04-12T05:56:23.130 INFO:tasks.workunit.client.1.smithi194.stdout:2/996: read d7/d12/d25/f129 [1620656,23381] 0 2024-04-12T05:56:23.130 INFO:tasks.workunit.client.1.smithi194.stdout:9/994: fsync d1/d41/d140/f14c 0 2024-04-12T05:56:23.133 INFO:tasks.workunit.client.1.smithi194.stdout:2/997: mkdir d7/d12/d25/d5d/d11e/d123/d155 0 2024-04-12T05:56:23.135 INFO:tasks.workunit.client.1.smithi194.stdout:8/763: dread d2/d8/d9/de/d38/d8f/f90 [0,4194304] 0 2024-04-12T05:56:23.135 INFO:tasks.workunit.client.1.smithi194.stdout:9/995: dread d1/da6/d10f/f9c [0,4194304] 0 2024-04-12T05:56:23.136 INFO:tasks.workunit.client.1.smithi194.stdout:2/998: readlink d7/d12/d25/d5d/d47/dac/db3/dd7/df8/d103/l114 0 2024-04-12T05:56:23.137 INFO:tasks.workunit.client.1.smithi194.stdout:8/764: chown d2/d8/d9/de/d38/c59 1821 1 2024-04-12T05:56:23.139 INFO:tasks.workunit.client.1.smithi194.stdout:2/999: mkdir d7/d22/dc9/d156 0 2024-04-12T05:56:23.139 INFO:tasks.workunit.client.1.smithi194.stdout:9/996: mknod d1/d41/d97/d131/c150 0 2024-04-12T05:56:23.141 INFO:tasks.workunit.client.1.smithi194.stdout:8/765: symlink d2/d8/d9/de/d10/d4e/lef 0 2024-04-12T05:56:23.143 INFO:tasks.workunit.client.1.smithi194.stdout:9/997: truncate d1/d4e/f7e 6579856 0 2024-04-12T05:56:23.143 INFO:tasks.workunit.client.1.smithi194.stdout:9/998: readlink d1/d4e/dea/dfc/d146/l13f 0 2024-04-12T05:56:23.144 INFO:tasks.workunit.client.1.smithi194.stdout:9/999: chown d1/d21/d49/fc5 26628 1 2024-04-12T05:56:23.146 INFO:tasks.workunit.client.1.smithi194.stdout:8/766: creat d2/ff0 x:0 0 0 2024-04-12T05:56:23.148 INFO:tasks.workunit.client.1.smithi194.stdout:8/767: mknod d2/d8/d66/dc1/cf1 0 2024-04-12T05:56:23.150 INFO:tasks.workunit.client.1.smithi194.stdout:8/768: fsync d2/d8/d15/d3b/f78 0 2024-04-12T05:56:23.155 INFO:tasks.workunit.client.1.smithi194.stdout:8/769: dread d2/d8/d9/d1a/faf [0,4194304] 0 2024-04-12T05:56:23.157 INFO:tasks.workunit.client.1.smithi194.stdout:8/770: creat d2/d8/d15/d35/d6c/ff2 x:0 0 0 2024-04-12T05:56:23.159 INFO:tasks.workunit.client.1.smithi194.stdout:8/771: creat d2/d8/d15/d33/d45/ff3 x:0 0 0 2024-04-12T05:56:23.161 INFO:tasks.workunit.client.1.smithi194.stdout:8/772: write d2/d8/d9/de/d38/d8f/fcd [1846986,6791] 0 2024-04-12T05:56:23.168 INFO:tasks.workunit.client.1.smithi194.stdout:8/773: dwrite d2/d8/d15/d35/d6c/ddf/dbe/f74 [0,4194304] 0 2024-04-12T05:56:23.170 INFO:tasks.workunit.client.1.smithi194.stdout:8/774: chown d2/d8/d9/de/d10/d4e/ld3 77 1 2024-04-12T05:56:23.191 INFO:tasks.workunit.client.1.smithi194.stdout:8/775: dread d2/d8/d15/d35/f4c [0,4194304] 0 2024-04-12T05:56:23.382 INFO:tasks.workunit.client.1.smithi194.stdout:8/776: sync 2024-04-12T05:56:23.384 INFO:tasks.workunit.client.1.smithi194.stdout:8/777: mknod d2/d8/d15/d35/cf4 0 2024-04-12T05:56:23.385 INFO:tasks.workunit.client.1.smithi194.stdout:8/778: chown d2/d8/d15/d33/d49/cb3 38151636 1 2024-04-12T05:56:23.388 INFO:tasks.workunit.client.1.smithi194.stdout:8/779: mkdir d2/d8/d9/de/d10/dcc/de3/df5 0 2024-04-12T05:56:23.393 INFO:tasks.workunit.client.1.smithi194.stdout:8/780: link d2/d8/d9/d1a/f5a d2/d8/d15/ff6 0 2024-04-12T05:56:23.396 INFO:tasks.workunit.client.1.smithi194.stdout:8/781: link d2/d8/d15/d33/d45/ff3 d2/d8/d66/da4/ff7 0 2024-04-12T05:56:23.401 INFO:tasks.workunit.client.1.smithi194.stdout:8/782: rename d2/d8/d15/d33/d45/fd1 to d2/d8/daa/ff8 0 2024-04-12T05:56:23.403 INFO:tasks.workunit.client.1.smithi194.stdout:8/783: mkdir d2/d8/d9/de/df9 0 2024-04-12T05:56:23.405 INFO:tasks.workunit.client.1.smithi194.stdout:8/784: mkdir d2/d8/d15/d33/d45/d52/dfa 0 2024-04-12T05:56:23.407 INFO:tasks.workunit.client.1.smithi194.stdout:8/785: write d2/d8/d9/de/d10/d4e/f8a [3957966,31313] 0 2024-04-12T05:56:23.409 INFO:tasks.workunit.client.1.smithi194.stdout:8/786: creat d2/d8/d9/de/ffb x:0 0 0 2024-04-12T05:56:23.411 INFO:tasks.workunit.client.1.smithi194.stdout:8/787: rmdir d2/d8/d9/de/d10 39 2024-04-12T05:56:23.412 INFO:tasks.workunit.client.1.smithi194.stdout:8/788: write d2/d8/d15/d35/d6c/ddf/f8d [496094,84508] 0 2024-04-12T05:56:23.414 INFO:tasks.workunit.client.1.smithi194.stdout:8/789: write d2/d8/d15/f16 [3207343,8103] 0 2024-04-12T05:56:23.418 INFO:tasks.workunit.client.1.smithi194.stdout:8/790: symlink d2/d8/d15/d33/d49/db9/lfc 0 2024-04-12T05:56:23.423 INFO:tasks.workunit.client.1.smithi194.stdout:8/791: dwrite d2/d8/d9/de/f98 [0,4194304] 0 2024-04-12T05:56:23.427 INFO:tasks.workunit.client.1.smithi194.stdout:8/792: rename d2/d8/c57 to d2/d8/d9/de/df9/cfd 0 2024-04-12T05:56:23.430 INFO:tasks.workunit.client.1.smithi194.stdout:8/793: rename d2/d8/d66/da4/ff7 to d2/d8/d15/d33/d45/de8/ffe 0 2024-04-12T05:56:23.435 INFO:tasks.workunit.client.1.smithi194.stdout:8/794: link d2/d8/d9/lba d2/d8/d9/de/d38/d8f/d94/lff 0 2024-04-12T05:56:23.436 INFO:tasks.workunit.client.1.smithi194.stdout:8/795: readlink d2/d8/d15/d33/d49/db9/lfc 0 2024-04-12T05:56:23.442 INFO:tasks.workunit.client.1.smithi194.stdout:8/796: dwrite d2/d8/d15/d33/f27 [0,4194304] 0 2024-04-12T05:56:23.444 INFO:tasks.workunit.client.1.smithi194.stdout:8/797: readlink d2/d8/d9/de/d38/d8f/d94/lff 0 2024-04-12T05:56:23.449 INFO:tasks.workunit.client.1.smithi194.stdout:8/798: link d2/d8/d9/d1a/l4d d2/d8/d15/d33/l100 0 2024-04-12T05:56:23.451 INFO:tasks.workunit.client.1.smithi194.stdout:8/799: mknod d2/d8/d15/d35/d6c/ddf/c101 0 2024-04-12T05:56:23.454 INFO:tasks.workunit.client.1.smithi194.stdout:8/800: mkdir d2/d8/d9/d1a/d102 0 2024-04-12T05:56:23.471 INFO:tasks.workunit.client.1.smithi194.stdout:8/801: dread d2/d8/d9/de/d10/f88 [4194304,4194304] 0 2024-04-12T05:56:23.480 INFO:tasks.workunit.client.1.smithi194.stdout:8/802: write d2/d8/d15/d33/d45/f55 [65492,121358] 0 2024-04-12T05:56:23.482 INFO:tasks.workunit.client.1.smithi194.stdout:8/803: write d2/d8/d9/de/d10/d4e/f8a [1236928,38918] 0 2024-04-12T05:56:23.496 INFO:tasks.workunit.client.1.smithi194.stdout:8/804: dread d2/d8/f54 [0,4194304] 0 2024-04-12T05:56:23.498 INFO:tasks.workunit.client.1.smithi194.stdout:8/805: unlink d2/d8/d15/d35/d6c/ddf/c101 0 2024-04-12T05:56:23.500 INFO:tasks.workunit.client.1.smithi194.stdout:8/806: unlink d2/f43 0 2024-04-12T05:56:23.503 INFO:tasks.workunit.client.1.smithi194.stdout:8/807: creat d2/d8/d9/de/d10/d7d/d84/d9f/f103 x:0 0 0 2024-04-12T05:56:23.503 INFO:tasks.workunit.client.1.smithi194.stdout:8/808: chown d2/d8/d15/d3b/c85 527656 1 2024-04-12T05:56:23.509 INFO:tasks.workunit.client.1.smithi194.stdout:8/809: write d2/f3e [978945,91477] 0 2024-04-12T05:56:23.510 INFO:tasks.workunit.client.1.smithi194.stdout:8/810: write d2/fbb [641811,51925] 0 2024-04-12T05:56:23.511 INFO:tasks.workunit.client.1.smithi194.stdout:8/811: stat d2/d8/d9/d1a/f3d 0 2024-04-12T05:56:23.515 INFO:tasks.workunit.client.1.smithi194.stdout:8/812: rename d2/d8/dc5/fe4 to d2/d8/d15/d35/d6c/ddf/dbe/f104 0 2024-04-12T05:56:23.516 INFO:tasks.workunit.client.1.smithi194.stdout:8/813: fsync d2/d8/f54 0 2024-04-12T05:56:23.517 INFO:tasks.workunit.client.1.smithi194.stdout:8/814: stat d2/d8/d9/f48 0 2024-04-12T05:56:23.521 INFO:tasks.workunit.client.1.smithi194.stdout:8/815: rename d2/d8/d9/de/d10/f88 to d2/d8/d15/d33/d45/f105 0 2024-04-12T05:56:23.527 INFO:tasks.workunit.client.1.smithi194.stdout:8/816: dwrite d2/d8/d15/d35/d6c/f91 [0,4194304] 0 2024-04-12T05:56:23.531 INFO:tasks.workunit.client.1.smithi194.stdout:8/817: write d2/d8/f26 [1947576,81015] 0 2024-04-12T05:56:23.535 INFO:tasks.workunit.client.1.smithi194.stdout:8/818: rename d2/d8/d15/d33/d45/d52/dfa to d2/d8/d15/d33/d49/db9/dd9/d106 0 2024-04-12T05:56:23.536 INFO:tasks.workunit.client.1.smithi194.stdout:8/819: chown d2/d8/d15/le0 9357238 1 2024-04-12T05:56:23.552 INFO:tasks.workunit.client.1.smithi194.stdout:8/820: dwrite d2/d8/d9/de/d10/f7c [0,4194304] 0 2024-04-12T05:56:23.768 INFO:tasks.workunit.client.1.smithi194.stdout:8/821: sync 2024-04-12T05:56:23.774 INFO:tasks.workunit.client.1.smithi194.stdout:8/822: dwrite d2/d8/d9/de/d38/feb [0,4194304] 0 2024-04-12T05:56:23.791 INFO:tasks.workunit.client.1.smithi194.stdout:8/823: dwrite d2/d8/d9/de/d38/d8f/f90 [4194304,4194304] 0 2024-04-12T05:56:23.793 INFO:tasks.workunit.client.1.smithi194.stdout:8/824: mknod d2/d8/d15/d33/c107 0 2024-04-12T05:56:23.797 INFO:tasks.workunit.client.1.smithi194.stdout:8/825: write d2/d8/d9/de/f37 [3733027,24223] 0 2024-04-12T05:56:23.800 INFO:tasks.workunit.client.1.smithi194.stdout:8/826: mknod d2/d8/d66/dc1/c108 0 2024-04-12T05:56:23.802 INFO:tasks.workunit.client.1.smithi194.stdout:8/827: creat d2/d8/d15/d3b/f109 x:0 0 0 2024-04-12T05:56:23.803 INFO:tasks.workunit.client.1.smithi194.stdout:8/828: readlink d2/d8/d15/d33/l100 0 2024-04-12T05:56:23.805 INFO:tasks.workunit.client.1.smithi194.stdout:8/829: rmdir d2/d8/d15/d35 39 2024-04-12T05:56:23.808 INFO:tasks.workunit.client.1.smithi194.stdout:8/830: fdatasync d2/d8/d9/d1a/f3d 0 2024-04-12T05:56:23.811 INFO:tasks.workunit.client.1.smithi194.stdout:8/831: mkdir d2/d8/d15/d10a 0 2024-04-12T05:56:23.815 INFO:tasks.workunit.client.1.smithi194.stdout:8/832: getdents d2/d8/d15/d35 0 2024-04-12T05:56:23.816 INFO:tasks.workunit.client.1.smithi194.stdout:8/833: write d2/d8/d9/de/f2e [2239996,19381] 0 2024-04-12T05:56:23.817 INFO:tasks.workunit.client.1.smithi194.stdout:8/834: chown d2/d8/d15/f16 118607698 1 2024-04-12T05:56:23.822 INFO:tasks.workunit.client.1.smithi194.stdout:8/835: write d2/d8/d15/d33/d49/f8e [461331,28820] 0 2024-04-12T05:56:23.822 INFO:tasks.workunit.client.1.smithi194.stdout:8/836: chown d2/d8/d9/de/d38/f3a 3889 1 2024-04-12T05:56:23.824 INFO:tasks.workunit.client.1.smithi194.stdout:8/837: creat d2/d8/d15/d33/d45/de8/f10b x:0 0 0 2024-04-12T05:56:23.826 INFO:tasks.workunit.client.1.smithi194.stdout:8/838: write d2/d8/daa/ff8 [343891,105908] 0 2024-04-12T05:56:23.827 INFO:tasks.workunit.client.1.smithi194.stdout:8/839: dread - d2/d8/d15/d35/d6c/ddf/dbe/f104 zero size 2024-04-12T05:56:23.832 INFO:tasks.workunit.client.1.smithi194.stdout:8/840: dwrite d2/d8/daa/ff8 [0,4194304] 0 2024-04-12T05:56:23.836 INFO:tasks.workunit.client.1.smithi194.stdout:8/841: creat d2/d8/d66/f10c x:0 0 0 2024-04-12T05:56:23.839 INFO:tasks.workunit.client.1.smithi194.stdout:8/842: mknod d2/c10d 0 2024-04-12T05:56:23.843 INFO:tasks.workunit.client.1.smithi194.stdout:8/843: dread d2/d8/f26 [0,4194304] 0 2024-04-12T05:56:23.845 INFO:tasks.workunit.client.1.smithi194.stdout:8/844: mknod d2/d8/d66/da4/dca/c10e 0 2024-04-12T05:56:23.846 INFO:tasks.workunit.client.1.smithi194.stdout:8/845: rmdir d2/d8/d66/da4 39 2024-04-12T05:56:23.850 INFO:tasks.workunit.client.1.smithi194.stdout:8/846: rename d2/d8/d9/de/d38/d8f/fc6 to d2/d8/d15/f10f 0 2024-04-12T05:56:23.852 INFO:tasks.workunit.client.1.smithi194.stdout:8/847: write d2/d8/d9/de/d38/f3a [1110215,127445] 0 2024-04-12T05:56:23.854 INFO:tasks.workunit.client.1.smithi194.stdout:8/848: mknod d2/d8/d9/d1a/d102/c110 0 2024-04-12T05:56:23.858 INFO:tasks.workunit.client.1.smithi194.stdout:8/849: dread d2/d8/d66/f70 [0,4194304] 0 2024-04-12T05:56:23.860 INFO:tasks.workunit.client.1.smithi194.stdout:8/850: creat d2/d8/d15/d33/ddd/f111 x:0 0 0 2024-04-12T05:56:23.862 INFO:tasks.workunit.client.1.smithi194.stdout:8/851: mknod d2/d8/d9/de/c112 0 2024-04-12T05:56:23.872 INFO:tasks.workunit.client.1.smithi194.stdout:8/852: write d2/d8/d9/de/d10/d4e/f60 [1051009,76872] 0 2024-04-12T05:56:23.873 INFO:tasks.workunit.client.1.smithi194.stdout:8/853: fsync d2/d8/d9/de/fa5 0 2024-04-12T05:56:23.877 INFO:tasks.workunit.client.1.smithi194.stdout:8/854: getdents d2/d8/d9/de/d10 0 2024-04-12T05:56:23.878 INFO:tasks.workunit.client.1.smithi194.stdout:8/855: chown d2/d8/d9/de/d10/d7d/d84/d9f 2 1 2024-04-12T05:56:23.886 INFO:tasks.workunit.client.1.smithi194.stdout:8/856: dwrite d2/d8/d9/d1a/f2b [0,4194304] 0 2024-04-12T05:56:23.892 INFO:tasks.workunit.client.1.smithi194.stdout:8/857: write d2/d8/d15/d33/d45/de8/ffe [987690,88801] 0 2024-04-12T05:56:23.893 INFO:tasks.workunit.client.1.smithi194.stdout:8/858: mknod d2/d8/d66/dc1/dc2/c113 0 2024-04-12T05:56:23.895 INFO:tasks.workunit.client.1.smithi194.stdout:8/859: write d2/d8/d9/de/d10/d7d/d84/d9f/f103 [65041,67386] 0 2024-04-12T05:56:23.896 INFO:tasks.workunit.client.1.smithi194.stdout:8/860: fsync d2/d8/d9/de/d10/d4e/fdb 0 2024-04-12T05:56:23.902 INFO:tasks.workunit.client.1.smithi194.stdout:8/861: dread d2/d8/d15/d33/f4b [0,4194304] 0 2024-04-12T05:56:23.904 INFO:tasks.workunit.client.1.smithi194.stdout:8/862: dread d2/d8/d9/de/f61 [0,4194304] 0 2024-04-12T05:56:23.907 INFO:tasks.workunit.client.1.smithi194.stdout:8/863: unlink d2/d8/d15/d35/d6c/f91 0 2024-04-12T05:56:23.909 INFO:tasks.workunit.client.1.smithi194.stdout:8/864: rename d2/d8/d9/de/d10/d7d/cb1 to d2/d8/d15/d33/ddd/c114 0 2024-04-12T05:56:23.916 INFO:tasks.workunit.client.1.smithi194.stdout:8/865: dwrite d2/d8/d15/d33/ddd/f111 [0,4194304] 0 2024-04-12T05:56:23.924 INFO:tasks.workunit.client.1.smithi194.stdout:8/866: dwrite d2/d8/d9/de/d10/d7d/d84/d9f/fc7 [0,4194304] 0 2024-04-12T05:56:23.933 INFO:tasks.workunit.client.1.smithi194.stdout:8/867: fdatasync d2/d8/daa/fd2 0 2024-04-12T05:56:23.937 INFO:tasks.workunit.client.1.smithi194.stdout:8/868: mkdir d2/d8/d15/d115 0 2024-04-12T05:56:23.942 INFO:tasks.workunit.client.1.smithi194.stdout:8/869: dread d2/f3e [0,4194304] 0 2024-04-12T05:56:23.952 INFO:tasks.workunit.client.1.smithi194.stdout:8/870: dwrite d2/d8/d15/d35/d6c/f6e [0,4194304] 0 2024-04-12T05:56:23.958 INFO:tasks.workunit.client.1.smithi194.stdout:8/871: getdents d2/d8/d15/d3b 0 2024-04-12T05:56:23.965 INFO:tasks.workunit.client.1.smithi194.stdout:8/872: dwrite d2/d8/d66/f10c [0,4194304] 0 2024-04-12T05:56:23.970 INFO:tasks.workunit.client.1.smithi194.stdout:8/873: getdents d2/d8/d15/d33/d49 0 2024-04-12T05:56:23.978 INFO:tasks.workunit.client.1.smithi194.stdout:8/874: dwrite d2/f56 [0,4194304] 0 2024-04-12T05:56:23.987 INFO:tasks.workunit.client.1.smithi194.stdout:8/875: dwrite d2/d8/d9/de/d10/f24 [0,4194304] 0 2024-04-12T05:56:23.994 INFO:tasks.workunit.client.1.smithi194.stdout:8/876: read d2/d8/d15/d33/f18 [1911771,93680] 0 2024-04-12T05:56:23.998 INFO:tasks.workunit.client.1.smithi194.stdout:8/877: symlink d2/d8/d15/d33/d49/db9/dd9/d106/l116 0 2024-04-12T05:56:24.000 INFO:tasks.workunit.client.1.smithi194.stdout:8/878: write d2/d8/d15/d33/f27 [1507051,15761] 0 2024-04-12T05:56:24.025 INFO:tasks.workunit.client.1.smithi194.stdout:8/879: dread d2/d8/d15/f62 [0,4194304] 0 2024-04-12T05:56:24.030 INFO:tasks.workunit.client.1.smithi194.stdout:8/880: rename d2/d8/d9/de/d38/f3a to d2/d8/d9/de/d10/d7d/d84/d9f/f117 0 2024-04-12T05:56:24.034 INFO:tasks.workunit.client.1.smithi194.stdout:8/881: rmdir d2/d8/d9/de/d38/d8f/d94/dea 0 2024-04-12T05:56:24.048 INFO:tasks.workunit.client.1.smithi194.stdout:8/882: dread d2/d8/d9/de/d38/d8f/d94/fbc [0,4194304] 0 2024-04-12T05:56:24.050 INFO:tasks.workunit.client.1.smithi194.stdout:8/883: mknod d2/d8/c118 0 2024-04-12T05:56:24.057 INFO:tasks.workunit.client.1.smithi194.stdout:8/884: dwrite d2/d8/daa/fd2 [0,4194304] 0 2024-04-12T05:56:24.064 INFO:tasks.workunit.client.1.smithi194.stdout:8/885: rename d2/d8/d15/d33/c4f to d2/d8/d9/de/c119 0 2024-04-12T05:56:24.066 INFO:tasks.workunit.client.1.smithi194.stdout:8/886: read d2/d8/d9/de/d38/d8f/d94/fbc [796205,14238] 0 2024-04-12T05:56:24.071 INFO:tasks.workunit.client.1.smithi194.stdout:8/887: creat d2/d8/d15/d33/d45/f11a x:0 0 0 2024-04-12T05:56:24.074 INFO:tasks.workunit.client.1.smithi194.stdout:8/888: dread d2/d8/d9/d1a/f5a [0,4194304] 0 2024-04-12T05:56:24.078 INFO:tasks.workunit.client.1.smithi194.stdout:8/889: rename d2/d8/d9/de/d10/d7d/d84/d9f/cd0 to d2/d8/dc8/c11b 0 2024-04-12T05:56:24.082 INFO:tasks.workunit.client.1.smithi194.stdout:8/890: rename d2/d8/d9/de/d10/d7d/d84/d9f/f117 to d2/d8/d9/de/df9/f11c 0 2024-04-12T05:56:24.084 INFO:tasks.workunit.client.1.smithi194.stdout:8/891: creat d2/d8/d15/d115/f11d x:0 0 0 2024-04-12T05:56:24.086 INFO:tasks.workunit.client.1.smithi194.stdout:8/892: creat d2/d8/d15/f11e x:0 0 0 2024-04-12T05:56:24.088 INFO:tasks.workunit.client.1.smithi194.stdout:8/893: unlink d2/d8/d15/d33/d49/l6d 0 2024-04-12T05:56:24.090 INFO:tasks.workunit.client.1.smithi194.stdout:8/894: mkdir d2/d8/d9/de/d10/d7d/d11f 0 2024-04-12T05:56:24.091 INFO:tasks.workunit.client.1.smithi194.stdout:8/895: read d2/d8/daa/fd2 [1320622,94007] 0 2024-04-12T05:56:24.094 INFO:tasks.workunit.client.1.smithi194.stdout:8/896: creat d2/f120 x:0 0 0 2024-04-12T05:56:24.099 INFO:tasks.workunit.client.1.smithi194.stdout:8/897: dread d2/d8/d9/de/f98 [0,4194304] 0 2024-04-12T05:56:24.102 INFO:tasks.workunit.client.1.smithi194.stdout:8/898: rename d2/d8/d9/de/d38/l65 to d2/d8/d9/de/d10/l121 0 2024-04-12T05:56:24.104 INFO:tasks.workunit.client.1.smithi194.stdout:8/899: fsync d2/f2d 0 2024-04-12T05:56:24.106 INFO:tasks.workunit.client.1.smithi194.stdout:8/900: creat d2/d8/dc8/f122 x:0 0 0 2024-04-12T05:56:24.111 INFO:tasks.workunit.client.1.smithi194.stdout:8/901: write d2/d8/daa/ff8 [4871830,90669] 0 2024-04-12T05:56:24.113 INFO:tasks.workunit.client.1.smithi194.stdout:8/902: mknod d2/d8/d15/d33/d45/c123 0 2024-04-12T05:56:24.114 INFO:tasks.workunit.client.1.smithi194.stdout:8/903: write d2/d8/d15/d33/ddd/f111 [4305972,61595] 0 2024-04-12T05:56:24.117 INFO:tasks.workunit.client.1.smithi194.stdout:8/904: creat d2/d8/d15/d33/d45/f124 x:0 0 0 2024-04-12T05:56:24.124 INFO:tasks.workunit.client.1.smithi194.stdout:8/905: dwrite d2/d8/d15/d33/d49/f8e [0,4194304] 0 2024-04-12T05:56:24.130 INFO:tasks.workunit.client.1.smithi194.stdout:8/906: mknod d2/d8/d9/de/d10/d7d/c125 0 2024-04-12T05:56:24.133 INFO:tasks.workunit.client.1.smithi194.stdout:8/907: mknod d2/d8/d9/de/df9/c126 0 2024-04-12T05:56:24.135 INFO:tasks.workunit.client.1.smithi194.stdout:8/908: mknod d2/d8/d9/de/d10/d7d/c127 0 2024-04-12T05:56:24.137 INFO:tasks.workunit.client.1.smithi194.stdout:8/909: mkdir d2/d8/d9/de/d10/d128 0 2024-04-12T05:56:24.138 INFO:tasks.workunit.client.1.smithi194.stdout:8/910: read - d2/d8/d9/de/fa8 zero size 2024-04-12T05:56:24.142 INFO:tasks.workunit.client.1.smithi194.stdout:8/911: rename d2/d8/d15/d35/c42 to d2/d8/d15/d33/d49/db9/c129 0 2024-04-12T05:56:24.149 INFO:tasks.workunit.client.1.smithi194.stdout:8/912: dread - d2/d8/d15/d35/f3f zero size 2024-04-12T05:56:24.157 INFO:tasks.workunit.client.1.smithi194.stdout:8/913: dwrite d2/d8/d15/d35/d6c/ddf/dbe/f74 [0,4194304] 0 2024-04-12T05:56:24.160 INFO:tasks.workunit.client.1.smithi194.stdout:8/914: symlink d2/d8/l12a 0 2024-04-12T05:56:24.169 INFO:tasks.workunit.client.1.smithi194.stdout:8/915: dwrite d2/d8/d15/d35/d6c/ddf/dbe/f71 [0,4194304] 0 2024-04-12T05:56:24.174 INFO:tasks.workunit.client.1.smithi194.stdout:8/916: rmdir d2/d8/d9/de/d10/d128 0 2024-04-12T05:56:24.176 INFO:tasks.workunit.client.1.smithi194.stdout:8/917: write d2/d8/d15/d3b/f78 [5120173,73434] 0 2024-04-12T05:56:24.179 INFO:tasks.workunit.client.1.smithi194.stdout:8/918: mkdir d2/d8/dc5/d12b 0 2024-04-12T05:56:24.180 INFO:tasks.workunit.client.1.smithi194.stdout:8/919: fsync d2/d8/d9/de/d10/d4e/f60 0 2024-04-12T05:56:24.182 INFO:tasks.workunit.client.1.smithi194.stdout:8/920: write d2/f3e [245077,52211] 0 2024-04-12T05:56:24.183 INFO:tasks.workunit.client.1.smithi194.stdout:8/921: write d2/d8/d15/d33/ddd/f111 [3708419,55921] 0 2024-04-12T05:56:24.185 INFO:tasks.workunit.client.1.smithi194.stdout:8/922: truncate d2/d8/d15/d33/dab/fe7 167523 0 2024-04-12T05:56:24.188 INFO:tasks.workunit.client.1.smithi194.stdout:8/923: getdents d2/d8/d15/d33/dab 0 2024-04-12T05:56:24.194 INFO:tasks.workunit.client.1.smithi194.stdout:8/924: dwrite d2/d8/d15/d3b/f78 [0,4194304] 0 2024-04-12T05:56:24.197 INFO:tasks.workunit.client.1.smithi194.stdout:8/925: mkdir d2/d8/dc8/d12c 0 2024-04-12T05:56:24.200 INFO:tasks.workunit.client.1.smithi194.stdout:8/926: link d2/d8/d9/de/d10/d4e/fd8 d2/d8/d9/de/d10/d7d/d11f/f12d 0 2024-04-12T05:56:24.201 INFO:tasks.workunit.client.1.smithi194.stdout:8/927: fsync d2/d8/d15/d33/ddd/f111 0 2024-04-12T05:56:24.203 INFO:tasks.workunit.client.1.smithi194.stdout:8/928: fdatasync d2/d8/d9/d1a/f25 0 2024-04-12T05:56:24.205 INFO:tasks.workunit.client.1.smithi194.stdout:8/929: write d2/d8/d9/de/d38/fa0 [3418792,46451] 0 2024-04-12T05:56:24.206 INFO:tasks.workunit.client.1.smithi194.stdout:8/930: read d2/d8/d66/f70 [329575,49584] 0 2024-04-12T05:56:24.208 INFO:tasks.workunit.client.1.smithi194.stdout:8/931: write d2/d8/d9/de/d10/d7d/fd4 [1129975,103936] 0 2024-04-12T05:56:24.215 INFO:tasks.workunit.client.1.smithi194.stdout:8/932: truncate d2/d8/d15/f16 4093762 0 2024-04-12T05:56:24.218 INFO:tasks.workunit.client.1.smithi194.stdout:8/933: getdents d2/d8/d15/d33 0 2024-04-12T05:56:24.220 INFO:tasks.workunit.client.1.smithi194.stdout:8/934: mknod d2/d8/c12e 0 2024-04-12T05:56:24.221 INFO:tasks.workunit.client.1.smithi194.stdout:8/935: fdatasync d2/d8/d15/d115/f11d 0 2024-04-12T05:56:24.223 INFO:tasks.workunit.client.1.smithi194.stdout:8/936: creat d2/d8/d15/d35/d6c/ddf/dbe/f12f x:0 0 0 2024-04-12T05:56:24.228 INFO:tasks.workunit.client.1.smithi194.stdout:8/937: write d2/f12 [5007666,49531] 0 2024-04-12T05:56:24.237 INFO:tasks.workunit.client.1.smithi194.stdout:8/938: dwrite d2/d8/d9/d1a/faf [0,4194304] 0 2024-04-12T05:56:24.238 INFO:tasks.workunit.client.1.smithi194.stdout:8/939: fsync d2/d8/d9/de/f37 0 2024-04-12T05:56:24.246 INFO:tasks.workunit.client.1.smithi194.stdout:8/940: dwrite d2/d8/d9/de/d10/d7d/fd4 [0,4194304] 0 2024-04-12T05:56:24.251 INFO:tasks.workunit.client.1.smithi194.stdout:8/941: symlink d2/d8/d15/d33/l130 0 2024-04-12T05:56:24.253 INFO:tasks.workunit.client.1.smithi194.stdout:8/942: readlink d2/d8/d9/de/d10/d7d/l8b 0 2024-04-12T05:56:24.255 INFO:tasks.workunit.client.1.smithi194.stdout:8/943: write d2/d8/d15/d33/d45/f11a [398411,68579] 0 2024-04-12T05:56:24.262 INFO:tasks.workunit.client.1.smithi194.stdout:8/944: dwrite d2/d8/d9/de/d38/d8f/f90 [4194304,4194304] 0 2024-04-12T05:56:24.275 INFO:tasks.workunit.client.1.smithi194.stdout:8/945: truncate d2/d8/d9/de/f98 573590 0 2024-04-12T05:56:24.276 INFO:tasks.workunit.client.1.smithi194.stdout:8/946: stat d2/d8/d15/d35/d6c/ff2 0 2024-04-12T05:56:24.299 INFO:tasks.workunit.client.1.smithi194.stdout:8/947: dread d2/d8/d15/d33/f18 [0,4194304] 0 2024-04-12T05:56:24.302 INFO:tasks.workunit.client.1.smithi194.stdout:8/948: creat d2/d8/daa/f131 x:0 0 0 2024-04-12T05:56:24.313 INFO:tasks.workunit.client.1.smithi194.stdout:8/949: write d2/d8/d66/f70 [251558,65949] 0 2024-04-12T05:56:24.322 INFO:tasks.workunit.client.1.smithi194.stdout:8/950: chown d2/d8/d9/de/d10/lcf 468 1 2024-04-12T05:56:24.346 INFO:tasks.workunit.client.1.smithi194.stdout:8/951: dread d2/d8/d9/d1a/f3d [0,4194304] 0 2024-04-12T05:56:24.349 INFO:tasks.workunit.client.1.smithi194.stdout:8/952: readlink d2/d8/d9/de/d10/l5c 0 2024-04-12T05:56:24.350 INFO:tasks.workunit.client.1.smithi194.stdout:8/953: chown d2/d8/d15/d33/d45/de8/f10b 265034 1 2024-04-12T05:56:24.351 INFO:tasks.workunit.client.1.smithi194.stdout:8/954: stat d2/d8/d9/d1a/f6f 0 2024-04-12T05:56:24.355 INFO:tasks.workunit.client.1.smithi194.stdout:8/955: truncate d2/d8/d9/de/d10/d4e/fd8 373514 0 2024-04-12T05:56:24.362 INFO:tasks.workunit.client.1.smithi194.stdout:8/956: chown d2/d8/d15/d33/d45/de8 88800 1 2024-04-12T05:56:24.367 INFO:tasks.workunit.client.1.smithi194.stdout:8/957: creat d2/d8/d15/d33/d45/dd7/f132 x:0 0 0 2024-04-12T05:56:24.370 INFO:tasks.workunit.client.1.smithi194.stdout:8/958: read d2/d8/d9/de/d10/d4e/fd8 [348993,82349] 0 2024-04-12T05:56:24.386 INFO:tasks.workunit.client.1.smithi194.stdout:8/959: link d2/d8/f54 d2/d8/d15/d33/d45/d52/f133 0 2024-04-12T05:56:24.389 INFO:tasks.workunit.client.1.smithi194.stdout:8/960: stat d2/d8/d15/d33/dab 0 2024-04-12T05:56:24.390 INFO:tasks.workunit.client.1.smithi194.stdout:8/961: chown d2/d8/d15/d33/d45/c7e 53 1 2024-04-12T05:56:24.396 INFO:tasks.workunit.client.1.smithi194.stdout:8/962: rename d2/d8/d9/de/f98 to d2/d8/d15/d33/d49/f134 0 2024-04-12T05:56:24.400 INFO:tasks.workunit.client.1.smithi194.stdout:8/963: link d2/d8/d66/f10c d2/d8/d15/d33/f135 0 2024-04-12T05:56:24.402 INFO:tasks.workunit.client.1.smithi194.stdout:8/964: read - d2/d8/d15/f9e zero size 2024-04-12T05:56:24.420 INFO:tasks.workunit.client.1.smithi194.stdout:8/965: dwrite d2/d8/d15/d3b/f109 [0,4194304] 0 2024-04-12T05:56:24.451 INFO:tasks.workunit.client.1.smithi194.stdout:8/966: dread d2/d8/d9/de/d10/d4e/f60 [0,4194304] 0 2024-04-12T05:56:24.455 INFO:tasks.workunit.client.1.smithi194.stdout:8/967: mkdir d2/d136 0 2024-04-12T05:56:24.461 INFO:tasks.workunit.client.1.smithi194.stdout:8/968: symlink d2/d8/d9/de/d10/l137 0 2024-04-12T05:56:24.461 INFO:tasks.workunit.client.1.smithi194.stdout:8/969: chown d2/d8/d9/de/d10/l5c 138752 1 2024-04-12T05:56:24.463 INFO:tasks.workunit.client.1.smithi194.stdout:8/970: stat d2/d8/d15/d33/d49/db9/dd9/d106 0 2024-04-12T05:56:24.463 INFO:tasks.workunit.client.1.smithi194.stdout:8/971: stat d2/d8/d15 0 2024-04-12T05:56:24.476 INFO:tasks.workunit.client.1.smithi194.stdout:8/972: write d2/d8/d15/d33/d49/f134 [263353,31264] 0 2024-04-12T05:56:24.478 INFO:tasks.workunit.client.1.smithi194.stdout:8/973: symlink d2/d8/d15/d33/l138 0 2024-04-12T05:56:24.483 INFO:tasks.workunit.client.1.smithi194.stdout:8/974: symlink d2/d8/d9/de/d10/d7d/d84/l139 0 2024-04-12T05:56:24.486 INFO:tasks.workunit.client.1.smithi194.stdout:8/975: creat d2/d8/d15/d10a/f13a x:0 0 0 2024-04-12T05:56:24.487 INFO:tasks.workunit.client.1.smithi194.stdout:8/976: readlink d2/d8/d15/d33/lc3 0 2024-04-12T05:56:24.491 INFO:tasks.workunit.client.1.smithi194.stdout:8/977: mkdir d2/d8/d15/d33/d49/db9/dd9/d13b 0 2024-04-12T05:56:24.494 INFO:tasks.workunit.client.1.smithi194.stdout:8/978: truncate d2/d8/d9/de/d38/fb0 526762 0 2024-04-12T05:56:24.500 INFO:tasks.workunit.client.1.smithi194.stdout:8/979: dwrite d2/d8/d15/d33/f27 [0,4194304] 0 2024-04-12T05:56:24.503 INFO:tasks.workunit.client.1.smithi194.stdout:8/980: mkdir d2/d8/d9/de/d10/d13c 0 2024-04-12T05:56:24.510 INFO:tasks.workunit.client.1.smithi194.stdout:8/981: dwrite d2/d8/d15/d33/d49/fcb [0,4194304] 0 2024-04-12T05:56:24.513 INFO:tasks.workunit.client.1.smithi194.stdout:8/982: rmdir d2/d8/d66/dc1 39 2024-04-12T05:56:24.517 INFO:tasks.workunit.client.1.smithi194.stdout:8/983: creat d2/d8/d9/de/d38/f13d x:0 0 0 2024-04-12T05:56:24.518 INFO:tasks.workunit.client.1.smithi194.stdout:8/984: write d2/d8/d15/d33/d49/fcb [389439,52812] 0 2024-04-12T05:56:24.522 INFO:tasks.workunit.client.1.smithi194.stdout:8/985: chown d2/d8/d15/d33/d49/db9 901 1 2024-04-12T05:56:24.525 INFO:tasks.workunit.client.1.smithi194.stdout:8/986: unlink d2/d8/d15/d35/cf4 0 2024-04-12T05:56:24.530 INFO:tasks.workunit.client.1.smithi194.stdout:8/987: dread d2/d8/d15/d35/fa2 [0,4194304] 0 2024-04-12T05:56:24.532 INFO:tasks.workunit.client.1.smithi194.stdout:8/988: symlink d2/d8/daa/l13e 0 2024-04-12T05:56:24.534 INFO:tasks.workunit.client.1.smithi194.stdout:8/989: mkdir d2/d8/d9/de/d10/d7d/d11f/d13f 0 2024-04-12T05:56:24.536 INFO:tasks.workunit.client.1.smithi194.stdout:8/990: mknod d2/d8/d9/d1a/c140 0 2024-04-12T05:56:24.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:24 smithi194 ceph-mon[49771]: pgmap v6: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:24 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:24 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:24 smithi008 ceph-mon[39722]: pgmap v6: 65 pgs: 65 active+clean; 1.8 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail 2024-04-12T05:56:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:24 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:24 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:25.093 INFO:tasks.workunit.client.1.smithi194.stdout:8/991: sync 2024-04-12T05:56:25.094 INFO:tasks.workunit.client.1.smithi194.stdout:8/992: read d2/f12 [3215156,93260] 0 2024-04-12T05:56:25.098 INFO:tasks.workunit.client.1.smithi194.stdout:8/993: rename d2/d8/d15/d35/fa2 to d2/d136/f141 0 2024-04-12T05:56:25.099 INFO:tasks.workunit.client.1.smithi194.stdout:8/994: read d2/d8/d9/d1a/f25 [837147,52466] 0 2024-04-12T05:56:25.101 INFO:tasks.workunit.client.1.smithi194.stdout:8/995: mkdir d2/d8/d9/d1a/d142 0 2024-04-12T05:56:25.103 INFO:tasks.workunit.client.1.smithi194.stdout:8/996: unlink d2/d8/d9/de/d10/d7d/c127 0 2024-04-12T05:56:25.106 INFO:tasks.workunit.client.1.smithi194.stdout:8/997: mknod d2/d8/d9/de/d38/d8f/c143 0 2024-04-12T05:56:25.130 INFO:tasks.workunit.client.1.smithi194.stdout:8/998: rmdir d2/d8/d15 39 2024-04-12T05:56:25.135 INFO:tasks.workunit.client.1.smithi194.stdout:8/999: getdents d2/d8/d9/de/d10/d7d 0 2024-04-12T05:56:25.144 INFO:tasks.workunit.client.1.smithi194.stderr:+ rm -rf -- ./tmp.5iwGLtTk4c 2024-04-12T05:56:26.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:26 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:26 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:26 smithi194 ceph-mon[49771]: pgmap v7: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 113 MiB/s wr, 423 op/s 2024-04-12T05:56:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:26 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:26 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:26 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:26 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:26 smithi008 ceph-mon[39722]: pgmap v7: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 48 MiB/s rd, 113 MiB/s wr, 423 op/s 2024-04-12T05:56:26.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:26 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:26.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:26 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:56:27.813 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:27 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:56:28.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:56:28.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:27 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/568198976' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:56:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:28 smithi194 ceph-mon[49771]: pgmap v8: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 88 MiB/s wr, 328 op/s 2024-04-12T05:56:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:28 smithi194 ceph-mon[49771]: mgrmap e30: smithi008.ixmnlp(active, since 10s), standbys: smithi194.xbiyaj 2024-04-12T05:56:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:28 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:28 smithi008 ceph-mon[39722]: pgmap v8: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 88 MiB/s wr, 328 op/s 2024-04-12T05:56:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:28 smithi008 ceph-mon[39722]: mgrmap e30: smithi008.ixmnlp(active, since 10s), standbys: smithi194.xbiyaj 2024-04-12T05:56:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:28 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr metadata", "who": "smithi194.xbiyaj", "id": "smithi194.xbiyaj"}]: dispatch 2024-04-12T05:56:30.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:30 smithi008 ceph-mon[39722]: pgmap v9: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 82 MiB/s wr, 430 op/s 2024-04-12T05:56:31.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:30 smithi194 ceph-mon[49771]: pgmap v9: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 82 MiB/s wr, 430 op/s 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 -- 172.21.15.8:0/63776468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 msgr2=0x7f93ec108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 --2- 172.21.15.8:0/63776468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec108210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93e000aa20 tx=0x7f93e000ad30 comp rx=0 tx=0).stop 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 -- 172.21.15.8:0/63776468 shutdown_connections 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 --2- 172.21.15.8:0/63776468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec108210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 --2- 172.21.15.8:0/63776468 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ec109130 0x7f93ec109550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 -- 172.21.15.8:0/63776468 >> 172.21.15.8:0/63776468 conn(0x7f93ec06da40 msgr2=0x7f93ec06fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 -- 172.21.15.8:0/63776468 shutdown_connections 2024-04-12T05:56:31.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.471+0000 7f93f1af8700 1 -- 172.21.15.8:0/63776468 wait complete. 2024-04-12T05:56:31.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 Processor -- start 2024-04-12T05:56:31.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 -- start start 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ec109130 0x7f93ec19d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19d6e0 con 0x7f93ec109130 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.472+0000 7f93f1af8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19d820 con 0x7f93ec107d90 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:44610/0 (socket says 172.21.15.8:44610) 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 -- 172.21.15.8:0/4250308291 learned_addr learned my addr 172.21.15.8:0/4250308291 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 -- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ec109130 msgr2=0x7f93ec19d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ec109130 0x7f93ec19d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 -- 172.21.15.8:0/4250308291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e000a700 con 0x7f93ec107d90 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.473+0000 7f93eb7fe700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00b480 tx=0x7f93dc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.474+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc0118c0 con 0x7f93ec107d90 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.474+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f93dc010380 con 0x7f93ec107d90 2024-04-12T05:56:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.474+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93dc00f510 con 0x7f93ec107d90 2024-04-12T05:56:31.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.474+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec1a02c0 con 0x7f93ec107d90 2024-04-12T05:56:31.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.474+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec1a0810 con 0x7f93ec107d90 2024-04-12T05:56:31.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.476+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec114180 con 0x7f93ec107d90 2024-04-12T05:56:31.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.477+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f93dc0104f0 con 0x7f93ec107d90 2024-04-12T05:56:31.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.477+0000 7f93e8ff9700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 0x7f93d406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:31.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.478+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f93dc08c2b0 con 0x7f93ec107d90 2024-04-12T05:56:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.480+0000 7f93eaffd700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 0x7f93d406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:31.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.480+0000 7f93eaffd700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 0x7f93d406f320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c6d0 tx=0x7f93e000af70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:31.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.485+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f93dc059d30 con 0x7f93ec107d90 2024-04-12T05:56:31.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.820+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93ec1a0450 con 0x7f93d406ce60 2024-04-12T05:56:31.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.821+0000 7f93e8ff9700 1 -- 172.21.15.8:0/4250308291 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f93ec1a0450 con 0x7f93d406ce60 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.823+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 msgr2=0x7f93d406f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.823+0000 7f93f1af8700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 0x7f93d406f320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93e000c6d0 tx=0x7f93e000af70 comp rx=0 tx=0).stop 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 msgr2=0x7f93ec19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00b480 tx=0x7f93dc00b840 comp rx=0 tx=0).stop 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 shutdown_connections 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f93d406ce60 0x7f93d406f320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f93ec107d90 0x7f93ec19cb80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 --2- 172.21.15.8:0/4250308291 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f93ec109130 0x7f93ec19d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:31.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 >> 172.21.15.8:0/4250308291 conn(0x7f93ec06da40 msgr2=0x7f93ec10bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 shutdown_connections 2024-04-12T05:56:31.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.824+0000 7f93f1af8700 1 -- 172.21.15.8:0/4250308291 wait complete. 2024-04-12T05:56:31.830 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 -- 172.21.15.8:0/4091022109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554100cb0 msgr2=0x7ff5541010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 --2- 172.21.15.8:0/4091022109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554100cb0 0x7ff5541010d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff550009910 tx=0x7ff550009c20 comp rx=0 tx=0).stop 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 -- 172.21.15.8:0/4091022109 shutdown_connections 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 --2- 172.21.15.8:0/4091022109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554101eb0 0x7ff554102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 --2- 172.21.15.8:0/4091022109 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554100cb0 0x7ff5541010d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 -- 172.21.15.8:0/4091022109 >> 172.21.15.8:0/4091022109 conn(0x7ff5540fc230 msgr2=0x7ff5540fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 -- 172.21.15.8:0/4091022109 shutdown_connections 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.962+0000 7ff55c188700 1 -- 172.21.15.8:0/4091022109 wait complete. 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 Processor -- start 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 -- start start 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 0x7ff5541963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554196f40 con 0x7ff554101eb0 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff55c188700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff554197080 con 0x7ff554100cb0 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49188/0 (socket says 172.21.15.8:49188) 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 -- 172.21.15.8:0/2603883337 learned_addr learned my addr 172.21.15.8:0/2603883337 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:32.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559f24700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 0x7ff5541963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 -- 172.21.15.8:0/2603883337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 msgr2=0x7ff5541963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 0x7ff5541963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559723700 1 -- 172.21.15.8:0/2603883337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5500095f0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.963+0000 7ff559f24700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 0x7ff5541963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff559723700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff54400d900 tx=0x7ff54400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5440041f0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff544004cc0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55419bb30 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5440056d0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.964+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55419c080 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.965+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5541096d0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.965+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7ff54400b7a0 con 0x7ff554101eb0 2024-04-12T05:56:32.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.966+0000 7ff54affd700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 0x7ff54006f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.966+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff54408c800 con 0x7ff554101eb0 2024-04-12T05:56:32.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.966+0000 7ff559f24700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 0x7ff54006f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.966+0000 7ff559f24700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 0x7ff54006f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff550006010 tx=0x7ff55000b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:31.971+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff54405a1a0 con 0x7ff554101eb0 2024-04-12T05:56:32.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.222+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff554106800 con 0x7ff54006cda0 2024-04-12T05:56:32.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.224+0000 7ff54affd700 1 -- 172.21.15.8:0/2603883337 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff554106800 con 0x7ff54006cda0 2024-04-12T05:56:32.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.227+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 msgr2=0x7ff54006f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.227+0000 7ff55c188700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 0x7ff54006f260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff550006010 tx=0x7ff55000b560 comp rx=0 tx=0).stop 2024-04-12T05:56:32.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.227+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 msgr2=0x7ff554196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.227+0000 7ff55c188700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff54400d900 tx=0x7ff54400dcc0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 shutdown_connections 2024-04-12T05:56:32.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff54006cda0 0x7ff54006f260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff554100cb0 0x7ff5541963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 --2- 172.21.15.8:0/2603883337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff554101eb0 0x7ff554196920 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 >> 172.21.15.8:0/2603883337 conn(0x7ff5540fc230 msgr2=0x7ff5541050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:32.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.228+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 shutdown_connections 2024-04-12T05:56:32.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.229+0000 7ff55c188700 1 -- 172.21.15.8:0/2603883337 wait complete. 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- 172.21.15.8:0/1599277393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 msgr2=0x7ffaf4101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 --2- 172.21.15.8:0/1599277393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4101270 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009910 tx=0x7ffae4009c20 comp rx=0 tx=0).stop 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- 172.21.15.8:0/1599277393 shutdown_connections 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 --2- 172.21.15.8:0/1599277393 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4101270 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 --2- 172.21.15.8:0/1599277393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 0x7ffaf4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- 172.21.15.8:0/1599277393 >> 172.21.15.8:0/1599277393 conn(0x7ffaf40fb190 msgr2=0x7ffaf40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- 172.21.15.8:0/1599277393 shutdown_connections 2024-04-12T05:56:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- 172.21.15.8:0/1599277393 wait complete. 2024-04-12T05:56:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 Processor -- start 2024-04-12T05:56:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.361+0000 7ffafbdde700 1 -- start start 2024-04-12T05:56:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffafbdde700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 0x7ffaf41963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffafbdde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffafbdde700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4196f20 con 0x7ffaf4100e10 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffafbdde700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4197060 con 0x7ffaf40ffc10 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49206/0 (socket says 172.21.15.8:49206) 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 -- 172.21.15.8:0/703821947 learned_addr learned my addr 172.21.15.8:0/703821947 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9b7a700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 0x7ffaf41963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 -- 172.21.15.8:0/703821947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 msgr2=0x7ffaf41963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 0x7ffaf41963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 -- 172.21.15.8:0/703821947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40095f0 con 0x7ffaf4100e10 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaf9379700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffae400bd70 tx=0x7ffae400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae401d070 con 0x7ffaf4100e10 2024-04-12T05:56:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ffae400f460 con 0x7ffaf4100e10 2024-04-12T05:56:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf419bab0 con 0x7ffaf4100e10 2024-04-12T05:56:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.362+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae4021640 con 0x7ffaf4100e10 2024-04-12T05:56:32.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.363+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf419bfa0 con 0x7ffaf4100e10 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.364+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf41085c0 con 0x7ffaf4100e10 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.364+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7ffae400fa20 con 0x7ffaf4100e10 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.364+0000 7ffaeaffd700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 0x7ffae006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.365+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffae408d8b0 con 0x7ffaf4100e10 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.365+0000 7ffaf9b7a700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 0x7ffae006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.365+0000 7ffaf9b7a700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 0x7ffae006f2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4100c70 tx=0x7ffaf0007400 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.369+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ffae405b2c0 con 0x7ffaf4100e10 2024-04-12T05:56:32.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.657+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaf4068c90 con 0x7ffae006ce20 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.670+0000 7ffaeaffd700 1 -- 172.21.15.8:0/703821947 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ffaf4068c90 con 0x7ffae006ce20 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (8m) 8s ago 9m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (10m) 8s ago 10m 7810k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (8m) 6s ago 8m 7742k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (10m) 8s ago 10m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:56:32.665 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (8m) 6s ago 8m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (7m) 8s ago 9m 88.9M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (3m) 8s ago 3m 188M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (3m) 8s ago 3m 121M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (3m) 6s ago 3m 45.2M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (3m) 6s ago 3m 107M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (31s) 8s ago 11m 473M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (93s) 6s ago 8m 61.6M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b06c49f54d22 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (11m) 8s ago 11m 53.5M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (8m) 6s ago 8m 40.9M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (9m) 8s ago 9m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (8m) 6s ago 8m 20.6M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 8s ago 7m 189M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (7m) 8s ago 7m 232M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:56:32.666 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (6m) 8s ago 6m 191M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:56:32.667 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (6m) 6s ago 6m 246M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:56:32.667 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (5m) 6s ago 5m 202M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:56:32.667 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (5m) 6s ago 5m 231M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:56:32.667 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (37s) 8s ago 9m 42.5M - 2.43.0 a07b618ecd1d 545875b9f866 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 msgr2=0x7ffae006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 0x7ffae006f2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4100c70 tx=0x7ffaf0007400 comp rx=0 tx=0).stop 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 msgr2=0x7ffaf4196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffae400bd70 tx=0x7ffae400bda0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 shutdown_connections 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffae006ce20 0x7ffae006f2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffaf40ffc10 0x7ffaf41963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 --2- 172.21.15.8:0/703821947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffaf4100e10 0x7ffaf4196900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 >> 172.21.15.8:0/703821947 conn(0x7ffaf40fb190 msgr2=0x7ffaf4068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:32.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 shutdown_connections 2024-04-12T05:56:32.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.673+0000 7ffafbdde700 1 -- 172.21.15.8:0/703821947 wait complete. 2024-04-12T05:56:32.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1153062216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 msgr2=0x7fa958102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1153062216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958102310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009910 tx=0x7fa94c009c20 comp rx=0 tx=0).stop 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1153062216 shutdown_connections 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1153062216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958102310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1153062216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 0x7fa9581010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1153062216 >> 172.21.15.8:0/1153062216 conn(0x7fa9580fc230 msgr2=0x7fa9580fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1153062216 shutdown_connections 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1153062216 wait complete. 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.790+0000 7fa95ea5c700 1 Processor -- start 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa95ea5c700 1 -- start start 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa95ea5c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 0x7fa9581963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa95ea5c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa95ea5c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958196f40 con 0x7fa958101eb0 2024-04-12T05:56:32.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa95ea5c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958197080 con 0x7fa958100cb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49234/0 (socket says 172.21.15.8:49234) 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 -- 172.21.15.8:0/1022188638 learned_addr learned my addr 172.21.15.8:0/1022188638 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa957fff700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 0x7fa9581963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 -- 172.21.15.8:0/1022188638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 msgr2=0x7fa9581963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 0x7fa9581963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 -- 172.21.15.8:0/1022188638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0095f0 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.791+0000 7fa9577fe700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bde0 tx=0x7fa94c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.792+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c01d070 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.792+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa94c005dc0 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.792+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95819bad0 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.792+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa94c003b10 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.792+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95819bf60 con 0x7fa958101eb0 2024-04-12T05:56:32.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.793+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9581096d0 con 0x7fa958101eb0 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.793+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7fa94c0048e0 con 0x7fa958101eb0 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.794+0000 7fa9557fa700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 0x7fa94006f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.794+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa94c08dcf0 con 0x7fa958101eb0 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.794+0000 7fa957fff700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 0x7fa94006f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.794+0000 7fa957fff700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 0x7fa94006f310 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa958101d10 tx=0x7fa948009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:32.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:32.798+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fa94c05b7a0 con 0x7fa958101eb0 2024-04-12T05:56:32.881 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:32 smithi008 ceph-mon[39722]: pgmap v10: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 75 MiB/s wr, 393 op/s 2024-04-12T05:56:32.881 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:32 smithi008 ceph-mon[39722]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:32.881 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:32 smithi194 ceph-mon[49771]: pgmap v10: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 75 MiB/s wr, 393 op/s 2024-04-12T05:56:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:32 smithi194 ceph-mon[49771]: from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:33.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:33.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.148+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa95819c650 con 0x7fa958101eb0 2024-04-12T05:56:33.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.148+0000 7fa9557fa700 1 -- 172.21.15.8:0/1022188638 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fa94c0262c0 con 0x7fa958101eb0 2024-04-12T05:56:33.143 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:56:33.144 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:56:33.145 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:56:33.145 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:56:33.145 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:56:33.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 msgr2=0x7fa94006f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 0x7fa94006f310 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa958101d10 tx=0x7fa948009260 comp rx=0 tx=0).stop 2024-04-12T05:56:33.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 msgr2=0x7fa958196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00bde0 tx=0x7fa94c00be10 comp rx=0 tx=0).stop 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 shutdown_connections 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa94006ce50 0x7fa94006f310 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa958100cb0 0x7fa9581963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 --2- 172.21.15.8:0/1022188638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa958101eb0 0x7fa958196920 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 >> 172.21.15.8:0/1022188638 conn(0x7fa9580fc230 msgr2=0x7fa9581050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 shutdown_connections 2024-04-12T05:56:33.147 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.151+0000 7fa95ea5c700 1 -- 172.21.15.8:0/1022188638 wait complete. 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3609762077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81017a0 msgr2=0x7fabc8103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3609762077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81017a0 0x7fabc8103b90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fabc000b0a0 tx=0x7fabc000b3b0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3609762077 shutdown_connections 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3609762077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81040d0 0x7fabc81064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3609762077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81017a0 0x7fabc8103b90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3609762077 >> 172.21.15.8:0/3609762077 conn(0x7fabc80fb190 msgr2=0x7fabc80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:33.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3609762077 shutdown_connections 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3609762077 wait complete. 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 Processor -- start 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 -- start start 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.289+0000 7fabcf0e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81017a0 0x7fabc8101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcf0e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcf0e8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80ffc90 con 0x7fabc81040d0 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcf0e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80ffdd0 con 0x7fabc81017a0 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49254/0 (socket says 172.21.15.8:49254) 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 -- 172.21.15.8:0/3474766968 learned_addr learned my addr 172.21.15.8:0/3474766968 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcce84700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81017a0 0x7fabc8101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 -- 172.21.15.8:0/3474766968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81017a0 msgr2=0x7fabc8101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81017a0 0x7fabc8101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 -- 172.21.15.8:0/3474766968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc0009d00 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc7fff700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c930 tx=0x7fabb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb0007ad0 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fabb000ce80 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc81000b0 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb0018630 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.290+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc819e3f0 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.291+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc81904d0 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.292+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7fabb0007c30 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.292+0000 7fabc5ffb700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 0x7fabb406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.292+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fabb008c6b0 con 0x7fabc81040d0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.292+0000 7fabcce84700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 0x7fabb406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.293+0000 7fabcce84700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 0x7fabb406f2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fabc0000f80 tx=0x7fabc00060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:56:33.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.297+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fabb005a0c0 con 0x7fabc81040d0 2024-04-12T05:56:33.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.620+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc80611d0 con 0x7fabb406ce20 2024-04-12T05:56:33.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.621+0000 7fabc5ffb700 1 -- 172.21.15.8:0/3474766968 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fabc80611d0 con 0x7fabb406ce20 2024-04-12T05:56:33.616 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr" 2024-04-12T05:56:33.617 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T05:56:33.618 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T05:56:33.618 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "", 2024-04-12T05:56:33.618 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:56:33.618 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:56:33.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 msgr2=0x7fabb406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 0x7fabb406f2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fabc0000f80 tx=0x7fabc00060c0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 msgr2=0x7fabc80ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c930 tx=0x7fabb000cc40 comp rx=0 tx=0).stop 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 shutdown_connections 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fabb406ce20 0x7fabb406f2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fabc81017a0 0x7fabc8101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 --2- 172.21.15.8:0/3474766968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fabc81040d0 0x7fabc80ff750 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:56:33.620 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 >> 172.21.15.8:0/3474766968 conn(0x7fabc80fb190 msgr2=0x7fabc80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:56:33.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.623+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 shutdown_connections 2024-04-12T05:56:33.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:56:33.624+0000 7fabcf0e8700 1 -- 172.21.15.8:0/3474766968 wait complete. 2024-04-12T05:56:33.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:33 smithi194 ceph-mon[49771]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:33 smithi194 ceph-mon[49771]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:33 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1022188638' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:56:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:33 smithi008 ceph-mon[39722]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:33 smithi008 ceph-mon[39722]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:33 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1022188638' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:56:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:34 smithi008 ceph-mon[39722]: pgmap v11: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 75 MiB/s wr, 393 op/s 2024-04-12T05:56:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:34 smithi008 ceph-mon[39722]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:35.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:34 smithi194 ceph-mon[49771]: pgmap v11: 65 pgs: 65 active+clean; 2.1 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 75 MiB/s wr, 393 op/s 2024-04-12T05:56:35.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:34 smithi194 ceph-mon[49771]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:56:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:36 smithi008 ceph-mon[39722]: pgmap v12: 65 pgs: 65 active+clean; 1.7 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 32 MiB/s rd, 76 MiB/s wr, 566 op/s 2024-04-12T05:56:37.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:36 smithi194 ceph-mon[49771]: pgmap v12: 65 pgs: 65 active+clean; 1.7 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 32 MiB/s rd, 76 MiB/s wr, 566 op/s 2024-04-12T05:56:38.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:38.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: pgmap v13: 65 pgs: 65 active+clean; 1.7 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 320 op/s 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:56:38.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:38 smithi194 ceph-mon[49771]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.conf 2024-04-12T05:56:38.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.conf 2024-04-12T05:56:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: pgmap v13: 65 pgs: 65 active+clean; 1.7 GiB data, 5.9 GiB used, 531 GiB / 536 GiB avail; 3.8 MiB/s rd, 9.9 MiB/s wr, 320 op/s 2024-04-12T05:56:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:56:38.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:38 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.conf 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:39 smithi194 ceph-mon[49771]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: Updating smithi008:/etc/ceph/ceph.client.admin.keyring 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: Updating smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: Updating smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/config/ceph.client.admin.keyring 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:39 smithi008 ceph-mon[39722]: Reconfiguring prometheus.smithi008 (dependencies changed)... 2024-04-12T05:56:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:40 smithi194 ceph-mon[49771]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:56:41.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:40 smithi194 ceph-mon[49771]: pgmap v14: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 454 op/s 2024-04-12T05:56:41.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:40 smithi008 ceph-mon[39722]: Reconfiguring daemon prometheus.smithi008 on smithi008 2024-04-12T05:56:41.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:40 smithi008 ceph-mon[39722]: pgmap v14: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 4.4 MiB/s rd, 10 MiB/s wr, 454 op/s 2024-04-12T05:56:43.012 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:42 smithi008 ceph-mon[39722]: pgmap v15: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 307 op/s 2024-04-12T05:56:43.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:42 smithi194 ceph-mon[49771]: pgmap v15: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 307 op/s 2024-04-12T05:56:43.726 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-12T05:56:43.726 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-12T05:56:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:44 smithi194 ceph-mon[49771]: pgmap v16: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 307 op/s 2024-04-12T05:56:45.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:44 smithi008 ceph-mon[39722]: pgmap v16: 65 pgs: 65 active+clean; 1.1 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 307 op/s 2024-04-12T05:56:47.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:46 smithi194 ceph-mon[49771]: pgmap v17: 65 pgs: 65 active+clean; 524 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 416 op/s 2024-04-12T05:56:47.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:46 smithi008 ceph-mon[39722]: pgmap v17: 65 pgs: 65 active+clean; 524 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 416 op/s 2024-04-12T05:56:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:48.072 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:56:49.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:48 smithi194 ceph-mon[49771]: pgmap v18: 65 pgs: 65 active+clean; 524 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-04-12T05:56:49.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:48 smithi008 ceph-mon[39722]: pgmap v18: 65 pgs: 65 active+clean; 524 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 243 op/s 2024-04-12T05:56:50.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: pgmap v19: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-04-12T05:56:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:56:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:56:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: pgmap v19: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 325 op/s 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:56:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:56:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:52 smithi194 ceph-mon[49771]: pgmap v20: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-12T05:56:52.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:52 smithi008 ceph-mon[39722]: pgmap v20: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-12T05:56:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:54 smithi194 ceph-mon[49771]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-12T05:56:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:56:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:56:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:54 smithi008 ceph-mon[39722]: pgmap v21: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-12T05:56:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:54 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi194.xbiyaj", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-12T05:56:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:54 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-12T05:56:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:54 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:56:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:55 smithi194 ceph-mon[49771]: Upgrade: Updating mgr.smithi194.xbiyaj 2024-04-12T05:56:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:55 smithi194 ceph-mon[49771]: Deploying daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:56:55.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:55 smithi008 ceph-mon[39722]: Upgrade: Updating mgr.smithi194.xbiyaj 2024-04-12T05:56:55.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:55 smithi008 ceph-mon[39722]: Deploying daemon mgr.smithi194.xbiyaj on smithi194 2024-04-12T05:56:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:56 smithi194 ceph-mon[49771]: pgmap v22: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 433 op/s 2024-04-12T05:56:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:56 smithi008 ceph-mon[39722]: pgmap v22: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 433 op/s 2024-04-12T05:56:58.813 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:56:58 smithi194 ceph-mon[49771]: pgmap v23: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2024-04-12T05:56:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:56:58 smithi008 ceph-mon[39722]: pgmap v23: 65 pgs: 65 active+clean; 295 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 323 op/s 2024-04-12T05:57:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:00 smithi194 ceph-mon[49771]: pgmap v24: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 544 op/s 2024-04-12T05:57:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:00 smithi008 ceph-mon[39722]: pgmap v24: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 544 op/s 2024-04-12T05:57:00.969 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-12T05:57:00.969 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-12T05:57:01.276 DEBUG:teuthology.parallel:result is None 2024-04-12T05:57:02.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:02.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:02.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:02.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:01 smithi008 ceph-mon[39722]: pgmap v25: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 462 op/s 2024-04-12T05:57:02.274 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:02.275 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:02.275 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:02.275 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:01 smithi194 ceph-mon[49771]: pgmap v25: 65 pgs: 65 active+clean; 296 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 462 op/s 2024-04-12T05:57:03.201 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:02 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:03.287 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:02 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 -- 172.21.15.8:0/3972673723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98104050 msgr2=0x7f2e981044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/3972673723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98104050 0x7f2e981044d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009910 tx=0x7f2e84009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 -- 172.21.15.8:0/3972673723 shutdown_connections 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/3972673723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98104050 0x7f2e981044d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/3972673723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98102e60 0x7f2e98103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 -- 172.21.15.8:0/3972673723 >> 172.21.15.8:0/3972673723 conn(0x7f2e980fe470 msgr2=0x7f2e98100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 -- 172.21.15.8:0/3972673723 shutdown_connections 2024-04-12T05:57:03.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.772+0000 7f2e9ec31700 1 -- 172.21.15.8:0/3972673723 wait complete. 2024-04-12T05:57:03.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 Processor -- start 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 -- start start 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98104050 0x7f2e9819d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819d660 con 0x7f2e98102e60 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.773+0000 7f2e9ec31700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9819d7a0 con 0x7f2e98104050 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60822/0 (socket says 172.21.15.8:60822) 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 -- 172.21.15.8:0/39409311 learned_addr learned my addr 172.21.15.8:0/39409311 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 -- 172.21.15.8:0/39409311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98104050 msgr2=0x7f2e9819d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98104050 0x7f2e9819d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 -- 172.21.15.8:0/39409311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840095f0 con 0x7f2e98102e60 2024-04-12T05:57:03.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e9c9cd700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00b480 tx=0x7f2e8c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.774+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e8c0118c0 con 0x7f2e98102e60 2024-04-12T05:57:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.775+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e981a2250 con 0x7f2e98102e60 2024-04-12T05:57:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.775+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e981a27a0 con 0x7f2e98102e60 2024-04-12T05:57:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.775+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2e8c010380 con 0x7f2e98102e60 2024-04-12T05:57:03.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.775+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e8c00f510 con 0x7f2e98102e60 2024-04-12T05:57:03.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.776+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e7c005320 con 0x7f2e98102e60 2024-04-12T05:57:03.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.778+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f2e8c00f740 con 0x7f2e98102e60 2024-04-12T05:57:03.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.778+0000 7f2e95ffb700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 0x7f2e8806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:03.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.778+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2e8c08c230 con 0x7f2e98102e60 2024-04-12T05:57:03.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.781+0000 7f2e97fff700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 0x7f2e8806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:03.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.782+0000 7f2e97fff700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 0x7f2e8806f320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400b5c0 tx=0x7f2e84005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:03.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:03.784+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2e8c059cb0 con 0x7f2e98102e60 2024-04-12T05:57:04.052 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.057+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e7c000bf0 con 0x7f2e8806ce60 2024-04-12T05:57:04.053 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.058+0000 7f2e95ffb700 1 -- 172.21.15.8:0/39409311 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7f2e7c000bf0 con 0x7f2e8806ce60 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 msgr2=0x7f2e8806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 0x7f2e8806f320 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400b5c0 tx=0x7f2e84005fd0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 msgr2=0x7f2e9819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00b480 tx=0x7f2e8c00b840 comp rx=0 tx=0).stop 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 shutdown_connections 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e8806ce60 0x7f2e8806f320 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.055 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e98102e60 0x7f2e9819cb00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 --2- 172.21.15.8:0/39409311 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e98104050 0x7f2e9819d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 >> 172.21.15.8:0/39409311 conn(0x7f2e980fe470 msgr2=0x7f2e98100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 shutdown_connections 2024-04-12T05:57:04.056 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.060+0000 7f2e9ec31700 1 -- 172.21.15.8:0/39409311 wait complete. 2024-04-12T05:57:04.066 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:57:04.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 -- 172.21.15.8:0/397686483 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 msgr2=0x7ff8ac0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/397686483 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac0fffc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009910 tx=0x7ff89c009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:04.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 -- 172.21.15.8:0/397686483 shutdown_connections 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/397686483 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 0x7ff8ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/397686483 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac0fffc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.167+0000 7ff8b16ce700 1 -- 172.21.15.8:0/397686483 >> 172.21.15.8:0/397686483 conn(0x7ff8ac0fb120 msgr2=0x7ff8ac0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 -- 172.21.15.8:0/397686483 shutdown_connections 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 -- 172.21.15.8:0/397686483 wait complete. 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 Processor -- start 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 -- start start 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 0x7ff8ac196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac196f30 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.168+0000 7ff8b16ce700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac197070 con 0x7ff8ac100da0 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60846/0 (socket says 172.21.15.8:60846) 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 -- 172.21.15.8:0/3406322833 learned_addr learned my addr 172.21.15.8:0/3406322833 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:04.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aa7fc700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 0x7ff8ac196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 -- 172.21.15.8:0/3406322833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 msgr2=0x7ff8ac196910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 0x7ff8ac196910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 -- 172.21.15.8:0/3406322833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c0095f0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8aaffd700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff89c000c00 tx=0x7ff89c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c01d070 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac19bac0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7ff89c0059e0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff89c003980 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.169+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac19bf50 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.170+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac1085c0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.171+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7ff89c0041f0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.171+0000 7ff893fff700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 0x7ff89406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.172+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff89c08de50 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.172+0000 7ff8aa7fc700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 0x7ff89406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.167 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.172+0000 7ff8aa7fc700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 0x7ff89406f2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009c60 tx=0x7ff8a0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.176+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff89c05b7e0 con 0x7ff8ac0ffba0 2024-04-12T05:57:04.173 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:03 smithi194 ceph-mon[49771]: pgmap v26: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 489 op/s 2024-04-12T05:57:04.243 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:03 smithi008 ceph-mon[39722]: pgmap v26: 65 pgs: 65 active+clean; 292 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 489 op/s 2024-04-12T05:57:04.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.403+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8ac19c620 con 0x7ff89406ce20 2024-04-12T05:57:04.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.404+0000 7ff893fff700 1 -- 172.21.15.8:0/3406322833 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7ff8ac19c620 con 0x7ff89406ce20 2024-04-12T05:57:04.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 msgr2=0x7ff89406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 0x7ff89406f2e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009c60 tx=0x7ff8a0009260 comp rx=0 tx=0).stop 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 msgr2=0x7ff8ac1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff89c000c00 tx=0x7ff89c00bde0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 shutdown_connections 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff89406ce20 0x7ff89406f2e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8ac0ffba0 0x7ff8ac1963d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 --2- 172.21.15.8:0/3406322833 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8ac100da0 0x7ff8ac196910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 >> 172.21.15.8:0/3406322833 conn(0x7ff8ac0fb120 msgr2=0x7ff8ac068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 shutdown_connections 2024-04-12T05:57:04.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.406+0000 7ff8b16ce700 1 -- 172.21.15.8:0/3406322833 wait complete. 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 -- 172.21.15.8:0/2785280286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a30100e10 msgr2=0x7f2a30101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 --2- 172.21.15.8:0/2785280286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a30100e10 0x7f2a30101270 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009910 tx=0x7f2a24009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 -- 172.21.15.8:0/2785280286 shutdown_connections 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 --2- 172.21.15.8:0/2785280286 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a30100e10 0x7f2a30101270 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 --2- 172.21.15.8:0/2785280286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a300ffc10 0x7f2a30100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 -- 172.21.15.8:0/2785280286 >> 172.21.15.8:0/2785280286 conn(0x7f2a300fb190 msgr2=0x7f2a300fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 -- 172.21.15.8:0/2785280286 shutdown_connections 2024-04-12T05:57:04.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.515+0000 7f2a3670d700 1 -- 172.21.15.8:0/2785280286 wait complete. 2024-04-12T05:57:04.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 Processor -- start 2024-04-12T05:57:04.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 -- start start 2024-04-12T05:57:04.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 0x7f2a3019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3019b6f0 con 0x7f2a300ffc10 2024-04-12T05:57:04.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a3670d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3019b830 con 0x7f2a30100e10 2024-04-12T05:57:04.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a2ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a2ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60866/0 (socket says 172.21.15.8:60866) 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a2ffff700 1 -- 172.21.15.8:0/3521203885 learned_addr learned my addr 172.21.15.8:0/3521203885 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.516+0000 7f2a2f7fe700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 0x7f2a3019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2ffff700 1 -- 172.21.15.8:0/3521203885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 msgr2=0x7f2a3019b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2ffff700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 0x7f2a3019b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2ffff700 1 -- 172.21.15.8:0/3521203885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a240095f0 con 0x7f2a300ffc10 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2f7fe700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 0x7f2a3019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2ffff700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000d900 tx=0x7f2a2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a200041f0 con 0x7f2a300ffc10 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f2a20004350 con 0x7f2a300ffc10 2024-04-12T05:57:04.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3019bb10 con 0x7f2a300ffc10 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a20003d20 con 0x7f2a300ffc10 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.517+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3019c060 con 0x7f2a300ffc10 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.518+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a301085c0 con 0x7f2a300ffc10 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.519+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f2a20009730 con 0x7f2a300ffc10 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.519+0000 7f2a2d7fa700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 0x7f2a1806f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.519+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2a2008c8c0 con 0x7f2a300ffc10 2024-04-12T05:57:04.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.519+0000 7f2a2f7fe700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 0x7f2a1806f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.520+0000 7f2a2f7fe700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 0x7f2a1806f2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a30069660 tx=0x7f2a2400b560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.524+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f2a2005a250 con 0x7f2a300ffc10 2024-04-12T05:57:04.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.784+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a30105760 con 0x7f2a1806ce10 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (8m) 40s ago 10m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (10m) 40s ago 10m 7810k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (9m) 39s ago 9m 7742k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (10m) 40s ago 10m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (9m) 39s ago 9m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (8m) 40s ago 10m 88.9M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (4m) 40s ago 4m 188M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (4m) 40s ago 4m 121M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (4m) 39s ago 4m 45.2M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:57:04.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (4m) 39s ago 4m 107M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (63s) 40s ago 11m 473M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 starting - - - - 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (11m) 40s ago 12m 53.5M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (9m) 39s ago 9m 40.9M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (10m) 40s ago 10m 17.2M - 1.5.0 0da6a335fe13 fc6d88e7fa97 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (9m) 39s ago 9m 20.6M - 1.5.0 0da6a335fe13 8bf2920b113f 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (8m) 40s ago 8m 189M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (7m) 40s ago 7m 232M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (7m) 40s ago 7m 191M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (6m) 39s ago 6m 246M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (6m) 39s ago 6m 202M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (5m) 39s ago 5m 231M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:57:04.797 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 starting - - - - 2024-04-12T05:57:04.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.800+0000 7f2a2d7fa700 1 -- 172.21.15.8:0/3521203885 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2a30105760 con 0x7f2a1806ce10 2024-04-12T05:57:04.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.803+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 msgr2=0x7f2a1806f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.803+0000 7f2a3670d700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 0x7f2a1806f2d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2a30069660 tx=0x7f2a2400b560 comp rx=0 tx=0).stop 2024-04-12T05:57:04.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.803+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 msgr2=0x7f2a300690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.803+0000 7f2a3670d700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000d900 tx=0x7f2a2000dcc0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 shutdown_connections 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a1806ce10 0x7f2a1806f2d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a300ffc10 0x7f2a300690e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 --2- 172.21.15.8:0/3521203885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a30100e10 0x7f2a3019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 >> 172.21.15.8:0/3521203885 conn(0x7f2a300fb190 msgr2=0x7f2a30104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 shutdown_connections 2024-04-12T05:57:04.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.804+0000 7f2a3670d700 1 -- 172.21.15.8:0/3521203885 wait complete. 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 -- 172.21.15.8:0/2433695458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 msgr2=0x7f1208103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 --2- 172.21.15.8:0/2433695458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f1208103c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009910 tx=0x7f11f8009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 -- 172.21.15.8:0/2433695458 shutdown_connections 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 --2- 172.21.15.8:0/2433695458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 0x7f1208106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 --2- 172.21.15.8:0/2433695458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f1208103c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.934+0000 7f120c859700 1 -- 172.21.15.8:0/2433695458 >> 172.21.15.8:0/2433695458 conn(0x7f12080fb190 msgr2=0x7f12080fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 -- 172.21.15.8:0/2433695458 shutdown_connections 2024-04-12T05:57:04.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 -- 172.21.15.8:0/2433695458 wait complete. 2024-04-12T05:57:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 Processor -- start 2024-04-12T05:57:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 -- start start 2024-04-12T05:57:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 0x7f1208198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208199130 con 0x7f1208101820 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.935+0000 7f120c859700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1208199270 con 0x7f1208104150 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60888/0 (socket says 172.21.15.8:60888) 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 -- 172.21.15.8:0/2909884188 learned_addr learned my addr 172.21.15.8:0/2909884188 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f1205d9b700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 0x7f1208198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 -- 172.21.15.8:0/2909884188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 msgr2=0x7f1208198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 0x7f1208198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 -- 172.21.15.8:0/2909884188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80095f0 con 0x7f1208101820 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120659c700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f11f800bde0 tx=0x7f11f800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f801d070 con 0x7f1208101820 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f11f8005dc0 con 0x7f1208101820 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120819dcc0 con 0x7f1208101820 2024-04-12T05:57:04.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f8003b10 con 0x7f1208101820 2024-04-12T05:57:04.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.936+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120819e150 con 0x7f1208101820 2024-04-12T05:57:04.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.937+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12081096c0 con 0x7f1208101820 2024-04-12T05:57:04.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.938+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f11f8003c70 con 0x7f1208101820 2024-04-12T05:57:04.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.938+0000 7f11ff7fe700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 0x7f11f406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:04.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.938+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f11f808de20 con 0x7f1208101820 2024-04-12T05:57:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.938+0000 7f1205d9b700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 0x7f11f406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:04.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.939+0000 7f1205d9b700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 0x7f11f406f2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009c60 tx=0x7f11f0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:04.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:04.943+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f11f805b8a0 con 0x7f1208101820 2024-04-12T05:57:05.225 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:04 smithi008 ceph-mon[39722]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.225 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:04 smithi008 ceph-mon[39722]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.225 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:04 smithi008 ceph-mon[39722]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.225 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:04 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:05.225 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:04 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:05.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.249+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f120819e7c0 con 0x7f1208101820 2024-04-12T05:57:05.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.249+0000 7f11ff7fe700 1 -- 172.21.15.8:0/2909884188 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f11f8026070 con 0x7f1208101820 2024-04-12T05:57:05.244 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:57:05.245 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:57:05.246 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.251+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 msgr2=0x7f11f406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.251+0000 7f120c859700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 0x7f11f406f2e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009c60 tx=0x7f11f0009260 comp rx=0 tx=0).stop 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.251+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 msgr2=0x7f12081985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.251+0000 7f120c859700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f11f800bde0 tx=0x7f11f800be10 comp rx=0 tx=0).stop 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 shutdown_connections 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f11f406ce20 0x7f11f406f2e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1208101820 0x7f12081985d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 --2- 172.21.15.8:0/2909884188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1208104150 0x7f1208198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 >> 172.21.15.8:0/2909884188 conn(0x7f12080fb190 msgr2=0x7f12080ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 shutdown_connections 2024-04-12T05:57:05.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.252+0000 7f120c859700 1 -- 172.21.15.8:0/2909884188 wait complete. 2024-04-12T05:57:05.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:04 smithi194 ceph-mon[49771]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:04 smithi194 ceph-mon[49771]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:04 smithi194 ceph-mon[49771]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:05.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:05.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 -- 172.21.15.8:0/751266810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 msgr2=0x7fab1c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 --2- 172.21.15.8:0/751266810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c0ff9a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fab0c009910 tx=0x7fab0c009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 -- 172.21.15.8:0/751266810 shutdown_connections 2024-04-12T05:57:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 --2- 172.21.15.8:0/751266810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 0x7fab1c100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 --2- 172.21.15.8:0/751266810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c0ff9a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 -- 172.21.15.8:0/751266810 >> 172.21.15.8:0/751266810 conn(0x7fab1c0fb120 msgr2=0x7fab1c0fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 -- 172.21.15.8:0/751266810 shutdown_connections 2024-04-12T05:57:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.364+0000 7fab20eec700 1 -- 172.21.15.8:0/751266810 wait complete. 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 Processor -- start 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 -- start start 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 0x7fab1c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c199110 con 0x7fab1c0ff580 2024-04-12T05:57:05.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab20eec700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c199250 con 0x7fab1c0ffee0 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60904/0 (socket says 172.21.15.8:60904) 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 -- 172.21.15.8:0/3726415755 learned_addr learned my addr 172.21.15.8:0/3726415755 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab19d9b700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 0x7fab1c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 -- 172.21.15.8:0/3726415755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 msgr2=0x7fab1c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 0x7fab1c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.365+0000 7fab1a59c700 1 -- 172.21.15.8:0/3726415755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0095f0 con 0x7fab1c0ff580 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab1a59c700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00bde0 tx=0x7fab0c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c01d070 con 0x7fab1c0ff580 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fab0c005dc0 con 0x7fab1c0ff580 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c19dca0 con 0x7fab1c0ff580 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0c003b10 con 0x7fab1c0ff580 2024-04-12T05:57:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.366+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c19e130 con 0x7fab1c0ff580 2024-04-12T05:57:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.367+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c10b8c0 con 0x7fab1c0ff580 2024-04-12T05:57:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.367+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 30) v1 ==== 90942+0+0 (secure 0 0 0) 0x7fab0c0048e0 con 0x7fab1c0ff580 2024-04-12T05:57:05.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.368+0000 7fab0b7fe700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 0x7fab0406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.368+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fab0c08ded0 con 0x7fab1c0ff580 2024-04-12T05:57:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.368+0000 7fab19d9b700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 0x7fab0406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:05.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.368+0000 7fab19d9b700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 0x7fab0406f2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab10009c60 tx=0x7fab10009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:05.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.372+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fab0c05b860 con 0x7fab1c0ff580 2024-04-12T05:57:05.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.612+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab1c1092d0 con 0x7fab0406ce20 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.613+0000 7fab0b7fe700 1 -- 172.21.15.8:0/3726415755 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fab1c1092d0 con 0x7fab0406ce20 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr" 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T05:57:05.608 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T05:57:05.609 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mgr daemons", 2024-04-12T05:57:05.609 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:57:05.609 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:57:05.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 msgr2=0x7fab0406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 0x7fab0406f2e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab10009c60 tx=0x7fab10009260 comp rx=0 tx=0).stop 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 msgr2=0x7fab1c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00bde0 tx=0x7fab0c00be10 comp rx=0 tx=0).stop 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 shutdown_connections 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fab0406ce20 0x7fab0406f2e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab1c0ff580 0x7fab1c1985b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 --2- 172.21.15.8:0/3726415755 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab1c0ffee0 0x7fab1c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 >> 172.21.15.8:0/3726415755 conn(0x7fab1c0fb120 msgr2=0x7fab1c107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 shutdown_connections 2024-04-12T05:57:05.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:05.615+0000 7fab20eec700 1 -- 172.21.15.8:0/3726415755 wait complete. 2024-04-12T05:57:06.218 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:05 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2909884188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:06.219 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:05 smithi194 ceph-mon[49771]: pgmap v27: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 687 op/s 2024-04-12T05:57:06.219 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:05 smithi194 ceph-mon[49771]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:06.219 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:05 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:06.219 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:05 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:06.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:05 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2909884188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:06.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:05 smithi008 ceph-mon[39722]: pgmap v27: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 687 op/s 2024-04-12T05:57:06.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:05 smithi008 ceph-mon[39722]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:06.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:05 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:06.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:05 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:08.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:08 smithi194 ceph-mon[49771]: pgmap v28: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 445 op/s 2024-04-12T05:57:08.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:08 smithi008 ceph-mon[39722]: pgmap v28: 65 pgs: 65 active+clean; 288 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 445 op/s 2024-04-12T05:57:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: pgmap v29: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-04-12T05:57:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj restarted 2024-04-12T05:57:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:57:10.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:57:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:57:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:57:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:57:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:10.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:10 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:10.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: pgmap v29: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 586 op/s 2024-04-12T05:57:10.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj restarted 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: Standby manager daemon smithi194.xbiyaj started 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/crt"}]: dispatch 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi194.xbiyaj/key"}]: dispatch 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.? 172.21.15.194:0/713725785' entity='mgr.smithi194.xbiyaj' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:10 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: mgrmap e31: smithi008.ixmnlp(active, since 53s), standbys: smithi194.xbiyaj 2024-04-12T05:57:11.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:57:11.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all mgr 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi008.ixmnlp"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi008.ixmnlp"}]': finished 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi194.xbiyaj"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi194.xbiyaj"}]': finished 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all rgw 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all iscsi 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all nfs 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all nvmeof 2024-04-12T05:57:11.655 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:11 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: mgrmap e31: smithi008.ixmnlp(active, since 53s), standbys: smithi194.xbiyaj 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all mgr 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi008.ixmnlp"}]: dispatch 2024-04-12T05:57:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi008.ixmnlp"}]': finished 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi194.xbiyaj"}]: dispatch 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi194.xbiyaj"}]': finished 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all rgw 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all iscsi 2024-04-12T05:57:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all nfs 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all nvmeof 2024-04-12T05:57:11.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:11 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:12.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:12 smithi008 ceph-mon[39722]: pgmap v30: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-04-12T05:57:13.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:12 smithi194 ceph-mon[49771]: pgmap v30: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-04-12T05:57:14.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:13 smithi008 ceph-mon[39722]: Upgrade: Updating node-exporter.smithi008 (1/2) 2024-04-12T05:57:14.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:13 smithi008 ceph-mon[39722]: Deploying daemon node-exporter.smithi008 on smithi008 2024-04-12T05:57:14.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:13 smithi194 ceph-mon[49771]: Upgrade: Updating node-exporter.smithi008 (1/2) 2024-04-12T05:57:14.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:13 smithi194 ceph-mon[49771]: Deploying daemon node-exporter.smithi008 on smithi008 2024-04-12T05:57:15.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:14 smithi008 ceph-mon[39722]: pgmap v31: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-04-12T05:57:15.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:14 smithi194 ceph-mon[49771]: pgmap v31: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 365 op/s 2024-04-12T05:57:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:16 smithi008 ceph-mon[39722]: pgmap v32: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 473 op/s 2024-04-12T05:57:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:16 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:16.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:16 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:16.787 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:16 smithi194 ceph-mon[49771]: pgmap v32: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 473 op/s 2024-04-12T05:57:16.787 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:16 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:16.787 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:16 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:18.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:18 smithi008 ceph-mon[39722]: Upgrade: Updating node-exporter.smithi194 (2/2) 2024-04-12T05:57:18.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:18 smithi008 ceph-mon[39722]: Deploying daemon node-exporter.smithi194 on smithi194 2024-04-12T05:57:18.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:18 smithi008 ceph-mon[39722]: pgmap v33: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 987 KiB/s wr, 275 op/s 2024-04-12T05:57:18.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:18 smithi194 ceph-mon[49771]: Upgrade: Updating node-exporter.smithi194 (2/2) 2024-04-12T05:57:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:18 smithi194 ceph-mon[49771]: Deploying daemon node-exporter.smithi194 on smithi194 2024-04-12T05:57:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:18 smithi194 ceph-mon[49771]: pgmap v33: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 952 KiB/s rd, 987 KiB/s wr, 275 op/s 2024-04-12T05:57:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:20.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:20 smithi008 ceph-mon[39722]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2024-04-12T05:57:20.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:20 smithi194 ceph-mon[49771]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2024-04-12T05:57:22.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:22 smithi194 ceph-mon[49771]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 945 KiB/s rd, 892 KiB/s wr, 248 op/s 2024-04-12T05:57:22.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:22 smithi008 ceph-mon[39722]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 945 KiB/s rd, 892 KiB/s wr, 248 op/s 2024-04-12T05:57:24.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:24 smithi194 ceph-mon[49771]: pgmap v36: 65 pgs: 65 active+clean; 284 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-04-12T05:57:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:24 smithi008 ceph-mon[39722]: pgmap v36: 65 pgs: 65 active+clean; 284 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 264 op/s 2024-04-12T05:57:26.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:26 smithi194 ceph-mon[49771]: pgmap v37: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 328 op/s 2024-04-12T05:57:26.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:26 smithi008 ceph-mon[39722]: pgmap v37: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 328 op/s 2024-04-12T05:57:28.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:28 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:28.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:28 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:28.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:28 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:28.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:28 smithi194 ceph-mon[49771]: pgmap v38: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 784 KiB/s rd, 739 KiB/s wr, 194 op/s 2024-04-12T05:57:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:28 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:28 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:28 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:28 smithi008 ceph-mon[39722]: pgmap v38: 65 pgs: 65 active+clean; 282 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 784 KiB/s rd, 739 KiB/s wr, 194 op/s 2024-04-12T05:57:30.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:30 smithi194 ceph-mon[49771]: pgmap v39: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-04-12T05:57:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:30 smithi008 ceph-mon[39722]: pgmap v39: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-04-12T05:57:31.075 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-12T05:57:31.076 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-12T05:57:31.365 DEBUG:teuthology.parallel:result is None 2024-04-12T05:57:31.366 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-12T05:57:31.406 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-12T05:57:31.406 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-12T05:57:31.450 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-12T05:57:31.451 DEBUG:teuthology.parallel:result is None 2024-04-12T05:57:32.248 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:31 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.248 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:31 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.249 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:31 smithi194 ceph-mon[49771]: pgmap v40: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 621 KiB/s rd, 685 KiB/s wr, 189 op/s 2024-04-12T05:57:32.249 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:31 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.249 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:31 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:31 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:31 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:31 smithi008 ceph-mon[39722]: pgmap v40: 65 pgs: 65 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 621 KiB/s rd, 685 KiB/s wr, 189 op/s 2024-04-12T05:57:32.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:31 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:32.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:31 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:33.275 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:33.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:34.294 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:33 smithi194 ceph-mon[49771]: pgmap v41: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 KiB/s rd, 819 KiB/s wr, 209 op/s 2024-04-12T05:57:34.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:33 smithi008 ceph-mon[39722]: pgmap v41: 65 pgs: 65 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 KiB/s rd, 819 KiB/s wr, 209 op/s 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- 172.21.15.8:0/44922538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 msgr2=0x7f46e4103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/44922538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e4103b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009910 tx=0x7f46e0009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- 172.21.15.8:0/44922538 shutdown_connections 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/44922538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 0x7f46e41064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/44922538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e4103b90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- 172.21.15.8:0/44922538 >> 172.21.15.8:0/44922538 conn(0x7f46e40fb150 msgr2=0x7f46e40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- 172.21.15.8:0/44922538 shutdown_connections 2024-04-12T05:57:35.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- 172.21.15.8:0/44922538 wait complete. 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 Processor -- start 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.738+0000 7f46ebb3c700 1 -- start start 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46ebb3c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46ebb3c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 0x7f46e4196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46ebb3c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4196f20 con 0x7f46e41017a0 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46ebb3c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4197060 con 0x7f46e41040d0 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44418/0 (socket says 172.21.15.8:44418) 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 -- 172.21.15.8:0/522387003 learned_addr learned my addr 172.21.15.8:0/522387003 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e90d7700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 0x7f46e4196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 -- 172.21.15.8:0/522387003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 msgr2=0x7f46e4196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 0x7f46e4196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 -- 172.21.15.8:0/522387003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00095f0 con 0x7f46e41017a0 2024-04-12T05:57:35.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46e98d8700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006010 tx=0x7f46e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:35.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.739+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e001d070 con 0x7f46e41017a0 2024-04-12T05:57:35.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.740+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e419bab0 con 0x7f46e41017a0 2024-04-12T05:57:35.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.740+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f46e000f460 con 0x7f46e41017a0 2024-04-12T05:57:35.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.740+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e0021640 con 0x7f46e41017a0 2024-04-12T05:57:35.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.740+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e419bfa0 con 0x7f46e41017a0 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.741+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e41096d0 con 0x7f46e41017a0 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.741+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 31) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f46e000f9b0 con 0x7f46e41017a0 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.742+0000 7f46daffd700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 0x7f46d006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.742+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f46e008d9a0 con 0x7f46e41017a0 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.742+0000 7f46e90d7700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 0x7f46d006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:35.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.742+0000 7f46e90d7700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 0x7f46d006f2e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f46d4005950 tx=0x7f46d400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:35.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.746+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f46e005b330 con 0x7f46e41017a0 2024-04-12T05:57:35.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.966+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46e419c5e0 con 0x7f46d006ce20 2024-04-12T05:57:35.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.967+0000 7f46daffd700 1 -- 172.21.15.8:0/522387003 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f46e419c5e0 con 0x7f46d006ce20 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 msgr2=0x7f46d006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 0x7f46d006f2e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f46d4005950 tx=0x7f46d400b410 comp rx=0 tx=0).stop 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 msgr2=0x7f46e41963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006010 tx=0x7f46e000bd70 comp rx=0 tx=0).stop 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 shutdown_connections 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46d006ce20 0x7f46d006f2e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46e41017a0 0x7f46e41963c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 --2- 172.21.15.8:0/522387003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46e41040d0 0x7f46e4196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:35.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 >> 172.21.15.8:0/522387003 conn(0x7f46e40fb150 msgr2=0x7f46e40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:35.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 shutdown_connections 2024-04-12T05:57:35.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:35.969+0000 7f46ebb3c700 1 -- 172.21.15.8:0/522387003 wait complete. 2024-04-12T05:57:35.974 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:57:36.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- 172.21.15.8:0/1257256475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 msgr2=0x7fa9e0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 --2- 172.21.15.8:0/1257256475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0101270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009910 tx=0x7fa9cc009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- 172.21.15.8:0/1257256475 shutdown_connections 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 --2- 172.21.15.8:0/1257256475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0101270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 --2- 172.21.15.8:0/1257256475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e0100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- 172.21.15.8:0/1257256475 >> 172.21.15.8:0/1257256475 conn(0x7fa9e00fb190 msgr2=0x7fa9e00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- 172.21.15.8:0/1257256475 shutdown_connections 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- 172.21.15.8:0/1257256475 wait complete. 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 Processor -- start 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.077+0000 7fa9e7226700 1 -- start start 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e7226700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e7226700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e7226700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0196f20 con 0x7fa9e0100e10 2024-04-12T05:57:36.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e7226700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0197060 con 0x7fa9e00ffc10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44436/0 (socket says 172.21.15.8:44436) 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 -- 172.21.15.8:0/254608714 learned_addr learned my addr 172.21.15.8:0/254608714 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 -- 172.21.15.8:0/254608714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 msgr2=0x7fa9e01963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e4fc2700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e01963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 -- 172.21.15.8:0/254608714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0095f0 con 0x7fa9e0100e10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e4fc2700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9dffff700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006010 tx=0x7fa9cc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9cc01d070 con 0x7fa9e0100e10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fa9cc00f460 con 0x7fa9e0100e10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e019bab0 con 0x7fa9e0100e10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.078+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9cc021640 con 0x7fa9e0100e10 2024-04-12T05:57:36.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.079+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e019bfa0 con 0x7fa9e0100e10 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.079+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e01085c0 con 0x7fa9e0100e10 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.080+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 31) v1 ==== 90942+0+0 (secure 0 0 0) 0x7fa9cc00f940 con 0x7fa9e0100e10 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.080+0000 7fa9ddffb700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 0x7fa9d006f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.080+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa9cc08d9a0 con 0x7fa9e0100e10 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.080+0000 7fa9e4fc2700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 0x7fa9d006f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.081+0000 7fa9e4fc2700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 0x7fa9d006f2e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005d90 tx=0x7fa9d4005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.085+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fa9cc05b3b0 con 0x7fa9e0100e10 2024-04-12T05:57:36.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.313+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e0068c90 con 0x7fa9d006ce20 2024-04-12T05:57:36.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.314+0000 7fa9ddffb700 1 -- 172.21.15.8:0/254608714 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fa9e0068c90 con 0x7fa9d006ce20 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.316+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 msgr2=0x7fa9d006f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.316+0000 7fa9e7226700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 0x7fa9d006f2e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005d90 tx=0x7fa9d4005d00 comp rx=0 tx=0).stop 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.316+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 msgr2=0x7fa9e0196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.316+0000 7fa9e7226700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc006010 tx=0x7fa9cc00be20 comp rx=0 tx=0).stop 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 shutdown_connections 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d006ce20 0x7fa9d006f2e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e00ffc10 0x7fa9e01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 --2- 172.21.15.8:0/254608714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e0100e10 0x7fa9e0196900 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 >> 172.21.15.8:0/254608714 conn(0x7fa9e00fb190 msgr2=0x7fa9e0068580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:36.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 shutdown_connections 2024-04-12T05:57:36.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.317+0000 7fa9e7226700 1 -- 172.21.15.8:0/254608714 wait complete. 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 -- 172.21.15.8:0/3728017228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 msgr2=0x7f59181064c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 --2- 172.21.15.8:0/3728017228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59181064c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5904009910 tx=0x7f5904009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 -- 172.21.15.8:0/3728017228 shutdown_connections 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 --2- 172.21.15.8:0/3728017228 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59181064c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 --2- 172.21.15.8:0/3728017228 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 0x7f5918103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 -- 172.21.15.8:0/3728017228 >> 172.21.15.8:0/3728017228 conn(0x7f59180fb190 msgr2=0x7f59180fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 -- 172.21.15.8:0/3728017228 shutdown_connections 2024-04-12T05:57:36.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.427+0000 7f591f117700 1 -- 172.21.15.8:0/3728017228 wait complete. 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 Processor -- start 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 -- start start 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 0x7f5918101100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180ffc90 con 0x7f59181040d0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f591f117700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59180ffdd0 con 0x7f59181017a0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f5917fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f5917fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44450/0 (socket says 172.21.15.8:44450) 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.428+0000 7f5917fff700 1 -- 172.21.15.8:0/133696309 learned_addr learned my addr 172.21.15.8:0/133696309 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f591ceb3700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 0x7f5918101100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5917fff700 1 -- 172.21.15.8:0/133696309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 msgr2=0x7f5918101100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5917fff700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 0x7f5918101100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5917fff700 1 -- 172.21.15.8:0/133696309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040095f0 con 0x7f59181040d0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5917fff700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5904006010 tx=0x7f590400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590401d070 con 0x7f59181040d0 2024-04-12T05:57:36.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5918100050 con 0x7f59181040d0 2024-04-12T05:57:36.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f590400f460 con 0x7f59181040d0 2024-04-12T05:57:36.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5904021640 con 0x7f59181040d0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.429+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5918100540 con 0x7f59181040d0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.430+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59181904d0 con 0x7f59181040d0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.431+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 31) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f590400f940 con 0x7f59181040d0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.431+0000 7f5915ffb700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 0x7f590806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.431+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f590408da40 con 0x7f59181040d0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.431+0000 7f591ceb3700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 0x7f590806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.432+0000 7f591ceb3700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 0x7f590806f2e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.437+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f590405b3b0 con 0x7f59181040d0 2024-04-12T05:57:36.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.656+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f59180611d0 con 0x7f590806ce20 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.667+0000 7f5915ffb700 1 -- 172.21.15.8:0/133696309 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f59180611d0 con 0x7f590806ce20 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (9m) 26s ago 10m 26.7M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (11m) 26s ago 11m 7860k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (9m) 5s ago 9m 7834k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (11m) 26s ago 11m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (9m) 5s ago 9m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (9m) 26s ago 10m 86.3M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:57:36.662 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (4m) 26s ago 4m 129M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (4m) 26s ago 4m 81.4M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (4m) 5s ago 4m 65.8M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (4m) 5s ago 4m 126M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (95s) 26s ago 12m 494M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (36s) 5s ago 9m 447M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (12m) 26s ago 12m 55.5M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (9m) 5s ago 9m 45.7M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 starting - - - - 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (12s) 5s ago 9m 3384k - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (8m) 26s ago 8m 232M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (8m) 26s ago 8m 230M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (7m) 26s ago 7m 183M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (7m) 5s ago 7m 260M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:57:36.663 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (6m) 5s ago 6m 226M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:57:36.664 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (6m) 5s ago 6m 208M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:57:36.664 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (47s) 26s ago 10m 48.5M - 2.43.0 a07b618ecd1d b69fc1c45f35 2024-04-12T05:57:36.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 msgr2=0x7f590806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 0x7f590806f2e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c0058e0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 msgr2=0x7f59180ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5904006010 tx=0x7f590400be20 comp rx=0 tx=0).stop 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 shutdown_connections 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ce20 0x7f590806f2e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59181017a0 0x7f5918101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 --2- 172.21.15.8:0/133696309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59181040d0 0x7f59180ff750 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 >> 172.21.15.8:0/133696309 conn(0x7f59180fb190 msgr2=0x7f59180fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 shutdown_connections 2024-04-12T05:57:36.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.669+0000 7f591f117700 1 -- 172.21.15.8:0/133696309 wait complete. 2024-04-12T05:57:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 -- 172.21.15.8:0/4004234425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101eb0 msgr2=0x7f5a90102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 --2- 172.21.15.8:0/4004234425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101eb0 0x7f5a90102310 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a80009910 tx=0x7f5a80009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 -- 172.21.15.8:0/4004234425 shutdown_connections 2024-04-12T05:57:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 --2- 172.21.15.8:0/4004234425 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90101eb0 0x7f5a90102310 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 --2- 172.21.15.8:0/4004234425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90100cb0 0x7f5a901010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.774+0000 7f5a9742e700 1 -- 172.21.15.8:0/4004234425 >> 172.21.15.8:0/4004234425 conn(0x7f5a900fc230 msgr2=0x7f5a900fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 -- 172.21.15.8:0/4004234425 shutdown_connections 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 -- 172.21.15.8:0/4004234425 wait complete. 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 Processor -- start 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 -- start start 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90101eb0 0x7f5a90196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90197030 con 0x7f5a90100cb0 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a9742e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a90197170 con 0x7f5a90101eb0 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a951ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a951ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44464/0 (socket says 172.21.15.8:44464) 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.775+0000 7f5a951ca700 1 -- 172.21.15.8:0/1769842906 learned_addr learned my addr 172.21.15.8:0/1769842906 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a949c9700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90101eb0 0x7f5a90196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a951ca700 1 -- 172.21.15.8:0/1769842906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90101eb0 msgr2=0x7f5a90196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a951ca700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90101eb0 0x7f5a90196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a951ca700 1 -- 172.21.15.8:0/1769842906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a800095f0 con 0x7f5a90100cb0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a951ca700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00b480 tx=0x7f5a8c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8c0118c0 con 0x7f5a90100cb0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f5a8c010380 con 0x7f5a90100cb0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9019bc20 con 0x7f5a90100cb0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8c00f510 con 0x7f5a90100cb0 2024-04-12T05:57:36.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.776+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9019c170 con 0x7f5a90100cb0 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.777+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a90109770 con 0x7f5a90100cb0 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.778+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 31) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f5a8c0104f0 con 0x7f5a90100cb0 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.778+0000 7f5a867fc700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 0x7f5a7c06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.778+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5a8c08c2e0 con 0x7f5a90100cb0 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.778+0000 7f5a949c9700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 0x7f5a7c06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:36.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.779+0000 7f5a949c9700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 0x7f5a7c06f2e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a80006010 tx=0x7f5a8000b580 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:36.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:36.782+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f5a8c059d80 con 0x7f5a90100cb0 2024-04-12T05:57:36.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:36 smithi194 ceph-mon[49771]: pgmap v42: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 785 KiB/s rd, 777 KiB/s wr, 282 op/s 2024-04-12T05:57:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:36 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:36 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:36 smithi008 ceph-mon[39722]: pgmap v42: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 785 KiB/s rd, 777 KiB/s wr, 282 op/s 2024-04-12T05:57:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:36 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:36 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:37.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.081+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a90109930 con 0x7f5a90100cb0 2024-04-12T05:57:37.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.081+0000 7f5a867fc700 1 -- 172.21.15.8:0/1769842906 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f5a8c0107a0 con 0x7f5a90100cb0 2024-04-12T05:57:37.076 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:57:37.076 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:57:37.077 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 msgr2=0x7f5a7c06f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 0x7f5a7c06f2e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a80006010 tx=0x7f5a8000b580 comp rx=0 tx=0).stop 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 msgr2=0x7f5a901964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00b480 tx=0x7f5a8c00b840 comp rx=0 tx=0).stop 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 shutdown_connections 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5a7c06ce20 0x7f5a7c06f2e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5a90100cb0 0x7f5a901964d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 --2- 172.21.15.8:0/1769842906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a90101eb0 0x7f5a90196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 >> 172.21.15.8:0/1769842906 conn(0x7f5a900fc230 msgr2=0x7f5a901050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 shutdown_connections 2024-04-12T05:57:37.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.083+0000 7f5a9742e700 1 -- 172.21.15.8:0/1769842906 wait complete. 2024-04-12T05:57:37.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 -- 172.21.15.8:0/2792900395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 msgr2=0x7f0f5c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 --2- 172.21.15.8:0/2792900395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c102310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009910 tx=0x7f0f50009c20 comp rx=0 tx=0).stop 2024-04-12T05:57:37.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 -- 172.21.15.8:0/2792900395 shutdown_connections 2024-04-12T05:57:37.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 --2- 172.21.15.8:0/2792900395 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c102310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 --2- 172.21.15.8:0/2792900395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 0x7f0f5c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 -- 172.21.15.8:0/2792900395 >> 172.21.15.8:0/2792900395 conn(0x7f0f5c0fc230 msgr2=0x7f0f5c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 -- 172.21.15.8:0/2792900395 shutdown_connections 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.190+0000 7f0f6345b700 1 -- 172.21.15.8:0/2792900395 wait complete. 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 Processor -- start 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 -- start start 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 0x7f0f5c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c196f40 con 0x7f0f5c101eb0 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f6345b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c197080 con 0x7f0f5c100cb0 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44480/0 (socket says 172.21.15.8:44480) 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 -- 172.21.15.8:0/1049111778 learned_addr learned my addr 172.21.15.8:0/1049111778 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:57:37.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f611f7700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 0x7f0f5c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 -- 172.21.15.8:0/1049111778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 msgr2=0x7f0f5c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 0x7f0f5c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.191+0000 7f0f609f6700 1 -- 172.21.15.8:0/1049111778 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500095f0 con 0x7f0f5c101eb0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f609f6700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006010 tx=0x7f0f5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f5001d070 con 0x7f0f5c101eb0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c19bad0 con 0x7f0f5c101eb0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f0f5000f460 con 0x7f0f5c101eb0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f50021640 con 0x7f0f5c101eb0 2024-04-12T05:57:37.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.192+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c19bfc0 con 0x7f0f5c101eb0 2024-04-12T05:57:37.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.193+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c1096d0 con 0x7f0f5c101eb0 2024-04-12T05:57:37.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.193+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 31) v1 ==== 90942+0+0 (secure 0 0 0) 0x7f0f5000f940 con 0x7f0f5c101eb0 2024-04-12T05:57:37.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.194+0000 7f0f4e7fc700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 0x7f0f4806f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:57:37.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.194+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0f5008da40 con 0x7f0f5c101eb0 2024-04-12T05:57:37.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.194+0000 7f0f611f7700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 0x7f0f4806f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:57:37.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.194+0000 7f0f611f7700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 0x7f0f4806f2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005950 tx=0x7f0f580058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:57:37.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.200+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f0f5005b3d0 con 0x7f0f5c101eb0 2024-04-12T05:57:37.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.432+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f5c106800 con 0x7f0f4806ce20 2024-04-12T05:57:37.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.433+0000 7f0f4e7fc700 1 -- 172.21.15.8:0/1049111778 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0f5c106800 con 0x7f0f4806ce20 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr" 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:57:37.428 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 msgr2=0x7f0f4806f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 0x7f0f4806f2e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005950 tx=0x7f0f580058e0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 msgr2=0x7f0f5c196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006010 tx=0x7f0f5000be20 comp rx=0 tx=0).stop 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 shutdown_connections 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f4806ce20 0x7f0f4806f2e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0f5c100cb0 0x7f0f5c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 --2- 172.21.15.8:0/1049111778 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0f5c101eb0 0x7f0f5c196920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 >> 172.21.15.8:0/1049111778 conn(0x7f0f5c0fc230 msgr2=0x7f0f5c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:57:37.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 shutdown_connections 2024-04-12T05:57:37.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:57:37.435+0000 7f0f6345b700 1 -- 172.21.15.8:0/1049111778 wait complete. 2024-04-12T05:57:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:37 smithi008 ceph-mon[39722]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:37 smithi008 ceph-mon[39722]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:37 smithi008 ceph-mon[39722]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:37 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/1769842906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:38.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:37 smithi194 ceph-mon[49771]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:38.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:37 smithi194 ceph-mon[49771]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:38.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:37 smithi194 ceph-mon[49771]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:38.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:37 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/1769842906' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:38 smithi008 ceph-mon[39722]: pgmap v43: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 665 KiB/s rd, 676 KiB/s wr, 218 op/s 2024-04-12T05:57:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:38 smithi008 ceph-mon[39722]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:38 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:38 smithi194 ceph-mon[49771]: pgmap v43: 65 pgs: 65 active+clean; 272 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 665 KiB/s rd, 676 KiB/s wr, 218 op/s 2024-04-12T05:57:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:38 smithi194 ceph-mon[49771]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:57:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:39.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:38 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:39.957 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:39.958 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:39 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:40.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:40.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:39 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:40 smithi194 ceph-mon[49771]: pgmap v44: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 696 KiB/s wr, 222 op/s 2024-04-12T05:57:41.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:40 smithi008 ceph-mon[39722]: pgmap v44: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 686 KiB/s rd, 696 KiB/s wr, 222 op/s 2024-04-12T05:57:42.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: pgmap v45: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 365 KiB/s rd, 299 KiB/s wr, 112 op/s 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: pgmap v45: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 365 KiB/s rd, 299 KiB/s wr, 112 op/s 2024-04-12T05:57:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:57:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:57:45.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:44 smithi008 ceph-mon[39722]: pgmap v46: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 365 KiB/s rd, 299 KiB/s wr, 112 op/s 2024-04-12T05:57:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:44 smithi194 ceph-mon[49771]: pgmap v46: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 365 KiB/s rd, 299 KiB/s wr, 112 op/s 2024-04-12T05:57:46.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:45 smithi194 ceph-mon[49771]: Upgrade: Updating prometheus.smithi008 2024-04-12T05:57:46.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:45 smithi194 ceph-mon[49771]: Deploying daemon prometheus.smithi008 on smithi008 2024-04-12T05:57:46.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:45 smithi008 ceph-mon[39722]: Upgrade: Updating prometheus.smithi008 2024-04-12T05:57:46.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:45 smithi008 ceph-mon[39722]: Deploying daemon prometheus.smithi008 on smithi008 2024-04-12T05:57:47.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:46 smithi008 ceph-mon[39722]: pgmap v47: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 165 KiB/s rd, 166 KiB/s wr, 95 op/s 2024-04-12T05:57:47.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:46 smithi194 ceph-mon[49771]: pgmap v47: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 165 KiB/s rd, 166 KiB/s wr, 95 op/s 2024-04-12T05:57:47.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:57:48.862 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:48 smithi008 ceph-mon[39722]: pgmap v48: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 20 KiB/s wr, 6 op/s 2024-04-12T05:57:49.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:48 smithi194 ceph-mon[49771]: pgmap v48: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 20 KiB/s wr, 6 op/s 2024-04-12T05:57:49.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:49 smithi194 ceph-mon[49771]: mgrmap e32: smithi008.ixmnlp(active, since 91s), standbys: smithi194.xbiyaj 2024-04-12T05:57:49.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:49 smithi008 ceph-mon[39722]: mgrmap e32: smithi008.ixmnlp(active, since 91s), standbys: smithi194.xbiyaj 2024-04-12T05:57:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: pgmap v49: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-04-12T05:57:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:50 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: pgmap v49: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 21 KiB/s wr, 8 op/s 2024-04-12T05:57:51.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:57:51.079 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:51.080 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:50 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:53.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:52 smithi194 ceph-mon[49771]: pgmap v50: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-12T05:57:53.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:52 smithi008 ceph-mon[39722]: pgmap v50: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-12T05:57:54.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:54 smithi194 ceph-mon[49771]: pgmap v51: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-12T05:57:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:54 smithi008 ceph-mon[39722]: pgmap v51: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-04-12T05:57:57.088 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:56 smithi008 ceph-mon[39722]: pgmap v52: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-12T05:57:57.089 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:56 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.089 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:56 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.089 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:56 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.089 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:56 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:56 smithi194 ceph-mon[49771]: pgmap v52: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.1 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-12T05:57:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:56 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:56 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:56 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:57.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:56 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:57:59.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:57:58 smithi008 ceph-mon[39722]: pgmap v53: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-12T05:57:59.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:57:58 smithi194 ceph-mon[49771]: pgmap v53: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: pgmap v54: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:00 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:01.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: pgmap v54: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-12T05:58:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:00 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:01.984 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:58:01.984 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:58:01.984 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:01.984 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:01.985 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:01 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:01 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:03.092 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:02 smithi008 ceph-mon[39722]: pgmap v55: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:03.092 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:02 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:03.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:02 smithi194 ceph-mon[49771]: pgmap v55: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:02 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:04.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:03 smithi008 ceph-mon[39722]: Upgrade: Updating alertmanager.smithi008 2024-04-12T05:58:04.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:03 smithi008 ceph-mon[39722]: Deploying daemon alertmanager.smithi008 on smithi008 2024-04-12T05:58:04.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:03 smithi194 ceph-mon[49771]: Upgrade: Updating alertmanager.smithi008 2024-04-12T05:58:04.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:03 smithi194 ceph-mon[49771]: Deploying daemon alertmanager.smithi008 on smithi008 2024-04-12T05:58:04.972 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:04 smithi008 ceph-mon[39722]: pgmap v56: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:05.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:04 smithi194 ceph-mon[49771]: pgmap v56: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:07.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:06 smithi008 ceph-mon[39722]: pgmap v57: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:07.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:06 smithi194 ceph-mon[49771]: pgmap v57: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- 172.21.15.8:0/3502908406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870104070 msgr2=0x7f1870106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 --2- 172.21.15.8:0/3502908406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870104070 0x7f1870106460 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1860009910 tx=0x7f1860009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- 172.21.15.8:0/3502908406 shutdown_connections 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 --2- 172.21.15.8:0/3502908406 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870104070 0x7f1870106460 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 --2- 172.21.15.8:0/3502908406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870101740 0x7f1870103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- 172.21.15.8:0/3502908406 >> 172.21.15.8:0/3502908406 conn(0x7f18700fb150 msgr2=0x7f18700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- 172.21.15.8:0/3502908406 shutdown_connections 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- 172.21.15.8:0/3502908406 wait complete. 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 Processor -- start 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- start start 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870104070 0x7f18701969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870197000 con 0x7f1870101740 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.554+0000 7f18784f0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1870197140 con 0x7f1870104070 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40558/0 (socket says 172.21.15.8:40558) 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 -- 172.21.15.8:0/1006105317 learned_addr learned my addr 172.21.15.8:0/1006105317 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f1875a8b700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870104070 0x7f18701969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 -- 172.21.15.8:0/1006105317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870104070 msgr2=0x7f18701969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870104070 0x7f18701969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 -- 172.21.15.8:0/1006105317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600095f0 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f187628c700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b4b0 tx=0x7f186c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f186c011800 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f186c011df0 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187019bbf0 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f186c010e40 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.555+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187019c140 con 0x7f1870101740 2024-04-12T05:58:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.556+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1870109750 con 0x7f1870101740 2024-04-12T05:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.557+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f186c010380 con 0x7f1870101740 2024-04-12T05:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.557+0000 7f18677fe700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 0x7f185c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.557+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f186c08c010 con 0x7f1870101740 2024-04-12T05:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.557+0000 7f1875a8b700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 0x7f185c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.558+0000 7f1875a8b700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 0x7f185c06f1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18600059d0 tx=0x7f1860005920 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:07.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.562+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f186c059c40 con 0x7f1870101740 2024-04-12T05:58:07.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.786+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1870104ce0 con 0x7f185c06cce0 2024-04-12T05:58:07.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.787+0000 7f18677fe700 1 -- 172.21.15.8:0/1006105317 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f1870104ce0 con 0x7f185c06cce0 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 msgr2=0x7f185c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 0x7f185c06f1a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f18600059d0 tx=0x7f1860005920 comp rx=0 tx=0).stop 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 msgr2=0x7f18701964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b4b0 tx=0x7f186c00b870 comp rx=0 tx=0).stop 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 shutdown_connections 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f185c06cce0 0x7f185c06f1a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1870101740 0x7f18701964a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 --2- 172.21.15.8:0/1006105317 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1870104070 0x7f18701969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 >> 172.21.15.8:0/1006105317 conn(0x7f18700fb150 msgr2=0x7f18700fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 shutdown_connections 2024-04-12T05:58:07.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.789+0000 7f18784f0700 1 -- 172.21.15.8:0/1006105317 wait complete. 2024-04-12T05:58:07.794 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:58:07.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- 172.21.15.8:0/3814463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc0ffc10 msgr2=0x7f26cc100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 --2- 172.21.15.8:0/3814463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc0ffc10 0x7f26cc100030 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009910 tx=0x7f26c0009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- 172.21.15.8:0/3814463141 shutdown_connections 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 --2- 172.21.15.8:0/3814463141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc100e10 0x7f26cc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 --2- 172.21.15.8:0/3814463141 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc0ffc10 0x7f26cc100030 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- 172.21.15.8:0/3814463141 >> 172.21.15.8:0/3814463141 conn(0x7f26cc0fb190 msgr2=0x7f26cc0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- 172.21.15.8:0/3814463141 shutdown_connections 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- 172.21.15.8:0/3814463141 wait complete. 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 Processor -- start 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.897+0000 7f26d414d700 1 -- start start 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc0ffc10 0x7f26cc0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc19b840 con 0x7f26cc100e10 2024-04-12T05:58:07.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc19b980 con 0x7f26cc0ffc10 2024-04-12T05:58:07.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40578/0 (socket says 172.21.15.8:40578) 2024-04-12T05:58:07.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 -- 172.21.15.8:0/3508907049 learned_addr learned my addr 172.21.15.8:0/3508907049 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:07.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d1ee9700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc0ffc10 0x7f26cc0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 -- 172.21.15.8:0/3508907049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc0ffc10 msgr2=0x7f26cc0690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:07.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc0ffc10 0x7f26cc0690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:07.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 -- 172.21.15.8:0/3508907049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00095f0 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d16e8700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b4b0 tx=0x7f26c800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c8011800 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f26c8011df0 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc19bc60 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c8010e70 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.898+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc19c1b0 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.899+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc108640 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.900+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f26c8010380 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.900+0000 7f26beffd700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 0x7f26b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.900+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f26c808bfe0 con 0x7f26cc100e10 2024-04-12T05:58:07.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.900+0000 7f26d1ee9700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 0x7f26b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:07.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.901+0000 7f26d1ee9700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 0x7f26b806f1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26c0006010 tx=0x7f26c000b580 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:07.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:07.905+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f26c8059c10 con 0x7f26cc100e10 2024-04-12T05:58:08.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.132+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26cc105760 con 0x7f26b806cce0 2024-04-12T05:58:08.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.133+0000 7f26beffd700 1 -- 172.21.15.8:0/3508907049 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f26cc105760 con 0x7f26b806cce0 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 msgr2=0x7f26b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 0x7f26b806f1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26c0006010 tx=0x7f26c000b580 comp rx=0 tx=0).stop 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 msgr2=0x7f26cc19b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f26c800b4b0 tx=0x7f26c800b870 comp rx=0 tx=0).stop 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 shutdown_connections 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26b806cce0 0x7f26b806f1a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26cc0ffc10 0x7f26cc0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 --2- 172.21.15.8:0/3508907049 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26cc100e10 0x7f26cc19b270 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 >> 172.21.15.8:0/3508907049 conn(0x7f26cc0fb190 msgr2=0x7f26cc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 shutdown_connections 2024-04-12T05:58:08.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.136+0000 7f26d414d700 1 -- 172.21.15.8:0/3508907049 wait complete. 2024-04-12T05:58:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- 172.21.15.8:0/3529818191 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 msgr2=0x7eff8c106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 --2- 172.21.15.8:0/3529818191 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c106460 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7eff88009910 tx=0x7eff88009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- 172.21.15.8:0/3529818191 shutdown_connections 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 --2- 172.21.15.8:0/3529818191 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c106460 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 --2- 172.21.15.8:0/3529818191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 0x7eff8c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- 172.21.15.8:0/3529818191 >> 172.21.15.8:0/3529818191 conn(0x7eff8c0fb150 msgr2=0x7eff8c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- 172.21.15.8:0/3529818191 shutdown_connections 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- 172.21.15.8:0/3529818191 wait complete. 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 Processor -- start 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.242+0000 7eff93aa7700 1 -- start start 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 0x7eff8c1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0ffc30 con 0x7eff8c104070 2024-04-12T05:58:08.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8c0ffd70 con 0x7eff8c101740 2024-04-12T05:58:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40604/0 (socket says 172.21.15.8:40604) 2024-04-12T05:58:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 -- 172.21.15.8:0/2601047405 learned_addr learned my addr 172.21.15.8:0/2601047405 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91843700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 0x7eff8c1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 -- 172.21.15.8:0/2601047405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 msgr2=0x7eff8c1010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 0x7eff8c1010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 -- 172.21.15.8:0/2601047405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff880095f0 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff91042700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff8800b5c0 tx=0x7eff8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8801d070 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7eff88022470 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c0ffff0 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8800f670 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.243+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8c1004e0 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.244+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8c109750 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.245+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7eff88022a30 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.245+0000 7eff7effd700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 0x7eff7806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.245+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7eff8808dc20 con 0x7eff8c104070 2024-04-12T05:58:08.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.245+0000 7eff91843700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 0x7eff7806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.246+0000 7eff91843700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 0x7eff7806f1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff80006fd0 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.250+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7eff8805b9a0 con 0x7eff8c104070 2024-04-12T05:58:08.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.466+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff8c104ce0 con 0x7eff7806cce0 2024-04-12T05:58:08.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.476+0000 7eff7effd700 1 -- 172.21.15.8:0/2601047405 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7eff8c104ce0 con 0x7eff7806cce0 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (9m) 12s ago 11m 27.0M - 0.25.0 c8568f914cd2 d20047aab769 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (11m) 12s ago 11m 7926k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (10m) 37s ago 10m 7834k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (11m) 12s ago 11m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (10m) 37s ago 10m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (9m) 12s ago 11m 87.5M - 9.4.7 954c08fa6188 b8572e37a524 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (5m) 12s ago 5m 122M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (5m) 12s ago 5m 71.3M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (5m) 37s ago 5m 65.8M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (5m) 37s ago 5m 126M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (2m) 12s ago 12m 501M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (68s) 37s ago 10m 447M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (13m) 12s ago 13m 57.0M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:58:08.472 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (10m) 37s ago 10m 45.7M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (53s) 12s ago 11m 8660k - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (44s) 37s ago 10m 3384k - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (9m) 12s ago 9m 239M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (8m) 12s ago 8m 239M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (8m) 12s ago 8m 195M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (7m) 37s ago 7m 260M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (7m) 37s ago 7m 226M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (6m) 37s ago 6m 208M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:58:08.473 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (18s) 12s ago 10m 39.1M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.478+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 msgr2=0x7eff7806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.478+0000 7eff93aa7700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 0x7eff7806f1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7eff80006fd0 tx=0x7eff80008040 comp rx=0 tx=0).stop 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.478+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 msgr2=0x7eff8c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.478+0000 7eff93aa7700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7eff8800b5c0 tx=0x7eff8800be20 comp rx=0 tx=0).stop 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 shutdown_connections 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7eff7806cce0 0x7eff7806f1a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7eff8c101740 0x7eff8c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 --2- 172.21.15.8:0/2601047405 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7eff8c104070 0x7eff8c0ff6f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 >> 172.21.15.8:0/2601047405 conn(0x7eff8c0fb150 msgr2=0x7eff8c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 shutdown_connections 2024-04-12T05:58:08.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.479+0000 7eff93aa7700 1 -- 172.21.15.8:0/2601047405 wait complete. 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 -- 172.21.15.8:0/833402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 msgr2=0x7f3ed01010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/833402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01010d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009910 tx=0x7f3ec4009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 -- 172.21.15.8:0/833402840 shutdown_connections 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/833402840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 0x7f3ed0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/833402840 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01010d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 -- 172.21.15.8:0/833402840 >> 172.21.15.8:0/833402840 conn(0x7f3ed00fc230 msgr2=0x7f3ed00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 -- 172.21.15.8:0/833402840 shutdown_connections 2024-04-12T05:58:08.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.585+0000 7f3ed4a32700 1 -- 172.21.15.8:0/833402840 wait complete. 2024-04-12T05:58:08.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 Processor -- start 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 -- start start 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 0x7f3ed0196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0196eb0 con 0x7f3ed0100cb0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ed4a32700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed0196ff0 con 0x7f3ed0101eb0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40622/0 (socket says 172.21.15.8:40622) 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 -- 172.21.15.8:0/2463632915 learned_addr learned my addr 172.21.15.8:0/2463632915 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ecdd9b700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 0x7f3ed0196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 -- 172.21.15.8:0/2463632915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 msgr2=0x7f3ed0196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 0x7f3ed0196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.586+0000 7f3ece59c700 1 -- 172.21.15.8:0/2463632915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40095f0 con 0x7f3ed0100cb0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ece59c700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400bda0 tx=0x7f3ec400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.583 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec401d070 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed019ba50 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f3ec400f460 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ec4017630 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.587+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed019bf10 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.588+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed01096d0 con 0x7f3ed0100cb0 2024-04-12T05:58:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.588+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f3ec400fa40 con 0x7f3ed0100cb0 2024-04-12T05:58:08.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.589+0000 7f3ebf7fe700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 0x7f3eb806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.589+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3ec408dab0 con 0x7f3ed0100cb0 2024-04-12T05:58:08.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.589+0000 7f3ecdd9b700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 0x7f3eb806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.589+0000 7f3ecdd9b700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 0x7f3eb806f1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000ac50 tx=0x7f3ec000a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.593+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f3ec405b660 con 0x7f3ed0100cb0 2024-04-12T05:58:08.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.883+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ed019c580 con 0x7f3ed0100cb0 2024-04-12T05:58:08.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.884+0000 7f3ebf7fe700 1 -- 172.21.15.8:0/2463632915 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f3ec405b1f0 con 0x7f3ed0100cb0 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:08.880 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:58:08.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 msgr2=0x7f3eb806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 0x7f3eb806f1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3ec000ac50 tx=0x7f3ec000a380 comp rx=0 tx=0).stop 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 msgr2=0x7f3ed01963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400bda0 tx=0x7f3ec400bdd0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 shutdown_connections 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb806cce0 0x7f3eb806f1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ed0100cb0 0x7f3ed01963e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 --2- 172.21.15.8:0/2463632915 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ed0101eb0 0x7f3ed0196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 >> 172.21.15.8:0/2463632915 conn(0x7f3ed00fc230 msgr2=0x7f3ed01050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 shutdown_connections 2024-04-12T05:58:08.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.886+0000 7f3ed4a32700 1 -- 172.21.15.8:0/2463632915 wait complete. 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.993+0000 7f67bfa36700 1 -- 172.21.15.8:0/121139702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 msgr2=0x7f67b8103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.993+0000 7f67bfa36700 1 --2- 172.21.15.8:0/121139702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8103b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009910 tx=0x7f67b4009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 -- 172.21.15.8:0/121139702 shutdown_connections 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 --2- 172.21.15.8:0/121139702 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 0x7f67b81064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 --2- 172.21.15.8:0/121139702 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8103b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 -- 172.21.15.8:0/121139702 >> 172.21.15.8:0/121139702 conn(0x7f67b80fb190 msgr2=0x7f67b80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 -- 172.21.15.8:0/121139702 shutdown_connections 2024-04-12T05:58:08.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 -- 172.21.15.8:0/121139702 wait complete. 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 Processor -- start 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.994+0000 7f67bfa36700 1 -- start start 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 0x7f67b81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8196fe0 con 0x7f67b81017a0 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b8197120 con 0x7f67b81040d0 2024-04-12T05:58:08.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40634/0 (socket says 172.21.15.8:40634) 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 -- 172.21.15.8:0/105462212 learned_addr learned my addr 172.21.15.8:0/105462212 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bcfd1700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 0x7f67b81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 -- 172.21.15.8:0/105462212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 msgr2=0x7f67b81969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 0x7f67b81969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 -- 172.21.15.8:0/105462212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40095f0 con 0x7f67b81017a0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bd7d2700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b401d070 con 0x7f67b81017a0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b819bb70 con 0x7f67b81017a0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7f67b4022470 con 0x7f67b81017a0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b400f690 con 0x7f67b81017a0 2024-04-12T05:58:08.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.995+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b819c060 con 0x7f67b81017a0 2024-04-12T05:58:08.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.997+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b8109740 con 0x7f67b81017a0 2024-04-12T05:58:08.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.997+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f67b4022980 con 0x7f67b81017a0 2024-04-12T05:58:08.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.997+0000 7f67ae7fc700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 0x7f67a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:08.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.998+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f67b408dc30 con 0x7f67b81017a0 2024-04-12T05:58:08.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.998+0000 7f67bcfd1700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 0x7f67a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:08.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:08.998+0000 7f67bcfd1700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 0x7f67a406f1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67a8006fd0 tx=0x7f67a8008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.004+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f67b405b900 con 0x7f67b81017a0 2024-04-12T05:58:09.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:08 smithi008 ceph-mon[39722]: pgmap v58: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:09.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:08 smithi008 ceph-mon[39722]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:09.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.234+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67b8104d40 con 0x7f67a406cce0 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.235+0000 7f67ae7fc700 1 -- 172.21.15.8:0/105462212 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f67b8104d40 con 0x7f67a406cce0 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-12T05:58:09.231 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr" 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "2/2 daemons upgraded", 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:58:09.232 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:58:09.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 msgr2=0x7f67a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:09.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 0x7f67a406f1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67a8006fd0 tx=0x7f67a8008040 comp rx=0 tx=0).stop 2024-04-12T05:58:09.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 msgr2=0x7f67b8196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006010 tx=0x7f67b400be20 comp rx=0 tx=0).stop 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 shutdown_connections 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a406cce0 0x7f67a406f1a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b81017a0 0x7f67b8196480 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 --2- 172.21.15.8:0/105462212 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b81040d0 0x7f67b81969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.237+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 >> 172.21.15.8:0/105462212 conn(0x7f67b80fb190 msgr2=0x7f67b80fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.238+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 shutdown_connections 2024-04-12T05:58:09.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:09.238+0000 7f67bfa36700 1 -- 172.21.15.8:0/105462212 wait complete. 2024-04-12T05:58:09.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:08 smithi194 ceph-mon[49771]: pgmap v58: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:09.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:08 smithi194 ceph-mon[49771]: from='client.14744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:10.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:09 smithi194 ceph-mon[49771]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:10.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:09 smithi194 ceph-mon[49771]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:10.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:09 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2463632915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:10.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:09 smithi008 ceph-mon[39722]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:10.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:09 smithi008 ceph-mon[39722]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:10.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:09 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2463632915' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:11.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:10 smithi008 ceph-mon[39722]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:11.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:10 smithi008 ceph-mon[39722]: pgmap v59: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:11.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:10 smithi194 ceph-mon[49771]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:11.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:10 smithi194 ceph-mon[49771]: pgmap v59: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:13.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:12 smithi008 ceph-mon[39722]: pgmap v60: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:13.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:12 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:13.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:12 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:13.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:12 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:13.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:12 smithi194 ceph-mon[49771]: pgmap v60: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:13.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:12 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:12 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:12 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:15.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:14 smithi194 ceph-mon[49771]: pgmap v61: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:15.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:14 smithi008 ceph-mon[39722]: pgmap v61: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:17.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:16 smithi008 ceph-mon[39722]: pgmap v62: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:17.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:16 smithi194 ceph-mon[49771]: pgmap v62: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:18.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:17 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:18.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:17 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:18.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:17 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:18.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:17 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:18.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:17 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:18.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:17 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:19.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:18 smithi008 ceph-mon[39722]: pgmap v63: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:19.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:19.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:18 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:19.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:18 smithi194 ceph-mon[49771]: pgmap v63: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:19.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:19.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:18 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:20 smithi008 ceph-mon[39722]: pgmap v64: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:21.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:20 smithi194 ceph-mon[49771]: pgmap v64: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:23.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: pgmap v65: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:22 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: pgmap v65: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:23.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:23.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:22 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:24.139 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:23 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:58:24.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:23 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-12T05:58:25.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:24 smithi008 ceph-mon[39722]: pgmap v66: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:25.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:24 smithi194 ceph-mon[49771]: pgmap v66: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:26.084 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:25 smithi008 ceph-mon[39722]: Upgrade: Updating grafana.smithi008 2024-04-12T05:58:26.084 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:25 smithi008 ceph-mon[39722]: Deploying daemon grafana.smithi008 on smithi008 2024-04-12T05:58:26.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:25 smithi194 ceph-mon[49771]: Upgrade: Updating grafana.smithi008 2024-04-12T05:58:26.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:25 smithi194 ceph-mon[49771]: Deploying daemon grafana.smithi008 on smithi008 2024-04-12T05:58:27.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:26 smithi008 ceph-mon[39722]: pgmap v67: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:27.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:26 smithi194 ceph-mon[49771]: pgmap v67: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:29.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:28 smithi008 ceph-mon[39722]: pgmap v68: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:29.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:28 smithi194 ceph-mon[49771]: pgmap v68: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:30 smithi008 ceph-mon[39722]: pgmap v69: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:30 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:30 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:30 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:30 smithi194 ceph-mon[49771]: pgmap v69: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:30 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:30 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:30 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:32.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:32 smithi194 ceph-mon[49771]: pgmap v70: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:32.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:32.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:32 smithi008 ceph-mon[39722]: pgmap v70: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:32.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:34 smithi008 ceph-mon[39722]: pgmap v71: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:34.951 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:34 smithi194 ceph-mon[49771]: pgmap v71: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:36.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:35 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:35 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:35 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:35 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:35 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:35 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:35 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:36.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:35 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:37.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:36 smithi008 ceph-mon[39722]: pgmap v72: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:37.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:36 smithi194 ceph-mon[49771]: pgmap v72: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:39.304 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:38 smithi008 ceph-mon[39722]: pgmap v73: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:39.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:38 smithi194 ceph-mon[49771]: pgmap v73: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:39.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/3079538494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 msgr2=0x7fb2b8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/3079538494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b8103b30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009910 tx=0x7fb2a8009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/3079538494 shutdown_connections 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/3079538494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b8106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/3079538494 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b8103b30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/3079538494 >> 172.21.15.8:0/3079538494 conn(0x7fb2b80fb150 msgr2=0x7fb2b80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/3079538494 shutdown_connections 2024-04-12T05:58:39.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.364+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/3079538494 wait complete. 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 Processor -- start 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 -- start start 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b81968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8196e70 con 0x7fb2b8101740 2024-04-12T05:58:39.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2bcdeb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8196fb0 con 0x7fb2b8104070 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37126/0 (socket says 172.21.15.8:37126) 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 -- 172.21.15.8:0/2990162440 learned_addr learned my addr 172.21.15.8:0/2990162440 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b5d9b700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b81968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 -- 172.21.15.8:0/2990162440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 msgr2=0x7fb2b81968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b81968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b659c700 1 -- 172.21.15.8:0/2990162440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80095f0 con 0x7fb2b8101740 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.365+0000 7fb2b5d9b700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b81968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2b659c700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b5c0 tx=0x7fb2a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:39.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a801d070 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1066+0+0 (secure 0 0 0) 0x7fb2a8022470 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b819ba10 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2a800f670 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.366+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b819bed0 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.367+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fb2a8022a30 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.367+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b81904e0 con 0x7fb2b8101740 2024-04-12T05:58:39.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.368+0000 7fb2a77fe700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 0x7fb2a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.368+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb2a808dcb0 con 0x7fb2b8101740 2024-04-12T05:58:39.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.368+0000 7fb2b5d9b700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 0x7fb2a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.368+0000 7fb2b5d9b700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 0x7fb2a006f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005fd0 tx=0x7fb2ac005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:39.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.373+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fb2a805bab0 con 0x7fb2b8101740 2024-04-12T05:58:39.604 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.610+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2b80611d0 con 0x7fb2a006cce0 2024-04-12T05:58:39.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.611+0000 7fb2a77fe700 1 -- 172.21.15.8:0/2990162440 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fb2b80611d0 con 0x7fb2a006cce0 2024-04-12T05:58:39.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 msgr2=0x7fb2a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 0x7fb2a006f1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac005fd0 tx=0x7fb2ac005e20 comp rx=0 tx=0).stop 2024-04-12T05:58:39.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 msgr2=0x7fb2b81963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb2a800b5c0 tx=0x7fb2a800be20 comp rx=0 tx=0).stop 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 shutdown_connections 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2a006cce0 0x7fb2a006f1a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2b8101740 0x7fb2b81963a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 --2- 172.21.15.8:0/2990162440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2b8104070 0x7fb2b81968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 >> 172.21.15.8:0/2990162440 conn(0x7fb2b80fb150 msgr2=0x7fb2b80fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:39.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 shutdown_connections 2024-04-12T05:58:39.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.613+0000 7fb2bcdeb700 1 -- 172.21.15.8:0/2990162440 wait complete. 2024-04-12T05:58:39.618 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:58:39.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 -- 172.21.15.8:0/1970936947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281017a0 msgr2=0x7f7428103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 --2- 172.21.15.8:0/1970936947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281017a0 0x7f7428103b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7424009910 tx=0x7f7424009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:39.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 -- 172.21.15.8:0/1970936947 shutdown_connections 2024-04-12T05:58:39.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 --2- 172.21.15.8:0/1970936947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281040d0 0x7f74281064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 --2- 172.21.15.8:0/1970936947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281017a0 0x7f7428103b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.732+0000 7f742f33f700 1 -- 172.21.15.8:0/1970936947 >> 172.21.15.8:0/1970936947 conn(0x7f74280fb190 msgr2=0x7f74280fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 -- 172.21.15.8:0/1970936947 shutdown_connections 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 -- 172.21.15.8:0/1970936947 wait complete. 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 Processor -- start 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 -- start start 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281017a0 0x7f7428196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428196ef0 con 0x7f74281040d0 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742f33f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428197030 con 0x7f74281017a0 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.733+0000 7f742c8da700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37142/0 (socket says 172.21.15.8:37142) 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 -- 172.21.15.8:0/2905157723 learned_addr learned my addr 172.21.15.8:0/2905157723 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742d0db700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281017a0 0x7f7428196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 -- 172.21.15.8:0/2905157723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281017a0 msgr2=0x7f7428196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281017a0 0x7f7428196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 -- 172.21.15.8:0/2905157723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240095f0 con 0x7f74281040d0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742c8da700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f741800e920 tx=0x7f741800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f741800c9d0 con 0x7f74281040d0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7418010430 con 0x7f74281040d0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742819bae0 con 0x7f74281040d0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7418018650 con 0x7f74281040d0 2024-04-12T05:58:39.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.734+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742819c030 con 0x7f74281040d0 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.735+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74281904d0 con 0x7f74281040d0 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.736+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f741800cb30 con 0x7f74281040d0 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.736+0000 7f741e7fc700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 0x7f741406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.736+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7418014070 con 0x7f74281040d0 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.736+0000 7f742d0db700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 0x7f741406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:39.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.737+0000 7f742d0db700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 0x7f741406f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7424009fd0 tx=0x7f742400b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:39.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.740+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f74180529b0 con 0x7f74281040d0 2024-04-12T05:58:39.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.963+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74280611d0 con 0x7f741406cce0 2024-04-12T05:58:39.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.964+0000 7f741e7fc700 1 -- 172.21.15.8:0/2905157723 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f74280611d0 con 0x7f741406cce0 2024-04-12T05:58:39.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 msgr2=0x7f741406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 0x7f741406f1a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7424009fd0 tx=0x7f742400b5a0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 msgr2=0x7f74281968d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f741800e920 tx=0x7f741800ec30 comp rx=0 tx=0).stop 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 shutdown_connections 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f741406cce0 0x7f741406f1a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f74281017a0 0x7f7428196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 --2- 172.21.15.8:0/2905157723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f74281040d0 0x7f74281968d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 >> 172.21.15.8:0/2905157723 conn(0x7f74280fb190 msgr2=0x7f74280fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 shutdown_connections 2024-04-12T05:58:39.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:39.966+0000 7f742f33f700 1 -- 172.21.15.8:0/2905157723 wait complete. 2024-04-12T05:58:40.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 -- 172.21.15.8:0/1484536601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940ff610 msgr2=0x7f46940ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 --2- 172.21.15.8:0/1484536601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940ff610 0x7f46940ffa30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4684009910 tx=0x7f4684009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:40.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 -- 172.21.15.8:0/1484536601 shutdown_connections 2024-04-12T05:58:40.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 --2- 172.21.15.8:0/1484536601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940fff70 0x7f46941003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 --2- 172.21.15.8:0/1484536601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940ff610 0x7f46940ffa30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 -- 172.21.15.8:0/1484536601 >> 172.21.15.8:0/1484536601 conn(0x7f46940fb190 msgr2=0x7f46940fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 -- 172.21.15.8:0/1484536601 shutdown_connections 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.088+0000 7f4698fac700 1 -- 172.21.15.8:0/1484536601 wait complete. 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 Processor -- start 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 -- start start 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940ff610 0x7f46941985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694199150 con 0x7f46940fff70 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4698fac700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4694199290 con 0x7f46940ff610 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4691d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4691d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37162/0 (socket says 172.21.15.8:37162) 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f4691d9b700 1 -- 172.21.15.8:0/1950938616 learned_addr learned my addr 172.21.15.8:0/1950938616 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:40.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.089+0000 7f469259c700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940ff610 0x7f46941985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4691d9b700 1 -- 172.21.15.8:0/1950938616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940ff610 msgr2=0x7f46941985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4691d9b700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940ff610 0x7f46941985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4691d9b700 1 -- 172.21.15.8:0/1950938616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840095f0 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4691d9b700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f468800e920 tx=0x7f468800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f468800c9d0 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469419dd40 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4688010430 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4688018650 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.090+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469419e290 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.091+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469410b8b0 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.092+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f46880105a0 con 0x7f46940fff70 2024-04-12T05:58:40.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.092+0000 7f46837fe700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 0x7f467c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.092+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4688014070 con 0x7f46940fff70 2024-04-12T05:58:40.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.092+0000 7f469259c700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 0x7f467c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.092+0000 7f469259c700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 0x7f467c06f1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4684009fd0 tx=0x7f468400b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.096+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f46880529b0 con 0x7f46940fff70 2024-04-12T05:58:40.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.317+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4694109360 con 0x7f467c06cce0 2024-04-12T05:58:40.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.328+0000 7f46837fe700 1 -- 172.21.15.8:0/1950938616 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4694109360 con 0x7f467c06cce0 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (32s) 5s ago 11m 14.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (12m) 5s ago 12m 7986k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (11m) 69s ago 11m 7834k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (12m) 5s ago 12m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (10m) 69s ago 10m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (11s) 5s ago 11m 41.2M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (5m) 5s ago 5m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (5m) 5s ago 5m 71.3M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (5m) 69s ago 5m 65.8M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (5m) 69s ago 5m 126M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (2m) 5s ago 13m 503M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (100s) 69s ago 10m 447M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T05:58:40.323 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (13m) 5s ago 13m 57.8M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (10m) 69s ago 10m 45.7M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (85s) 5s ago 12m 9286k - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (76s) 69s ago 10m 3384k - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (9m) 5s ago 9m 240M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (9m) 5s ago 9m 240M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (8m) 5s ago 8m 196M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (8m) 69s ago 8m 260M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (7m) 69s ago 7m 226M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (7m) 69s ago 7m 208M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:58:40.324 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (50s) 5s ago 11m 46.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.329+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 msgr2=0x7f467c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 0x7f467c06f1a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4684009fd0 tx=0x7f468400b5a0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 msgr2=0x7f4694198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f468800e920 tx=0x7f468800ec30 comp rx=0 tx=0).stop 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 shutdown_connections 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f467c06cce0 0x7f467c06f1a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46940ff610 0x7f46941985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 --2- 172.21.15.8:0/1950938616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46940fff70 0x7f4694198b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 >> 172.21.15.8:0/1950938616 conn(0x7f46940fb190 msgr2=0x7f4694107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:40.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 shutdown_connections 2024-04-12T05:58:40.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.330+0000 7f4698fac700 1 -- 172.21.15.8:0/1950938616 wait complete. 2024-04-12T05:58:40.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 -- 172.21.15.8:0/1988282297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680fff50 msgr2=0x7f26681003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 --2- 172.21.15.8:0/1988282297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680fff50 0x7f26681003d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f265c009910 tx=0x7f265c009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:40.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 -- 172.21.15.8:0/1988282297 shutdown_connections 2024-04-12T05:58:40.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 --2- 172.21.15.8:0/1988282297 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680fff50 0x7f26681003d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 --2- 172.21.15.8:0/1988282297 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680ff5f0 0x7f26680ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 -- 172.21.15.8:0/1988282297 >> 172.21.15.8:0/1988282297 conn(0x7f26680fb190 msgr2=0x7f26680fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:40.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 -- 172.21.15.8:0/1988282297 shutdown_connections 2024-04-12T05:58:40.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.436+0000 7f266c9e2700 1 -- 172.21.15.8:0/1988282297 wait complete. 2024-04-12T05:58:40.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 Processor -- start 2024-04-12T05:58:40.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 -- start start 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680fff50 0x7f2668198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668199120 con 0x7f26680ff5f0 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266c9e2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668199260 con 0x7f26680fff50 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37176/0 (socket says 172.21.15.8:37176) 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f266659c700 1 -- 172.21.15.8:0/703745868 learned_addr learned my addr 172.21.15.8:0/703745868 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.437+0000 7f2665d9b700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680fff50 0x7f2668198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266659c700 1 -- 172.21.15.8:0/703745868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680fff50 msgr2=0x7f2668198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266659c700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680fff50 0x7f2668198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266659c700 1 -- 172.21.15.8:0/703745868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c0095f0 con 0x7f26680ff5f0 2024-04-12T05:58:40.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266659c700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f265800e920 tx=0x7f265800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f265800c9d0 con 0x7f26680ff5f0 2024-04-12T05:58:40.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266819dd10 con 0x7f26680ff5f0 2024-04-12T05:58:40.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2658010430 con 0x7f26680ff5f0 2024-04-12T05:58:40.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2658018650 con 0x7f26680ff5f0 2024-04-12T05:58:40.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.438+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266819e200 con 0x7f26680ff5f0 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.439+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266810b8b0 con 0x7f26680ff5f0 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.440+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f26580105a0 con 0x7f26680ff5f0 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.440+0000 7f26577fe700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 0x7f265006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.440+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2658014070 con 0x7f26680ff5f0 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.440+0000 7f2665d9b700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 0x7f265006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.440+0000 7f2665d9b700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 0x7f265006f1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f265c009fd0 tx=0x7f265c00b5a0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.444+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f26580529b0 con 0x7f26680ff5f0 2024-04-12T05:58:40.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.732+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f266819e810 con 0x7f26680ff5f0 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.733+0000 7f26577fe700 1 -- 172.21.15.8:0/703745868 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f265805a350 con 0x7f26680ff5f0 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:58:40.728 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:58:40.729 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:58:40.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.734+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 msgr2=0x7f265006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.734+0000 7f266c9e2700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 0x7f265006f1a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f265c009fd0 tx=0x7f265c00b5a0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.734+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 msgr2=0x7f26681985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.734+0000 7f266c9e2700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f265800e920 tx=0x7f265800ec30 comp rx=0 tx=0).stop 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 shutdown_connections 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f265006cce0 0x7f265006f1a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26680ff5f0 0x7f26681985c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 --2- 172.21.15.8:0/703745868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26680fff50 0x7f2668198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 >> 172.21.15.8:0/703745868 conn(0x7f26680fb190 msgr2=0x7f2668107c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 shutdown_connections 2024-04-12T05:58:40.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.735+0000 7f266c9e2700 1 -- 172.21.15.8:0/703745868 wait complete. 2024-04-12T05:58:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: pgmap v74: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: Upgrade: Finalizing container_image settings 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T05:58:40.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: Upgrade: Complete! 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:40 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- 172.21.15.8:0/2954259131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c101740 msgr2=0x7fe31c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 --2- 172.21.15.8:0/2954259131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c101740 0x7fe31c103b30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe30c009910 tx=0x7fe30c009c20 comp rx=0 tx=0).stop 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- 172.21.15.8:0/2954259131 shutdown_connections 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 --2- 172.21.15.8:0/2954259131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c104070 0x7fe31c106460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 --2- 172.21.15.8:0/2954259131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c101740 0x7fe31c103b30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- 172.21.15.8:0/2954259131 >> 172.21.15.8:0/2954259131 conn(0x7fe31c0fb150 msgr2=0x7fe31c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:40.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- 172.21.15.8:0/2954259131 shutdown_connections 2024-04-12T05:58:40.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- 172.21.15.8:0/2954259131 wait complete. 2024-04-12T05:58:40.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 Processor -- start 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.842+0000 7fe321ecb700 1 -- start start 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 0x7fe31c1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ffc30 con 0x7fe31c104070 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c0ffd70 con 0x7fe31c101740 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37186/0 (socket says 172.21.15.8:37186) 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 -- 172.21.15.8:0/4232305125 learned_addr learned my addr 172.21.15.8:0/4232305125 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31b7fe700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 0x7fe31c1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 -- 172.21.15.8:0/4232305125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 msgr2=0x7fe31c1010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 0x7fe31c1010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 -- 172.21.15.8:0/4232305125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0095f0 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31b7fe700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 0x7fe31c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe31affd700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe30400b4b0 tx=0x7fe30400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304011800 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe304011e00 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe304010e50 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c100050 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.843+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c19e4c0 con 0x7fe31c104070 2024-04-12T05:58:40.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.844+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe31c190560 con 0x7fe31c104070 2024-04-12T05:58:40.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.845+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fe304010380 con 0x7fe31c104070 2024-04-12T05:58:40.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.846+0000 7fe318ff9700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:58:40.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.846+0000 7fe31b7fe700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:58:40.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.846+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe30408c540 con 0x7fe31c104070 2024-04-12T05:58:40.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.846+0000 7fe31b7fe700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 0x7fe30806f1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe30c006010 tx=0x7fe30c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:58:40.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:40.850+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fe30405a150 con 0x7fe31c104070 2024-04-12T05:58:40.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: pgmap v74: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:40.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: Upgrade: Finalizing container_image settings 2024-04-12T05:58:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: Upgrade: Complete! 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:58:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:40 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:41.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.070+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe31c0611d0 con 0x7fe30806cce0 2024-04-12T05:58:41.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.071+0000 7fe318ff9700 1 -- 172.21.15.8:0/4232305125 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fe31c0611d0 con 0x7fe30806cce0 2024-04-12T05:58:41.066 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": null, 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": false, 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "", 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": null, 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "", 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:58:41.067 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:58:41.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 msgr2=0x7fe30806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 0x7fe30806f1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe30c006010 tx=0x7fe30c00b540 comp rx=0 tx=0).stop 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 msgr2=0x7fe31c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe30400b4b0 tx=0x7fe30400b870 comp rx=0 tx=0).stop 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 shutdown_connections 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe30806cce0 0x7fe30806f1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe31c101740 0x7fe31c1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 --2- 172.21.15.8:0/4232305125 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe31c104070 0x7fe31c0ff6f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.073+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 >> 172.21.15.8:0/4232305125 conn(0x7fe31c0fb150 msgr2=0x7fe31c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.074+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 shutdown_connections 2024-04-12T05:58:41.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:58:41.074+0000 7fe321ecb700 1 -- 172.21.15.8:0/4232305125 wait complete. 2024-04-12T05:58:41.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:41 smithi008 ceph-mon[39722]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:41.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:41 smithi008 ceph-mon[39722]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:41.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:41 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/703745868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:42.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:41 smithi194 ceph-mon[49771]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:42.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:41 smithi194 ceph-mon[49771]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:42.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:41 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/703745868' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:58:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:42 smithi008 ceph-mon[39722]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:42 smithi008 ceph-mon[39722]: pgmap v75: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:42.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:42 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:43.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:42 smithi194 ceph-mon[49771]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:58:43.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:42 smithi194 ceph-mon[49771]: pgmap v75: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:43.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:42 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:58:44.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:44 smithi008 ceph-mon[39722]: pgmap v76: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:45.055 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:44 smithi194 ceph-mon[49771]: pgmap v76: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:46.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:46 smithi008 ceph-mon[39722]: pgmap v77: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:47.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:46 smithi194 ceph-mon[49771]: pgmap v77: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:47.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:58:48.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:48 smithi008 ceph-mon[39722]: pgmap v78: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:49.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:48 smithi194 ceph-mon[49771]: pgmap v78: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:50 smithi008 ceph-mon[39722]: pgmap v79: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:50 smithi194 ceph-mon[49771]: pgmap v79: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:52.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:52 smithi008 ceph-mon[39722]: pgmap v80: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:53.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:52 smithi194 ceph-mon[49771]: pgmap v80: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:54 smithi008 ceph-mon[39722]: pgmap v81: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:55.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:54 smithi194 ceph-mon[49771]: pgmap v81: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:56 smithi008 ceph-mon[39722]: pgmap v82: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:57.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:56 smithi194 ceph-mon[49771]: pgmap v82: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:58:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:58:58 smithi008 ceph-mon[39722]: pgmap v83: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:58:59.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:58:58 smithi194 ceph-mon[49771]: pgmap v83: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:00 smithi008 ceph-mon[39722]: pgmap v84: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:01.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:00 smithi194 ceph-mon[49771]: pgmap v84: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:02 smithi008 ceph-mon[39722]: pgmap v85: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:02 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:03.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:02 smithi194 ceph-mon[49771]: pgmap v85: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:03.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:02 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:04 smithi008 ceph-mon[39722]: pgmap v86: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:05.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:04 smithi194 ceph-mon[49771]: pgmap v86: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:06 smithi008 ceph-mon[39722]: pgmap v87: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:07.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:06 smithi194 ceph-mon[49771]: pgmap v87: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:08.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:08 smithi008 ceph-mon[39722]: pgmap v88: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:09.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:08 smithi194 ceph-mon[49771]: pgmap v88: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:10 smithi008 ceph-mon[39722]: pgmap v89: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:11.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:10 smithi194 ceph-mon[49771]: pgmap v89: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:11.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1919016658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 msgr2=0x7f13e81010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:11.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1919016658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e81010d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009910 tx=0x7f13d8009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1919016658 shutdown_connections 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1919016658 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1919016658 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e81010d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1919016658 >> 172.21.15.8:0/1919016658 conn(0x7f13e80fc230 msgr2=0x7f13e80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1919016658 shutdown_connections 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.194+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1919016658 wait complete. 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 Processor -- start 2024-04-12T05:59:11.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 -- start start 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8196ee0 con 0x7f13e8100cb0 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13ee7d6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e8197020 con 0x7f13e8101eb0 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41004/0 (socket says 172.21.15.8:41004) 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 -- 172.21.15.8:0/1785946306 learned_addr learned my addr 172.21.15.8:0/1785946306 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e77fe700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:11.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 -- 172.21.15.8:0/1785946306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 msgr2=0x7f13e8196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e7fff700 1 -- 172.21.15.8:0/1785946306 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80095f0 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.195+0000 7f13e77fe700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13e7fff700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13d800bda0 tx=0x7f13d800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d801d070 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e819ba80 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13d8005940 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d800f460 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.196+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e819bf40 con 0x7f13e8100cb0 2024-04-12T05:59:11.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.197+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e81096d0 con 0x7f13e8100cb0 2024-04-12T05:59:11.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.197+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f13d8022850 con 0x7f13e8100cb0 2024-04-12T05:59:11.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.198+0000 7f13e57fa700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 0x7f13d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:11.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.198+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f13d808ddf0 con 0x7f13e8100cb0 2024-04-12T05:59:11.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.198+0000 7f13e77fe700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 0x7f13d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:11.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.198+0000 7f13e77fe700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 0x7f13d406f1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005fd0 tx=0x7f13d0005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:11.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.203+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f13d805bb70 con 0x7f13e8100cb0 2024-04-12T05:59:11.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.422+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e8106800 con 0x7f13d406cce0 2024-04-12T05:59:11.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.423+0000 7f13e57fa700 1 -- 172.21.15.8:0/1785946306 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f13e8106800 con 0x7f13d406cce0 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 msgr2=0x7f13d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 0x7f13d406f1a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f13d0005fd0 tx=0x7f13d0005e20 comp rx=0 tx=0).stop 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 msgr2=0x7f13e8196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13d800bda0 tx=0x7f13d800be80 comp rx=0 tx=0).stop 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 shutdown_connections 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13d406cce0 0x7f13d406f1a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13e8100cb0 0x7f13e8196410 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 --2- 172.21.15.8:0/1785946306 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13e8101eb0 0x7f13e8196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:11.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 >> 172.21.15.8:0/1785946306 conn(0x7f13e80fc230 msgr2=0x7f13e81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:11.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 shutdown_connections 2024-04-12T05:59:11.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:11.425+0000 7f13ee7d6700 1 -- 172.21.15.8:0/1785946306 wait complete. 2024-04-12T05:59:12.341 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-12T05:59:12.630 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:12.889 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:12 smithi008 ceph-mon[39722]: pgmap v90: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:12.889 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:12 smithi008 ceph-mon[39722]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:13.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:12 smithi194 ceph-mon[49771]: pgmap v90: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:13.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:12 smithi194 ceph-mon[49771]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:14.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 -- 172.21.15.8:0/1942391940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 msgr2=0x7fb16c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 --2- 172.21.15.8:0/1942391940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c105890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb154009910 tx=0x7fb154009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 -- 172.21.15.8:0/1942391940 shutdown_connections 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 --2- 172.21.15.8:0/1942391940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c105890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 --2- 172.21.15.8:0/1942391940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 -- 172.21.15.8:0/1942391940 >> 172.21.15.8:0/1942391940 conn(0x7fb16c0fc740 msgr2=0x7fb16c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 -- 172.21.15.8:0/1942391940 shutdown_connections 2024-04-12T05:59:14.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.569+0000 7fb171227700 1 -- 172.21.15.8:0/1942391940 wait complete. 2024-04-12T05:59:14.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.570+0000 7fb171227700 1 Processor -- start 2024-04-12T05:59:14.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.570+0000 7fb171227700 1 -- start start 2024-04-12T05:59:14.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.570+0000 7fb171227700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c19c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:14.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.570+0000 7fb171227700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:14.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb171227700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c19d240 con 0x7fb16c101840 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb171227700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c1966f0 con 0x7fb16c100e90 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36752/0 (socket says 172.21.15.8:36752) 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 -- 172.21.15.8:0/2977964170 learned_addr learned my addr 172.21.15.8:0/2977964170 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16ad9d700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c19c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 -- 172.21.15.8:0/2977964170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 msgr2=0x7fb16c19c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c19c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 -- 172.21.15.8:0/2977964170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1540095f0 con 0x7fb16c101840 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16ad9d700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb16a59c700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1540092e0 tx=0x7fb15400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb15401d070 con 0x7fb16c101840 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb15400f460 con 0x7fb16c101840 2024-04-12T05:59:14.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c196970 con 0x7fb16c101840 2024-04-12T05:59:14.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb154021600 con 0x7fb16c101840 2024-04-12T05:59:14.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.571+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c196e60 con 0x7fb16c101840 2024-04-12T05:59:14.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.572+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c1091e0 con 0x7fb16c101840 2024-04-12T05:59:14.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.573+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fb15400f5d0 con 0x7fb16c101840 2024-04-12T05:59:14.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.573+0000 7fb163fff700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 0x7fb15806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:14.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.573+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb15408d6f0 con 0x7fb16c101840 2024-04-12T05:59:14.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.573+0000 7fb16ad9d700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 0x7fb15806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:14.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.574+0000 7fb16ad9d700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 0x7fb15806f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb15c007900 tx=0x7fb15c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:14.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.578+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fb15405b2a0 con 0x7fb16c101840 2024-04-12T05:59:14.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.866+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb16c04f2e0 con 0x7fb16c101840 2024-04-12T05:59:14.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.866+0000 7fb163fff700 1 -- 172.21.15.8:0/2977964170 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fb15405ae30 con 0x7fb16c101840 2024-04-12T05:59:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 msgr2=0x7fb15806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 0x7fb15806f1a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb15c007900 tx=0x7fb15c008040 comp rx=0 tx=0).stop 2024-04-12T05:59:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 msgr2=0x7fb16c19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1540092e0 tx=0x7fb15400be20 comp rx=0 tx=0).stop 2024-04-12T05:59:14.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 shutdown_connections 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb15806cce0 0x7fb15806f1a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb16c100e90 0x7fb16c19c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 --2- 172.21.15.8:0/2977964170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb16c101840 0x7fb16c19cbb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 >> 172.21.15.8:0/2977964170 conn(0x7fb16c0fc740 msgr2=0x7fb16c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 shutdown_connections 2024-04-12T05:59:14.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:14.868+0000 7fb171227700 1 -- 172.21.15.8:0/2977964170 wait complete. 2024-04-12T05:59:14.874 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:59:14.889 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:14 smithi008 ceph-mon[39722]: pgmap v91: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:15.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:14 smithi194 ceph-mon[49771]: pgmap v91: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:15.620 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-12T05:59:15.863 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:15.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:15 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2977964170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:16.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:15 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2977964170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:16 smithi008 ceph-mon[39722]: pgmap v92: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:17.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:16 smithi194 ceph-mon[49771]: pgmap v92: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 -- 172.21.15.8:0/2031920509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc106cd0 msgr2=0x7fbabc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:17.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 --2- 172.21.15.8:0/2031920509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc106cd0 0x7fbabc1070b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009910 tx=0x7fbab8009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 -- 172.21.15.8:0/2031920509 shutdown_connections 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 --2- 172.21.15.8:0/2031920509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc068580 0x7fbabc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 --2- 172.21.15.8:0/2031920509 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc106cd0 0x7fbabc1070b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 -- 172.21.15.8:0/2031920509 >> 172.21.15.8:0/2031920509 conn(0x7fbabc075670 msgr2=0x7fbabc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 -- 172.21.15.8:0/2031920509 shutdown_connections 2024-04-12T05:59:17.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.770+0000 7fbac4772700 1 -- 172.21.15.8:0/2031920509 wait complete. 2024-04-12T05:59:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 Processor -- start 2024-04-12T05:59:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 -- start start 2024-04-12T05:59:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 0x7fbabc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc19d130 con 0x7fbabc068580 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac4772700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc196630 con 0x7fbabc106cd0 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36770/0 (socket says 172.21.15.8:36770) 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 -- 172.21.15.8:0/3362921542 learned_addr learned my addr 172.21.15.8:0/3362921542 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac1d0d700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 0x7fbabc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 -- 172.21.15.8:0/3362921542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 msgr2=0x7fbabc19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 0x7fbabc19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac250e700 1 -- 172.21.15.8:0/3362921542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab80095f0 con 0x7fbabc068580 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.771+0000 7fbac1d0d700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 0x7fbabc19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbac250e700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbab8006010 tx=0x7fbab800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab801d070 con 0x7fbabc068580 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbab800f460 con 0x7fbabc068580 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc1968b0 con 0x7fbabc068580 2024-04-12T05:59:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbab8021620 con 0x7fbabc068580 2024-04-12T05:59:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.772+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc196da0 con 0x7fbabc068580 2024-04-12T05:59:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.773+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc1091f0 con 0x7fbabc068580 2024-04-12T05:59:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.773+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fbab800fa40 con 0x7fbabc068580 2024-04-12T05:59:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.774+0000 7fbab37fe700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 0x7fbaa806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.774+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbab808d770 con 0x7fbabc068580 2024-04-12T05:59:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.774+0000 7fbac1d0d700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 0x7fbaa806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.774+0000 7fbac1d0d700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 0x7fbaa806f1a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005950 tx=0x7fbaac0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:17.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:17.778+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fbab805b280 con 0x7fbabc068580 2024-04-12T05:59:17.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:17 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:18.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.063+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbabc04f2e0 con 0x7fbabc068580 2024-04-12T05:59:18.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.063+0000 7fbab37fe700 1 -- 172.21.15.8:0/3362921542 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fbab808be60 con 0x7fbabc068580 2024-04-12T05:59:18.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 msgr2=0x7fbaa806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 0x7fbaa806f1a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005950 tx=0x7fbaac0058e0 comp rx=0 tx=0).stop 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 msgr2=0x7fbabc19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbab8006010 tx=0x7fbab800be20 comp rx=0 tx=0).stop 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 shutdown_connections 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbaa806cce0 0x7fbaa806f1a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbabc068580 0x7fbabc19c560 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 --2- 172.21.15.8:0/3362921542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc106cd0 0x7fbabc19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 >> 172.21.15.8:0/3362921542 conn(0x7fbabc075670 msgr2=0x7fbabc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 shutdown_connections 2024-04-12T05:59:18.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:18.065+0000 7fbac4772700 1 -- 172.21.15.8:0/3362921542 wait complete. 2024-04-12T05:59:18.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:17 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:18.072 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)" 2024-04-12T05:59:18.838 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:18 smithi008 ceph-mon[39722]: pgmap v93: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:18.839 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:18 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/3362921542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:18.864 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-12T05:59:19.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:18 smithi194 ceph-mon[49771]: pgmap v93: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:19.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:18 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/3362921542' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:19.106 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:20.785 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:20 smithi008 ceph-mon[39722]: pgmap v94: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:20.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:20 smithi194 ceph-mon[49771]: pgmap v94: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.048+0000 7fbc3c60e700 1 -- 172.21.15.8:0/2007329337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 msgr2=0x7fbc3410d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.048+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/2007329337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3410d290 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009910 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- 172.21.15.8:0/2007329337 shutdown_connections 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/2007329337 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3410d290 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/2007329337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc34068580 0x7fbc34068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- 172.21.15.8:0/2007329337 >> 172.21.15.8:0/2007329337 conn(0x7fbc34075670 msgr2=0x7fbc34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- 172.21.15.8:0/2007329337 shutdown_connections 2024-04-12T05:59:21.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- 172.21.15.8:0/2007329337 wait complete. 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 Processor -- start 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- start start 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc34068580 0x7fbc3419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3419f340 con 0x7fbc34068ea0 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.049+0000 7fbc3c60e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34198880 con 0x7fbc34068580 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36784/0 (socket says 172.21.15.8:36784) 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 -- 172.21.15.8:0/4084074751 learned_addr learned my addr 172.21.15.8:0/4084074751 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 -- 172.21.15.8:0/4084074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc34068580 msgr2=0x7fbc3419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc34068580 0x7fbc3419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 -- 172.21.15.8:0/4084074751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280095f0 con 0x7fbc34068ea0 2024-04-12T05:59:21.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc39ba9700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006010 tx=0x7fbc2800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2801d070 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc2800f460 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc34198b00 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc28021640 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.050+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc34198ff0 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.051+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3410aa00 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.052+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fbc2800fa20 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.052+0000 7fbc277fe700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 0x7fbc2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.053+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc2808dc00 con 0x7fbc34068ea0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.053+0000 7fbc3a3aa700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 0x7fbc2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.053+0000 7fbc3a3aa700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 0x7fbc2006f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005fd0 tx=0x7fbc30005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:21.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.056+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fbc2805b950 con 0x7fbc34068ea0 2024-04-12T05:59:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.342+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc34199df0 con 0x7fbc34068ea0 2024-04-12T05:59:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.342+0000 7fbc277fe700 1 -- 172.21.15.8:0/4084074751 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fbc2805b4e0 con 0x7fbc34068ea0 2024-04-12T05:59:21.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 msgr2=0x7fbc2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:21.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 0x7fbc2006f1a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc30005fd0 tx=0x7fbc30005d00 comp rx=0 tx=0).stop 2024-04-12T05:59:21.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 msgr2=0x7fbc3419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbc28006010 tx=0x7fbc2800be20 comp rx=0 tx=0).stop 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 shutdown_connections 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.344+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc2006cce0 0x7fbc2006f1a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.345+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc34068580 0x7fbc3419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.345+0000 7fbc3c60e700 1 --2- 172.21.15.8:0/4084074751 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc34068ea0 0x7fbc3419ed40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.345+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 >> 172.21.15.8:0/4084074751 conn(0x7fbc34075670 msgr2=0x7fbc340fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.345+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 shutdown_connections 2024-04-12T05:59:21.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:21.345+0000 7fbc3c60e700 1 -- 172.21.15.8:0/4084074751 wait complete. 2024-04-12T05:59:21.351 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:59:21.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:21 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/4084074751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:22.019 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-12T05:59:22.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:21 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/4084074751' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:22.262 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:22.807 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:22 smithi008 ceph-mon[39722]: pgmap v95: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:22 smithi194 ceph-mon[49771]: pgmap v95: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:24.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 -- 172.21.15.8:0/1820598206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 msgr2=0x7fa9e4100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1820598206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e4100180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009910 tx=0x7fa9d8009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 -- 172.21.15.8:0/1820598206 shutdown_connections 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1820598206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e4100750 0x7fa9e41047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1820598206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e4100180 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 -- 172.21.15.8:0/1820598206 >> 172.21.15.8:0/1820598206 conn(0x7fa9e40fb650 msgr2=0x7fa9e40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.225+0000 7fa9eafec700 1 -- 172.21.15.8:0/1820598206 shutdown_connections 2024-04-12T05:59:24.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 -- 172.21.15.8:0/1820598206 wait complete. 2024-04-12T05:59:24.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 Processor -- start 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 -- start start 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e4100750 0x7fa9e419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e419d2b0 con 0x7fa9e40ffda0 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9eafec700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e4196760 con 0x7fa9e4100750 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9e8d88700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9e8d88700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38998/0 (socket says 172.21.15.8:38998) 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.226+0000 7fa9e8d88700 1 -- 172.21.15.8:0/1022795472 learned_addr learned my addr 172.21.15.8:0/1022795472 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e8d88700 1 -- 172.21.15.8:0/1022795472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e4100750 msgr2=0x7fa9e419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e8d88700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e4100750 0x7fa9e419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:24.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e8d88700 1 -- 172.21.15.8:0/1022795472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80095f0 con 0x7fa9e40ffda0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e8d88700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8006010 tx=0x7fa9d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d801d070 con 0x7fa9e40ffda0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa9d800f460 con 0x7fa9e40ffda0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e41969e0 con 0x7fa9e40ffda0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9d8021640 con 0x7fa9e40ffda0 2024-04-12T05:59:24.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.227+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e4196ed0 con 0x7fa9e40ffda0 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.228+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e404f2e0 con 0x7fa9e40ffda0 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.229+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fa9d800fa60 con 0x7fa9e40ffda0 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.229+0000 7fa9e1ffb700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 0x7fa9d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.229+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa9d808d720 con 0x7fa9e40ffda0 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.229+0000 7fa9e3fff700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 0x7fa9d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:24.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.230+0000 7fa9e3fff700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 0x7fa9d406f1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4197bd0 tx=0x7fa9d0006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:24.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.234+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fa9d805b350 con 0x7fa9e40ffda0 2024-04-12T05:59:24.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:24.452+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9e41971e0 con 0x7fa9d406cce0 2024-04-12T05:59:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:24 smithi008 ceph-mon[39722]: pgmap v96: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:25.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:24 smithi194 ceph-mon[49771]: pgmap v96: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:25.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:25 smithi194 ceph-mon[49771]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:25.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:25 smithi008 ceph-mon[39722]: from='client.14800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:26.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.072+0000 7fa9e1ffb700 1 -- 172.21.15.8:0/1022795472 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5393 (secure 0 0 0) 0x7fa9e41971e0 con 0x7fa9d406cce0 2024-04-12T05:59:26.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 msgr2=0x7fa9d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:26.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 0x7fa9d406f1a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4197bd0 tx=0x7fa9d0006d20 comp rx=0 tx=0).stop 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 msgr2=0x7fa9e419c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8006010 tx=0x7fa9d800be20 comp rx=0 tx=0).stop 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 shutdown_connections 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa9d406cce0 0x7fa9d406f1a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa9e40ffda0 0x7fa9e419c6e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.074+0000 7fa9eafec700 1 --2- 172.21.15.8:0/1022795472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa9e4100750 0x7fa9e419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.075+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 >> 172.21.15.8:0/1022795472 conn(0x7fa9e40fb650 msgr2=0x7fa9e4104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.075+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 shutdown_connections 2024-04-12T05:59:26.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:26.075+0000 7fa9eafec700 1 -- 172.21.15.8:0/1022795472 wait complete. 2024-04-12T05:59:26.081 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:59:26.699 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:26 smithi008 ceph-mon[39722]: pgmap v97: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:26.725 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch ps' 2024-04-12T05:59:26.969 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:27.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:26 smithi194 ceph-mon[49771]: pgmap v97: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 -- 172.21.15.8:0/1513763631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 msgr2=0x7f884c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 --2- 172.21.15.8:0/1513763631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c1070b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f883c009910 tx=0x7f883c009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 -- 172.21.15.8:0/1513763631 shutdown_connections 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 --2- 172.21.15.8:0/1513763631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884c068580 0x7f884c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 --2- 172.21.15.8:0/1513763631 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c1070b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 -- 172.21.15.8:0/1513763631 >> 172.21.15.8:0/1513763631 conn(0x7f884c075670 msgr2=0x7f884c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:28.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 -- 172.21.15.8:0/1513763631 shutdown_connections 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 -- 172.21.15.8:0/1513763631 wait complete. 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.850+0000 7f8851dad700 1 Processor -- start 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8851dad700 1 -- start start 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8851dad700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884c068580 0x7f884c19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8851dad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8851dad700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c19d250 con 0x7f884c106cd0 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8851dad700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c196700 con 0x7f884c068580 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39020/0 (socket says 172.21.15.8:39020) 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 -- 172.21.15.8:0/1803115616 learned_addr learned my addr 172.21.15.8:0/1803115616 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 -- 172.21.15.8:0/1803115616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884c068580 msgr2=0x7f884c19c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884c068580 0x7f884c19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 -- 172.21.15.8:0/1803115616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883c0095f0 con 0x7f884c106cd0 2024-04-12T05:59:28.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f884affd700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f883400b4b0 tx=0x7f883400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.851+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8834011800 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.852+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8834011e00 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.852+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8834010e50 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.852+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c1969e0 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.852+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c196f30 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.853+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c1091d0 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.853+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f8834010380 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.853+0000 7f8848ff9700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 0x7f883806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.854+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f883408c0f0 con 0x7f884c106cd0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.854+0000 7f884b7fe700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 0x7f883806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.854+0000 7f884b7fe700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 0x7f883806f1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f883c006010 tx=0x7f883c00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:28.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:28.858+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f8834059ca0 con 0x7f884c106cd0 2024-04-12T05:59:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:28 smithi008 ceph-mon[39722]: pgmap v98: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:29.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.068+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f884c1971b0 con 0x7f883806cce0 2024-04-12T05:59:29.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:28 smithi194 ceph-mon[49771]: pgmap v98: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.079+0000 7f8848ff9700 1 -- 172.21.15.8:0/1803115616 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f884c1971b0 con 0x7f883806cce0 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (81s) 54s ago 12m 14.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (13m) 54s ago 13m 7986k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (11m) 118s ago 11m 7834k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (12m) 54s ago 12m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (11m) 118s ago 11m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (60s) 54s ago 12m 41.2M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T05:59:29.074 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (6m) 54s ago 6m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (6m) 54s ago 6m 71.3M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (6m) 118s ago 6m 65.8M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (6m) 118s ago 6m 126M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (3m) 54s ago 14m 503M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (2m) 118s ago 11m 447M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (14m) 54s ago 14m 57.8M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (11m) 118s ago 11m 45.7M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (2m) 54s ago 12m 9286k - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (2m) 118s ago 11m 3384k - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (10m) 54s ago 10m 240M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (10m) 54s ago 10m 240M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (9m) 54s ago 9m 196M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (9m) 118s ago 9m 260M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:59:29.075 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (8m) 118s ago 8m 226M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:59:29.076 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (8m) 118s ago 8m 208M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:59:29.076 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (99s) 54s ago 12m 46.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T05:59:29.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 msgr2=0x7f883806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:29.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 0x7f883806f1a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f883c006010 tx=0x7f883c00b540 comp rx=0 tx=0).stop 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 msgr2=0x7f884c19cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f883400b4b0 tx=0x7f883400b870 comp rx=0 tx=0).stop 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 shutdown_connections 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f883806cce0 0x7f883806f1a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f884c068580 0x7f884c19c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 --2- 172.21.15.8:0/1803115616 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f884c106cd0 0x7f884c19cbc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 >> 172.21.15.8:0/1803115616 conn(0x7f884c075670 msgr2=0x7f884c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 shutdown_connections 2024-04-12T05:59:29.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:29.081+0000 7f8851dad700 1 -- 172.21.15.8:0/1803115616 wait complete. 2024-04-12T05:59:29.803 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T05:59:29.804 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:59:29.805 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-12T05:59:30.051 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:30 smithi194 ceph-mon[49771]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:30 smithi194 ceph-mon[49771]: pgmap v99: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:31.039 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:30 smithi008 ceph-mon[39722]: from='client.14804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:31.040 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:30 smithi008 ceph-mon[39722]: pgmap v99: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:31.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- 172.21.15.8:0/3055835696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068ea0 msgr2=0x7f1bb810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:31.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/3055835696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068ea0 0x7f1bb810d290 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009910 tx=0x7f1ba4009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:31.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- 172.21.15.8:0/3055835696 shutdown_connections 2024-04-12T05:59:31.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/3055835696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068ea0 0x7f1bb810d290 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:31.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/3055835696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068580 0x7f1bb8068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- 172.21.15.8:0/3055835696 >> 172.21.15.8:0/3055835696 conn(0x7f1bb8075670 msgr2=0x7f1bb8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- 172.21.15.8:0/3055835696 shutdown_connections 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- 172.21.15.8:0/3055835696 wait complete. 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 Processor -- start 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.906+0000 7f1bbf00e700 1 -- start start 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbf00e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbf00e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068ea0 0x7f1bb819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbf00e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb819f3f0 con 0x7f1bb8068580 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbf00e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb81988a0 con 0x7f1bb8068ea0 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:31.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39032/0 (socket says 172.21.15.8:39032) 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 -- 172.21.15.8:0/729081578 learned_addr learned my addr 172.21.15.8:0/729081578 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 -- 172.21.15.8:0/729081578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068ea0 msgr2=0x7f1bb819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bb7fff700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068ea0 0x7f1bb819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068ea0 0x7f1bb819ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 -- 172.21.15.8:0/729081578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba40095f0 con 0x7f1bb8068580 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbcdaa700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00d8d0 tx=0x7f1bac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bac009940 con 0x7f1bb8068580 2024-04-12T05:59:31.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1bac010460 con 0x7f1bb8068580 2024-04-12T05:59:31.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.907+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb8198b80 con 0x7f1bb8068580 2024-04-12T05:59:31.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.908+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bac00f5f0 con 0x7f1bb8068580 2024-04-12T05:59:31.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.908+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81990d0 con 0x7f1bb8068580 2024-04-12T05:59:31.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.909+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb810aa00 con 0x7f1bb8068580 2024-04-12T05:59:31.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.910+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f1bac0105d0 con 0x7f1bb8068580 2024-04-12T05:59:31.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.910+0000 7f1bb5ffb700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 0x7f1ba806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:31.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.910+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1bac08c490 con 0x7f1bb8068580 2024-04-12T05:59:31.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.910+0000 7f1bb7fff700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 0x7f1ba806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:31.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.911+0000 7f1bb7fff700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 0x7f1ba806f1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b5c0 tx=0x7f1ba4005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:31.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:31.914+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f1bac05a170 con 0x7f1bb8068580 2024-04-12T05:59:32.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.113+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f1bb804f2e0 con 0x7f1bb8068580 2024-04-12T05:59:32.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.117+0000 7f1bb5ffb700 1 -- 172.21.15.8:0/729081578 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v45)=0 v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1bac010880 con 0x7f1bb8068580 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 msgr2=0x7f1ba806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 0x7f1ba806f1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400b5c0 tx=0x7f1ba4005fd0 comp rx=0 tx=0).stop 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 msgr2=0x7f1bb819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00d8d0 tx=0x7f1bac00dc90 comp rx=0 tx=0).stop 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 shutdown_connections 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba806cce0 0x7f1ba806f1a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bb8068580 0x7f1bb819e820 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 --2- 172.21.15.8:0/729081578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bb8068ea0 0x7f1bb819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:32.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 >> 172.21.15.8:0/729081578 conn(0x7f1bb8075670 msgr2=0x7f1bb80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:32.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 shutdown_connections 2024-04-12T05:59:32.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:32.120+0000 7f1bbf00e700 1 -- 172.21.15.8:0/729081578 wait complete. 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: pgmap v100: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/729081578' entity='client.admin' 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:32.803 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:32 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:32.825 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T05:59:32.825 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:59:32.826 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-12T05:59:33.067 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: pgmap v100: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/729081578' entity='client.admin' 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:33.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:32 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/2132028145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28101850 msgr2=0x7f6b281058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/2132028145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28101850 0x7f6b281058a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009910 tx=0x7f6b14009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/2132028145 shutdown_connections 2024-04-12T05:59:34.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/2132028145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28101850 0x7f6b281058a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/2132028145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28100ea0 0x7f6b28101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/2132028145 >> 172.21.15.8:0/2132028145 conn(0x7f6b280fc750 msgr2=0x7f6b280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/2132028145 shutdown_connections 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.991+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/2132028145 wait complete. 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 Processor -- start 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 -- start start 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 0x7f6b2819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2819f3f0 con 0x7f6b28100ea0 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2f0e1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b281988a0 con 0x7f6b28101850 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2ce7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:34.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2ce7d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34874/0 (socket says 172.21.15.8:34874) 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.992+0000 7f6b2ce7d700 1 -- 172.21.15.8:0/1880231035 learned_addr learned my addr 172.21.15.8:0/1880231035 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b27fff700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 0x7f6b2819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2ce7d700 1 -- 172.21.15.8:0/1880231035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 msgr2=0x7f6b2819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2ce7d700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 0x7f6b2819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2ce7d700 1 -- 172.21.15.8:0/1880231035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b140095f0 con 0x7f6b28100ea0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b27fff700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 0x7f6b2819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2ce7d700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00d900 tx=0x7f6b1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c004a00 con 0x7f6b28100ea0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b28198b80 con 0x7f6b28100ea0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6b1c005020 con 0x7f6b28100ea0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c009c00 con 0x7f6b28100ea0 2024-04-12T05:59:34.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.993+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b281990d0 con 0x7f6b28100ea0 2024-04-12T05:59:34.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.994+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b28105020 con 0x7f6b28100ea0 2024-04-12T05:59:34.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.995+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f6b1c005190 con 0x7f6b28100ea0 2024-04-12T05:59:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.995+0000 7f6b25ffb700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 0x7f6b1806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.995+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6b1c08c6b0 con 0x7f6b28100ea0 2024-04-12T05:59:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.995+0000 7f6b27fff700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 0x7f6b1806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:34.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:34.996+0000 7f6b27fff700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 0x7f6b1806f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b280fcee0 tx=0x7f6b1401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:34.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.000+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f6b1c05a260 con 0x7f6b28100ea0 2024-04-12T05:59:35.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:34 smithi194 ceph-mon[49771]: pgmap v101: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:35.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:34 smithi008 ceph-mon[39722]: pgmap v101: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:35.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.201+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f6b2804f2e0 con 0x7f6b28100ea0 2024-04-12T05:59:35.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.201+0000 7f6b25ffb700 1 -- 172.21.15.8:0/1880231035 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v45)=0 v45) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6b1c005440 con 0x7f6b28100ea0 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 msgr2=0x7f6b1806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 0x7f6b1806f1a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b280fcee0 tx=0x7f6b1401a040 comp rx=0 tx=0).stop 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 msgr2=0x7f6b2819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00d900 tx=0x7f6b1c00dcc0 comp rx=0 tx=0).stop 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 shutdown_connections 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6b1806cce0 0x7f6b1806f1a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6b28100ea0 0x7f6b2819e820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 --2- 172.21.15.8:0/1880231035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6b28101850 0x7f6b2819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 >> 172.21.15.8:0/1880231035 conn(0x7f6b280fc750 msgr2=0x7f6b280fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 shutdown_connections 2024-04-12T05:59:35.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:35.204+0000 7f6b2f0e1700 1 -- 172.21.15.8:0/1880231035 wait complete. 2024-04-12T05:59:35.890 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-12T05:59:36.133 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:36.704 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:36 smithi008 ceph-mon[39722]: pgmap v102: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:37.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:36 smithi194 ceph-mon[49771]: pgmap v102: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/1975389230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c4105bd0 msgr2=0x7fe0c4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/1975389230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c4105bd0 0x7fe0c4105fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009910 tx=0x7fe0b4009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/1975389230 shutdown_connections 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/1975389230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c40ffc20 0x7fe0c4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/1975389230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c4105bd0 0x7fe0c4105fb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/1975389230 >> 172.21.15.8:0/1975389230 conn(0x7fe0c4075670 msgr2=0x7fe0c4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:38.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/1975389230 shutdown_connections 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.077+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/1975389230 wait complete. 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 Processor -- start 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 -- start start 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c4105bd0 0x7fe0c419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c419d210 con 0x7fe0c40ffc20 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.078+0000 7fe0c9ff3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c41966c0 con 0x7fe0c4105bd0 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34888/0 (socket says 172.21.15.8:34888) 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 -- 172.21.15.8:0/2472787670 learned_addr learned my addr 172.21.15.8:0/2472787670 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 -- 172.21.15.8:0/2472787670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c4105bd0 msgr2=0x7fe0c419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c4105bd0 0x7fe0c419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 -- 172.21.15.8:0/2472787670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b40095f0 con 0x7fe0c40ffc20 2024-04-12T05:59:38.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c37fe700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006010 tx=0x7fe0b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0b401d070 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c4196940 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0b400f460 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0b4021640 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.079+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c4196e30 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.080+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c404f2e0 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.081+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fe0b400f9e0 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.081+0000 7fe0c0ff9700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 0x7fe0b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.081+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe0b408d840 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.081+0000 7fe0c2ffd700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 0x7fe0b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.082+0000 7fe0c2ffd700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 0x7fe0b006f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4197b30 tx=0x7fe0ac006d20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.086+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fe0b405b3f0 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.289+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fe0c4197050 con 0x7fe0c40ffc20 2024-04-12T05:59:38.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.289+0000 7fe0c0ff9700 1 -- 172.21.15.8:0/2472787670 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v45)=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7fe0b405af80 con 0x7fe0c40ffc20 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 msgr2=0x7fe0b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 0x7fe0b006f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4197b30 tx=0x7fe0ac006d20 comp rx=0 tx=0).stop 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 msgr2=0x7fe0c419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4006010 tx=0x7fe0b400be20 comp rx=0 tx=0).stop 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 shutdown_connections 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe0b006cce0 0x7fe0b006f1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe0c40ffc20 0x7fe0c419c640 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 --2- 172.21.15.8:0/2472787670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c4105bd0 0x7fe0c419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 >> 172.21.15.8:0/2472787670 conn(0x7fe0c4075670 msgr2=0x7fe0c41040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.291+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 shutdown_connections 2024-04-12T05:59:38.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:38.292+0000 7fe0c9ff3700 1 -- 172.21.15.8:0/2472787670 wait complete. 2024-04-12T05:59:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:38 smithi008 ceph-mon[39722]: pgmap v103: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:38.960 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-12T05:59:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:38 smithi194 ceph-mon[49771]: pgmap v103: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:39.199 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 -- 172.21.15.8:0/1383034334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40ffc20 msgr2=0x7f67b4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 --2- 172.21.15.8:0/1383034334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40ffc20 0x7f67b4100080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009910 tx=0x7f67a4009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 -- 172.21.15.8:0/1383034334 shutdown_connections 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 --2- 172.21.15.8:0/1383034334 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b40ffc20 0x7f67b4100080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 --2- 172.21.15.8:0/1383034334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b4105bd0 0x7f67b4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 -- 172.21.15.8:0/1383034334 >> 172.21.15.8:0/1383034334 conn(0x7f67b4075670 msgr2=0x7f67b4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 -- 172.21.15.8:0/1383034334 shutdown_connections 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.928+0000 7f67bc136700 1 -- 172.21.15.8:0/1383034334 wait complete. 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 Processor -- start 2024-04-12T05:59:40.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 -- start start 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b40ffc20 0x7f67b4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b41982a0 con 0x7f67b4105bd0 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67bc136700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b4198410 con 0x7f67b40ffc20 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34906/0 (socket says 172.21.15.8:34906) 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 -- 172.21.15.8:0/4107315932 learned_addr learned my addr 172.21.15.8:0/4107315932 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b9ed2700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b40ffc20 0x7f67b4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 -- 172.21.15.8:0/4107315932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b40ffc20 msgr2=0x7f67b4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b40ffc20 0x7f67b4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.929+0000 7f67b96d1700 1 -- 172.21.15.8:0/4107315932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a40095f0 con 0x7f67b4105bd0 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67b96d1700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67a4006010 tx=0x7f67a400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:40.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67a401d070 con 0x7f67b4105bd0 2024-04-12T05:59:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b41a07b0 con 0x7f67b4105bd0 2024-04-12T05:59:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f67a400f460 con 0x7f67b4105bd0 2024-04-12T05:59:40.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67a4021640 con 0x7f67b4105bd0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.930+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b41a0c20 con 0x7f67b4105bd0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.931+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b41080f0 con 0x7f67b4105bd0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.931+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f67a400f9e0 con 0x7f67b4105bd0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.932+0000 7f67aaffd700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 0x7f67a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.932+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f67a408d770 con 0x7f67b4105bd0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.932+0000 7f67b9ed2700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 0x7f67a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:40.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.932+0000 7f67b9ed2700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 0x7f67a006f1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009a10 tx=0x7f67b0009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:40.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:40.937+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f67a4057890 con 0x7f67b4105bd0 2024-04-12T05:59:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:40 smithi194 ceph-mon[49771]: pgmap v104: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:41.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.137+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f67b4107ad0 con 0x7f67b4105bd0 2024-04-12T05:59:41.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.138+0000 7f67aaffd700 1 -- 172.21.15.8:0/4107315932 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v45)=0 v45) v1 ==== 135+0+0 (secure 0 0 0) 0x7f67a405aeb0 con 0x7f67b4105bd0 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 msgr2=0x7f67a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 0x7f67a006f1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009a10 tx=0x7f67b0009190 comp rx=0 tx=0).stop 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 msgr2=0x7f67b4197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f67a4006010 tx=0x7f67a400bd70 comp rx=0 tx=0).stop 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 shutdown_connections 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f67a006cce0 0x7f67a006f1a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67b40ffc20 0x7f67b4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 --2- 172.21.15.8:0/4107315932 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f67b4105bd0 0x7f67b4197c80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 >> 172.21.15.8:0/4107315932 conn(0x7f67b4075670 msgr2=0x7f67b40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 shutdown_connections 2024-04-12T05:59:41.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:41.140+0000 7f67bc136700 1 -- 172.21.15.8:0/4107315932 wait complete. 2024-04-12T05:59:41.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:40 smithi008 ceph-mon[39722]: pgmap v104: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:41.869 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-12T05:59:42.116 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:43.020 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:42 smithi008 ceph-mon[39722]: pgmap v105: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:43.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:42 smithi194 ceph-mon[49771]: pgmap v105: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 -- 172.21.15.8:0/236290155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c74101e80 msgr2=0x7f0c74102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/236290155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c74101e80 0x7f0c74102260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009910 tx=0x7f0c68009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 -- 172.21.15.8:0/236290155 shutdown_connections 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/236290155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c741027a0 0x7f0c7410ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/236290155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c74101e80 0x7f0c74102260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 -- 172.21.15.8:0/236290155 >> 172.21.15.8:0/236290155 conn(0x7f0c740fb650 msgr2=0x7f0c740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 -- 172.21.15.8:0/236290155 shutdown_connections 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.966+0000 7f0c79a2b700 1 -- 172.21.15.8:0/236290155 wait complete. 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 Processor -- start 2024-04-12T05:59:43.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 -- start start 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c74101e80 0x7f0c7419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7419f300 con 0x7f0c741027a0 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.967+0000 7f0c79a2b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74198890 con 0x7f0c74101e80 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56682/0 (socket says 172.21.15.8:56682) 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 -- 172.21.15.8:0/1955154669 learned_addr learned my addr 172.21.15.8:0/1955154669 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c72ffd700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c74101e80 0x7f0c7419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 -- 172.21.15.8:0/1955154669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c74101e80 msgr2=0x7f0c7419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c74101e80 0x7f0c7419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 -- 172.21.15.8:0/1955154669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680095f0 con 0x7f0c741027a0 2024-04-12T05:59:43.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c727fc700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c960 tx=0x7f0c6400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:43.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c64007a70 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c74198b70 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0c64007bd0 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c64018630 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.968+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c741990c0 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.969+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7410a790 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.970+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f0c6400f450 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.970+0000 7f0c78a29700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 0x7f0c5c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.970+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0c6408c540 con 0x7f0c741027a0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.970+0000 7f0c72ffd700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 0x7f0c5c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:43.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.971+0000 7f0c72ffd700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 0x7f0c5c06f1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c6801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:43.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:43.975+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f0c6405a050 con 0x7f0c741027a0 2024-04-12T05:59:44.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.196+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c74199340 con 0x7f0c5c06cce0 2024-04-12T05:59:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.202+0000 7f0c78a29700 1 -- 172.21.15.8:0/1955154669 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f0c74199340 con 0x7f0c5c06cce0 2024-04-12T05:59:44.196 INFO:teuthology.orchestra.run.smithi008.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:59:44.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 msgr2=0x7f0c5c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 0x7f0c5c06f1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c6801a040 comp rx=0 tx=0).stop 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 msgr2=0x7f0c7419ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400c960 tx=0x7f0c6400cc70 comp rx=0 tx=0).stop 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 shutdown_connections 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0c5c06cce0 0x7f0c5c06f1a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0c74101e80 0x7f0c7419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 --2- 172.21.15.8:0/1955154669 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0c741027a0 0x7f0c7419ed00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 >> 172.21.15.8:0/1955154669 conn(0x7f0c740fb650 msgr2=0x7f0c741002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 shutdown_connections 2024-04-12T05:59:44.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:44.204+0000 7f0c79a2b700 1 -- 172.21.15.8:0/1955154669 wait complete. 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: pgmap v106: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:59:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:44 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:44.964 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-12T05:59:44.965 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T05:59:44.965 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-12T05:59:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: pgmap v106: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:45.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:45.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T05:59:45.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:44 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:45.899 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T05:59:46.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:45 smithi194 ceph-mon[49771]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:46.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:45 smithi194 ceph-mon[49771]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:59:46.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:45 smithi194 ceph-mon[49771]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:59:46.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:45 smithi008 ceph-mon[39722]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:46.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:45 smithi008 ceph-mon[39722]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:59:46.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:45 smithi008 ceph-mon[39722]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d540ebaca6b131a1dd560e7f69e024b133bbaa42 2024-04-12T05:59:47.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:46 smithi194 ceph-mon[49771]: pgmap v107: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:47.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:46 smithi008 ceph-mon[39722]: pgmap v107: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:48.024 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: Upgrade: Target is version 18.2.2-1187-gd540ebac (reef) 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3'] 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: Upgrade: Setting container_image for all mgr 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T05:59:48.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:47 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: Upgrade: Target is version 18.2.2-1187-gd540ebac (reef) 2024-04-12T05:59:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3'] 2024-04-12T05:59:48.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: Upgrade: Setting container_image for all mgr 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T05:59:48.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:47 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T05:59:48.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 -- 172.21.15.8:0/731188106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4102700 msgr2=0x7f87f410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 --2- 172.21.15.8:0/731188106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4102700 0x7f87f410abf0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009910 tx=0x7f87e8009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 -- 172.21.15.8:0/731188106 shutdown_connections 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 --2- 172.21.15.8:0/731188106 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4102700 0x7f87f410abf0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 --2- 172.21.15.8:0/731188106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4101de0 0x7f87f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 -- 172.21.15.8:0/731188106 >> 172.21.15.8:0/731188106 conn(0x7f87f40fb610 msgr2=0x7f87f40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 -- 172.21.15.8:0/731188106 shutdown_connections 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 -- 172.21.15.8:0/731188106 wait complete. 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.495+0000 7f87f90d8700 1 Processor -- start 2024-04-12T05:59:48.491 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f90d8700 1 -- start start 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f90d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f90d8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4102700 0x7f87f419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f90d8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f419d190 con 0x7f87f4101de0 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f90d8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f4196640 con 0x7f87f4102700 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56692/0 (socket says 172.21.15.8:56692) 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 -- 172.21.15.8:0/1950120161 learned_addr learned my addr 172.21.15.8:0/1950120161 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f259c700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4102700 0x7f87f419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 -- 172.21.15.8:0/1950120161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4102700 msgr2=0x7f87f419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4102700 0x7f87f419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.493 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 -- 172.21.15.8:0/1950120161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80095f0 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.496+0000 7f87f2d9d700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c8a0 tx=0x7f87e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.497+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e400cea0 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.497+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f4196920 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.497+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f87e4004830 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.497+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87e4005500 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.497+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f4196e70 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.498+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f41082f0 con 0x7f87f4101de0 2024-04-12T05:59:48.494 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.498+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f87e40049a0 con 0x7f87f4101de0 2024-04-12T05:59:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.498+0000 7f87dbfff700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 0x7f87dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.499+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f87e408c1f0 con 0x7f87f4101de0 2024-04-12T05:59:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.499+0000 7f87f259c700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 0x7f87dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.499+0000 7f87f259c700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 0x7f87dc06f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b5c0 tx=0x7f87e8005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:48.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.503+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f87e4059da0 con 0x7f87f4101de0 2024-04-12T05:59:48.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.724+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87f41970f0 con 0x7f87dc06cce0 2024-04-12T05:59:48.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.725+0000 7f87dbfff700 1 -- 172.21.15.8:0/1950120161 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f87f41970f0 con 0x7f87dc06cce0 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 msgr2=0x7f87dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 0x7f87dc06f1a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b5c0 tx=0x7f87e8005fb0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 msgr2=0x7f87f419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c8a0 tx=0x7f87e400cc60 comp rx=0 tx=0).stop 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 shutdown_connections 2024-04-12T05:59:48.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f87dc06cce0 0x7f87dc06f1a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f87f4101de0 0x7f87f419c5c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 --2- 172.21.15.8:0/1950120161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f87f4102700 0x7f87f419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 >> 172.21.15.8:0/1950120161 conn(0x7f87f40fb610 msgr2=0x7f87f4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 shutdown_connections 2024-04-12T05:59:48.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.727+0000 7f87f90d8700 1 -- 172.21.15.8:0/1950120161 wait complete. 2024-04-12T05:59:48.732 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T05:59:48.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 -- 172.21.15.8:0/3420144048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc105bd0 msgr2=0x7ff9bc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/3420144048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc105fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009910 tx=0x7ff9ac009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:48.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 -- 172.21.15.8:0/3420144048 shutdown_connections 2024-04-12T05:59:48.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/3420144048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/3420144048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc105fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 -- 172.21.15.8:0/3420144048 >> 172.21.15.8:0/3420144048 conn(0x7ff9bc075670 msgr2=0x7ff9bc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 -- 172.21.15.8:0/3420144048 shutdown_connections 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.836+0000 7ff9c23c0700 1 -- 172.21.15.8:0/3420144048 wait complete. 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 Processor -- start 2024-04-12T05:59:48.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 -- start start 2024-04-12T05:59:48.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc19d260 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9c23c0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc1966e0 con 0x7ff9bc105bd0 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56706/0 (socket says 172.21.15.8:56706) 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 -- 172.21.15.8:0/2738473989 learned_addr learned my addr 172.21.15.8:0/2738473989 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bb7fe700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 -- 172.21.15.8:0/2738473989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 msgr2=0x7ff9bc19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc19cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bbfff700 1 -- 172.21.15.8:0/2738473989 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac0095f0 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.837+0000 7ff9bb7fe700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T05:59:48.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9bbfff700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006010 tx=0x7ff9ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9ac01d070 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc196960 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9ac00f460 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9ac021640 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.838+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc196e50 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.839+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc04f2e0 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.839+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7ff9ac00fa20 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.840+0000 7ff9b97fa700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 0x7ff9a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:48.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.840+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff9ac08d800 con 0x7ff9bc0ffc20 2024-04-12T05:59:48.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.840+0000 7ff9bb7fe700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 0x7ff9a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:48.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.840+0000 7ff9bb7fe700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 0x7ff9a406f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc197ed0 tx=0x7ff9b0008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:48.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:48.844+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7ff9ac05b3b0 con 0x7ff9bc0ffc20 2024-04-12T05:59:49.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.067+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9bc1971b0 con 0x7ff9a406cce0 2024-04-12T05:59:49.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.068+0000 7ff9b97fa700 1 -- 172.21.15.8:0/2738473989 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7ff9bc1971b0 con 0x7ff9a406cce0 2024-04-12T05:59:49.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.070+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 msgr2=0x7ff9a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.070+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 0x7ff9a406f1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc197ed0 tx=0x7ff9b0008040 comp rx=0 tx=0).stop 2024-04-12T05:59:49.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.070+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 msgr2=0x7ff9bc19c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.070+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac006010 tx=0x7ff9ac00be20 comp rx=0 tx=0).stop 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 shutdown_connections 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9a406cce0 0x7ff9a406f1a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff9bc0ffc20 0x7ff9bc19c640 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 --2- 172.21.15.8:0/2738473989 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9bc105bd0 0x7ff9bc19cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 >> 172.21.15.8:0/2738473989 conn(0x7ff9bc075670 msgr2=0x7ff9bc068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 shutdown_connections 2024-04-12T05:59:49.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.071+0000 7ff9c23c0700 1 -- 172.21.15.8:0/2738473989 wait complete. 2024-04-12T05:59:49.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:48 smithi194 ceph-mon[49771]: pgmap v108: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:49.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:48 smithi008 ceph-mon[39722]: pgmap v108: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:49.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 -- 172.21.15.8:0/3378241183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea8100330 msgr2=0x7f7ea810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 --2- 172.21.15.8:0/3378241183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea8100330 0x7f7ea810d080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009910 tx=0x7f7e98009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:49.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 -- 172.21.15.8:0/3378241183 shutdown_connections 2024-04-12T05:59:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 --2- 172.21.15.8:0/3378241183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea8100330 0x7f7ea810d080 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 --2- 172.21.15.8:0/3378241183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea80ffa10 0x7f7ea80ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 -- 172.21.15.8:0/3378241183 >> 172.21.15.8:0/3378241183 conn(0x7f7ea80fb650 msgr2=0x7f7ea80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 -- 172.21.15.8:0/3378241183 shutdown_connections 2024-04-12T05:59:49.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.179+0000 7f7eb0683700 1 -- 172.21.15.8:0/3378241183 wait complete. 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.180+0000 7f7eb0683700 1 Processor -- start 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.180+0000 7f7eb0683700 1 -- start start 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eb0683700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eb0683700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea8100330 0x7f7ea819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eb0683700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea819f370 con 0x7f7ea80ffa10 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eb0683700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8198820 con 0x7f7ea8100330 2024-04-12T05:59:49.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56724/0 (socket says 172.21.15.8:56724) 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 -- 172.21.15.8:0/2719636097 learned_addr learned my addr 172.21.15.8:0/2719636097 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eadc1e700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea8100330 0x7f7ea819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 -- 172.21.15.8:0/2719636097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea8100330 msgr2=0x7f7ea819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea8100330 0x7f7ea819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 -- 172.21.15.8:0/2719636097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980095f0 con 0x7f7ea80ffa10 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eae41f700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c8a0 tx=0x7f7ea400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea400cea0 con 0x7f7ea80ffa10 2024-04-12T05:59:49.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea8198b00 con 0x7f7ea80ffa10 2024-04-12T05:59:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ea4004830 con 0x7f7ea80ffa10 2024-04-12T05:59:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.181+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea4005500 con 0x7f7ea80ffa10 2024-04-12T05:59:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.182+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea8199050 con 0x7f7ea80ffa10 2024-04-12T05:59:49.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.182+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea810a7f0 con 0x7f7ea80ffa10 2024-04-12T05:59:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.184+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f7ea40049a0 con 0x7f7ea80ffa10 2024-04-12T05:59:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.184+0000 7f7e9f7fe700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 0x7f7e9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.184+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7ea408c530 con 0x7f7ea80ffa10 2024-04-12T05:59:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.184+0000 7f7eadc1e700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 0x7f7e9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.185+0000 7f7eadc1e700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 0x7f7e9406f1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80fbea0 tx=0x7f7e98005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.188+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f7ea405a200 con 0x7f7ea80ffa10 2024-04-12T05:59:49.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.404+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ea8199320 con 0x7f7e9406cce0 2024-04-12T05:59:49.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.415+0000 7f7e9f7fe700 1 -- 172.21.15.8:0/2719636097 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7ea8199320 con 0x7f7e9406cce0 2024-04-12T05:59:49.410 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T05:59:49.410 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (101s) 74s ago 13m 14.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (13m) 74s ago 13m 7986k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (12m) 2m ago 12m 7834k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (13m) 74s ago 13m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (12m) 2m ago 12m 6647k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (80s) 74s ago 12m 41.2M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (6m) 74s ago 6m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (7m) 74s ago 7m 71.3M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (6m) 2m ago 6m 65.8M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (6m) 2m ago 6m 126M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (3m) 74s ago 14m 503M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (2m) 2m ago 11m 447M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (14m) 74s ago 14m 57.8M 2048M 18.2.1 5be31c24972a 281a8a1d6515 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (11m) 2m ago 11m 45.7M 2048M 18.2.1 5be31c24972a 423eb70e605f 2024-04-12T05:59:49.411 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (2m) 74s ago 13m 9286k - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (2m) 2m ago 12m 3384k - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (11m) 74s ago 11m 240M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (10m) 74s ago 10m 240M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (10m) 74s ago 10m 196M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (9m) 2m ago 9m 260M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (9m) 2m ago 9m 226M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (8m) 2m ago 8m 208M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T05:59:49.412 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (119s) 74s ago 12m 46.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.417+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 msgr2=0x7f7e9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.417+0000 7f7eb0683700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 0x7f7e9406f1a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80fbea0 tx=0x7f7e98005940 comp rx=0 tx=0).stop 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.417+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 msgr2=0x7f7ea819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.417+0000 7f7eb0683700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400c8a0 tx=0x7f7ea400cc60 comp rx=0 tx=0).stop 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 shutdown_connections 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e9406cce0 0x7f7e9406f1a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ea80ffa10 0x7f7ea819e7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 --2- 172.21.15.8:0/2719636097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ea8100330 0x7f7ea819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 >> 172.21.15.8:0/2719636097 conn(0x7f7ea80fb650 msgr2=0x7f7ea80fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 shutdown_connections 2024-04-12T05:59:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.418+0000 7f7eb0683700 1 -- 172.21.15.8:0/2719636097 wait complete. 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.529+0000 7fb352d58700 1 -- 172.21.15.8:0/4153297531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 msgr2=0x7fb34c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.529+0000 7fb352d58700 1 --2- 172.21.15.8:0/4153297531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c100180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb340009910 tx=0x7fb340009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 -- 172.21.15.8:0/4153297531 shutdown_connections 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 --2- 172.21.15.8:0/4153297531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 0x7fb34c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 --2- 172.21.15.8:0/4153297531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c100180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 -- 172.21.15.8:0/4153297531 >> 172.21.15.8:0/4153297531 conn(0x7fb34c0fb650 msgr2=0x7fb34c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 -- 172.21.15.8:0/4153297531 shutdown_connections 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 -- 172.21.15.8:0/4153297531 wait complete. 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 Processor -- start 2024-04-12T05:59:49.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 -- start start 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.530+0000 7fb352d58700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 0x7fb34c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb352d58700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c19d170 con 0x7fb34c0ffda0 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb352d58700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c196620 con 0x7fb34c100750 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56738/0 (socket says 172.21.15.8:56738) 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 -- 172.21.15.8:0/82137870 learned_addr learned my addr 172.21.15.8:0/82137870 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb34bfff700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 0x7fb34c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 -- 172.21.15.8:0/82137870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 msgr2=0x7fb34c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 0x7fb34c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 -- 172.21.15.8:0/82137870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400095f0 con 0x7fb34c0ffda0 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb350af4700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb340006010 tx=0x7fb34000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34001d070 con 0x7fb34c0ffda0 2024-04-12T05:59:49.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb340005710 con 0x7fb34c0ffda0 2024-04-12T05:59:49.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c1968a0 con 0x7fb34c0ffda0 2024-04-12T05:59:49.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.531+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb34000fe20 con 0x7fb34c0ffda0 2024-04-12T05:59:49.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.532+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c196d90 con 0x7fb34c0ffda0 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.532+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c04f2e0 con 0x7fb34c0ffda0 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.533+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fb340003a60 con 0x7fb34c0ffda0 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.533+0000 7fb349ffb700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 0x7fb33c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.533+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb34008d630 con 0x7fb34c0ffda0 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.534+0000 7fb34bfff700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 0x7fb33c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.534+0000 7fb34bfff700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 0x7fb33c06f1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0fe090 tx=0x7fb33800b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.538+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fb34005b1e0 con 0x7fb34c0ffda0 2024-04-12T05:59:49.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.829+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb34c197040 con 0x7fb34c0ffda0 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.830+0000 7fb349ffb700 1 -- 172.21.15.8:0/82137870 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fb34005ad70 con 0x7fb34c0ffda0 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T05:59:49.825 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T05:59:49.826 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 msgr2=0x7fb33c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 0x7fb33c06f1a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0fe090 tx=0x7fb33800b3c0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 msgr2=0x7fb34c19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb340006010 tx=0x7fb34000bd70 comp rx=0 tx=0).stop 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 shutdown_connections 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb33c06cce0 0x7fb33c06f1a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb34c0ffda0 0x7fb34c19c5a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 --2- 172.21.15.8:0/82137870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb34c100750 0x7fb34c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 >> 172.21.15.8:0/82137870 conn(0x7fb34c0fb650 msgr2=0x7fb34c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 shutdown_connections 2024-04-12T05:59:49.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.832+0000 7fb352d58700 1 -- 172.21.15.8:0/82137870 wait complete. 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 -- 172.21.15.8:0/3074795700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8106cd0 msgr2=0x7fefc81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 --2- 172.21.15.8:0/3074795700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8106cd0 0x7fefc81070b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fefb8009910 tx=0x7fefb8009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 -- 172.21.15.8:0/3074795700 shutdown_connections 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 --2- 172.21.15.8:0/3074795700 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8068580 0x7fefc80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 --2- 172.21.15.8:0/3074795700 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8106cd0 0x7fefc81070b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 -- 172.21.15.8:0/3074795700 >> 172.21.15.8:0/3074795700 conn(0x7fefc8075670 msgr2=0x7fefc8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 -- 172.21.15.8:0/3074795700 shutdown_connections 2024-04-12T05:59:49.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.940+0000 7fefce073700 1 -- 172.21.15.8:0/3074795700 wait complete. 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 Processor -- start 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 -- start start 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8106cd0 0x7fefc819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc819d130 con 0x7fefc8068580 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefce073700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc8196630 con 0x7fefc8106cd0 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefc77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefc77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56758/0 (socket says 172.21.15.8:56758) 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefc77fe700 1 -- 172.21.15.8:0/2252218143 learned_addr learned my addr 172.21.15.8:0/2252218143 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.941+0000 7fefc6ffd700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8106cd0 0x7fefc819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc77fe700 1 -- 172.21.15.8:0/2252218143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8106cd0 msgr2=0x7fefc819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc77fe700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8106cd0 0x7fefc819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc77fe700 1 -- 172.21.15.8:0/2252218143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80095f0 con 0x7fefc8068580 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc77fe700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fefb8006010 tx=0x7fefb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb801d070 con 0x7fefc8068580 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fefb8005710 con 0x7fefc8068580 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefb800fe20 con 0x7fefc8068580 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc81968b0 con 0x7fefc8068580 2024-04-12T05:59:49.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.942+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc8196da0 con 0x7fefc8068580 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.943+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc81091f0 con 0x7fefc8068580 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.943+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fefb8003a60 con 0x7fefc8068580 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.944+0000 7fefc4ff9700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 0x7fefb006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.944+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fefb808d640 con 0x7fefc8068580 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.944+0000 7fefc6ffd700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 0x7fefb006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:49.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.944+0000 7fefc6ffd700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 0x7fefb006f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefc80fe390 tx=0x7fefbc00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:49.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:49.948+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fefb805b1f0 con 0x7fefc8068580 2024-04-12T05:59:50.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:49 smithi008 ceph-mon[39722]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:50.025 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:49 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:50.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:49 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:59:50.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:49 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:59:50.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:49 smithi008 ceph-mon[39722]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:50.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:49 smithi194 ceph-mon[49771]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:50.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:49 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:50.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:49 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T05:59:50.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:49 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T05:59:50.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:49 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T05:59:50.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.214+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fefc804f2e0 con 0x7fefc8068580 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.215+0000 7fefc4ff9700 1 -- 172.21.15.8:0/2252218143 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1879 (secure 0 0 0) 0x7fefb8061070 con 0x7fefc8068580 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:e23 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T05:59:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 23 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T05:55:45.534456+0000 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:59:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:50.213 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:50.213 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:50.213 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:59:50.213 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 msgr2=0x7fefb006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 0x7fefb006f1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fefc80fe390 tx=0x7fefbc00b410 comp rx=0 tx=0).stop 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 msgr2=0x7fefc819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fefb8006010 tx=0x7fefb800bd70 comp rx=0 tx=0).stop 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 shutdown_connections 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fefb006cce0 0x7fefb006f1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fefc8068580 0x7fefc819c560 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 --2- 172.21.15.8:0/2252218143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefc8106cd0 0x7fefc819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 >> 172.21.15.8:0/2252218143 conn(0x7fefc8075670 msgr2=0x7fefc80fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 shutdown_connections 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.217+0000 7fefce073700 1 -- 172.21.15.8:0/2252218143 wait complete. 2024-04-12T05:59:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 -- 172.21.15.8:0/2120663583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 msgr2=0x7fef540ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 --2- 172.21.15.8:0/2120663583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef540ff980 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fef44009910 tx=0x7fef44009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 -- 172.21.15.8:0/2120663583 shutdown_connections 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 --2- 172.21.15.8:0/2120663583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef540fff50 0x7fef54105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 --2- 172.21.15.8:0/2120663583 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef540ff980 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 -- 172.21.15.8:0/2120663583 >> 172.21.15.8:0/2120663583 conn(0x7fef54074e30 msgr2=0x7fef54075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 -- 172.21.15.8:0/2120663583 shutdown_connections 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.327+0000 7fef5962c700 1 -- 172.21.15.8:0/2120663583 wait complete. 2024-04-12T05:59:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 Processor -- start 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 -- start start 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef540fff50 0x7fef5419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5419d170 con 0x7fef540ff5a0 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.328+0000 7fef5962c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54196620 con 0x7fef540fff50 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56782/0 (socket says 172.21.15.8:56782) 2024-04-12T05:59:50.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 -- 172.21.15.8:0/2665323994 learned_addr learned my addr 172.21.15.8:0/2665323994 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 -- 172.21.15.8:0/2665323994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef540fff50 msgr2=0x7fef5419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef540fff50 0x7fef5419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 -- 172.21.15.8:0/2665323994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440095f0 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef52ffd700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef44006010 tx=0x7fef4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef4401d070 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef4400f460 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef44021600 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef541968a0 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.329+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef54196d40 con 0x7fef540ff5a0 2024-04-12T05:59:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.330+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef54109270 con 0x7fef540ff5a0 2024-04-12T05:59:50.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.331+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7fef4400f5d0 con 0x7fef540ff5a0 2024-04-12T05:59:50.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.331+0000 7fef3bfff700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 0x7fef3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.331+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fef4408d6c0 con 0x7fef540ff5a0 2024-04-12T05:59:50.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.331+0000 7fef527fc700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 0x7fef3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:50.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.332+0000 7fef527fc700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 0x7fef3c06f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef48005950 tx=0x7fef480058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:50.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.336+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7fef4405b2f0 con 0x7fef540ff5a0 2024-04-12T05:59:50.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.565+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef54196fb0 con 0x7fef3c06cce0 2024-04-12T05:59:50.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.566+0000 7fef3bfff700 1 -- 172.21.15.8:0/2665323994 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fef54196fb0 con 0x7fef3c06cce0 2024-04-12T05:59:50.562 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr" 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "2/23 daemons upgraded", 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mon daemons", 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T05:59:50.563 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 msgr2=0x7fef3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 0x7fef3c06f1a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fef48005950 tx=0x7fef480058e0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 msgr2=0x7fef5419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fef44006010 tx=0x7fef4400be20 comp rx=0 tx=0).stop 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 shutdown_connections 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef3c06cce0 0x7fef3c06f1a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef540ff5a0 0x7fef5419c5a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 --2- 172.21.15.8:0/2665323994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef540fff50 0x7fef5419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 >> 172.21.15.8:0/2665323994 conn(0x7fef54074e30 msgr2=0x7fef540fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:50.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 shutdown_connections 2024-04-12T05:59:50.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.568+0000 7fef5962c700 1 -- 172.21.15.8:0/2665323994 wait complete. 2024-04-12T05:59:50.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 -- 172.21.15.8:0/1343105243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc101850 msgr2=0x7f31bc1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 --2- 172.21.15.8:0/1343105243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc101850 0x7f31bc1058a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f31b0009910 tx=0x7f31b0009c20 comp rx=0 tx=0).stop 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 -- 172.21.15.8:0/1343105243 shutdown_connections 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 --2- 172.21.15.8:0/1343105243 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc101850 0x7f31bc1058a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 --2- 172.21.15.8:0/1343105243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc100ea0 0x7f31bc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 -- 172.21.15.8:0/1343105243 >> 172.21.15.8:0/1343105243 conn(0x7f31bc0fc750 msgr2=0x7f31bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 -- 172.21.15.8:0/1343105243 shutdown_connections 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.675+0000 7f31c26b2700 1 -- 172.21.15.8:0/1343105243 wait complete. 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 Processor -- start 2024-04-12T05:59:50.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 -- start start 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc101850 0x7f31bc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc19f500 con 0x7f31bc100ea0 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31c26b2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1989b0 con 0x7f31bc101850 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56802/0 (socket says 172.21.15.8:56802) 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.676+0000 7f31bbfff700 1 -- 172.21.15.8:0/216523881 learned_addr learned my addr 172.21.15.8:0/216523881 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31bb7fe700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc101850 0x7f31bc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31bbfff700 1 -- 172.21.15.8:0/216523881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc101850 msgr2=0x7f31bc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31bbfff700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc101850 0x7f31bc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31bbfff700 1 -- 172.21.15.8:0/216523881 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b00095f0 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31bbfff700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b4b0 tx=0x7f31ac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac011800 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31ac011e00 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc198c90 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac010e50 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.677+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1991b0 con 0x7f31bc100ea0 2024-04-12T05:59:50.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.678+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc105020 con 0x7f31bc100ea0 2024-04-12T05:59:50.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.679+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90840+0+0 (secure 0 0 0) 0x7f31ac010380 con 0x7f31bc100ea0 2024-04-12T05:59:50.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.679+0000 7f31b97fa700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T05:59:50.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.679+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f31ac08c0d0 con 0x7f31bc100ea0 2024-04-12T05:59:50.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.679+0000 7f31bb7fe700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T05:59:50.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.680+0000 7f31bb7fe700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 0x7f31a406f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006010 tx=0x7f31b001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T05:59:50.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.684+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182676 (secure 0 0 0) 0x7f31ac059c80 con 0x7f31bc100ea0 2024-04-12T05:59:50.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.988+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f31bc04f2e0 con 0x7f31bc100ea0 2024-04-12T05:59:50.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.989+0000 7f31b97fa700 1 -- 172.21.15.8:0/216523881 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f31ac059810 con 0x7f31bc100ea0 2024-04-12T05:59:50.984 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 msgr2=0x7f31a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 0x7f31a406f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006010 tx=0x7f31b001a040 comp rx=0 tx=0).stop 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 msgr2=0x7f31bc19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00b4b0 tx=0x7f31ac00b870 comp rx=0 tx=0).stop 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 shutdown_connections 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f31a406cce0 0x7f31a406f1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f31bc100ea0 0x7f31bc19e930 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 --2- 172.21.15.8:0/216523881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f31bc101850 0x7f31bc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 >> 172.21.15.8:0/216523881 conn(0x7f31bc0fc750 msgr2=0x7f31bc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T05:59:50.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 shutdown_connections 2024-04-12T05:59:50.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T05:59:50.991+0000 7f31c26b2700 1 -- 172.21.15.8:0/216523881 wait complete. 2024-04-12T05:59:51.037 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:51.037 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: Upgrade: Updating mon.smithi008 2024-04-12T05:59:51.037 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: Deploying daemon mon.smithi008 on smithi008 2024-04-12T05:59:51.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: pgmap v109: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:51.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:51.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/82137870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:51.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:50 smithi008 ceph-mon[39722]: from='client.? 172.21.15.8:0/2252218143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: Upgrade: Updating mon.smithi008 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: Deploying daemon mon.smithi008 on smithi008 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: pgmap v109: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T05:59:51.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/82137870' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T05:59:51.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:50 smithi194 ceph-mon[49771]: from='client.? 172.21.15.8:0/2252218143' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T05:59:51.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:51 smithi008 systemd[1]: Stopping Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T05:59:51.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[39699]: 2024-04-12T05:59:51.282+0000 7f8e5e69b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi008 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T05:59:51.298 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:51 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[39699]: 2024-04-12T05:59:51.282+0000 7f8e5e69b700 -1 mon.smithi008@0(leader) e2 *** Got Signal Terminated *** 2024-04-12T05:59:52.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:51 smithi008 bash[145880]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008 2024-04-12T05:59:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:52 smithi008 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service: Succeeded. 2024-04-12T05:59:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:52 smithi008 systemd[1]: Stopped Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:59:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:52 smithi008 systemd[1]: Starting Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable), process ceph-mon, pid 7 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: pidfile_write: ignore empty --pid-file 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: load: jerasure load: lrc 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: RocksDB version: 7.9.2 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Git sha 0 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Compile date 2024-04-08 23:21:21 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: DB SUMMARY 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: DB Session ID: HYIAI9DORA0I48972Z2E 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: CURRENT file: CURRENT 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T05:59:53.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1510 Bytes 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi008/store.db dir, Total Num: 1, files: 000032.sst 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi008/store.db: 000030.log size: 5240923 ; 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.error_if_exists: 0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.create_if_missing: 0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.paranoid_checks: 1 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.env: 0x5629e455d7a0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.fs: PosixFileSystem 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.info_log: 0x5629e75a93a0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.statistics: (nil) 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.use_fsync: 0 2024-04-12T05:59:53.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_log_file_size: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_fallocate: 1 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.use_direct_reads: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.db_log_dir: 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.wal_dir: 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T05:59:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.write_buffer_manager: 0x5629e681e140 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T05:59:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.unordered_write: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.row_cache: None 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.wal_filter: None 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.two_write_queues: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.wal_compression: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.atomic_flush: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T05:59:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.log_readahead_size: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_background_jobs: 2 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_background_compactions: -1 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_subcompactions: 1 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T05:59:53.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_open_files: -1 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_background_flushes: -1 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Compression algorithms supported: 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T05:59:53.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kZSTD supported: 0 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kXpressCompression supported: 0 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kZlibCompression supported: 1 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kSnappyCompression supported: 1 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kLZ4Compression supported: 1 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: kBZip2Compression supported: 0 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi008/store.db/MANIFEST-000015 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.merge_operator: 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_filter: None 2024-04-12T05:59:53.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5629e75a94c0) 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: cache_index_and_filter_blocks: 1 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: pin_top_level_index_and_filter: 1 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_type: 0 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: data_block_index_type: 0 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_shortening: 1 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: checksum: 4 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: no_block_cache: 0 2024-04-12T05:59:53.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache: 0x5629e68bf1f0 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_name: BinnedLRUCache 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_options: 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: capacity : 536870912 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: num_shard_bits : 4 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: strict_capacity_limit : 0 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: high_pri_pool_ratio: 0.000 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_cache_compressed: (nil) 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: persistent_cache: (nil) 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_size: 4096 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_size_deviation: 10 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_restart_interval: 16 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: index_block_restart_interval: 1 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: metadata_block_size: 4096 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: partition_filters: 0 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: use_delta_encoding: 1 2024-04-12T05:59:53.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: filter_policy: bloomfilter 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: whole_key_filtering: 1 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: verify_compression: 0 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: read_amp_bytes_per_bit: 0 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: format_version: 5 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: enable_index_compression: 1 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: block_align: 0 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: max_auto_readahead_size: 262144 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: prepopulate_block_cache: 0 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: initial_auto_readahead_size: 8192 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: num_file_reads_for_auto_readahead: 2 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression: NoCompression 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T05:59:53.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.num_levels: 7 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T05:59:53.415 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T05:59:53.416 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.arena_block_size: 1048576 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T05:59:53.417 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T05:59:53.418 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T05:59:53.418 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T05:59:53.418 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T05:59:53.418 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.table_properties_collectors: 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.inplace_update_support: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.bloom_locality: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.max_successive_merges: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T05:59:53.419 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.ttl: 2592000 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enable_blob_files: false 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.min_blob_size: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.blob_file_starting_level: 0 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-12T05:59:53.420 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi008/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12024, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 68c12ccb-cf4a-4751-83b9-27e5c20b0020 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901593033787, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901593053746, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4644323, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12025, "largest_seqno": 15843, "table_properties": {"data_size": 4629718, "index_size": 9424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4165, "raw_key_size": 36532, "raw_average_key_size": 21, "raw_value_size": 4596746, "raw_average_value_size": 2762, "num_data_blocks": 426, "num_entries": 1664, "num_filter_entries": 1664, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712901593, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "68c12ccb-cf4a-4751-83b9-27e5c20b0020", "db_session_id": "HYIAI9DORA0I48972Z2E", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901593053831, "job": 1, "event": "recovery_finished"} 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi008/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5629e695e000 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: DB pointer 0x5629e694c000 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-12T05:59:53.421 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** DB Stats ** 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** Compaction Stats [default] ** 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: L0 1/0 4.43 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.8 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: L6 1/0 4.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-12T05:59:53.422 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Sum 2/0 9.21 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.8 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 646.8 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** Compaction Stats [default] ** 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 646.8 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T05:59:53.423 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Cumulative compaction: 0.00 GB write, 192.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Interval compaction: 0.00 GB write, 192.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Block cache BinnedLRUCache@0x5629e68bf1f0#7 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.9e-05 secs_since: 0 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: starting mon.smithi008 rank 0 at public addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] at bind addrs [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi008 fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???) e2 preinit fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).mds e23 new map 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).mds e23 print_map 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: e23 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: legacy client fscid: 1 2024-04-12T05:59:53.424 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: Filesystem 'cephfs' (1) 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: fs_name cephfs 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: epoch 23 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: created 2024-04-12T05:52:43.616162+0000 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: modified 2024-04-12T05:55:45.534456+0000 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: tableserver 0 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: root 0 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: session_timeout 60 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: session_autoclose 300 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: max_file_size 1099511627776 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: max_xattr_size 65536 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: required_client_features {} 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: last_failure 0 2024-04-12T05:59:53.425 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: last_failure_osd_epoch 46 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: max_mds 2 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: in 0,1 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: up {0=14476,1=24277} 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: failed 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: damaged 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: stopped 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: data_pools [3] 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: metadata_pool 2 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: inline_data disabled 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: balancer 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: bal_rank_mask -1 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: standby_count_wanted 1 2024-04-12T05:59:53.426 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: [mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: [mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: [mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: [mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout: 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 ceph-mon[146084]: mon.smithi008@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 bash[146046]: 69cc2ae6e9155c56347fe8f904cc8d224066a5119d0b1147ef157e8ce9b01940 2024-04-12T05:59:53.427 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:53 smithi008 systemd[1]: Started Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T05:59:54.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:59:54.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: pgmap v110: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: pgmap v111: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: mon.smithi008 calling monitor election 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: mgrmap e32: smithi008.ixmnlp(active, since 3m), standbys: smithi194.xbiyaj 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: overall HEALTH_OK 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='' 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:54 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T05:59:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T05:59:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: pgmap v110: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: pgmap v111: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: mon.smithi008 calling monitor election 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: mgrmap e32: smithi008.ixmnlp(active, since 3m), standbys: smithi194.xbiyaj 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: overall HEALTH_OK 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='' 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T05:59:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:54 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T05:59:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:56 smithi194 ceph-mon[49771]: pgmap v112: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:56.896 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:56 smithi008 ceph-mon[146084]: pgmap v112: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T05:59:58.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 05:59:58 smithi194 ceph-mon[49771]: pgmap v113: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T05:59:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 05:59:58 smithi008 ceph-mon[146084]: pgmap v113: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:00 smithi194 ceph-mon[49771]: pgmap v114: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:00 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:00 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:00 smithi194 ceph-mon[49771]: overall HEALTH_OK 2024-04-12T06:00:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:00 smithi008 ceph-mon[146084]: pgmap v114: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:00 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:00 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:00 smithi008 ceph-mon[146084]: overall HEALTH_OK 2024-04-12T06:00:01.731 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:01 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:01.732 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:01 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:01.732 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:01 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:02.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:01 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:02.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:01 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:02.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:01 smithi194 ceph-mon[49771]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:02 smithi008 ceph-mon[146084]: pgmap v115: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:03.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:02 smithi194 ceph-mon[49771]: pgmap v115: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:03.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:02 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: pgmap v116: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:04.903 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:04 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: pgmap v116: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:04.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-12T06:00:05.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:05 smithi008 ceph-mon[146084]: Detected new or changed devices on smithi008 2024-04-12T06:00:05.920 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:05 smithi194 ceph-mon[49771]: Detected new or changed devices on smithi008 2024-04-12T06:00:06.889 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:06 smithi194 ceph-mon[49771]: pgmap v117: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:06.890 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:06 smithi194 ceph-mon[49771]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:06.890 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:06 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T06:00:06.890 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:06 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T06:00:06.890 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:06 smithi194 ceph-mon[49771]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:06 smithi008 ceph-mon[146084]: pgmap v117: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:06 smithi008 ceph-mon[146084]: from='mgr.14660 ' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-12T06:00:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-12T06:00:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:07.910 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:07 smithi194 ceph-mon[49771]: Upgrade: Updating mon.smithi194 2024-04-12T06:00:07.910 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:07 smithi194 ceph-mon[49771]: Deploying daemon mon.smithi194 on smithi194 2024-04-12T06:00:08.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:07 smithi008 ceph-mon[146084]: Upgrade: Updating mon.smithi194 2024-04-12T06:00:08.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:07 smithi008 ceph-mon[146084]: Deploying daemon mon.smithi194 on smithi194 2024-04-12T06:00:08.180 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:07 smithi194 systemd[1]: Stopping Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:00:08.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:08 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194[49739]: 2024-04-12T06:00:08.184+0000 7f168ee23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi194 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:00:08.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:08 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194[49739]: 2024-04-12T06:00:08.184+0000 7f168ee23700 -1 mon.smithi194@1(peon) e2 *** Got Signal Terminated *** 2024-04-12T06:00:09.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:08 smithi194 bash[106790]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194 2024-04-12T06:00:09.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:09 smithi194 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi194.service: Succeeded. 2024-04-12T06:00:09.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:09 smithi194 systemd[1]: Stopped Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:00:09.569 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:09 smithi194 systemd[1]: Starting Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:00:10.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: set uid:gid to 167:167 (ceph:ceph) 2024-04-12T06:00:10.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable), process ceph-mon, pid 7 2024-04-12T06:00:10.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: pidfile_write: ignore empty --pid-file 2024-04-12T06:00:10.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: load: jerasure load: lrc 2024-04-12T06:00:10.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: RocksDB version: 7.9.2 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Git sha 0 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Compile date 2024-04-08 23:21:21 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: DB SUMMARY 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: DB Session ID: EZSP40BD7RJK25NL4AKW 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: CURRENT file: CURRENT 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: IDENTITY file: IDENTITY 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1404 Bytes 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi194/store.db dir, Total Num: 1, files: 000027.sst 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi194/store.db: 000025.log size: 8445358 ; 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.error_if_exists: 0 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.create_if_missing: 0 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.paranoid_checks: 1 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-12T06:00:10.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.env: 0x55d471b3d7a0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.fs: PosixFileSystem 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.info_log: 0x55d4742893a0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_file_opening_threads: 16 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.statistics: (nil) 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.use_fsync: 0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_log_file_size: 0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.keep_log_file_num: 1000 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.recycle_log_file_num: 0 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_fallocate: 1 2024-04-12T06:00:10.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_mmap_reads: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_mmap_writes: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.use_direct_reads: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.create_missing_column_families: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.db_log_dir: 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.wal_dir: 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.advise_random_on_open: 1 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.db_write_buffer_size: 0 2024-04-12T06:00:10.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.write_buffer_manager: 0x55d4734fe140 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.rate_limiter: (nil) 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.wal_recovery_mode: 2 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enable_thread_tracking: 0 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enable_pipelined_write: 0 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.unordered_write: 0 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.row_cache: None 2024-04-12T06:00:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.wal_filter: None 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_ingest_behind: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.two_write_queues: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.manual_wal_flush: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.wal_compression: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.atomic_flush: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.log_readahead_size: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.best_efforts_recovery: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.allow_data_in_errors: 0 2024-04-12T06:00:10.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.db_host_id: __hostname__ 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_background_jobs: 2 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_background_compactions: -1 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_subcompactions: 1 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_total_wal_size: 0 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-12T06:00:10.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_open_files: -1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bytes_per_sync: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_readahead_size: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_background_flushes: -1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Compression algorithms supported: 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kZSTD supported: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kXpressCompression supported: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kLZ4HCCompression supported: 1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kZlibCompression supported: 1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kSnappyCompression supported: 1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kLZ4Compression supported: 1 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: kBZip2Compression supported: 0 2024-04-12T06:00:10.327 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi194/store.db/MANIFEST-000010 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.merge_operator: 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_filter: None 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_filter_factory: None 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.sst_partitioner_factory: None 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d4742894c0) 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: cache_index_and_filter_blocks: 1 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-12T06:00:10.328 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: pin_top_level_index_and_filter: 1 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: index_type: 0 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: data_block_index_type: 0 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: index_shortening: 1 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: checksum: 4 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: no_block_cache: 0 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_cache: 0x55d47359f1f0 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_cache_name: BinnedLRUCache 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_cache_options: 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: capacity : 536870912 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: num_shard_bits : 4 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: strict_capacity_limit : 0 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: high_pri_pool_ratio: 0.000 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_cache_compressed: (nil) 2024-04-12T06:00:10.329 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: persistent_cache: (nil) 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_size: 4096 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_size_deviation: 10 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_restart_interval: 16 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: index_block_restart_interval: 1 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: metadata_block_size: 4096 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: partition_filters: 0 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: use_delta_encoding: 1 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: filter_policy: bloomfilter 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: whole_key_filtering: 1 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: verify_compression: 0 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: read_amp_bytes_per_bit: 0 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: format_version: 5 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: enable_index_compression: 1 2024-04-12T06:00:10.330 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: block_align: 0 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: max_auto_readahead_size: 262144 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: prepopulate_block_cache: 0 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: initial_auto_readahead_size: 8192 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: num_file_reads_for_auto_readahead: 2 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.write_buffer_size: 33554432 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_write_buffer_number: 2 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression: NoCompression 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression: Disabled 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.prefix_extractor: nullptr 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.num_levels: 7 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-12T06:00:10.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.level: 32767 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.strategy: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-12T06:00:10.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.enabled: false 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.target_file_size_base: 67108864 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-12T06:00:10.333 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.arena_block_size: 1048576 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.disable_auto_compactions: 0 2024-04-12T06:00:10.334 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-12T06:00:10.335 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.table_properties_collectors: 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.inplace_update_support: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.bloom_locality: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.max_successive_merges: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.paranoid_file_checks: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.force_consistency_checks: 1 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.report_bg_io_stats: 0 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.ttl: 2592000 2024-04-12T06:00:10.336 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enable_blob_files: false 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.min_blob_size: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_file_size: 268435456 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.blob_file_starting_level: 0 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T06:00:10.337 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi194/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 11974, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4a08bf92-e683-4bca-99d3-4ea29f77ca52 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901610020793, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901610044446, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4966835, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11980, "largest_seqno": 16511, "table_properties": {"data_size": 4951114, "index_size": 10220, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4485, "raw_key_size": 39137, "raw_average_key_size": 22, "raw_value_size": 4915658, "raw_average_value_size": 2764, "num_data_blocks": 462, "num_entries": 1778, "num_filter_entries": 1778, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712901610, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4a08bf92-e683-4bca-99d3-4ea29f77ca52", "db_session_id": "EZSP40BD7RJK25NL4AKW", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712901610044600, "job": 1, "event": "recovery_finished"} 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi194/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d47363e000 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: DB pointer 0x55d47362c000 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-12T06:00:10.338 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: ** DB Stats ** 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: ** Compaction Stats [default] ** 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: L0 1/0 4.74 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-12T06:00:10.339 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: L6 1/0 4.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Sum 2/0 9.51 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 561.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: ** Compaction Stats [default] ** 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 561.4 0.01 0.00 1 0.008 0 0 0.0 0.0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-12T06:00:10.340 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Cumulative compaction: 0.00 GB write, 177.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Interval compaction: 0.00 GB write, 177.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Block cache BinnedLRUCache@0x55d47359f1f0#7 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: starting mon.smithi194 rank 1 at public addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] at bind addrs [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi194 fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???) e2 preinit fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).mds e24 new map 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).mds e24 print_map 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: e24 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:00:10.341 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: legacy client fscid: 1 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: Filesystem 'cephfs' (1) 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: fs_name cephfs 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: epoch 24 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: modified 2024-04-12T06:00:00.958556+0000 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: tableserver 0 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: root 0 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: session_timeout 60 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: session_autoclose 300 2024-04-12T06:00:10.342 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: max_file_size 1099511627776 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: max_xattr_size 65536 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: required_client_features {} 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: last_failure 0 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: last_failure_osd_epoch 46 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: max_mds 2 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: in 0,1 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: up {0=14476,1=24277} 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: failed 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: damaged 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: stopped 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: data_pools [3] 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: metadata_pool 2 2024-04-12T06:00:10.343 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: inline_data disabled 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: balancer 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: bal_rank_mask -1 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: standby_count_wanted 1 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: [mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: [mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: [mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: [mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout: 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 ceph-mon[106982]: mon.smithi194@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-12T06:00:10.344 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 bash[106944]: e7818c807a5175e8170ec361c7477c1a1a1eb1cbf6557d893dd14356959846f1 2024-04-12T06:00:10.345 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:10 smithi194 systemd[1]: Started Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:00:11.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:11 smithi194 ceph-mon[106982]: pgmap v118: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:11.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T06:00:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T06:00:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:11 smithi008 ceph-mon[146084]: pgmap v118: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi008"}]: dispatch 2024-04-12T06:00:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mon metadata", "id": "smithi194"}]: dispatch 2024-04-12T06:00:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: pgmap v119: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: pgmap v120: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: mon.smithi194 calling monitor election 2024-04-12T06:00:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: mon.smithi008 calling monitor election 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: mgrmap e32: smithi008.ixmnlp(active, since 3m), standbys: smithi194.xbiyaj 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: overall HEALTH_OK 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: pgmap v119: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: pgmap v120: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: mon.smithi194 calling monitor election 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: mon.smithi008 calling monitor election 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: mon.smithi008 is new leader, mons smithi008,smithi194 in quorum (ranks 0,1) 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: monmap e2: 2 mons at {smithi008=[v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0],smithi194=[v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-12T06:00:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: osdmap e48: 6 total, 6 up, 6 in 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: mgrmap e32: smithi008.ixmnlp(active, since 3m), standbys: smithi194.xbiyaj 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: overall HEALTH_OK 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:14.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:14 smithi194 ceph-mon[106982]: pgmap v121: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:14.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:14 smithi008 ceph-mon[146084]: pgmap v121: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:15.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:15 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:15.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:15 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:15.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:15 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:15.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:15 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2 up:standby-replay 2024-04-12T06:00:15.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:15 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:15.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:15 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:16.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:16 smithi194 ceph-mon[106982]: pgmap v122: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:16 smithi008 ceph-mon[146084]: pgmap v122: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:17.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:17.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:18.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:18 smithi194 ceph-mon[106982]: pgmap v123: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:18.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:18 smithi008 ceph-mon[146084]: pgmap v123: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:21.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: pgmap v124: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: Detected new or changed devices on smithi194 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi008"}]: dispatch 2024-04-12T06:00:21.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi008"}]': finished 2024-04-12T06:00:21.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi194"}]: dispatch 2024-04-12T06:00:21.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi194"}]': finished 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.107+0000 7f63ee385700 1 -- 172.21.15.8:0/1195265199 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 msgr2=0x7f63e81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.107+0000 7f63ee385700 1 --2- 172.21.15.8:0/1195265199 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e81058a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009910 tx=0x7f63dc009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 -- 172.21.15.8:0/1195265199 shutdown_connections 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 --2- 172.21.15.8:0/1195265199 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e81058a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 --2- 172.21.15.8:0/1195265199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 0x7f63e8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 -- 172.21.15.8:0/1195265199 >> 172.21.15.8:0/1195265199 conn(0x7f63e80fc750 msgr2=0x7f63e80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 -- 172.21.15.8:0/1195265199 shutdown_connections 2024-04-12T06:00:21.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 -- 172.21.15.8:0/1195265199 wait complete. 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 Processor -- start 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.108+0000 7f63ee385700 1 -- start start 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63ee385700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 0x7f63e819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63ee385700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63ee385700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e819d1f0 con 0x7f63e8101850 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63ee385700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e81966a0 con 0x7f63e8100ea0 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.105 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56098/0 (socket says 172.21.15.8:56098) 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 -- 172.21.15.8:0/3241987899 learned_addr learned my addr 172.21.15.8:0/3241987899 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e7fff700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 0x7f63e819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 -- 172.21.15.8:0/3241987899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 msgr2=0x7f63e819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 0x7f63e819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.109+0000 7f63e77fe700 1 -- 172.21.15.8:0/3241987899 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc0095f0 con 0x7f63e8101850 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63e77fe700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005b40 tx=0x7f63dc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63dc01d070 con 0x7f63e8101850 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63dc005dc0 con 0x7f63e8101850 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e8196920 con 0x7f63e8101850 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63dc0039b0 con 0x7f63e8101850 2024-04-12T06:00:21.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.110+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e8196db0 con 0x7f63e8101850 2024-04-12T06:00:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.111+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e81092a0 con 0x7f63e8101850 2024-04-12T06:00:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.112+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f63dc003ba0 con 0x7f63e8101850 2024-04-12T06:00:21.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.112+0000 7f63e57fa700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 0x7f63d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.112+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f63dc08e2c0 con 0x7f63e8101850 2024-04-12T06:00:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.112+0000 7f63e7fff700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 0x7f63d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.113+0000 7f63e7fff700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 0x7f63d006f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c60 tx=0x7f63d8009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.116+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f63dc05bef0 con 0x7f63e8101850 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: pgmap v124: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: Detected new or changed devices on smithi194 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi008"}]: dispatch 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi008"}]': finished 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi194"}]: dispatch 2024-04-12T06:00:21.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi194"}]': finished 2024-04-12T06:00:21.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.339+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63e8197030 con 0x7f63d006cce0 2024-04-12T06:00:21.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.340+0000 7f63e57fa700 1 -- 172.21.15.8:0/3241987899 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f63e8197030 con 0x7f63d006cce0 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 msgr2=0x7f63d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 0x7f63d006f1a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009c60 tx=0x7f63d8009260 comp rx=0 tx=0).stop 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 msgr2=0x7f63e819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63dc005b40 tx=0x7f63dc00bde0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 shutdown_connections 2024-04-12T06:00:21.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d006cce0 0x7f63d006f1a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63e8100ea0 0x7f63e819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 --2- 172.21.15.8:0/3241987899 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63e8101850 0x7f63e819cb60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 >> 172.21.15.8:0/3241987899 conn(0x7f63e80fc750 msgr2=0x7f63e80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 shutdown_connections 2024-04-12T06:00:21.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.342+0000 7f63ee385700 1 -- 172.21.15.8:0/3241987899 wait complete. 2024-04-12T06:00:21.348 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 -- 172.21.15.8:0/1945734579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8068580 msgr2=0x7fc5b80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/1945734579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8068580 0x7fc5b80689e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009910 tx=0x7fc5a0009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 -- 172.21.15.8:0/1945734579 shutdown_connections 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/1945734579 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8068580 0x7fc5b80689e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/1945734579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8106cd0 0x7fc5b81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 -- 172.21.15.8:0/1945734579 >> 172.21.15.8:0/1945734579 conn(0x7fc5b8075670 msgr2=0x7fc5b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 -- 172.21.15.8:0/1945734579 shutdown_connections 2024-04-12T06:00:21.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 -- 172.21.15.8:0/1945734579 wait complete. 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.451+0000 7fc5bd50c700 1 Processor -- start 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5bd50c700 1 -- start start 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5bd50c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 0x7fc5b80ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5bd50c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5bd50c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b8100770 con 0x7fc5b8106cd0 2024-04-12T06:00:21.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5bd50c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b81008b0 con 0x7fc5b8068580 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56112/0 (socket says 172.21.15.8:56112) 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 -- 172.21.15.8:0/2442188206 learned_addr learned my addr 172.21.15.8:0/2442188206 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b6ffd700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 0x7fc5b80ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 -- 172.21.15.8:0/2442188206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 msgr2=0x7fc5b80ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 0x7fc5b80ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b67fc700 1 -- 172.21.15.8:0/2442188206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a00095f0 con 0x7fc5b8106cd0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.452+0000 7fc5b6ffd700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 0x7fc5b80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5b67fc700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b5c0 tx=0x7fc5a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a001d070 con 0x7fc5b8106cd0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5a000f460 con 0x7fc5b8106cd0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b8100b30 con 0x7fc5b8106cd0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5a0021640 con 0x7fc5b8106cd0 2024-04-12T06:00:21.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.453+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b81a2df0 con 0x7fc5b8106cd0 2024-04-12T06:00:21.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.454+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b81091f0 con 0x7fc5b8106cd0 2024-04-12T06:00:21.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.455+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc5a000f9b0 con 0x7fc5b8106cd0 2024-04-12T06:00:21.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.455+0000 7fc5affff700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 0x7fc5a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.456+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc5a008dc30 con 0x7fc5b8106cd0 2024-04-12T06:00:21.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.456+0000 7fc5b6ffd700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 0x7fc5a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.456+0000 7fc5b6ffd700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 0x7fc5a406f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80099d0 tx=0x7fc5a8008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.459+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc5a0057d50 con 0x7fc5b8106cd0 2024-04-12T06:00:21.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.690+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b802cce0 con 0x7fc5a406cce0 2024-04-12T06:00:21.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.691+0000 7fc5affff700 1 -- 172.21.15.8:0/2442188206 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fc5b802cce0 con 0x7fc5a406cce0 2024-04-12T06:00:21.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 msgr2=0x7fc5a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 0x7fc5a406f1a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80099d0 tx=0x7fc5a8008040 comp rx=0 tx=0).stop 2024-04-12T06:00:21.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 msgr2=0x7fc5b8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000b5c0 tx=0x7fc5a000be20 comp rx=0 tx=0).stop 2024-04-12T06:00:21.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 shutdown_connections 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5a406cce0 0x7fc5a406f1a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5b8068580 0x7fc5b80ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 --2- 172.21.15.8:0/2442188206 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5b8106cd0 0x7fc5b8100230 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 >> 172.21.15.8:0/2442188206 conn(0x7fc5b8075670 msgr2=0x7fc5b80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 shutdown_connections 2024-04-12T06:00:21.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.693+0000 7fc5bd50c700 1 -- 172.21.15.8:0/2442188206 wait complete. 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 -- 172.21.15.8:0/1695070005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 msgr2=0x7f488c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 --2- 172.21.15.8:0/1695070005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c0fffc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4880009910 tx=0x7f4880009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 -- 172.21.15.8:0/1695070005 shutdown_connections 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 --2- 172.21.15.8:0/1695070005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c0fffc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 --2- 172.21.15.8:0/1695070005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 0x7f488c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 -- 172.21.15.8:0/1695070005 >> 172.21.15.8:0/1695070005 conn(0x7f488c0fb610 msgr2=0x7f488c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 -- 172.21.15.8:0/1695070005 shutdown_connections 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.811+0000 7f4892beb700 1 -- 172.21.15.8:0/1695070005 wait complete. 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 Processor -- start 2024-04-12T06:00:21.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 -- start start 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 0x7f488c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c1980f0 con 0x7f488c0ffb40 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4892beb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c198230 con 0x7f488c105bf0 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4890987700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4890987700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56116/0 (socket says 172.21.15.8:56116) 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.812+0000 7f4890987700 1 -- 172.21.15.8:0/3635017819 learned_addr learned my addr 172.21.15.8:0/3635017819 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:21.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4890987700 1 -- 172.21.15.8:0/3635017819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 msgr2=0x7f488c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f488bfff700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 0x7f488c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4890987700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 0x7f488c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4890987700 1 -- 172.21.15.8:0/3635017819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800095f0 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4890987700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e920 tx=0x7f487c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c009790 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f487c00cba0 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c1a0600 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f487c018650 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.813+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c1a0ac0 con 0x7f488c0ffb40 2024-04-12T06:00:21.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.814+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c1080a0 con 0x7f488c0ffb40 2024-04-12T06:00:21.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.815+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f487c00cd10 con 0x7f488c0ffb40 2024-04-12T06:00:21.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.815+0000 7f4889ffb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 0x7f487406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:21.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.815+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f487c014070 con 0x7f488c0ffb40 2024-04-12T06:00:21.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.815+0000 7f488bfff700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 0x7f487406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:21.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.816+0000 7f488bfff700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 0x7f487406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f488000b6e0 tx=0x7f4880009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:21.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:21.819+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f487c052ab0 con 0x7f488c0ffb40 2024-04-12T06:00:22.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.034+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f488c0611d0 con 0x7f487406cce0 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.046+0000 7f4889ffb700 1 -- 172.21.15.8:0/3635017819 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f488c0611d0 con 0x7f487406cce0 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (2m) 22s ago 13m 17.5M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (13m) 22s ago 13m 8094k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (12m) 7s ago 12m 8056k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:00:22.041 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (13m) 22s ago 13m 6656k - 18.2.1 5be31c24972a 8e1a162f4d32 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (12m) 7s ago 12m 6651k - 18.2.1 5be31c24972a 229be7e92684 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (113s) 22s ago 13m 79.9M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (7m) 22s ago 7m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (7m) 22s ago 7m 71.6M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (7m) 7s ago 7m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (7m) 7s ago 7m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (4m) 22s ago 15m 505M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (3m) 7s ago 12m 449M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (29s) 22s ago 15m 25.7M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (12s) 7s ago 12m 24.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (3m) 22s ago 13m 17.0M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (2m) 7s ago 12m 18.0M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:00:22.042 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (11m) 22s ago 11m 241M 1228M 18.2.1 5be31c24972a aff7e206cf55 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (11m) 22s ago 11m 241M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (10m) 22s ago 10m 196M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (10m) 7s ago 10m 266M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (9m) 7s ago 9m 231M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (9m) 7s ago 9m 212M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:00:22.043 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (2m) 22s ago 13m 54.0M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.048+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 msgr2=0x7f487406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.048+0000 7f4892beb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 0x7f487406f1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f488000b6e0 tx=0x7f4880009f70 comp rx=0 tx=0).stop 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.048+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 msgr2=0x7f488c197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.048+0000 7f4892beb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e920 tx=0x7f487c00ece0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 shutdown_connections 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f487406cce0 0x7f487406f1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f488c0ffb40 0x7f488c197590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 --2- 172.21.15.8:0/3635017819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f488c105bf0 0x7f488c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 >> 172.21.15.8:0/3635017819 conn(0x7f488c0fb610 msgr2=0x7f488c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 shutdown_connections 2024-04-12T06:00:22.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.049+0000 7f4892beb700 1 -- 172.21.15.8:0/3635017819 wait complete. 2024-04-12T06:00:22.065 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:21 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all mon 2024-04-12T06:00:22.066 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:21 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:22.066 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:21 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T06:00:22.066 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:21 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:22.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:21 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all mon 2024-04-12T06:00:22.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:21 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:22.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:21 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi008", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T06:00:22.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:21 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:22.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 -- 172.21.15.8:0/3445570053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 msgr2=0x7f2ef0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/3445570053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0101280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009910 tx=0x7f2ee4009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:22.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 -- 172.21.15.8:0/3445570053 shutdown_connections 2024-04-12T06:00:22.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/3445570053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 0x7f2ef01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/3445570053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0101280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 -- 172.21.15.8:0/3445570053 >> 172.21.15.8:0/3445570053 conn(0x7f2ef00fc750 msgr2=0x7f2ef00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 -- 172.21.15.8:0/3445570053 shutdown_connections 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 -- 172.21.15.8:0/3445570053 wait complete. 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.157+0000 7f2ef71f1700 1 Processor -- start 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 -- start start 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 0x7f2ef00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0073b10 con 0x7f2ef0100ea0 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef0073c50 con 0x7f2ef0101850 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56134/0 (socket says 172.21.15.8:56134) 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 -- 172.21.15.8:0/1666032308 learned_addr learned my addr 172.21.15.8:0/1666032308 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2eeffff700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 0x7f2ef00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 -- 172.21.15.8:0/1666032308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 msgr2=0x7f2ef00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 0x7f2ef00735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 -- 172.21.15.8:0/1666032308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40095f0 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef4f8d700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006010 tx=0x7f2ee400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee401d070 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ee400f460 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.158+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef0073ed0 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.159+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee4021620 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.159+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef01a2c90 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.160+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef0109240 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.160+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2ee400fa40 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.161+0000 7f2eedffb700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 0x7f2ee006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.161+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2ee408d650 con 0x7f2ef0100ea0 2024-04-12T06:00:22.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.161+0000 7f2eeffff700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 0x7f2ee006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.161+0000 7f2eeffff700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 0x7f2ee006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005950 tx=0x7f2edc0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.165+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2ee4057790 con 0x7f2ef0100ea0 2024-04-12T06:00:22.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.459+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2ef0074660 con 0x7f2ef0100ea0 2024-04-12T06:00:22.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.459+0000 7f2eedffb700 1 -- 172.21.15.8:0/1666032308 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7f2ee405adb0 con 0x7f2ef0100ea0 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:00:22.455 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.461+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 msgr2=0x7f2ee006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.461+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 0x7f2ee006f1a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005950 tx=0x7f2edc0058e0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.461+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 msgr2=0x7f2ef0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.461+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006010 tx=0x7f2ee400be20 comp rx=0 tx=0).stop 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 shutdown_connections 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ee006cce0 0x7f2ee006f1a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2ef0100ea0 0x7f2ef0073090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 --2- 172.21.15.8:0/1666032308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2ef0101850 0x7f2ef00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 >> 172.21.15.8:0/1666032308 conn(0x7f2ef00fc750 msgr2=0x7f2ef00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 shutdown_connections 2024-04-12T06:00:22.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.462+0000 7f2ef71f1700 1 -- 172.21.15.8:0/1666032308 wait complete. 2024-04-12T06:00:22.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 -- 172.21.15.8:0/990081569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 msgr2=0x7f2d64101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/990081569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64101280 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009910 tx=0x7f2d58009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:22.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 -- 172.21.15.8:0/990081569 shutdown_connections 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/990081569 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 0x7f2d641058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/990081569 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64101280 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 -- 172.21.15.8:0/990081569 >> 172.21.15.8:0/990081569 conn(0x7f2d640fc750 msgr2=0x7f2d640feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 -- 172.21.15.8:0/990081569 shutdown_connections 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.568+0000 7f2d6abcd700 1 -- 172.21.15.8:0/990081569 wait complete. 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 Processor -- start 2024-04-12T06:00:22.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 -- start start 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 0x7f2d640735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64073b10 con 0x7f2d64100ea0 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d6abcd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64073c50 con 0x7f2d64101850 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56158/0 (socket says 172.21.15.8:56158) 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 -- 172.21.15.8:0/1146607810 learned_addr learned my addr 172.21.15.8:0/1146607810 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d63fff700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 0x7f2d640735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 -- 172.21.15.8:0/1146607810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 msgr2=0x7f2d640735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 0x7f2d640735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 -- 172.21.15.8:0/1146607810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580095f0 con 0x7f2d64100ea0 2024-04-12T06:00:22.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.569+0000 7f2d68969700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b5c0 tx=0x7f2d5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.570+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5801d070 con 0x7f2d64100ea0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.570+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d5800f460 con 0x7f2d64100ea0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.570+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d64073ed0 con 0x7f2d64100ea0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.570+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d58021640 con 0x7f2d64100ea0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.570+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d641a2c40 con 0x7f2d64100ea0 2024-04-12T06:00:22.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.571+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d64109240 con 0x7f2d64100ea0 2024-04-12T06:00:22.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.571+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2d5800f9b0 con 0x7f2d64100ea0 2024-04-12T06:00:22.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.572+0000 7f2d61ffb700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 0x7f2d5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.572+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2d5808d6f0 con 0x7f2d64100ea0 2024-04-12T06:00:22.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.572+0000 7f2d63fff700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 0x7f2d5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.572+0000 7f2d63fff700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 0x7f2d5406f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005ea0 tx=0x7f2d50005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.576+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2d58057810 con 0x7f2d64100ea0 2024-04-12T06:00:22.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.827+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d64108c20 con 0x7f2d64100ea0 2024-04-12T06:00:22.822 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.827+0000 7f2d61ffb700 1 -- 172.21.15.8:0/1146607810 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f2d5805ae30 con 0x7f2d64100ea0 2024-04-12T06:00:22.822 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:00:22.822 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:00:22.823 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:00:22.824 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:22.825 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 msgr2=0x7f2d5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 0x7f2d5406f1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d50005ea0 tx=0x7f2d50005e30 comp rx=0 tx=0).stop 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 msgr2=0x7f2d64073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800b5c0 tx=0x7f2d5800be20 comp rx=0 tx=0).stop 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 shutdown_connections 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2d5406cce0 0x7f2d5406f1a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2d64100ea0 0x7f2d64073090 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 --2- 172.21.15.8:0/1146607810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2d64101850 0x7f2d640735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 >> 172.21.15.8:0/1146607810 conn(0x7f2d640fc750 msgr2=0x7f2d640fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 shutdown_connections 2024-04-12T06:00:22.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.829+0000 7f2d6abcd700 1 -- 172.21.15.8:0/1146607810 wait complete. 2024-04-12T06:00:22.827 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 -- 172.21.15.8:0/2067958561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244068580 msgr2=0x7f42440689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2067958561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244068580 0x7f42440689e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4234009910 tx=0x7f4234009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 -- 172.21.15.8:0/2067958561 shutdown_connections 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2067958561 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244068580 0x7f42440689e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2067958561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244106cd0 0x7f42441070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 -- 172.21.15.8:0/2067958561 >> 172.21.15.8:0/2067958561 conn(0x7f4244075670 msgr2=0x7f4244075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 -- 172.21.15.8:0/2067958561 shutdown_connections 2024-04-12T06:00:22.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.938+0000 7f424bf5d700 1 -- 172.21.15.8:0/2067958561 wait complete. 2024-04-12T06:00:22.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 Processor -- start 2024-04-12T06:00:22.935 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 -- start start 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 0x7f424419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424419d290 con 0x7f4244106cd0 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f424bf5d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244196740 con 0x7f4244068580 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56178/0 (socket says 172.21.15.8:56178) 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 -- 172.21.15.8:0/2655901135 learned_addr learned my addr 172.21.15.8:0/2655901135 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f4249cf9700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 0x7f424419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 -- 172.21.15.8:0/2655901135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 msgr2=0x7f424419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 0x7f424419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f42494f8700 1 -- 172.21.15.8:0/2655901135 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42340095f0 con 0x7f4244106cd0 2024-04-12T06:00:22.936 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.939+0000 7f4249cf9700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 0x7f424419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f42494f8700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4234006010 tx=0x7f423400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423401d070 con 0x7f4244106cd0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42441969c0 con 0x7f4244106cd0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4244196eb0 con 0x7f4244106cd0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f423400f460 con 0x7f4244106cd0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.940+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4234017630 con 0x7f4244106cd0 2024-04-12T06:00:22.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.941+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4228005320 con 0x7f4244106cd0 2024-04-12T06:00:22.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.941+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4234003a60 con 0x7f4244106cd0 2024-04-12T06:00:22.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.942+0000 7f423affd700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 0x7f423006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:22.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.942+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f423408d6b0 con 0x7f4244106cd0 2024-04-12T06:00:22.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.942+0000 7f4249cf9700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 0x7f423006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:22.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.942+0000 7f4249cf9700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 0x7f423006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4240007900 tx=0x7f4240008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:22.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:22.946+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f423405b310 con 0x7f4244106cd0 2024-04-12T06:00:23.043 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:23.043 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: pgmap v125: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:23.043 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: Upgrade: Updating crash.smithi008 (1/2) 2024-04-12T06:00:23.043 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: Deploying daemon crash.smithi008 on smithi008 2024-04-12T06:00:23.044 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:23.044 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:22 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1666032308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: pgmap v125: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: Upgrade: Updating crash.smithi008 (1/2) 2024-04-12T06:00:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: Deploying daemon crash.smithi008 on smithi008 2024-04-12T06:00:23.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:23.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:22 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1666032308' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:23.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.183+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4228000bf0 con 0x7f423006cce0 2024-04-12T06:00:23.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.185+0000 7f423affd700 1 -- 172.21.15.8:0/2655901135 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f4228000bf0 con 0x7f423006cce0 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "4/23 daemons upgraded", 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading crash daemons", 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:00:23.180 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:00:23.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 msgr2=0x7f423006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 0x7f423006f1a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4240007900 tx=0x7f4240008040 comp rx=0 tx=0).stop 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 msgr2=0x7f424419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4234006010 tx=0x7f423400be20 comp rx=0 tx=0).stop 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 shutdown_connections 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f423006cce0 0x7f423006f1a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4244068580 0x7f424419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 --2- 172.21.15.8:0/2655901135 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4244106cd0 0x7f424419cc00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 >> 172.21.15.8:0/2655901135 conn(0x7f4244075670 msgr2=0x7f42440fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 shutdown_connections 2024-04-12T06:00:23.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.187+0000 7f424bf5d700 1 -- 172.21.15.8:0/2655901135 wait complete. 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 -- 172.21.15.8:0/1880799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 msgr2=0x7fb9ec0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 --2- 172.21.15.8:0/1880799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec0fffc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009910 tx=0x7fb9dc009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 -- 172.21.15.8:0/1880799428 shutdown_connections 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 --2- 172.21.15.8:0/1880799428 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec0fffc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 --2- 172.21.15.8:0/1880799428 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 0x7fb9ec105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 -- 172.21.15.8:0/1880799428 >> 172.21.15.8:0/1880799428 conn(0x7fb9ec0fb610 msgr2=0x7fb9ec0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 -- 172.21.15.8:0/1880799428 shutdown_connections 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.299+0000 7fb9f3749700 1 -- 172.21.15.8:0/1880799428 wait complete. 2024-04-12T06:00:23.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 Processor -- start 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 -- start start 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 0x7fb9ec101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec102230 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f3749700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec102370 con 0x7fb9ec105bf0 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f14e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f14e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56192/0 (socket says 172.21.15.8:56192) 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f14e5700 1 -- 172.21.15.8:0/428851824 learned_addr learned my addr 172.21.15.8:0/428851824 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.300+0000 7fb9f0ce4700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 0x7fb9ec101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f14e5700 1 -- 172.21.15.8:0/428851824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 msgr2=0x7fb9ec101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f14e5700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 0x7fb9ec101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f14e5700 1 -- 172.21.15.8:0/428851824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc0095f0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f14e5700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e920 tx=0x7fb9e800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e800c9d0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9e8010430 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9e8018650 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec1a28f0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.301+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec1a2ca0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.302+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb9e80105a0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.303+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec04f2e0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.303+0000 7fb9e27fc700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 0x7fb9d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.303+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb9e8014070 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.303+0000 7fb9f0ce4700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 0x7fb9d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:23.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.303+0000 7fb9f0ce4700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 0x7fb9d806f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec102fa0 tx=0x7fb9dc005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:23.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.311+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb9e8052ab0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.615+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb9ec102ce0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.616+0000 7fb9e27fc700 1 -- 172.21.15.8:0/428851824 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb9e80568d0 con 0x7fb9ec0ffb40 2024-04-12T06:00:23.611 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:00:23.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 msgr2=0x7fb9d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 0x7fb9d806f1a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec102fa0 tx=0x7fb9dc005a90 comp rx=0 tx=0).stop 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 msgr2=0x7fb9ec1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800e920 tx=0x7fb9e800ec30 comp rx=0 tx=0).stop 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 shutdown_connections 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb9d806cce0 0x7fb9d806f1a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb9ec0ffb40 0x7fb9ec1017b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 --2- 172.21.15.8:0/428851824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9ec105bf0 0x7fb9ec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 >> 172.21.15.8:0/428851824 conn(0x7fb9ec0fb610 msgr2=0x7fb9ec068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 shutdown_connections 2024-04-12T06:00:23.613 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:23.618+0000 7fb9f3749700 1 -- 172.21.15.8:0/428851824 wait complete. 2024-04-12T06:00:24.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:23 smithi194 ceph-mon[106982]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:24.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:23 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1146607810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:00:24.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:23 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/428851824' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:00:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:23 smithi008 ceph-mon[146084]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:23 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1146607810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:00:24.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:23 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/428851824' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:00:25.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:24 smithi194 ceph-mon[106982]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:25.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:24 smithi194 ceph-mon[106982]: pgmap v126: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:25.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:24 smithi008 ceph-mon[146084]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:25.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:24 smithi008 ceph-mon[146084]: pgmap v126: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:26.517 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:26.680 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:26.681 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:26 smithi194 ceph-mon[106982]: pgmap v127: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:26.681 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:26.681 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:26.681 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:26 smithi008 ceph-mon[146084]: pgmap v127: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:28.271 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: Upgrade: Updating crash.smithi194 (2/2) 2024-04-12T06:00:28.271 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:28.271 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T06:00:28.271 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:28.271 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: Deploying daemon crash.smithi194 on smithi194 2024-04-12T06:00:28.272 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:28 smithi194 ceph-mon[106982]: pgmap v128: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: Upgrade: Updating crash.smithi194 (2/2) 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi194", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: Deploying daemon crash.smithi194 on smithi194 2024-04-12T06:00:28.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:28 smithi008 ceph-mon[146084]: pgmap v128: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:30.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:30 smithi008 ceph-mon[146084]: pgmap v129: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:30.715 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:30 smithi194 ceph-mon[106982]: pgmap v129: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:32.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:32.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:32.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:32.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:32 smithi008 ceph-mon[146084]: pgmap v130: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:32.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:32.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:32.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:32.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:32 smithi194 ceph-mon[106982]: pgmap v130: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:33.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:33.323 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:34.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:34 smithi194 ceph-mon[106982]: pgmap v131: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:34.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:34 smithi008 ceph-mon[146084]: pgmap v131: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:35.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:35.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:36.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:36 smithi194 ceph-mon[106982]: pgmap v132: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:36.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:36.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:36 smithi008 ceph-mon[146084]: pgmap v132: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:38.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:38.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:38.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:38 smithi008 ceph-mon[146084]: pgmap v133: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:38.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:38.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:38.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:38 smithi194 ceph-mon[106982]: pgmap v133: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:40.800 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:40 smithi008 ceph-mon[146084]: pgmap v134: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:40.800 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:40.800 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:40 smithi194 ceph-mon[106982]: pgmap v134: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:41.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi008"}]: dispatch 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi008"}]': finished 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi194"}]: dispatch 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi194"}]': finished 2024-04-12T06:00:42.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:41 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T06:00:42.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi008"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi008"}]': finished 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi194"}]: dispatch 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi194"}]': finished 2024-04-12T06:00:42.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:41 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all crash 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: Upgrade: osd.0 is safe to restart 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: pgmap v135: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:43.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T06:00:43.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:43.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all crash 2024-04-12T06:00:43.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-12T06:00:43.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: Upgrade: osd.0 is safe to restart 2024-04-12T06:00:43.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: pgmap v135: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-12T06:00:43.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:00:44.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:43 smithi008 ceph-mon[146084]: Upgrade: Updating osd.0 2024-04-12T06:00:44.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:43 smithi008 ceph-mon[146084]: Deploying daemon osd.0 on smithi008 2024-04-12T06:00:44.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:43 smithi194 ceph-mon[106982]: Upgrade: Updating osd.0 2024-04-12T06:00:44.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:43 smithi194 ceph-mon[106982]: Deploying daemon osd.0 on smithi008 2024-04-12T06:00:45.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:44 smithi008 ceph-mon[146084]: pgmap v136: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:45.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:44 smithi194 ceph-mon[106982]: pgmap v136: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:00:46.293 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:46 smithi008 systemd[1]: Stopping Ceph osd.0 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:00:46.653 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:46 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[63659]: 2024-04-12T06:00:46.297+0000 7f4399a30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:00:46.653 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:46 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[63659]: 2024-04-12T06:00:46.297+0000 7f4399a30700 -1 osd.0 48 *** Got signal Terminated *** 2024-04-12T06:00:46.653 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:46 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[63659]: 2024-04-12T06:00:46.297+0000 7f4399a30700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:00:47.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:46 smithi008 ceph-mon[146084]: pgmap v137: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:47.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:46 smithi008 ceph-mon[146084]: osd.0 marked itself down and dead 2024-04-12T06:00:47.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:46 smithi194 ceph-mon[106982]: pgmap v137: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:00:47.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:46 smithi194 ceph-mon[106982]: osd.0 marked itself down and dead 2024-04-12T06:00:47.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:47 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:00:47.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:47 smithi008 ceph-mon[146084]: osdmap e49: 6 total, 5 up, 6 in 2024-04-12T06:00:47.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:47.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:47.901 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:47 smithi008 ceph-mon[146084]: osdmap e50: 6 total, 5 up, 6 in 2024-04-12T06:00:47.901 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:47 smithi008 bash[157421]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0 2024-04-12T06:00:48.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:47 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:00:48.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:47 smithi194 ceph-mon[106982]: osdmap e49: 6 total, 5 up, 6 in 2024-04-12T06:00:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:00:48.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:47 smithi194 ceph-mon[106982]: osdmap e50: 6 total, 5 up, 6 in 2024-04-12T06:00:49.119 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:48 smithi008 ceph-mon[146084]: pgmap v139: 65 pgs: 9 stale+active+clean, 56 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:00:49.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:48 smithi194 ceph-mon[106982]: pgmap v139: 65 pgs: 9 stale+active+clean, 56 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:00:49.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:49 smithi008 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.0.service: Succeeded. 2024-04-12T06:00:49.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:49 smithi008 systemd[1]: Stopped Ceph osd.0 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:00:49.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:49 smithi008 systemd[1]: Starting Ceph osd.0 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:00:50.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:49 smithi194 ceph-mon[106982]: pgmap v141: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:50.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:49 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded (PG_DEGRADED) 2024-04-12T06:00:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:49 smithi008 ceph-mon[146084]: pgmap v141: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:49 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded (PG_DEGRADED) 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-12T06:00:50.903 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:50 smithi008 bash[157700]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-12T06:00:52.153 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:51 smithi008 ceph-osd[158003]: -- 172.21.15.8:0/176794073 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55632e1b51e0 con 0x55632e185000 2024-04-12T06:00:52.153 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:51 smithi008 bash[157965]: eb3bf9318105d679f60c6fd1d0fdde2ca9f842d16cf7cda8ac7711684298ee8e 2024-04-12T06:00:52.153 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:51 smithi008 systemd[1]: Started Ceph osd.0 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:00:52.653 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:52 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:00:52.389+0000 7f68a91fe7c0 -1 Falling back to public interface 2024-04-12T06:00:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:52 smithi008 ceph-mon[146084]: pgmap v142: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 65/333 objects degraded (19.520%) 2024-04-12T06:00:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:52 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:52 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:52.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:52 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:52.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:52 smithi194 ceph-mon[106982]: pgmap v142: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 65/333 objects degraded (19.520%) 2024-04-12T06:00:52.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:52 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:52 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:52 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:00:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 -- 172.21.15.8:0/2070299197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30080ffc20 msgr2=0x7f3008100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2070299197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30080ffc20 0x7f3008100080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009910 tx=0x7f2ffc009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 -- 172.21.15.8:0/2070299197 shutdown_connections 2024-04-12T06:00:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2070299197 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30080ffc20 0x7f3008100080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2070299197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3008105bd0 0x7f3008105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 -- 172.21.15.8:0/2070299197 >> 172.21.15.8:0/2070299197 conn(0x7f3008075670 msgr2=0x7f3008075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:53.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 -- 172.21.15.8:0/2070299197 shutdown_connections 2024-04-12T06:00:53.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 -- 172.21.15.8:0/2070299197 wait complete. 2024-04-12T06:00:53.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.728+0000 7f300d1ae700 1 Processor -- start 2024-04-12T06:00:53.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300d1ae700 1 -- start start 2024-04-12T06:00:53.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300d1ae700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30080ffc20 0x7f300819c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300d1ae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300d1ae700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300819d1c0 con 0x7f3008105bd0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300d1ae700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3008196670 con 0x7f30080ffc20 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60472/0 (socket says 172.21.15.8:60472) 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 -- 172.21.15.8:0/2600235610 learned_addr learned my addr 172.21.15.8:0/2600235610 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f3006d9d700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30080ffc20 0x7f300819c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 -- 172.21.15.8:0/2600235610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30080ffc20 msgr2=0x7f300819c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30080ffc20 0x7f300819c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.729+0000 7f300659c700 1 -- 172.21.15.8:0/2600235610 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0095f0 con 0x7f3008105bd0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f300659c700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006010 tx=0x7f2ffc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ffc01d070 con 0x7f3008105bd0 2024-04-12T06:00:53.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2ffc00f460 con 0x7f3008105bd0 2024-04-12T06:00:53.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30081968f0 con 0x7f3008105bd0 2024-04-12T06:00:53.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ffc021640 con 0x7f3008105bd0 2024-04-12T06:00:53.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.730+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3008196de0 con 0x7f3008105bd0 2024-04-12T06:00:53.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.731+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300804f2e0 con 0x7f3008105bd0 2024-04-12T06:00:53.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.731+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2ffc00fa60 con 0x7f3008105bd0 2024-04-12T06:00:53.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.732+0000 7f2feffff700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 0x7f2ff006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:53.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.732+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2ffc08d770 con 0x7f3008105bd0 2024-04-12T06:00:53.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.732+0000 7f3006d9d700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 0x7f2ff006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:53.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.732+0000 7f3006d9d700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 0x7f2ff006f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005fd0 tx=0x7f2ff800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:53.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.737+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2ffc05b320 con 0x7f3008105bd0 2024-04-12T06:00:53.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.968+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3008197070 con 0x7f2ff006cce0 2024-04-12T06:00:53.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.969+0000 7f2feffff700 1 -- 172.21.15.8:0/2600235610 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f3008197070 con 0x7f2ff006cce0 2024-04-12T06:00:53.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 msgr2=0x7f2ff006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:53.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 0x7f2ff006f1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005fd0 tx=0x7f2ff800a560 comp rx=0 tx=0).stop 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 msgr2=0x7f300819cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006010 tx=0x7f2ffc00be20 comp rx=0 tx=0).stop 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 shutdown_connections 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2ff006cce0 0x7f2ff006f1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30080ffc20 0x7f300819c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 --2- 172.21.15.8:0/2600235610 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3008105bd0 0x7f300819cb30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.971+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 >> 172.21.15.8:0/2600235610 conn(0x7f3008075670 msgr2=0x7f3008103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.972+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 shutdown_connections 2024-04-12T06:00:53.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:53.972+0000 7f300d1ae700 1 -- 172.21.15.8:0/2600235610 wait complete. 2024-04-12T06:00:53.979 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 -- 172.21.15.8:0/578199728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064100e90 msgr2=0x7fb064101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/578199728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064100e90 0x7fb064101270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb054009910 tx=0x7fb054009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 -- 172.21.15.8:0/578199728 shutdown_connections 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/578199728 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064101840 0x7fb064105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/578199728 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064100e90 0x7fb064101270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 -- 172.21.15.8:0/578199728 >> 172.21.15.8:0/578199728 conn(0x7fb0640fc740 msgr2=0x7fb0640feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 -- 172.21.15.8:0/578199728 shutdown_connections 2024-04-12T06:00:54.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.094+0000 7fb06a0e7700 1 -- 172.21.15.8:0/578199728 wait complete. 2024-04-12T06:00:54.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 Processor -- start 2024-04-12T06:00:54.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 -- start start 2024-04-12T06:00:54.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064100e90 0x7fb06419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06419d190 con 0x7fb064101840 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb06a0e7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064196640 con 0x7fb064100e90 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb062ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb062ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60486/0 (socket says 172.21.15.8:60486) 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb062ffd700 1 -- 172.21.15.8:0/2700052522 learned_addr learned my addr 172.21.15.8:0/2700052522 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb0637fe700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064100e90 0x7fb06419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.095+0000 7fb062ffd700 1 -- 172.21.15.8:0/2700052522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064100e90 msgr2=0x7fb06419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb062ffd700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064100e90 0x7fb06419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb062ffd700 1 -- 172.21.15.8:0/2700052522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0540095f0 con 0x7fb064101840 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb062ffd700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb058007e90 tx=0x7fb05800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb05800dda0 con 0x7fb064101840 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb05800f040 con 0x7fb064101840 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064196920 con 0x7fb064101840 2024-04-12T06:00:54.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb058012580 con 0x7fb064101840 2024-04-12T06:00:54.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.096+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb064196e70 con 0x7fb064101840 2024-04-12T06:00:54.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.097+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb064109230 con 0x7fb064101840 2024-04-12T06:00:54.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.098+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb058012800 con 0x7fb064101840 2024-04-12T06:00:54.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.098+0000 7fb060ff9700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 0x7fb04c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.098+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb05808be10 con 0x7fb064101840 2024-04-12T06:00:54.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.098+0000 7fb0637fe700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 0x7fb04c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.098+0000 7fb0637fe700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 0x7fb04c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb054006010 tx=0x7fb05401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.103+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb0580599c0 con 0x7fb064101840 2024-04-12T06:00:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.325+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb064197090 con 0x7fb04c06cce0 2024-04-12T06:00:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.326+0000 7fb060ff9700 1 -- 172.21.15.8:0/2700052522 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb064197090 con 0x7fb04c06cce0 2024-04-12T06:00:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 msgr2=0x7fb04c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 0x7fb04c06f1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb054006010 tx=0x7fb05401a040 comp rx=0 tx=0).stop 2024-04-12T06:00:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 msgr2=0x7fb06419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb058007e90 tx=0x7fb05800d3b0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 shutdown_connections 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb04c06cce0 0x7fb04c06f1a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb064100e90 0x7fb06419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 --2- 172.21.15.8:0/2700052522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb064101840 0x7fb06419cb00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.328+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 >> 172.21.15.8:0/2700052522 conn(0x7fb0640fc740 msgr2=0x7fb0640fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.329+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 shutdown_connections 2024-04-12T06:00:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.329+0000 7fb06a0e7700 1 -- 172.21.15.8:0/2700052522 wait complete. 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 -- 172.21.15.8:0/3788624453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8106cd0 msgr2=0x7f95b81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 --2- 172.21.15.8:0/3788624453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8106cd0 0x7f95b81070b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95b4009910 tx=0x7f95b4009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 -- 172.21.15.8:0/3788624453 shutdown_connections 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 --2- 172.21.15.8:0/3788624453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8068580 0x7f95b80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 --2- 172.21.15.8:0/3788624453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8106cd0 0x7f95b81070b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 -- 172.21.15.8:0/3788624453 >> 172.21.15.8:0/3788624453 conn(0x7f95b8075670 msgr2=0x7f95b8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 -- 172.21.15.8:0/3788624453 shutdown_connections 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.438+0000 7f95c07e8700 1 -- 172.21.15.8:0/3788624453 wait complete. 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 Processor -- start 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 -- start start 2024-04-12T06:00:54.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8106cd0 0x7f95b819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b819d140 con 0x7f95b8068580 2024-04-12T06:00:54.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95c07e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b8196640 con 0x7f95b8106cd0 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95be584700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95be584700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60500/0 (socket says 172.21.15.8:60500) 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95be584700 1 -- 172.21.15.8:0/468646553 learned_addr learned my addr 172.21.15.8:0/468646553 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.439+0000 7f95bdd83700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8106cd0 0x7f95b819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95be584700 1 -- 172.21.15.8:0/468646553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8106cd0 msgr2=0x7f95b819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95be584700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8106cd0 0x7f95b819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95be584700 1 -- 172.21.15.8:0/468646553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b40095f0 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95be584700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95b40092e0 tx=0x7f95b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b401d070 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f95b4005e00 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b81968c0 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95b400fe90 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.440+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b8196db0 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.441+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b81091f0 con 0x7f95b8068580 2024-04-12T06:00:54.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.442+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f95b400f6d0 con 0x7f95b8068580 2024-04-12T06:00:54.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.442+0000 7f95af7fe700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 0x7f95a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.442+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f95b408d650 con 0x7f95b8068580 2024-04-12T06:00:54.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.442+0000 7f95bdd83700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 0x7f95a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.443+0000 7f95bdd83700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 0x7f95a406f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95b80fe390 tx=0x7f95a800b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.447+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f95b405b280 con 0x7f95b8068580 2024-04-12T06:00:54.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:54 smithi008 ceph-mon[146084]: pgmap v143: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:54.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.662+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b8197020 con 0x7f95a406cce0 2024-04-12T06:00:54.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.672+0000 7f95af7fe700 1 -- 172.21.15.8:0/468646553 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f95b8197020 con 0x7f95a406cce0 2024-04-12T06:00:54.667 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:00:54.667 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (2m) 18s ago 14m 17.6M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:00:54.667 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (14m) 18s ago 14m 8144k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:00:54.667 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (13m) 20s ago 13m 8082k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (30s) 18s ago 14m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (24s) 20s ago 13m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (2m) 18s ago 13m 80.0M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (8m) 18s ago 8m 122M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (8m) 18s ago 8m 71.5M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (7m) 20s ago 7m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (8m) 20s ago 8m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (4m) 18s ago 15m 506M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (3m) 20s ago 13m 450M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (62s) 18s ago 15m 43.5M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (45s) 20s ago 12m 30.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (3m) 18s ago 14m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (3m) 20s ago 13m 18.0M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:00:54.668 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 starting - - - 1228M 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (11m) 18s ago 11m 241M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (11m) 18s ago 11m 197M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (10m) 20s ago 10m 266M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (10m) 20s ago 10m 231M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (9m) 20s ago 9m 213M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:00:54.669 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (3m) 18s ago 13m 66.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 msgr2=0x7f95a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 0x7f95a406f1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95b80fe390 tx=0x7f95a800b3c0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 msgr2=0x7f95b819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95b40092e0 tx=0x7f95b400bd70 comp rx=0 tx=0).stop 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 shutdown_connections 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f95a406cce0 0x7f95a406f1a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95b8068580 0x7f95b819c570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 --2- 172.21.15.8:0/468646553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f95b8106cd0 0x7f95b819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.674+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 >> 172.21.15.8:0/468646553 conn(0x7f95b8075670 msgr2=0x7f95b80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.675+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 shutdown_connections 2024-04-12T06:00:54.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.675+0000 7f95c07e8700 1 -- 172.21.15.8:0/468646553 wait complete. 2024-04-12T06:00:54.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- 172.21.15.8:0/1835768866 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80068580 msgr2=0x7f3d800689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1835768866 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80068580 0x7f3d800689e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009910 tx=0x7f3d70009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- 172.21.15.8:0/1835768866 shutdown_connections 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1835768866 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80068580 0x7f3d800689e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1835768866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80106cd0 0x7f3d801070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- 172.21.15.8:0/1835768866 >> 172.21.15.8:0/1835768866 conn(0x7f3d80075670 msgr2=0x7f3d80075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- 172.21.15.8:0/1835768866 shutdown_connections 2024-04-12T06:00:54.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- 172.21.15.8:0/1835768866 wait complete. 2024-04-12T06:00:54.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 Processor -- start 2024-04-12T06:00:54.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.780+0000 7f3d879b6700 1 -- start start 2024-04-12T06:00:54.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80068580 0x7f3d8019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8019d110 con 0x7f3d80106cd0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d801965c0 con 0x7f3d80068580 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60524/0 (socket says 172.21.15.8:60524) 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 -- 172.21.15.8:0/1961699600 learned_addr learned my addr 172.21.15.8:0/1961699600 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d85752700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80068580 0x7f3d8019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 -- 172.21.15.8:0/1961699600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80068580 msgr2=0x7f3d8019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80068580 0x7f3d8019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 -- 172.21.15.8:0/1961699600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d700095f0 con 0x7f3d80106cd0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d84f51700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d700092e0 tx=0x7f3d7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d7001d070 con 0x7f3d80106cd0 2024-04-12T06:00:54.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d80196840 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d70005e00 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d7000fe60 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.781+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d80196d30 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.782+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d801091d0 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.783+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3d7000f6d0 con 0x7f3d80106cd0 2024-04-12T06:00:54.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.783+0000 7f3d767fc700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 0x7f3d6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:54.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.783+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3d7008d7a0 con 0x7f3d80106cd0 2024-04-12T06:00:54.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.783+0000 7f3d85752700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 0x7f3d6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:54.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.784+0000 7f3d85752700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 0x7f3d6c06f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005950 tx=0x7f3d7c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:54.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:54.788+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3d7005b350 con 0x7f3d80106cd0 2024-04-12T06:00:54.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:54 smithi194 ceph-mon[106982]: pgmap v143: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:55.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.094+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3d80197b70 con 0x7f3d80106cd0 2024-04-12T06:00:55.089 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.095+0000 7f3d767fc700 1 -- 172.21.15.8:0/1961699600 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+693 (secure 0 0 0) 0x7f3d7005aee0 con 0x7f3d80106cd0 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5 2024-04-12T06:00:55.090 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:00:55.091 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 msgr2=0x7f3d6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 0x7f3d6c06f1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005950 tx=0x7f3d7c0058e0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 msgr2=0x7f3d8019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d700092e0 tx=0x7f3d7000be20 comp rx=0 tx=0).stop 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 shutdown_connections 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6c06cce0 0x7f3d6c06f1a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d80068580 0x7f3d8019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 --2- 172.21.15.8:0/1961699600 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d80106cd0 0x7f3d8019ca80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 >> 172.21.15.8:0/1961699600 conn(0x7f3d80075670 msgr2=0x7f3d800fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 shutdown_connections 2024-04-12T06:00:55.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.097+0000 7f3d879b6700 1 -- 172.21.15.8:0/1961699600 wait complete. 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 -- 172.21.15.8:0/4150429034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0ff5a0 msgr2=0x7f97dc0ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 --2- 172.21.15.8:0/4150429034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc0ff980 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009910 tx=0x7f97d8009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 -- 172.21.15.8:0/4150429034 shutdown_connections 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 --2- 172.21.15.8:0/4150429034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0fff50 0x7f97dc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 --2- 172.21.15.8:0/4150429034 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc0ff980 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 -- 172.21.15.8:0/4150429034 >> 172.21.15.8:0/4150429034 conn(0x7f97dc074e30 msgr2=0x7f97dc075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 -- 172.21.15.8:0/4150429034 shutdown_connections 2024-04-12T06:00:55.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.207+0000 7f97e4322700 1 -- 172.21.15.8:0/4150429034 wait complete. 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 Processor -- start 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 -- start start 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc19d130 con 0x7f97dc0fff50 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e4322700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1965e0 con 0x7f97dc0ff5a0 2024-04-12T06:00:55.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60556/0 (socket says 172.21.15.8:60556) 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 -- 172.21.15.8:0/893195690 learned_addr learned my addr 172.21.15.8:0/893195690 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e20be700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 -- 172.21.15.8:0/893195690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0ff5a0 msgr2=0x7f97dc19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 -- 172.21.15.8:0/893195690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d80095f0 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.208+0000 7f97e18bd700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97d000d900 tx=0x7f97d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.209+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d0009900 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.209+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97d0010460 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.209+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc1968c0 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.209+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc196e10 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.209+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d000f5f0 con 0x7f97dc0fff50 2024-04-12T06:00:55.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.210+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0005320 con 0x7f97dc0fff50 2024-04-12T06:00:55.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.211+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f97d0009a60 con 0x7f97dc0fff50 2024-04-12T06:00:55.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.211+0000 7f97cf7fe700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.211+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f97d008c160 con 0x7f97dc0fff50 2024-04-12T06:00:55.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.212+0000 7f97e20be700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.212+0000 7f97e20be700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.215+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f97d0059e30 con 0x7f97dc0fff50 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.504+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f97c0005cc0 con 0x7f97dc0fff50 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.504+0000 7f97cf7fe700 1 -- 172.21.15.8:0/893195690 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f97d0017760 con 0x7f97dc0fff50 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:00:55.499 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:00:55.500 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:55.501 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:00:55.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 msgr2=0x7f97c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d801a040 comp rx=0 tx=0).stop 2024-04-12T06:00:55.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 msgr2=0x7f97dc19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97d000d900 tx=0x7f97d000dcc0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 shutdown_connections 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc0ff5a0 0x7f97dc19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 --2- 172.21.15.8:0/893195690 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc0fff50 0x7f97dc19caa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 >> 172.21.15.8:0/893195690 conn(0x7f97dc074e30 msgr2=0x7f97dc0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 shutdown_connections 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.506+0000 7f97e4322700 1 -- 172.21.15.8:0/893195690 wait complete. 2024-04-12T06:00:55.503 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:00:55.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 -- 172.21.15.8:0/2888320453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 msgr2=0x7fb5d4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/2888320453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d4105840 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009910 tx=0x7fb5c4009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:55.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 -- 172.21.15.8:0/2888320453 shutdown_connections 2024-04-12T06:00:55.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/2888320453 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d4105840 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/2888320453 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 0x7fb5d40ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 -- 172.21.15.8:0/2888320453 >> 172.21.15.8:0/2888320453 conn(0x7fb5d4074e30 msgr2=0x7fb5d4075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 -- 172.21.15.8:0/2888320453 shutdown_connections 2024-04-12T06:00:55.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.619+0000 7fb5dbaef700 1 -- 172.21.15.8:0/2888320453 wait complete. 2024-04-12T06:00:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 Processor -- start 2024-04-12T06:00:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 -- start start 2024-04-12T06:00:55.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 0x7fb5d419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d419d1d0 con 0x7fb5d40fff50 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5dbaef700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4196710 con 0x7fb5d40ff5a0 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5d908a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5d908a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60572/0 (socket says 172.21.15.8:60572) 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5d908a700 1 -- 172.21.15.8:0/3093108063 learned_addr learned my addr 172.21.15.8:0/3093108063 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.620+0000 7fb5d988b700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 0x7fb5d419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5d908a700 1 -- 172.21.15.8:0/3093108063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 msgr2=0x7fb5d419c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5d908a700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 0x7fb5d419c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5d908a700 1 -- 172.21.15.8:0/3093108063 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c40095f0 con 0x7fb5d40fff50 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5d908a700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400bda0 tx=0x7fb5c400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c401d070 con 0x7fb5d40fff50 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5c400f460 con 0x7fb5d40fff50 2024-04-12T06:00:55.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4196990 con 0x7fb5d40fff50 2024-04-12T06:00:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c4021640 con 0x7fb5d40fff50 2024-04-12T06:00:55.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.621+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4196e80 con 0x7fb5d40fff50 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.622+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4109200 con 0x7fb5d40fff50 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.623+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb5c400fa60 con 0x7fb5d40fff50 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.623+0000 7fb5caffd700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 0x7fb5c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.623+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb5c408d5e0 con 0x7fb5d40fff50 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.623+0000 7fb5d988b700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 0x7fb5c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.624+0000 7fb5d988b700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 0x7fb5c006f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000ac50 tx=0x7fb5d000a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.628+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb5c405b210 con 0x7fb5d40fff50 2024-04-12T06:00:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:55 smithi008 ceph-mon[146084]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:55 smithi008 ceph-mon[146084]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:55 smithi008 ceph-mon[146084]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:55 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1961699600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:55 smithi194 ceph-mon[106982]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:55 smithi194 ceph-mon[106982]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:55 smithi194 ceph-mon[106982]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:55.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:55 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1961699600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:00:55.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.876+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5d4197ce0 con 0x7fb5c006cce0 2024-04-12T06:00:55.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.877+0000 7fb5caffd700 1 -- 172.21.15.8:0/3093108063 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb5d4197ce0 con 0x7fb5c006cce0 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "6/23 daemons upgraded", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:00:55.873 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 msgr2=0x7fb5c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 0x7fb5c006f1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000ac50 tx=0x7fb5d000a380 comp rx=0 tx=0).stop 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 msgr2=0x7fb5d419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400bda0 tx=0x7fb5c400bdd0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 shutdown_connections 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb5c006cce0 0x7fb5c006f1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb5d40ff5a0 0x7fb5d419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 --2- 172.21.15.8:0/3093108063 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb5d40fff50 0x7fb5d419cbd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 >> 172.21.15.8:0/3093108063 conn(0x7fb5d4074e30 msgr2=0x7fb5d40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 shutdown_connections 2024-04-12T06:00:55.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.879+0000 7fb5dbaef700 1 -- 172.21.15.8:0/3093108063 wait complete. 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/1026641027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e0105bd0 msgr2=0x7fc4e0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/1026641027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e0105bd0 0x7fc4e0105fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009910 tx=0x7fc4d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/1026641027 shutdown_connections 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/1026641027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e00ffc20 0x7fc4e0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/1026641027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e0105bd0 0x7fc4e0105fb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/1026641027 >> 172.21.15.8:0/1026641027 conn(0x7fc4e0075670 msgr2=0x7fc4e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:55.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/1026641027 shutdown_connections 2024-04-12T06:00:55.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.987+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/1026641027 wait complete. 2024-04-12T06:00:55.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 Processor -- start 2024-04-12T06:00:55.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 -- start start 2024-04-12T06:00:55.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0105bd0 0x7fc4e019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d160 con 0x7fc4e00ffc20 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4e5dc6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e0196610 con 0x7fc4e0105bd0 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60582/0 (socket says 172.21.15.8:60582) 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 -- 172.21.15.8:0/3164884452 learned_addr learned my addr 172.21.15.8:0/3164884452 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4deffd700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0105bd0 0x7fc4e019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 -- 172.21.15.8:0/3164884452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0105bd0 msgr2=0x7fc4e019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0105bd0 0x7fc4e019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.988+0000 7fc4df7fe700 1 -- 172.21.15.8:0/3164884452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00095f0 con 0x7fc4e00ffc20 2024-04-12T06:00:55.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4df7fe700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00092e0 tx=0x7fc4d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d001d070 con 0x7fc4e00ffc20 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc4d0005e00 con 0x7fc4e00ffc20 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e0196890 con 0x7fc4e00ffc20 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4d000fe60 con 0x7fc4e00ffc20 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.989+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e0196d80 con 0x7fc4e00ffc20 2024-04-12T06:00:55.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.990+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e004f2e0 con 0x7fc4e00ffc20 2024-04-12T06:00:55.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.990+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc4d000f6d0 con 0x7fc4e00ffc20 2024-04-12T06:00:55.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.991+0000 7fc4dcff9700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 0x7fc4cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:00:55.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.991+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc4d008d7a0 con 0x7fc4e00ffc20 2024-04-12T06:00:55.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.991+0000 7fc4deffd700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 0x7fc4cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:00:55.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.991+0000 7fc4deffd700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 0x7fc4cc06f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005950 tx=0x7fc4c80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:00:55.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:55.995+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc4d005b350 con 0x7fc4e00ffc20 2024-04-12T06:00:56.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.295+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc4e0197030 con 0x7fc4e00ffc20 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.296+0000 7fc4dcff9700 1 -- 172.21.15.8:0/3164884452 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7fc4d005aee0 con 0x7fc4e00ffc20 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: osd.0 (root=default,host=smithi008) is down 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-12T06:00:56.292 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-12T06:00:56.293 INFO:teuthology.orchestra.run.smithi008.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-12T06:00:56.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 msgr2=0x7fc4cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:56.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 0x7fc4cc06f1a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8005950 tx=0x7fc4c80058e0 comp rx=0 tx=0).stop 2024-04-12T06:00:56.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 msgr2=0x7fc4e019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:00:56.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00092e0 tx=0x7fc4d000be20 comp rx=0 tx=0).stop 2024-04-12T06:00:56.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 shutdown_connections 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc4cc06cce0 0x7fc4cc06f1a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc4e00ffc20 0x7fc4e019c590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 --2- 172.21.15.8:0/3164884452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc4e0105bd0 0x7fc4e019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 >> 172.21.15.8:0/3164884452 conn(0x7fc4e0075670 msgr2=0x7fc4e0068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 shutdown_connections 2024-04-12T06:00:56.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:00:56.298+0000 7fc4e5dc6700 1 -- 172.21.15.8:0/3164884452 wait complete. 2024-04-12T06:00:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:56 smithi008 ceph-mon[146084]: pgmap v144: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:56 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/893195690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:00:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:56 smithi008 ceph-mon[146084]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:56 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3164884452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:00:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:56 smithi194 ceph-mon[106982]: pgmap v144: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:56 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/893195690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:00:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:56 smithi194 ceph-mon[106982]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:00:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:56 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3164884452' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:00:57.153 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:56 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:00:56.894+0000 7f68a91fe7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-12T06:00:57.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:57 smithi008 ceph-mon[146084]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T06:00:57.774 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:57 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:00:57.332+0000 7f68a91fe7c0 -1 osd.0 48 log_to_monitors true 2024-04-12T06:00:57.774 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:00:57 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:00:57.437+0000 7f689a658700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:00:57.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:57 smithi194 ceph-mon[106982]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-12T06:00:58.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: pgmap v145: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:58.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: osdmap e51: 6 total, 5 up, 6 in 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: pgmap v145: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: osdmap e51: 6 total, 5 up, 6 in 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:58.900 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:00:59.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:59 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:00:59.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:59 smithi194 ceph-mon[106982]: osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533] boot 2024-04-12T06:00:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:59 smithi194 ceph-mon[106982]: osdmap e52: 6 total, 6 up, 6 in 2024-04-12T06:00:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T06:00:59.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:00:59 smithi194 ceph-mon[106982]: osdmap e53: 6 total, 6 up, 6 in 2024-04-12T06:00:59.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:59 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:00:59.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:59 smithi008 ceph-mon[146084]: osd.0 [v2:172.21.15.8:6802/1483673533,v1:172.21.15.8:6803/1483673533] boot 2024-04-12T06:00:59.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:59 smithi008 ceph-mon[146084]: osdmap e52: 6 total, 6 up, 6 in 2024-04-12T06:00:59.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-12T06:00:59.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:00:59 smithi008 ceph-mon[146084]: osdmap e53: 6 total, 6 up, 6 in 2024-04-12T06:01:00.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:00 smithi008 ceph-mon[146084]: pgmap v148: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:01:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:00 smithi194 ceph-mon[106982]: pgmap v148: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:01:02.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: pgmap v150: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:02.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: pgmap v150: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 65/333 objects degraded (19.520%) 2024-04-12T06:01:02.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.910 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.910 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:02.910 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:01:02.910 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:02.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:03.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:03 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:03.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:03 smithi194 ceph-mon[106982]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-12T06:01:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:03 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:03 smithi008 ceph-mon[146084]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-12T06:01:04.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:04 smithi194 ceph-mon[106982]: pgmap v151: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-12T06:01:04.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:04 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded) 2024-04-12T06:01:04.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:04 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:01:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:04 smithi008 ceph-mon[146084]: pgmap v151: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-12T06:01:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:04 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 65/333 objects degraded (19.520%), 19 pgs degraded) 2024-04-12T06:01:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:04 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:01:06.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:06 smithi194 ceph-mon[106982]: pgmap v152: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:06.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:06 smithi008 ceph-mon[146084]: pgmap v152: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:08.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:08 smithi194 ceph-mon[106982]: pgmap v153: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-12T06:01:08.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:08 smithi008 ceph-mon[146084]: pgmap v153: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-12T06:01:10.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:10 smithi194 ceph-mon[106982]: pgmap v154: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:01:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:10 smithi008 ceph-mon[146084]: pgmap v154: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:01:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:12 smithi194 ceph-mon[106982]: pgmap v155: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:01:12.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:12 smithi008 ceph-mon[146084]: pgmap v155: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:01:12.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:14.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:14 smithi194 ceph-mon[106982]: pgmap v156: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:14.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:14 smithi008 ceph-mon[146084]: pgmap v156: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:16.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:16 smithi194 ceph-mon[106982]: pgmap v157: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:16 smithi008 ceph-mon[146084]: pgmap v157: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: pgmap v158: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:18.603 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:18 smithi008 ceph-mon[146084]: Upgrade: osd.1 is safe to restart 2024-04-12T06:01:18.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: pgmap v158: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:18.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:18.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:18.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-12T06:01:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:18 smithi194 ceph-mon[106982]: Upgrade: osd.1 is safe to restart 2024-04-12T06:01:20.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: Upgrade: Updating osd.1 2024-04-12T06:01:20.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:20.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T06:01:20.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:20.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: Deploying daemon osd.1 on smithi008 2024-04-12T06:01:20.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:20 smithi194 ceph-mon[106982]: pgmap v159: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: Upgrade: Updating osd.1 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: Deploying daemon osd.1 on smithi008 2024-04-12T06:01:20.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:20 smithi008 ceph-mon[146084]: pgmap v159: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:22.563 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:22 smithi008 ceph-mon[146084]: pgmap v160: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:22.563 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:22 smithi008 systemd[1]: Stopping Ceph osd.1 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:01:22.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:22 smithi194 ceph-mon[106982]: pgmap v160: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:22.902 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:22 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[73600]: 2024-04-12T06:01:22.567+0000 7fa384d67700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:01:22.903 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:22 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[73600]: 2024-04-12T06:01:22.567+0000 7fa384d67700 -1 osd.1 53 *** Got signal Terminated *** 2024-04-12T06:01:22.903 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:22 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[73600]: 2024-04-12T06:01:22.567+0000 7fa384d67700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:01:23.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:23 smithi194 ceph-mon[106982]: osd.1 marked itself down and dead 2024-04-12T06:01:23.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:23 smithi008 ceph-mon[146084]: osd.1 marked itself down and dead 2024-04-12T06:01:24.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:24 smithi008 bash[164123]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1 2024-04-12T06:01:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:24 smithi008 ceph-mon[146084]: pgmap v161: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:24 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:01:24.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:24 smithi008 ceph-mon[146084]: osdmap e54: 6 total, 5 up, 6 in 2024-04-12T06:01:24.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:24 smithi194 ceph-mon[106982]: pgmap v161: 65 pgs: 65 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:24.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:24 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:01:24.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:24 smithi194 ceph-mon[106982]: osdmap e54: 6 total, 5 up, 6 in 2024-04-12T06:01:25.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:25 smithi194 ceph-mon[106982]: osdmap e55: 6 total, 5 up, 6 in 2024-04-12T06:01:25.863 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:25 smithi008 ceph-mon[146084]: osdmap e55: 6 total, 5 up, 6 in 2024-04-12T06:01:26.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 -- 172.21.15.8:0/1725132304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 msgr2=0x7f69481070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1725132304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f69481070b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6938009910 tx=0x7f6938009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 -- 172.21.15.8:0/1725132304 shutdown_connections 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1725132304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 0x7f69480689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1725132304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f69481070b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 -- 172.21.15.8:0/1725132304 >> 172.21.15.8:0/1725132304 conn(0x7f6948075670 msgr2=0x7f6948075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 -- 172.21.15.8:0/1725132304 shutdown_connections 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.414+0000 7f694d1a0700 1 -- 172.21.15.8:0/1725132304 wait complete. 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 Processor -- start 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 -- start start 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 0x7f694819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819d130 con 0x7f6948106cd0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694d1a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6948196630 con 0x7f6948068580 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44908/0 (socket says 172.21.15.8:44908) 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f694659c700 1 -- 172.21.15.8:0/1845435950 learned_addr learned my addr 172.21.15.8:0/1845435950 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.415+0000 7f6946d9d700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 0x7f694819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694659c700 1 -- 172.21.15.8:0/1845435950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 msgr2=0x7f694819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694659c700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 0x7f694819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694659c700 1 -- 172.21.15.8:0/1845435950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69380095f0 con 0x7f6948106cd0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694659c700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d8d0 tx=0x7f693c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693c0098a0 con 0x7f6948106cd0 2024-04-12T06:01:26.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948196910 con 0x7f6948106cd0 2024-04-12T06:01:26.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f693c010460 con 0x7f6948106cd0 2024-04-12T06:01:26.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693c009ea0 con 0x7f6948106cd0 2024-04-12T06:01:26.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.416+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948196e60 con 0x7f6948106cd0 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.418+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69481091d0 con 0x7f6948106cd0 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.419+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f693c0105d0 con 0x7f6948106cd0 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.419+0000 7f692ffff700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 0x7f693006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.420+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f693c08c460 con 0x7f6948106cd0 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.420+0000 7f6946d9d700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 0x7f693006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.420+0000 7f6946d9d700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 0x7f693006f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f693801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:26.419 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.424+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f693c05a090 con 0x7f6948106cd0 2024-04-12T06:01:26.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:26 smithi008 ceph-mon[146084]: pgmap v164: 65 pgs: 12 stale+active+clean, 53 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:26.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:26 smithi008 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.1.service: Succeeded. 2024-04-12T06:01:26.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:26 smithi008 systemd[1]: Stopped Ceph osd.1 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:01:26.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:26 smithi008 systemd[1]: Starting Ceph osd.1 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:01:26.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.660+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6948197080 con 0x7f693006cce0 2024-04-12T06:01:26.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.662+0000 7f692ffff700 1 -- 172.21.15.8:0/1845435950 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6948197080 con 0x7f693006cce0 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 msgr2=0x7f693006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 0x7f693006f1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6938006010 tx=0x7f693801a040 comp rx=0 tx=0).stop 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 msgr2=0x7f694819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d8d0 tx=0x7f693c00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 shutdown_connections 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f693006cce0 0x7f693006f1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6948068580 0x7f694819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 --2- 172.21.15.8:0/1845435950 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6948106cd0 0x7f694819caa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 >> 172.21.15.8:0/1845435950 conn(0x7f6948075670 msgr2=0x7f69480fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 shutdown_connections 2024-04-12T06:01:26.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.664+0000 7f694d1a0700 1 -- 172.21.15.8:0/1845435950 wait complete. 2024-04-12T06:01:26.669 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:01:26.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:26 smithi194 ceph-mon[106982]: pgmap v164: 65 pgs: 12 stale+active+clean, 53 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:26.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 -- 172.21.15.8:0/3046212216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc105bd0 msgr2=0x7f62dc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 --2- 172.21.15.8:0/3046212216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc105bd0 0x7f62dc105fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009910 tx=0x7f62cc009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:26.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 -- 172.21.15.8:0/3046212216 shutdown_connections 2024-04-12T06:01:26.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 --2- 172.21.15.8:0/3046212216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc0ffc20 0x7f62dc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 --2- 172.21.15.8:0/3046212216 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc105bd0 0x7f62dc105fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 -- 172.21.15.8:0/3046212216 >> 172.21.15.8:0/3046212216 conn(0x7f62dc075670 msgr2=0x7f62dc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:26.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 -- 172.21.15.8:0/3046212216 shutdown_connections 2024-04-12T06:01:26.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.772+0000 7f62e115a700 1 -- 172.21.15.8:0/3046212216 wait complete. 2024-04-12T06:01:26.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 Processor -- start 2024-04-12T06:01:26.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 -- start start 2024-04-12T06:01:26.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc105bd0 0x7f62dc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc19d150 con 0x7f62dc0ffc20 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.773+0000 7f62e115a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc196600 con 0x7f62dc105bd0 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44924/0 (socket says 172.21.15.8:44924) 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 -- 172.21.15.8:0/3459695638 learned_addr learned my addr 172.21.15.8:0/3459695638 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62da59c700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc105bd0 0x7f62dc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 -- 172.21.15.8:0/3459695638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc105bd0 msgr2=0x7f62dc19cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:26.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc105bd0 0x7f62dc19cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 -- 172.21.15.8:0/3459695638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0095f0 con 0x7f62dc0ffc20 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62dad9d700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc01d070 con 0x7f62dc0ffc20 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f62cc005dc0 con 0x7f62dc0ffc20 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62cc003940 con 0x7f62dc0ffc20 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc196880 con 0x7f62dc0ffc20 2024-04-12T06:01:26.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.774+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc196cc0 con 0x7f62dc0ffc20 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.775+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc04f2e0 con 0x7f62dc0ffc20 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.776+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f62cc004ae0 con 0x7f62dc0ffc20 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.776+0000 7f62d3fff700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 0x7f62c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.776+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f62cc08db50 con 0x7f62dc0ffc20 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.776+0000 7f62da59c700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 0x7f62c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:26.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.777+0000 7f62da59c700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 0x7f62c806f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0ffb20 tx=0x7f62c4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:26.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:26.781+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f62cc05b660 con 0x7f62dc0ffc20 2024-04-12T06:01:27.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.014+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62dc196fc0 con 0x7f62c806cce0 2024-04-12T06:01:27.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.015+0000 7f62d3fff700 1 -- 172.21.15.8:0/3459695638 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f62dc196fc0 con 0x7f62c806cce0 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 msgr2=0x7f62c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 0x7f62c806f1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0ffb20 tx=0x7f62c4009260 comp rx=0 tx=0).stop 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 msgr2=0x7f62dc19c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62cc000c00 tx=0x7f62cc00bde0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 shutdown_connections 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f62c806cce0 0x7f62c806f1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f62dc0ffc20 0x7f62dc19c580 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 --2- 172.21.15.8:0/3459695638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62dc105bd0 0x7f62dc19cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 >> 172.21.15.8:0/3459695638 conn(0x7f62dc075670 msgr2=0x7f62dc068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 shutdown_connections 2024-04-12T06:01:27.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.017+0000 7f62e115a700 1 -- 172.21.15.8:0/3459695638 wait complete. 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3854600724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 msgr2=0x7fd4881058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3854600724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4881058a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd474009910 tx=0x7fd474009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3854600724 shutdown_connections 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3854600724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4881058a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3854600724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 0x7fd488101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3854600724 >> 172.21.15.8:0/3854600724 conn(0x7fd4880fc750 msgr2=0x7fd4880feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3854600724 shutdown_connections 2024-04-12T06:01:27.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.127+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3854600724 wait complete. 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 Processor -- start 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 -- start start 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 0x7fd488073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488073b10 con 0x7fd488101850 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48ea7a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd488073c50 con 0x7fd488100ea0 2024-04-12T06:01:27.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44944/0 (socket says 172.21.15.8:44944) 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 -- 172.21.15.8:0/3558699342 learned_addr learned my addr 172.21.15.8:0/3558699342 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd48c816700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 0x7fd488073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 -- 172.21.15.8:0/3558699342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 msgr2=0x7fd488073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 0x7fd488073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.128+0000 7fd487fff700 1 -- 172.21.15.8:0/3558699342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740095f0 con 0x7fd488101850 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd487fff700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4740092e0 tx=0x7fd47400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47401d070 con 0x7fd488101850 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd474005dc0 con 0x7fd488101850 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488073ed0 con 0x7fd488101850 2024-04-12T06:01:27.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47400fea0 con 0x7fd488101850 2024-04-12T06:01:27.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.129+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4881a2d50 con 0x7fd488101850 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.130+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4881092a0 con 0x7fd488101850 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.131+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd474003a60 con 0x7fd488101850 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.131+0000 7fd485ffb700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 0x7fd47806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.131+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd47408d680 con 0x7fd488101850 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.131+0000 7fd48c816700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 0x7fd47806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.132+0000 7fd48c816700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 0x7fd47806f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005950 tx=0x7fd47c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.136+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd4740577a0 con 0x7fd488101850 2024-04-12T06:01:27.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.350+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4880746d0 con 0x7fd47806cce0 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.361+0000 7fd485ffb700 1 -- 172.21.15.8:0/3558699342 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd4880746d0 con 0x7fd47806cce0 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (3m) 29s ago 14m 22.2M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (14m) 29s ago 14m 8170k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (13m) 52s ago 13m 8082k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (62s) 29s ago 14m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (56s) 52s ago 13m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:01:27.356 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (2m) 29s ago 14m 80.0M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (8m) 29s ago 8m 122M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (8m) 29s ago 8m 71.2M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (8m) 52s ago 8m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (8m) 52s ago 8m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (5m) 29s ago 16m 508M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (4m) 52s ago 13m 450M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (94s) 29s ago 16m 43.4M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (77s) 52s ago 13m 30.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (4m) 29s ago 14m 16.9M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (4m) 52s ago 13m 18.0M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (35s) 29s ago 12m 11.5M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (12m) 29s ago 12m 240M 1228M 18.2.1 5be31c24972a 37ac4528bcb6 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (11m) 29s ago 11m 197M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T06:01:27.357 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (11m) 52s ago 11m 266M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T06:01:27.358 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (10m) 52s ago 10m 231M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:01:27.358 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (10m) 52s ago 10m 213M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:01:27.358 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (3m) 29s ago 14m 66.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:01:27.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 msgr2=0x7fd47806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 0x7fd47806f1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005950 tx=0x7fd47c00b410 comp rx=0 tx=0).stop 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 msgr2=0x7fd4880735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4740092e0 tx=0x7fd47400bd70 comp rx=0 tx=0).stop 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 shutdown_connections 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd47806cce0 0x7fd47806f1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd488100ea0 0x7fd488073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 --2- 172.21.15.8:0/3558699342 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd488101850 0x7fd4880735d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 >> 172.21.15.8:0/3558699342 conn(0x7fd4880fc750 msgr2=0x7fd4880fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.363+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 shutdown_connections 2024-04-12T06:01:27.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.364+0000 7fd48ea7a700 1 -- 172.21.15.8:0/3558699342 wait complete. 2024-04-12T06:01:27.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 -- 172.21.15.8:0/2290785096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 msgr2=0x7fb6e40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/2290785096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e40689e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8009910 tx=0x7fb6d8009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:27.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 -- 172.21.15.8:0/2290785096 shutdown_connections 2024-04-12T06:01:27.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/2290785096 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e40689e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/2290785096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 -- 172.21.15.8:0/2290785096 >> 172.21.15.8:0/2290785096 conn(0x7fb6e4075670 msgr2=0x7fb6e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 -- 172.21.15.8:0/2290785096 shutdown_connections 2024-04-12T06:01:27.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.475+0000 7fb6e97d4700 1 -- 172.21.15.8:0/2290785096 wait complete. 2024-04-12T06:01:27.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 Processor -- start 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 -- start start 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e419d110 con 0x7fb6e4068580 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e97d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41965c0 con 0x7fb6e4106cd0 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44958/0 (socket says 172.21.15.8:44958) 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e2ffd700 1 -- 172.21.15.8:0/496072120 learned_addr learned my addr 172.21.15.8:0/496072120 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.476+0000 7fb6e27fc700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e2ffd700 1 -- 172.21.15.8:0/496072120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 msgr2=0x7fb6e419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e2ffd700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e2ffd700 1 -- 172.21.15.8:0/496072120 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80095f0 con 0x7fb6e4068580 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e27fc700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e2ffd700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e920 tx=0x7fb6d400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d4009790 con 0x7fb6e4068580 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb6d400cba0 con 0x7fb6e4068580 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e41968a0 con 0x7fb6e4068580 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d4018650 con 0x7fb6e4068580 2024-04-12T06:01:27.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.477+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e4196df0 con 0x7fb6e4068580 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.478+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e41091d0 con 0x7fb6e4068580 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.479+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb6d400cd10 con 0x7fb6e4068580 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.479+0000 7fb6cbfff700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 0x7fb6cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.479+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb6d4014070 con 0x7fb6e4068580 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.479+0000 7fb6e27fc700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 0x7fb6cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.480+0000 7fb6e27fc700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 0x7fb6cc06f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80059d0 tx=0x7fb6d8005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.485+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb6d4052ab0 con 0x7fb6e4068580 2024-04-12T06:01:27.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.791+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb6e404f2e0 con 0x7fb6e4068580 2024-04-12T06:01:27.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.792+0000 7fb6cbfff700 1 -- 172.21.15.8:0/496072120 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fb6d405a1b0 con 0x7fb6e4068580 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T06:01:27.787 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 5 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:01:27.788 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 msgr2=0x7fb6cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 0x7fb6cc06f1a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80059d0 tx=0x7fb6d8005940 comp rx=0 tx=0).stop 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 msgr2=0x7fb6e419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400e920 tx=0x7fb6d400ece0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 shutdown_connections 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb6cc06cce0 0x7fb6cc06f1a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb6e4068580 0x7fb6e419c540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 --2- 172.21.15.8:0/496072120 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb6e4106cd0 0x7fb6e419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 >> 172.21.15.8:0/496072120 conn(0x7fb6e4075670 msgr2=0x7fb6e40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 shutdown_connections 2024-04-12T06:01:27.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.794+0000 7fb6e97d4700 1 -- 172.21.15.8:0/496072120 wait complete. 2024-04-12T06:01:27.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:27 smithi194 ceph-mon[106982]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:27.872 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:27 smithi008 ceph-mon[146084]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-12T06:01:27.873 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:27 smithi008 bash[164421]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 -- 172.21.15.8:0/3044769582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 msgr2=0x7f48041058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 --2- 172.21.15.8:0/3044769582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f48041058a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009910 tx=0x7f47f4009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 -- 172.21.15.8:0/3044769582 shutdown_connections 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 --2- 172.21.15.8:0/3044769582 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f48041058a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 --2- 172.21.15.8:0/3044769582 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f4804101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 -- 172.21.15.8:0/3044769582 >> 172.21.15.8:0/3044769582 conn(0x7f48040fc750 msgr2=0x7f48040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 -- 172.21.15.8:0/3044769582 shutdown_connections 2024-04-12T06:01:27.897 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.902+0000 7f480b822700 1 -- 172.21.15.8:0/3044769582 wait complete. 2024-04-12T06:01:27.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 Processor -- start 2024-04-12T06:01:27.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 -- start start 2024-04-12T06:01:27.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f480419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480419f370 con 0x7f4804101850 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f480b822700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4804198820 con 0x7f4804100ea0 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44980/0 (socket says 172.21.15.8:44980) 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 -- 172.21.15.8:0/1466015815 learned_addr learned my addr 172.21.15.8:0/1466015815 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f48095be700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f480419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 -- 172.21.15.8:0/1466015815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 msgr2=0x7f480419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f480419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f4808dbd700 1 -- 172.21.15.8:0/1466015815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40095f0 con 0x7f4804101850 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.903+0000 7f48095be700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f480419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f4808dbd700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47f400bd70 tx=0x7f47f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f401d070 con 0x7f4804101850 2024-04-12T06:01:27.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47f400f460 con 0x7f4804101850 2024-04-12T06:01:27.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4804198aa0 con 0x7f4804101850 2024-04-12T06:01:27.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47f4021640 con 0x7f4804101850 2024-04-12T06:01:27.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.904+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4804198f90 con 0x7f4804101850 2024-04-12T06:01:27.901 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.905+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480404f2e0 con 0x7f4804101850 2024-04-12T06:01:27.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.905+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f47f400f9e0 con 0x7f4804101850 2024-04-12T06:01:27.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.906+0000 7f47fa7fc700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 0x7f47f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:27.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.906+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f47f408d6d0 con 0x7f4804101850 2024-04-12T06:01:27.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.906+0000 7f48095be700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 0x7f47f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:27.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.906+0000 7f48095be700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 0x7f47f006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4800005950 tx=0x7f48000058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:27.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:27.910+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f47f405b280 con 0x7f4804101850 2024-04-12T06:01:28.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.154+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f48041991b0 con 0x7f4804101850 2024-04-12T06:01:28.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.154+0000 7f47fa7fc700 1 -- 172.21.15.8:0/1466015815 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f47f408be60 con 0x7f4804101850 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:01:28.150 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:01:28.151 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:01:28.152 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:01:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 msgr2=0x7f47f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 0x7f47f006f1a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4800005950 tx=0x7f48000058e0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 msgr2=0x7f480419ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f47f400bd70 tx=0x7f47f400be50 comp rx=0 tx=0).stop 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 shutdown_connections 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47f006cce0 0x7f47f006f1a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4804100ea0 0x7f480419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 --2- 172.21.15.8:0/1466015815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4804101850 0x7f480419ece0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.156+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 >> 172.21.15.8:0/1466015815 conn(0x7f48040fc750 msgr2=0x7f4804104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.157+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 shutdown_connections 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.157+0000 7f480b822700 1 -- 172.21.15.8:0/1466015815 wait complete. 2024-04-12T06:01:28.154 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:01:28.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 -- 172.21.15.8:0/2718462559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 msgr2=0x7f4d0c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 --2- 172.21.15.8:0/2718462559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c1070b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:28.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 -- 172.21.15.8:0/2718462559 shutdown_connections 2024-04-12T06:01:28.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 --2- 172.21.15.8:0/2718462559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 --2- 172.21.15.8:0/2718462559 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c1070b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.264+0000 7f4d132d4700 1 -- 172.21.15.8:0/2718462559 >> 172.21.15.8:0/2718462559 conn(0x7f4d0c075670 msgr2=0x7f4d0c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:28.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 -- 172.21.15.8:0/2718462559 shutdown_connections 2024-04-12T06:01:28.260 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 -- 172.21.15.8:0/2718462559 wait complete. 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 Processor -- start 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 -- start start 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c19d130 con 0x7f4d0c106cd0 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.265+0000 7f4d132d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c196630 con 0x7f4d0c068580 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44996/0 (socket says 172.21.15.8:44996) 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 -- 172.21.15.8:0/455859900 learned_addr learned my addr 172.21.15.8:0/455859900 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d11070700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.262 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 -- 172.21.15.8:0/455859900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 msgr2=0x7f4d0c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 -- 172.21.15.8:0/455859900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d080095f0 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d11070700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d1086f700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e920 tx=0x7f4cfc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cfc009790 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4cfc00cba0 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cfc018650 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c196910 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.266+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c196e60 con 0x7f4d0c106cd0 2024-04-12T06:01:28.263 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.267+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c1091d0 con 0x7f4d0c106cd0 2024-04-12T06:01:28.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.268+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4cfc00cd10 con 0x7f4d0c106cd0 2024-04-12T06:01:28.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.268+0000 7f4d027fc700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 0x7f4cf806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.268+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4cfc014070 con 0x7f4d0c106cd0 2024-04-12T06:01:28.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.268+0000 7f4d11070700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 0x7f4cf806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.269+0000 7f4d11070700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 0x7f4cf806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:28.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.274+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4cfc052ab0 con 0x7f4d0c106cd0 2024-04-12T06:01:28.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.527+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d0c1970d0 con 0x7f4cf806cce0 2024-04-12T06:01:28.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.528+0000 7f4d027fc700 1 -- 172.21.15.8:0/455859900 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4d0c1970d0 con 0x7f4cf806cce0 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "7/23 daemons upgraded", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:01:28.523 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 msgr2=0x7f4cf806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 0x7f4cf806f1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009910 tx=0x7f4d08009f70 comp rx=0 tx=0).stop 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 msgr2=0x7f4d0c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00e920 tx=0x7f4cfc00ece0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 shutdown_connections 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4cf806cce0 0x7f4cf806f1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d0c068580 0x7f4d0c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 --2- 172.21.15.8:0/455859900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d0c106cd0 0x7f4d0c19caa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 >> 172.21.15.8:0/455859900 conn(0x7f4d0c075670 msgr2=0x7f4d0c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 shutdown_connections 2024-04-12T06:01:28.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.530+0000 7f4d132d4700 1 -- 172.21.15.8:0/455859900 wait complete. 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/1095070908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc0ffc20 msgr2=0x7f1bbc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/1095070908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc100080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009910 tx=0x7f1bb0009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/1095070908 shutdown_connections 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/1095070908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc100080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/1095070908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/1095070908 >> 172.21.15.8:0/1095070908 conn(0x7f1bbc075670 msgr2=0x7f1bbc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/1095070908 shutdown_connections 2024-04-12T06:01:28.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.639+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/1095070908 wait complete. 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 Processor -- start 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 -- start start 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc1022c0 con 0x7f1bbc105bd0 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bc0bf4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bbc102400 con 0x7f1bbc0ffc20 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bb9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bb9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45012/0 (socket says 172.21.15.8:45012) 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bb9d9b700 1 -- 172.21.15.8:0/3924084013 learned_addr learned my addr 172.21.15.8:0/3924084013 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bba59c700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.640+0000 7f1bb9d9b700 1 -- 172.21.15.8:0/3924084013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc0ffc20 msgr2=0x7f1bbc101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bb9d9b700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bb9d9b700 1 -- 172.21.15.8:0/3924084013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb00095f0 con 0x7f1bbc105bd0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bb9d9b700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb001d070 con 0x7f1bbc105bd0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bbc102680 con 0x7f1bbc105bd0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1bb000f460 con 0x7f1bbc105bd0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bb0021640 con 0x7f1bbc105bd0 2024-04-12T06:01:28.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.641+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bbc1a2c60 con 0x7f1bbc105bd0 2024-04-12T06:01:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.642+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bbc04f2e0 con 0x7f1bbc105bd0 2024-04-12T06:01:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.643+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1bb0003a60 con 0x7f1bbc105bd0 2024-04-12T06:01:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.643+0000 7f1bab7fe700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 0x7f1ba406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.643+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1bb008d7e0 con 0x7f1bbc105bd0 2024-04-12T06:01:28.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.643+0000 7f1bba59c700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 0x7f1ba406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:28.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.644+0000 7f1bba59c700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 0x7f1ba406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005ea0 tx=0x7f1bac005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:28.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.648+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1bb0057900 con 0x7f1bbc105bd0 2024-04-12T06:01:28.788 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-mon[146084]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:28.788 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-mon[146084]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:28.788 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-mon[146084]: pgmap v165: 65 pgs: 12 stale+active+clean, 53 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:01:28.788 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/496072120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:28.789 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1466015815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:01:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:28 smithi194 ceph-mon[106982]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:28 smithi194 ceph-mon[106982]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:28 smithi194 ceph-mon[106982]: pgmap v165: 65 pgs: 12 stale+active+clean, 53 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:01:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:28 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/496072120' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:28 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1466015815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:01:28.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.946+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1bbc1a2e50 con 0x7f1bbc105bd0 2024-04-12T06:01:28.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.946+0000 7f1bab7fe700 1 -- 172.21.15.8:0/3924084013 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f1bb0003d10 con 0x7f1bbc105bd0 2024-04-12T06:01:28.942 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_WARN 1 osds down 2024-04-12T06:01:28.942 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-12T06:01:28.942 INFO:teuthology.orchestra.run.smithi008.stdout: osd.1 (root=default,host=smithi008) is down 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 msgr2=0x7f1ba406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 0x7f1ba406f1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005ea0 tx=0x7f1bac005e30 comp rx=0 tx=0).stop 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 msgr2=0x7f1bbc101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0006010 tx=0x7f1bb000be20 comp rx=0 tx=0).stop 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 shutdown_connections 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1ba406cce0 0x7f1ba406f1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1bbc0ffc20 0x7f1bbc101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 --2- 172.21.15.8:0/3924084013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1bbc105bd0 0x7f1bbc101d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 >> 172.21.15.8:0/3924084013 conn(0x7f1bbc075670 msgr2=0x7f1bbc068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 shutdown_connections 2024-04-12T06:01:28.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:28.949+0000 7f1bc0bf4700 1 -- 172.21.15.8:0/3924084013 wait complete. 2024-04-12T06:01:29.152 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:28 smithi008 ceph-osd[164883]: -- 172.21.15.8:0/1862893302 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a3b9e0f1e0 con 0x55a3b9ddec00 2024-04-12T06:01:29.153 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:28 smithi008 bash[164823]: da712394028418218fd1751cc9736c5a1bf68ae193ce06b8958c189068f0db62 2024-04-12T06:01:29.153 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:28 smithi008 systemd[1]: Started Ceph osd.1 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:01:29.653 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:01:29.319+0000 7f0eb53757c0 -1 Falling back to public interface 2024-04-12T06:01:30.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-mon[146084]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:30.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:30.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:30.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3924084013' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:01:30.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:30.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:29 smithi194 ceph-mon[106982]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:01:30.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:30.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:30.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:29 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3924084013' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:01:30.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:31.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:30 smithi194 ceph-mon[106982]: pgmap v166: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:31.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:30 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T06:01:31.328 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:30 smithi008 ceph-mon[146084]: pgmap v166: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:31.328 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:30 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T06:01:33.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:32 smithi194 ceph-mon[106982]: pgmap v167: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:33.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:33.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:33.402 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:32 smithi008 ceph-mon[146084]: pgmap v167: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:33.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:33.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:34.152 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:33 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:01:33.882+0000 7f0eb53757c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-12T06:01:34.903 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:34 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:01:34.443+0000 7f0eb53757c0 -1 osd.1 53 log_to_monitors true 2024-04-12T06:01:35.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:34 smithi194 ceph-mon[106982]: pgmap v168: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:35.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:34 smithi194 ceph-mon[106982]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T06:01:35.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:34 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:35.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:34 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:35.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:34 smithi008 ceph-mon[146084]: pgmap v168: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:35.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:34 smithi008 ceph-mon[146084]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-12T06:01:35.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:34 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:35.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:34 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:36.151 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:01:35.753+0000 7f0ea27c7700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: osdmap e56: 6 total, 5 up, 6 in 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:36.151 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:35 smithi008 ceph-mon[146084]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' 2024-04-12T06:01:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-12T06:01:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: osdmap e56: 6 total, 5 up, 6 in 2024-04-12T06:01:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:01:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:36.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:36.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:35 smithi194 ceph-mon[106982]: from='osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324]' entity='osd.1' 2024-04-12T06:01:37.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:36 smithi194 ceph-mon[106982]: pgmap v170: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:37.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:36 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:01:37.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:36 smithi194 ceph-mon[106982]: osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324] boot 2024-04-12T06:01:37.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:36 smithi194 ceph-mon[106982]: osdmap e57: 6 total, 6 up, 6 in 2024-04-12T06:01:37.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T06:01:37.378 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:36 smithi008 ceph-mon[146084]: pgmap v170: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-04-12T06:01:37.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:36 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:01:37.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:36 smithi008 ceph-mon[146084]: osd.1 [v2:172.21.15.8:6810/202303324,v1:172.21.15.8:6811/202303324] boot 2024-04-12T06:01:37.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:36 smithi008 ceph-mon[146084]: osdmap e57: 6 total, 6 up, 6 in 2024-04-12T06:01:37.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-12T06:01:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:38 smithi008 ceph-mon[146084]: pgmap v172: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/333 objects degraded (15.616%) 2024-04-12T06:01:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:38 smithi008 ceph-mon[146084]: osdmap e58: 6 total, 6 up, 6 in 2024-04-12T06:01:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:38 smithi194 ceph-mon[106982]: pgmap v172: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 52/333 objects degraded (15.616%) 2024-04-12T06:01:39.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:38 smithi194 ceph-mon[106982]: osdmap e58: 6 total, 6 up, 6 in 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: pgmap v174: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 22/333 objects degraded (6.607%) 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:01:40.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-04-12T06:01:40.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:40 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 22/333 objects degraded (6.607%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T06:01:40.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: pgmap v174: 65 pgs: 2 peering, 7 active+undersized+degraded, 5 active+undersized, 51 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 22/333 objects degraded (6.607%) 2024-04-12T06:01:40.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-04-12T06:01:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:40 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 22/333 objects degraded (6.607%), 7 pgs degraded (PG_DEGRADED) 2024-04-12T06:01:41.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:41 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/333 objects degraded (6.607%), 7 pgs degraded) 2024-04-12T06:01:41.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:41 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:01:41.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:41 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/333 objects degraded (6.607%), 7 pgs degraded) 2024-04-12T06:01:41.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:41 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:01:42.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:42 smithi194 ceph-mon[106982]: pgmap v175: 65 pgs: 2 peering, 63 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:01:42.902 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:42 smithi008 ceph-mon[146084]: pgmap v175: 65 pgs: 2 peering, 63 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:01:44.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:44 smithi194 ceph-mon[106982]: pgmap v176: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-12T06:01:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:44 smithi008 ceph-mon[146084]: pgmap v176: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-12T06:01:46.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:46 smithi008 ceph-mon[146084]: pgmap v177: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:01:46.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:46 smithi194 ceph-mon[106982]: pgmap v177: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:01:48.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:48 smithi194 ceph-mon[106982]: pgmap v178: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-12T06:01:48.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:48.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:48.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:48 smithi008 ceph-mon[146084]: pgmap v178: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-12T06:01:48.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:48.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:01:50.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:50 smithi194 ceph-mon[106982]: pgmap v179: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:50.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:50 smithi008 ceph-mon[146084]: pgmap v179: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:01:52.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:52 smithi194 ceph-mon[106982]: pgmap v180: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:52.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:52 smithi008 ceph-mon[146084]: pgmap v180: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:53.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:53 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:53.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:53 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:54.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:54 smithi194 ceph-mon[106982]: pgmap v181: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:54 smithi008 ceph-mon[146084]: pgmap v181: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:01:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:55 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:55.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:55 smithi008 ceph-mon[146084]: Upgrade: osd.2 is safe to restart 2024-04-12T06:01:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:55 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-12T06:01:55.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:55 smithi194 ceph-mon[106982]: Upgrade: osd.2 is safe to restart 2024-04-12T06:01:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:56 smithi008 ceph-mon[146084]: pgmap v182: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T06:01:56.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:56 smithi194 ceph-mon[106982]: pgmap v182: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:01:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-12T06:01:56.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:01:57.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:57 smithi194 ceph-mon[106982]: Upgrade: Updating osd.2 2024-04-12T06:01:57.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:57 smithi194 ceph-mon[106982]: Deploying daemon osd.2 on smithi008 2024-04-12T06:01:57.820 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:57 smithi008 ceph-mon[146084]: Upgrade: Updating osd.2 2024-04-12T06:01:57.820 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:57 smithi008 ceph-mon[146084]: Deploying daemon osd.2 on smithi008 2024-04-12T06:01:58.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:01:58 smithi194 ceph-mon[106982]: pgmap v183: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:58.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:01:58 smithi008 ceph-mon[146084]: pgmap v183: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.061+0000 7f5cf5bab700 1 -- 172.21.15.8:0/3584468956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 msgr2=0x7f5cf01058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.061+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/3584468956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf01058a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009910 tx=0x7f5ce4009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 -- 172.21.15.8:0/3584468956 shutdown_connections 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/3584468956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf01058a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/3584468956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 0x7f5cf0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 -- 172.21.15.8:0/3584468956 >> 172.21.15.8:0/3584468956 conn(0x7f5cf00fc750 msgr2=0x7f5cf00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 -- 172.21.15.8:0/3584468956 shutdown_connections 2024-04-12T06:01:59.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 -- 172.21.15.8:0/3584468956 wait complete. 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.062+0000 7f5cf5bab700 1 Processor -- start 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cf5bab700 1 -- start start 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cf5bab700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 0x7f5cf019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cf5bab700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cf5bab700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf019d1f0 con 0x7f5cf0101850 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cf5bab700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf01966a0 con 0x7f5cf0100ea0 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35718/0 (socket says 172.21.15.8:35718) 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 -- 172.21.15.8:0/455841791 learned_addr learned my addr 172.21.15.8:0/455841791 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5cef7fe700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 0x7f5cf019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 -- 172.21.15.8:0/455841791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 msgr2=0x7f5cf019c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 0x7f5cf019c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 -- 172.21.15.8:0/455841791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40095f0 con 0x7f5cf0101850 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.063+0000 7f5ceeffd700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4005b40 tx=0x7f5ce400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.064+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce401d070 con 0x7f5cf0101850 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.064+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ce4005dc0 con 0x7f5cf0101850 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.064+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf0196920 con 0x7f5cf0101850 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.064+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce40039b0 con 0x7f5cf0101850 2024-04-12T06:01:59.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.064+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf0196db0 con 0x7f5cf0101850 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.065+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf01092a0 con 0x7f5cf0101850 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.065+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f5ce4003ba0 con 0x7f5cf0101850 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.066+0000 7f5cecff9700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 0x7f5cd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.066+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5ce408db80 con 0x7f5cf0101850 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.066+0000 7f5cef7fe700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 0x7f5cd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.066+0000 7f5cef7fe700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 0x7f5cd806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009c60 tx=0x7f5ce0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.070+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5ce405b730 con 0x7f5cf0101850 2024-04-12T06:01:59.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.295+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cf0197c50 con 0x7f5cd806cce0 2024-04-12T06:01:59.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.296+0000 7f5cecff9700 1 -- 172.21.15.8:0/455841791 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5cf0197c50 con 0x7f5cd806cce0 2024-04-12T06:01:59.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 msgr2=0x7f5cd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.293 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 0x7f5cd806f1a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009c60 tx=0x7f5ce0009260 comp rx=0 tx=0).stop 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 msgr2=0x7f5cf019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4005b40 tx=0x7f5ce400bde0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 shutdown_connections 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5cd806cce0 0x7f5cd806f1a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5cf0100ea0 0x7f5cf019c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 --2- 172.21.15.8:0/455841791 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5cf0101850 0x7f5cf019cb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 >> 172.21.15.8:0/455841791 conn(0x7f5cf00fc750 msgr2=0x7f5cf00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 shutdown_connections 2024-04-12T06:01:59.294 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.298+0000 7f5cf5bab700 1 -- 172.21.15.8:0/455841791 wait complete. 2024-04-12T06:01:59.304 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 -- 172.21.15.8:0/2861898455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 msgr2=0x7f7e0c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/2861898455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c105fb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009910 tx=0x7f7e00009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 -- 172.21.15.8:0/2861898455 shutdown_connections 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/2861898455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 0x7f7e0c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/2861898455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c105fb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 -- 172.21.15.8:0/2861898455 >> 172.21.15.8:0/2861898455 conn(0x7f7e0c075670 msgr2=0x7f7e0c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 -- 172.21.15.8:0/2861898455 shutdown_connections 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.407+0000 7f7e12f3b700 1 -- 172.21.15.8:0/2861898455 wait complete. 2024-04-12T06:01:59.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 Processor -- start 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 -- start start 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 0x7f7e0c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c198160 con 0x7f7e0c105bd0 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e12f3b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c1982a0 con 0x7f7e0c0ffc20 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35730/0 (socket says 172.21.15.8:35730) 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 -- 172.21.15.8:0/1387512535 learned_addr learned my addr 172.21.15.8:0/1387512535 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e10cd7700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 0x7f7e0c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 -- 172.21.15.8:0/1387512535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 msgr2=0x7f7e0c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 0x7f7e0c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.408+0000 7f7e0bfff700 1 -- 172.21.15.8:0/1387512535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000095f0 con 0x7f7e0c105bd0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e0bfff700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d8d0 tx=0x7f7df800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df8009940 con 0x7f7e0c105bd0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7df8010460 con 0x7f7e0c105bd0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c1a0670 con 0x7f7e0c105bd0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7df800f5f0 con 0x7f7e0c105bd0 2024-04-12T06:01:59.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.409+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c1a0b30 con 0x7f7e0c105bd0 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.410+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c1080d0 con 0x7f7e0c105bd0 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.410+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7df80105d0 con 0x7f7e0c105bd0 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.411+0000 7f7e09ffb700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 0x7f7dfc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.411+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7df808c410 con 0x7f7e0c105bd0 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.411+0000 7f7e10cd7700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 0x7f7dfc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.411+0000 7f7e10cd7700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 0x7f7dfc06f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e00006010 tx=0x7f7e0001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.415+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7df80565b0 con 0x7f7e0c105bd0 2024-04-12T06:01:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.668+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e0c0611d0 con 0x7f7dfc06cce0 2024-04-12T06:01:59.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.669+0000 7f7e09ffb700 1 -- 172.21.15.8:0/1387512535 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7e0c0611d0 con 0x7f7dfc06cce0 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 msgr2=0x7f7dfc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 0x7f7dfc06f1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e00006010 tx=0x7f7e0001a040 comp rx=0 tx=0).stop 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 msgr2=0x7f7e0c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d8d0 tx=0x7f7df800dc90 comp rx=0 tx=0).stop 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 shutdown_connections 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7dfc06cce0 0x7f7dfc06f1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e0c0ffc20 0x7f7e0c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 --2- 172.21.15.8:0/1387512535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e0c105bd0 0x7f7e0c197b40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 >> 172.21.15.8:0/1387512535 conn(0x7f7e0c075670 msgr2=0x7f7e0c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 shutdown_connections 2024-04-12T06:01:59.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.672+0000 7f7e12f3b700 1 -- 172.21.15.8:0/1387512535 wait complete. 2024-04-12T06:01:59.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 -- 172.21.15.8:0/382779320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 msgr2=0x7f7a4c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 --2- 172.21.15.8:0/382779320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c105fb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009910 tx=0x7f7a40009c20 comp rx=0 tx=0).stop 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 -- 172.21.15.8:0/382779320 shutdown_connections 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 --2- 172.21.15.8:0/382779320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 0x7f7a4c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 --2- 172.21.15.8:0/382779320 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c105fb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 -- 172.21.15.8:0/382779320 >> 172.21.15.8:0/382779320 conn(0x7f7a4c075670 msgr2=0x7f7a4c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.810+0000 7f7a52e54700 1 -- 172.21.15.8:0/382779320 shutdown_connections 2024-04-12T06:01:59.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 -- 172.21.15.8:0/382779320 wait complete. 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 Processor -- start 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 -- start start 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 0x7f7a4c195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c196260 con 0x7f7a4c105bd0 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.811+0000 7f7a52e54700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c198f90 con 0x7f7a4c0ffc20 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35748/0 (socket says 172.21.15.8:35748) 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 -- 172.21.15.8:0/1947735645 learned_addr learned my addr 172.21.15.8:0/1947735645 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:01:59.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a50bf0700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 0x7f7a4c195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 -- 172.21.15.8:0/1947735645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 msgr2=0x7f7a4c195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 0x7f7a4c195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 -- 172.21.15.8:0/1947735645 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400095f0 con 0x7f7a4c105bd0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a4bfff700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800eab0 tx=0x7f7a3800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a38013070 con 0x7f7a4c105bd0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c199210 con 0x7f7a4c105bd0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7a38017410 con 0x7f7a4c105bd0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.812+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a380165e0 con 0x7f7a4c105bd0 2024-04-12T06:01:59.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.813+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c199760 con 0x7f7a4c105bd0 2024-04-12T06:01:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.814+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c04f2e0 con 0x7f7a4c105bd0 2024-04-12T06:01:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.814+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7a38016800 con 0x7f7a4c105bd0 2024-04-12T06:01:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.815+0000 7f7a49ffb700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 0x7f7a3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:01:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.815+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7a3808dcb0 con 0x7f7a4c105bd0 2024-04-12T06:01:59.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.815+0000 7f7a50bf0700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 0x7f7a3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:01:59.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.815+0000 7f7a50bf0700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 0x7f7a3c06f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009fd0 tx=0x7f7a40005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:01:59.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:01:59.822+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7a38057f70 con 0x7f7a4c105bd0 2024-04-12T06:01:59.903 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:01:59 smithi008 systemd[1]: Stopping Ceph osd.2 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:02:00.035 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.040+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a4c199a50 con 0x7f7a3c06cce0 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.051+0000 7f7a49ffb700 1 -- 172.21.15.8:0/1947735645 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7a4c199a50 con 0x7f7a3c06cce0 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (3m) 25s ago 15m 27.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (15m) 25s ago 15m 8216k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (14m) 85s ago 14m 8082k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:02:00.046 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (95s) 25s ago 15m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (89s) 85s ago 14m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (3m) 25s ago 14m 81.5M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (9m) 25s ago 9m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (9m) 25s ago 9m 71.2M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (9m) 85s ago 9m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (9m) 85s ago 9m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (5m) 25s ago 16m 509M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (4m) 85s ago 14m 450M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (2m) 25s ago 17m 45.7M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (110s) 85s ago 14m 30.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (4m) 25s ago 15m 16.9M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (4m) 85s ago 14m 18.0M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (68s) 25s ago 13m 94.6M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (31s) 25s ago 12m 16.0M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:02:00.047 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (12m) 25s ago 12m 196M 1228M 18.2.1 5be31c24972a b7d5bd11237a 2024-04-12T06:02:00.048 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (11m) 85s ago 11m 266M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T06:02:00.048 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (11m) 85s ago 11m 231M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:02:00.048 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (10m) 85s ago 10m 213M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:02:00.048 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (4m) 25s ago 14m 66.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:02:00.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 msgr2=0x7f7a3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 0x7f7a3c06f1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009fd0 tx=0x7f7a40005940 comp rx=0 tx=0).stop 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 msgr2=0x7f7a4c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800eab0 tx=0x7f7a3800edc0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 shutdown_connections 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7a3c06cce0 0x7f7a3c06f1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a4c0ffc20 0x7f7a4c195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 --2- 172.21.15.8:0/1947735645 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7a4c105bd0 0x7f7a4c195bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 >> 172.21.15.8:0/1947735645 conn(0x7f7a4c075670 msgr2=0x7f7a4c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 shutdown_connections 2024-04-12T06:02:00.049 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.053+0000 7f7a52e54700 1 -- 172.21.15.8:0/1947735645 wait complete. 2024-04-12T06:02:00.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 -- 172.21.15.8:0/3405522401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c0ffc20 msgr2=0x7f295c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 --2- 172.21.15.8:0/3405522401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c0ffc20 0x7f295c100080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f294c009910 tx=0x7f294c009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:00.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 -- 172.21.15.8:0/3405522401 shutdown_connections 2024-04-12T06:02:00.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 --2- 172.21.15.8:0/3405522401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c0ffc20 0x7f295c100080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 --2- 172.21.15.8:0/3405522401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c105bd0 0x7f295c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 -- 172.21.15.8:0/3405522401 >> 172.21.15.8:0/3405522401 conn(0x7f295c075670 msgr2=0x7f295c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 -- 172.21.15.8:0/3405522401 shutdown_connections 2024-04-12T06:02:00.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.159+0000 7f2964148700 1 -- 172.21.15.8:0/3405522401 wait complete. 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.160+0000 7f2964148700 1 Processor -- start 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.160+0000 7f2964148700 1 -- start start 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.160+0000 7f2964148700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 0x7f295c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.160+0000 7f2964148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2964148700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1982a0 con 0x7f295c105bd0 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2964148700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c198410 con 0x7f295c0ffc20 2024-04-12T06:02:00.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35778/0 (socket says 172.21.15.8:35778) 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 -- 172.21.15.8:0/1537444360 learned_addr learned my addr 172.21.15.8:0/1537444360 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2961ee4700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 0x7f295c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 -- 172.21.15.8:0/1537444360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 msgr2=0x7f295c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 0x7f295c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 -- 172.21.15.8:0/1537444360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0095f0 con 0x7f295c105bd0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2961ee4700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 0x7f295c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f29616e3700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f294c006010 tx=0x7f294c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f294c01d070 con 0x7f295c105bd0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f294c00f460 con 0x7f295c105bd0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f294c021640 con 0x7f295c105bd0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1a07b0 con 0x7f295c105bd0 2024-04-12T06:02:00.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.161+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1a0c20 con 0x7f295c105bd0 2024-04-12T06:02:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.162+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c1080f0 con 0x7f295c105bd0 2024-04-12T06:02:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.163+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f294c00fa60 con 0x7f295c105bd0 2024-04-12T06:02:00.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.163+0000 7f2952ffd700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 0x7f294806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.163+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f294c08d6d0 con 0x7f295c105bd0 2024-04-12T06:02:00.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.163+0000 7f2961ee4700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 0x7f294806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.164+0000 7f2961ee4700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 0x7f294806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2958007900 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.168+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f294c057870 con 0x7f295c105bd0 2024-04-12T06:02:00.403 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:01:59 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[83405]: 2024-04-12T06:01:59.925+0000 7f51006ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:02:00.403 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:01:59 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[83405]: 2024-04-12T06:01:59.925+0000 7f51006ed700 -1 osd.2 58 *** Got signal Terminated *** 2024-04-12T06:02:00.403 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:01:59 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[83405]: 2024-04-12T06:01:59.925+0000 7f51006ed700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:02:00.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.454+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f295c04f2e0 con 0x7f295c105bd0 2024-04-12T06:02:00.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.455+0000 7f2952ffd700 1 -- 172.21.15.8:0/1537444360 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f294c08be60 con 0x7f295c105bd0 2024-04-12T06:02:00.449 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:02:00.450 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-04-12T06:02:00.451 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:02:00.451 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:02:00.451 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:02:00.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 msgr2=0x7f294806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 0x7f294806f1a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2958007900 tx=0x7f2958008040 comp rx=0 tx=0).stop 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 msgr2=0x7f295c197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f294c006010 tx=0x7f294c00be20 comp rx=0 tx=0).stop 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 shutdown_connections 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f294806cce0 0x7f294806f1a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f295c0ffc20 0x7f295c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 --2- 172.21.15.8:0/1537444360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f295c105bd0 0x7f295c197c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 >> 172.21.15.8:0/1537444360 conn(0x7f295c075670 msgr2=0x7f295c0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 shutdown_connections 2024-04-12T06:02:00.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.457+0000 7f2964148700 1 -- 172.21.15.8:0/1537444360 wait complete. 2024-04-12T06:02:00.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 -- 172.21.15.8:0/974533027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8068580 msgr2=0x7f64d80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 --2- 172.21.15.8:0/974533027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8068580 0x7f64d80689e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009910 tx=0x7f64c0009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:00.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 -- 172.21.15.8:0/974533027 shutdown_connections 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 --2- 172.21.15.8:0/974533027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8068580 0x7f64d80689e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 --2- 172.21.15.8:0/974533027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8106cd0 0x7f64d81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 -- 172.21.15.8:0/974533027 >> 172.21.15.8:0/974533027 conn(0x7f64d8075670 msgr2=0x7f64d8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 -- 172.21.15.8:0/974533027 shutdown_connections 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.571+0000 7f64ddd26700 1 -- 172.21.15.8:0/974533027 wait complete. 2024-04-12T06:02:00.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 Processor -- start 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 -- start start 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8068580 0x7f64d819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d819d110 con 0x7f64d8106cd0 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64ddd26700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81965c0 con 0x7f64d8068580 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64d6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35790/0 (socket says 172.21.15.8:35790) 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64d6ffd700 1 -- 172.21.15.8:0/3364674727 learned_addr learned my addr 172.21.15.8:0/3364674727 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.572+0000 7f64d77fe700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8068580 0x7f64d819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d6ffd700 1 -- 172.21.15.8:0/3364674727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8068580 msgr2=0x7f64d819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d6ffd700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8068580 0x7f64d819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d6ffd700 1 -- 172.21.15.8:0/3364674727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00095f0 con 0x7f64d8106cd0 2024-04-12T06:02:00.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d6ffd700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006010 tx=0x7f64c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c001d070 con 0x7f64d8106cd0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8196840 con 0x7f64d8106cd0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64c0005710 con 0x7f64d8106cd0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c000fe20 con 0x7f64d8106cd0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.573+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d8196d30 con 0x7f64d8106cd0 2024-04-12T06:02:00.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.574+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d81091d0 con 0x7f64d8106cd0 2024-04-12T06:02:00.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.575+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f64c0003a60 con 0x7f64d8106cd0 2024-04-12T06:02:00.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.576+0000 7f64d4ff9700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 0x7f64c406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.576+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f64c008dae0 con 0x7f64d8106cd0 2024-04-12T06:02:00.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.576+0000 7f64d77fe700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 0x7f64c406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.576+0000 7f64d77fe700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 0x7f64c406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64c8007be0 tx=0x7f64c80073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.580+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f64c005b5f0 con 0x7f64d8106cd0 2024-04-12T06:02:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:00 smithi194 ceph-mon[106982]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:00 smithi194 ceph-mon[106982]: pgmap v184: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:02:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:00 smithi194 ceph-mon[106982]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:00 smithi194 ceph-mon[106982]: osd.2 marked itself down and dead 2024-04-12T06:02:00.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:00 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1537444360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:00.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.828+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64d804f2e0 con 0x7f64d8106cd0 2024-04-12T06:02:00.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.828+0000 7f64d4ff9700 1 -- 172.21.15.8:0/3364674727 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f64c005b180 con 0x7f64d8106cd0 2024-04-12T06:02:00.823 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:02:00.824 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:02:00.825 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:00.826 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.830+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 msgr2=0x7f64c406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.830+0000 7f64ddd26700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 0x7f64c406f1a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64c8007be0 tx=0x7f64c80073d0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.830+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 msgr2=0x7f64d819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.830+0000 7f64ddd26700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006010 tx=0x7f64c000bd70 comp rx=0 tx=0).stop 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 shutdown_connections 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64c406cce0 0x7f64c406f1a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64d8068580 0x7f64d819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 --2- 172.21.15.8:0/3364674727 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64d8106cd0 0x7f64d819ca80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 >> 172.21.15.8:0/3364674727 conn(0x7f64d8075670 msgr2=0x7f64d80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 shutdown_connections 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.831+0000 7f64ddd26700 1 -- 172.21.15.8:0/3364674727 wait complete. 2024-04-12T06:02:00.827 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:02:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:00 smithi008 ceph-mon[146084]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:00 smithi008 ceph-mon[146084]: pgmap v184: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:02:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:00 smithi008 ceph-mon[146084]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:00 smithi008 ceph-mon[146084]: osd.2 marked itself down and dead 2024-04-12T06:02:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:00 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1537444360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 -- 172.21.15.8:0/3227043688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 msgr2=0x7f0ac41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 --2- 172.21.15.8:0/3227043688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac41047a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009910 tx=0x7f0ab8009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 -- 172.21.15.8:0/3227043688 shutdown_connections 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 --2- 172.21.15.8:0/3227043688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac41047a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 --2- 172.21.15.8:0/3227043688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 -- 172.21.15.8:0/3227043688 >> 172.21.15.8:0/3227043688 conn(0x7f0ac40fb650 msgr2=0x7f0ac40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 -- 172.21.15.8:0/3227043688 shutdown_connections 2024-04-12T06:02:00.937 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 -- 172.21.15.8:0/3227043688 wait complete. 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.942+0000 7f0aca02d700 1 Processor -- start 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0aca02d700 1 -- start start 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0aca02d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0aca02d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0aca02d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac419d180 con 0x7f0ac4100750 2024-04-12T06:02:00.938 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0aca02d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4196630 con 0x7f0ac40ffda0 2024-04-12T06:02:00.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0ac2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0ac37fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.939 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0ac2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35804/0 (socket says 172.21.15.8:35804) 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0ac2ffd700 1 -- 172.21.15.8:0/2785350160 learned_addr learned my addr 172.21.15.8:0/2785350160 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.943+0000 7f0ac37fe700 1 -- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 msgr2=0x7f0ac419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac37fe700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac37fe700 1 -- 172.21.15.8:0/2785350160 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80095f0 con 0x7f0ac40ffda0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac37fe700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac419c5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009c50 tx=0x7f0ab4009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab4015070 con 0x7f0ac40ffda0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac41968b0 con 0x7f0ac40ffda0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0ab400bc50 con 0x7f0ac40ffda0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ab40185c0 con 0x7f0ac40ffda0 2024-04-12T06:02:00.940 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.944+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac4196e00 con 0x7f0ac40ffda0 2024-04-12T06:02:00.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.945+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac404f2e0 con 0x7f0ac40ffda0 2024-04-12T06:02:00.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.946+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f0ab4019410 con 0x7f0ac40ffda0 2024-04-12T06:02:00.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.946+0000 7f0ac0ff9700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 0x7f0aac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:00.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.946+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0ab408fe90 con 0x7f0ac40ffda0 2024-04-12T06:02:00.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.946+0000 7f0ac2ffd700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 0x7f0aac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:00.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.947+0000 7f0ac2ffd700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 0x7f0aac06f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40ff000 tx=0x7f0ab8005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:00.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:00.952+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0ab405daf0 con 0x7f0ac40ffda0 2024-04-12T06:02:01.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.176+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac4197260 con 0x7f0aac06cce0 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.177+0000 7f0ac0ff9700 1 -- 172.21.15.8:0/2785350160 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0ac4197260 con 0x7f0aac06cce0 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:02:01.172 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:02:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:02:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "8/23 daemons upgraded", 2024-04-12T06:02:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:02:01.173 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:02:01.173 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:02:01.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 msgr2=0x7f0aac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 0x7f0aac06f1a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40ff000 tx=0x7f0ab8005940 comp rx=0 tx=0).stop 2024-04-12T06:02:01.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 msgr2=0x7f0ac419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac419c5b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009c50 tx=0x7f0ab4009f60 comp rx=0 tx=0).stop 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 shutdown_connections 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0aac06cce0 0x7f0aac06f1a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0ac40ffda0 0x7f0ac419c5b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 --2- 172.21.15.8:0/2785350160 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0ac4100750 0x7f0ac419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 >> 172.21.15.8:0/2785350160 conn(0x7f0ac40fb650 msgr2=0x7f0ac4103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 shutdown_connections 2024-04-12T06:02:01.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.179+0000 7f0aca02d700 1 -- 172.21.15.8:0/2785350160 wait complete. 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 -- 172.21.15.8:0/4244342440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 msgr2=0x7f88381070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 --2- 172.21.15.8:0/4244342440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f88381070b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f882800b0a0 tx=0x7f882800b3b0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 -- 172.21.15.8:0/4244342440 shutdown_connections 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 --2- 172.21.15.8:0/4244342440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 0x7f88380689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 --2- 172.21.15.8:0/4244342440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f88381070b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 -- 172.21.15.8:0/4244342440 >> 172.21.15.8:0/4244342440 conn(0x7f8838075670 msgr2=0x7f8838075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 -- 172.21.15.8:0/4244342440 shutdown_connections 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.286+0000 7f883fb3e700 1 -- 172.21.15.8:0/4244342440 wait complete. 2024-04-12T06:02:01.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 Processor -- start 2024-04-12T06:02:01.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 -- start start 2024-04-12T06:02:01.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 0x7f883819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:01.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:01.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883819d130 con 0x7f8838106cd0 2024-04-12T06:02:01.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883fb3e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838196630 con 0x7f8838068580 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883d0d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883d0d9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35832/0 (socket says 172.21.15.8:35832) 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883d0d9700 1 -- 172.21.15.8:0/855808828 learned_addr learned my addr 172.21.15.8:0/855808828 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.287+0000 7f883d8da700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 0x7f883819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883d0d9700 1 -- 172.21.15.8:0/855808828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 msgr2=0x7f883819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883d0d9700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 0x7f883819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883d0d9700 1 -- 172.21.15.8:0/855808828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009d00 con 0x7f8838106cd0 2024-04-12T06:02:01.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883d0d9700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c8a0 tx=0x7f882c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c00cea0 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f882c004830 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c005500 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8838196910 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.288+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838196e60 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.289+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88381091d0 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.289+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f882c0049a0 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.290+0000 7f8836ffd700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 0x7f882006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.290+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f882c08c110 con 0x7f8838106cd0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.290+0000 7f883d8da700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 0x7f882006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:01.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.290+0000 7f883d8da700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 0x7f882006f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8828009850 tx=0x7f88280097c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:01.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.295+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f882c059d40 con 0x7f8838106cd0 2024-04-12T06:02:01.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.654+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f883804f2e0 con 0x7f8838106cd0 2024-04-12T06:02:01.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.654+0000 7f8836ffd700 1 -- 172.21.15.8:0/855808828 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f882c0598d0 con 0x7f8838106cd0 2024-04-12T06:02:01.650 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_WARN 1 osds down 2024-04-12T06:02:01.650 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-12T06:02:01.650 INFO:teuthology.orchestra.run.smithi008.stdout: osd.2 (root=default,host=smithi008) is down 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 msgr2=0x7f882006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 0x7f882006f1a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8828009850 tx=0x7f88280097c0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 msgr2=0x7f883819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f882c00c8a0 tx=0x7f882c00cc60 comp rx=0 tx=0).stop 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 shutdown_connections 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f882006cce0 0x7f882006f1a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8838068580 0x7f883819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 --2- 172.21.15.8:0/855808828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8838106cd0 0x7f883819caa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 >> 172.21.15.8:0/855808828 conn(0x7f8838075670 msgr2=0x7f88380fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 shutdown_connections 2024-04-12T06:02:01.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:01.657+0000 7f883fb3e700 1 -- 172.21.15.8:0/855808828 wait complete. 2024-04-12T06:02:01.653 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:01 smithi008 bash[171669]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2 2024-04-12T06:02:01.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:01 smithi008 ceph-mon[146084]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:01.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:01 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:02:01.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:01 smithi008 ceph-mon[146084]: osdmap e59: 6 total, 5 up, 6 in 2024-04-12T06:02:01.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:01 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3364674727' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:02:01.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:01 smithi194 ceph-mon[106982]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:01.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:01 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:02:01.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:01 smithi194 ceph-mon[106982]: osdmap e59: 6 total, 5 up, 6 in 2024-04-12T06:02:01.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:01 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3364674727' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:02:02.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: from='client.44187 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:02.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: pgmap v186: 65 pgs: 9 stale+active+clean, 56 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:02:02.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: osdmap e60: 6 total, 5 up, 6 in 2024-04-12T06:02:02.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/855808828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:02:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:02.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:02.851 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: from='client.44187 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:02.851 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: pgmap v186: 65 pgs: 9 stale+active+clean, 56 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:02:02.851 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: osdmap e60: 6 total, 5 up, 6 in 2024-04-12T06:02:02.852 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/855808828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:02:02.852 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:02.852 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:03.521 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:03 smithi008 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.2.service: Succeeded. 2024-04-12T06:02:03.521 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:03 smithi008 systemd[1]: Stopped Ceph osd.2 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:02:03.521 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:03 smithi008 systemd[1]: Starting Ceph osd.2 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:02:03.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:03 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 28/333 objects degraded (8.408%), 9 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:03 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 28/333 objects degraded (8.408%), 9 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:04.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:04 smithi008 ceph-mon[146084]: pgmap v188: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/333 objects degraded (8.408%) 2024-04-12T06:02:04.654 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-12T06:02:04.655 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:04 smithi008 bash[172057]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-12T06:02:04.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:04 smithi194 ceph-mon[106982]: pgmap v188: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 28/333 objects degraded (8.408%) 2024-04-12T06:02:05.903 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:05 smithi008 ceph-osd[172381]: -- 172.21.15.8:0/2459073946 <== mon.0 v2:172.21.15.8:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5597f7e651e0 con 0x5597f7e35000 2024-04-12T06:02:05.903 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:05 smithi008 bash[172343]: 566586cc138fe9c7ee23ef051c48d554ec1aa7a6c04e9559dc51f2bd7eeb3617 2024-04-12T06:02:05.903 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:05 smithi008 systemd[1]: Started Ceph osd.2 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:02:06.375 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:06 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:02:06.055+0000 7f3a385c47c0 -1 Falling back to public interface 2024-04-12T06:02:07.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:06 smithi194 ceph-mon[106982]: pgmap v189: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:07.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:07.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:07.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:07.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:06 smithi008 ceph-mon[146084]: pgmap v189: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:07.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:07.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:07.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:09.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:08 smithi194 ceph-mon[106982]: pgmap v190: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:09.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:08 smithi008 ceph-mon[146084]: pgmap v190: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:10.653 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:10 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:02:10.247+0000 7f3a385c47c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-12T06:02:11.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:10 smithi194 ceph-mon[106982]: pgmap v191: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:11.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:10 smithi194 ceph-mon[106982]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T06:02:11.153 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:10 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:02:10.665+0000 7f3a385c47c0 -1 osd.2 58 log_to_monitors true 2024-04-12T06:02:11.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:10 smithi008 ceph-mon[146084]: pgmap v191: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:11.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:10 smithi008 ceph-mon[146084]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-12T06:02:11.903 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:02:11.462+0000 7f3a25a16700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:02:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T06:02:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: osdmap e61: 6 total, 5 up, 6 in 2024-04-12T06:02:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:02:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:11 smithi008 ceph-mon[146084]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' 2024-04-12T06:02:12.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-12T06:02:12.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: osdmap e61: 6 total, 5 up, 6 in 2024-04-12T06:02:12.069 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi008", "root=default"]}]: dispatch 2024-04-12T06:02:12.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:12.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:12.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:11 smithi194 ceph-mon[106982]: from='osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174]' entity='osd.2' 2024-04-12T06:02:13.128 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: pgmap v193: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:13.128 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:13.129 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:13.129 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:02:13.129 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174] boot 2024-04-12T06:02:13.129 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: osdmap e62: 6 total, 6 up, 6 in 2024-04-12T06:02:13.129 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T06:02:13.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: pgmap v193: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: osd.2 [v2:172.21.15.8:6818/2007171174,v1:172.21.15.8:6819/2007171174] boot 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: osdmap e62: 6 total, 6 up, 6 in 2024-04-12T06:02:13.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-12T06:02:14.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:13 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 41/333 objects degraded (12.312%), 12 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:14.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:13 smithi008 ceph-mon[146084]: osdmap e63: 6 total, 6 up, 6 in 2024-04-12T06:02:14.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:13 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 41/333 objects degraded (12.312%), 12 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:14.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:13 smithi194 ceph-mon[106982]: osdmap e63: 6 total, 6 up, 6 in 2024-04-12T06:02:15.153 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:14 smithi008 ceph-mon[146084]: pgmap v195: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 26/333 objects degraded (7.808%) 2024-04-12T06:02:15.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:14 smithi194 ceph-mon[106982]: pgmap v195: 65 pgs: 10 peering, 7 active+undersized+degraded, 10 active+undersized, 38 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 26/333 objects degraded (7.808%) 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: pgmap v197: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 23/333 objects degraded (6.907%) 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:02:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:16 smithi008 ceph-mon[146084]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: pgmap v197: 65 pgs: 10 peering, 6 active+undersized+degraded, 8 active+undersized, 41 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 23/333 objects degraded (6.907%) 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:16 smithi194 ceph-mon[106982]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-04-12T06:02:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:18 smithi194 ceph-mon[106982]: pgmap v198: 65 pgs: 10 peering, 2 active+undersized+degraded, 3 active+undersized, 50 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 8/333 objects degraded (2.402%) 2024-04-12T06:02:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:18 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:18.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:18 smithi008 ceph-mon[146084]: pgmap v198: 65 pgs: 10 peering, 2 active+undersized+degraded, 3 active+undersized, 50 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 8/333 objects degraded (2.402%) 2024-04-12T06:02:18.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:18.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:18.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:18 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:19.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:19 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded) 2024-04-12T06:02:19.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:19 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:02:19.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:19 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded) 2024-04-12T06:02:19.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:19 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:02:20.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:20 smithi194 ceph-mon[106982]: pgmap v199: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:02:20.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:20 smithi008 ceph-mon[146084]: pgmap v199: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:02:22.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:22 smithi194 ceph-mon[106982]: pgmap v200: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:02:22.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:22 smithi008 ceph-mon[146084]: pgmap v200: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:02:24.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:24 smithi194 ceph-mon[106982]: pgmap v201: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:02:24.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:24 smithi008 ceph-mon[146084]: pgmap v201: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-12T06:02:26.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:26 smithi194 ceph-mon[106982]: pgmap v202: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:02:26.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:26 smithi008 ceph-mon[146084]: pgmap v202: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:02:28.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:28 smithi194 ceph-mon[106982]: pgmap v203: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:02:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:28 smithi008 ceph-mon[146084]: pgmap v203: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:02:28.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:30.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:30 smithi194 ceph-mon[106982]: pgmap v204: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:02:30.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:30 smithi008 ceph-mon[146084]: pgmap v204: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:02:31.699 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:31.699 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:31 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:31.699 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:31 smithi194 ceph-mon[106982]: Upgrade: osd.3 is safe to restart 2024-04-12T06:02:31.775 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:31.775 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:31 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-12T06:02:31.775 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:31 smithi008 ceph-mon[146084]: Upgrade: osd.3 is safe to restart 2024-04-12T06:02:31.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 -- 172.21.15.8:0/2129010098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 msgr2=0x7f7848100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 --2- 172.21.15.8:0/2129010098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f7848100080 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f783c009910 tx=0x7f783c009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 -- 172.21.15.8:0/2129010098 shutdown_connections 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 --2- 172.21.15.8:0/2129010098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f7848100080 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 --2- 172.21.15.8:0/2129010098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 0x7f7848105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 -- 172.21.15.8:0/2129010098 >> 172.21.15.8:0/2129010098 conn(0x7f7848075670 msgr2=0x7f7848075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 -- 172.21.15.8:0/2129010098 shutdown_connections 2024-04-12T06:02:31.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.782+0000 7f784d07f700 1 -- 172.21.15.8:0/2129010098 wait complete. 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.783+0000 7f784d07f700 1 Processor -- start 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.783+0000 7f784d07f700 1 -- start start 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.783+0000 7f784d07f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f784d07f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 0x7f784819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f784d07f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784819d150 con 0x7f78480ffc20 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f784d07f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7848196600 con 0x7f7848105bd0 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47176/0 (socket says 172.21.15.8:47176) 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 -- 172.21.15.8:0/1073132039 learned_addr learned my addr 172.21.15.8:0/1073132039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f784659c700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 0x7f784819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 -- 172.21.15.8:0/1073132039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 msgr2=0x7f784819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 0x7f784819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:31.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 -- 172.21.15.8:0/1073132039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783c0095f0 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f7846d9d700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f783800d8d0 tx=0x7f783800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.784+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78380098a0 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.785+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78481968e0 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.785+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7838010460 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.785+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7838009ea0 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.785+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7848196e00 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.786+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784804f2e0 con 0x7f78480ffc20 2024-04-12T06:02:31.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.786+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f78380105d0 con 0x7f78480ffc20 2024-04-12T06:02:31.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.787+0000 7f782ffff700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 0x7f783006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:31.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.787+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f783808c390 con 0x7f78480ffc20 2024-04-12T06:02:31.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.787+0000 7f784659c700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 0x7f783006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:31.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.787+0000 7f784659c700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 0x7f783006f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f783c00b5c0 tx=0x7f783c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:31.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:31.793+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7838059f40 con 0x7f78480ffc20 2024-04-12T06:02:32.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.019+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7848197110 con 0x7f783006cce0 2024-04-12T06:02:32.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.021+0000 7f782ffff700 1 -- 172.21.15.8:0/1073132039 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7848197110 con 0x7f783006cce0 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 msgr2=0x7f783006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 0x7f783006f1a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f783c00b5c0 tx=0x7f783c01a040 comp rx=0 tx=0).stop 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 msgr2=0x7f784819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f783800d8d0 tx=0x7f783800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 shutdown_connections 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f783006cce0 0x7f783006f1a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f78480ffc20 0x7f784819c580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 --2- 172.21.15.8:0/1073132039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7848105bd0 0x7f784819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 >> 172.21.15.8:0/1073132039 conn(0x7f7848075670 msgr2=0x7f7848068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 shutdown_connections 2024-04-12T06:02:32.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.023+0000 7f784d07f700 1 -- 172.21.15.8:0/1073132039 wait complete. 2024-04-12T06:02:32.029 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.130+0000 7f27a8682700 1 -- 172.21.15.8:0/2180820802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0100ea0 msgr2=0x7f27a0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.130+0000 7f27a8682700 1 --2- 172.21.15.8:0/2180820802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0100ea0 0x7f27a0101280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f279c009910 tx=0x7f279c009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 -- 172.21.15.8:0/2180820802 shutdown_connections 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 --2- 172.21.15.8:0/2180820802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0101850 0x7f27a01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 --2- 172.21.15.8:0/2180820802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0100ea0 0x7f27a0101280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 -- 172.21.15.8:0/2180820802 >> 172.21.15.8:0/2180820802 conn(0x7f27a00fc750 msgr2=0x7f27a00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 -- 172.21.15.8:0/2180820802 shutdown_connections 2024-04-12T06:02:32.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 -- 172.21.15.8:0/2180820802 wait complete. 2024-04-12T06:02:32.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 Processor -- start 2024-04-12T06:02:32.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.131+0000 7f27a8682700 1 -- start start 2024-04-12T06:02:32.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0100ea0 0x7f27a0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0073b10 con 0x7f27a0101850 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a0073c50 con 0x7f27a0100ea0 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47192/0 (socket says 172.21.15.8:47192) 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 -- 172.21.15.8:0/3125884736 learned_addr learned my addr 172.21.15.8:0/3125884736 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a641e700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0100ea0 0x7f27a0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 -- 172.21.15.8:0/3125884736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0100ea0 msgr2=0x7f27a0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0100ea0 0x7f27a0073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 -- 172.21.15.8:0/3125884736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0095f0 con 0x7f27a0101850 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a5c1d700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f279000d900 tx=0x7f279000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2790009900 con 0x7f27a0101850 2024-04-12T06:02:32.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2790010460 con 0x7f27a0101850 2024-04-12T06:02:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a01a2960 con 0x7f27a0101850 2024-04-12T06:02:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f279000f5f0 con 0x7f27a0101850 2024-04-12T06:02:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.132+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a01a2d70 con 0x7f27a0101850 2024-04-12T06:02:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.133+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a0109240 con 0x7f27a0101850 2024-04-12T06:02:32.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.134+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f27900105d0 con 0x7f27a0101850 2024-04-12T06:02:32.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.134+0000 7f27977fe700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 0x7f278c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.134+0000 7f27a641e700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 0x7f278c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.134+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f279008c200 con 0x7f27a0101850 2024-04-12T06:02:32.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.135+0000 7f27a641e700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 0x7f278c06f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f279c006010 tx=0x7f279c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.139+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f27900563a0 con 0x7f27a0101850 2024-04-12T06:02:32.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.358+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27a00611d0 con 0x7f278c06cce0 2024-04-12T06:02:32.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.359+0000 7f27977fe700 1 -- 172.21.15.8:0/3125884736 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f27a00611d0 con 0x7f278c06cce0 2024-04-12T06:02:32.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 msgr2=0x7f278c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 0x7f278c06f1a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f279c006010 tx=0x7f279c01a040 comp rx=0 tx=0).stop 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 msgr2=0x7f27a00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f279000d900 tx=0x7f279000dcc0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 shutdown_connections 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f278c06cce0 0x7f278c06f1a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27a0100ea0 0x7f27a0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 --2- 172.21.15.8:0/3125884736 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f27a0101850 0x7f27a00735d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 >> 172.21.15.8:0/3125884736 conn(0x7f27a00fc750 msgr2=0x7f27a00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 shutdown_connections 2024-04-12T06:02:32.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.361+0000 7f27a8682700 1 -- 172.21.15.8:0/3125884736 wait complete. 2024-04-12T06:02:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.469+0000 7fa379dec700 1 -- 172.21.15.8:0/2113587475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 msgr2=0x7fa3741057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.469+0000 7fa379dec700 1 --2- 172.21.15.8:0/2113587475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa3741057a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa368009910 tx=0x7fa368009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:32.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 -- 172.21.15.8:0/2113587475 shutdown_connections 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 --2- 172.21.15.8:0/2113587475 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa3741057a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 --2- 172.21.15.8:0/2113587475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 0x7fa3740ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 -- 172.21.15.8:0/2113587475 >> 172.21.15.8:0/2113587475 conn(0x7fa374074cb0 msgr2=0x7fa3740750c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 -- 172.21.15.8:0/2113587475 shutdown_connections 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 -- 172.21.15.8:0/2113587475 wait complete. 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 Processor -- start 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.470+0000 7fa379dec700 1 -- start start 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa379dec700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 0x7fa374196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa379dec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa379dec700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374197240 con 0x7fa3740ffe70 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa379dec700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419afd0 con 0x7fa3740ff4c0 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 0x7fa374196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47206/0 (socket says 172.21.15.8:47206) 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 -- 172.21.15.8:0/1601001636 learned_addr learned my addr 172.21.15.8:0/1601001636 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 -- 172.21.15.8:0/1601001636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 msgr2=0x7fa374196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 0x7fa374196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 -- 172.21.15.8:0/1601001636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3680095f0 con 0x7fa3740ffe70 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa372ffd700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa368005b40 tx=0x7fa36800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa36801d070 con 0x7fa3740ffe70 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa368005dc0 con 0x7fa3740ffe70 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37419b250 con 0x7fa3740ffe70 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.471+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3680039b0 con 0x7fa3740ffe70 2024-04-12T06:02:32.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.472+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa37419b6e0 con 0x7fa3740ffe70 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.473+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3741090a0 con 0x7fa3740ffe70 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.473+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa368003ba0 con 0x7fa3740ffe70 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.473+0000 7fa370ff9700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 0x7fa35c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.473+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa36808dbb0 con 0x7fa3740ffe70 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.473+0000 7fa3737fe700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 0x7fa35c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.474+0000 7fa3737fe700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 0x7fa35c06f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa364009c60 tx=0x7fa364009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.478+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa368094080 con 0x7fa3740ffe70 2024-04-12T06:02:32.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.690+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa37402cd30 con 0x7fa35c06cce0 2024-04-12T06:02:32.696 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.701+0000 7fa370ff9700 1 -- 172.21.15.8:0/1601001636 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fa37402cd30 con 0x7fa35c06cce0 2024-04-12T06:02:32.696 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (4m) 21s ago 15m 27.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (16m) 21s ago 16m 8262k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (14m) 118s ago 14m 8082k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (2m) 21s ago 16m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (2m) 118s ago 14m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (4m) 21s ago 15m 80.8M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (9m) 21s ago 9m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (9m) 21s ago 9m 71.2M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (9m) 118s ago 9m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (9m) 118s ago 9m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (6m) 21s ago 17m 510M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (5m) 118s ago 14m 450M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (2m) 21s ago 17m 47.3M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:02:32.697 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (2m) 118s ago 14m 30.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (5m) 21s ago 15m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (5m) 118s ago 14m 18.0M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (101s) 21s ago 13m 95.6M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (64s) 21s ago 13m 103M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (27s) 21s ago 12m 11.2M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (12m) 118s ago 12m 266M 2251M 18.2.1 5be31c24972a d4847c7e4473 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (11m) 118s ago 11m 231M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (11m) 118s ago 11m 213M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:02:32.698 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (4m) 21s ago 15m 75.0M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 msgr2=0x7fa35c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 0x7fa35c06f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa364009c60 tx=0x7fa364009260 comp rx=0 tx=0).stop 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 msgr2=0x7fa374196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa368005b40 tx=0x7fa36800bde0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 shutdown_connections 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa35c06cce0 0x7fa35c06f1a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3740ff4c0 0x7fa374196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 --2- 172.21.15.8:0/1601001636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa3740ffe70 0x7fa374196b60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.703+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 >> 172.21.15.8:0/1601001636 conn(0x7fa374074cb0 msgr2=0x7fa3740fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.704+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 shutdown_connections 2024-04-12T06:02:32.699 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.704+0000 7fa379dec700 1 -- 172.21.15.8:0/1601001636 wait complete. 2024-04-12T06:02:32.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 -- 172.21.15.8:0/3050544143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 msgr2=0x7feaa80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 --2- 172.21.15.8:0/3050544143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa80fffc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009910 tx=0x7fea9c009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 -- 172.21.15.8:0/3050544143 shutdown_connections 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 --2- 172.21.15.8:0/3050544143 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa80fffc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 --2- 172.21.15.8:0/3050544143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 0x7feaa8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 -- 172.21.15.8:0/3050544143 >> 172.21.15.8:0/3050544143 conn(0x7feaa80fb610 msgr2=0x7feaa80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 -- 172.21.15.8:0/3050544143 shutdown_connections 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.809+0000 7feaae5df700 1 -- 172.21.15.8:0/3050544143 wait complete. 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 Processor -- start 2024-04-12T06:02:32.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 -- start start 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 0x7feaa8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81022c0 con 0x7feaa80ffb40 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaae5df700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa8102430 con 0x7feaa8105bf0 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47224/0 (socket says 172.21.15.8:47224) 2024-04-12T06:02:32.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 -- 172.21.15.8:0/2301418684 learned_addr learned my addr 172.21.15.8:0/2301418684 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa77fe700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 0x7feaa8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 -- 172.21.15.8:0/2301418684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 msgr2=0x7feaa8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 0x7feaa8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.810+0000 7feaa7fff700 1 -- 172.21.15.8:0/2301418684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0095f0 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaa7fff700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d8d0 tx=0x7fea9800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea980098a0 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea98010460 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81a28f0 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea98009ea0 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.811+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81a2d60 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.812+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa804f2e0 con 0x7feaa80ffb40 2024-04-12T06:02:32.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.812+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fea980105d0 con 0x7feaa80ffb40 2024-04-12T06:02:32.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.813+0000 7feaa57fa700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 0x7fea9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:32.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.813+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fea9808c430 con 0x7feaa80ffb40 2024-04-12T06:02:32.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.813+0000 7feaa77fe700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 0x7fea9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:32.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.813+0000 7feaa77fe700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 0x7fea9006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feaa8102fa0 tx=0x7fea9c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:32.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:32.817+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fea980564b0 con 0x7feaa80ffb40 2024-04-12T06:02:32.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:32 smithi194 ceph-mon[106982]: pgmap v205: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:02:32.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:32.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:32.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:32 smithi008 ceph-mon[146084]: pgmap v205: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:02:32.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:32.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:33.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.101+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaa81a2ff0 con 0x7feaa80ffb40 2024-04-12T06:02:33.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.102+0000 7feaa57fa700 1 -- 172.21.15.8:0/2301418684 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fea98010880 con 0x7feaa80ffb40 2024-04-12T06:02:33.097 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:02:33.097 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 3 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-04-12T06:02:33.098 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 7 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 msgr2=0x7fea9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 0x7fea9006f1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feaa8102fa0 tx=0x7fea9c01a040 comp rx=0 tx=0).stop 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 msgr2=0x7feaa81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d8d0 tx=0x7fea9800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 shutdown_connections 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea9006cce0 0x7fea9006f1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feaa80ffb40 0x7feaa81017b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 --2- 172.21.15.8:0/2301418684 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feaa8105bf0 0x7feaa8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 >> 172.21.15.8:0/2301418684 conn(0x7feaa80fb610 msgr2=0x7feaa8068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 shutdown_connections 2024-04-12T06:02:33.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.104+0000 7feaae5df700 1 -- 172.21.15.8:0/2301418684 wait complete. 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 -- 172.21.15.8:0/2098407820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 msgr2=0x7f2fdc10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/2098407820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc10ac50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009940 tx=0x7f2fd0009c50 comp rx=0 tx=0).stop 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 -- 172.21.15.8:0/2098407820 shutdown_connections 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/2098407820 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc10ac50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/2098407820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 0x7f2fdc102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 -- 172.21.15.8:0/2098407820 >> 172.21.15.8:0/2098407820 conn(0x7f2fdc0fb610 msgr2=0x7f2fdc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 -- 172.21.15.8:0/2098407820 shutdown_connections 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.210+0000 7f2fe2f68700 1 -- 172.21.15.8:0/2098407820 wait complete. 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 Processor -- start 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 -- start start 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 0x7f2fdc0ffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc100520 con 0x7f2fdc102760 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe2f68700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc100660 con 0x7f2fdc101e40 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47246/0 (socket says 172.21.15.8:47246) 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 -- 172.21.15.8:0/1322890192 learned_addr learned my addr 172.21.15.8:0/1322890192 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fe0d04700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 0x7f2fdc0ffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 -- 172.21.15.8:0/1322890192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 msgr2=0x7f2fdc0ffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 0x7f2fdc0ffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 -- 172.21.15.8:0/1322890192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00095f0 con 0x7f2fdc102760 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.211+0000 7f2fdbfff700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0005b40 tx=0x7f2fd00055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.212+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd001d070 con 0x7f2fdc102760 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.212+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1008e0 con 0x7f2fdc102760 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.212+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2fd0005dc0 con 0x7f2fdc102760 2024-04-12T06:02:33.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.212+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fd000fe20 con 0x7f2fdc102760 2024-04-12T06:02:33.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.212+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1a2cd0 con 0x7f2fdc102760 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.213+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc108350 con 0x7f2fdc102760 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.213+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2fd000f460 con 0x7f2fdc102760 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.214+0000 7f2fd9ffb700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 0x7f2fc406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.214+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2fd008da10 con 0x7f2fdc102760 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.214+0000 7f2fe0d04700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 0x7f2fc406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.214+0000 7f2fe0d04700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 0x7f2fc406f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009550 tx=0x7f2fd40091b0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.218+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2fd0057bd0 con 0x7f2fdc102760 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.466+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2fdc02cc70 con 0x7f2fdc102760 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.466+0000 7f2fd9ffb700 1 -- 172.21.15.8:0/1322890192 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f2fd005b1f0 con 0x7f2fdc102760 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:02:33.461 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:02:33.462 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:02:33.463 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:02:33.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 msgr2=0x7f2fc406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 0x7f2fc406f1a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009550 tx=0x7f2fd40091b0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 msgr2=0x7f2fdc0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0005b40 tx=0x7f2fd00055f0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 shutdown_connections 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2fc406cce0 0x7f2fc406f1a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fdc101e40 0x7f2fdc0ffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 --2- 172.21.15.8:0/1322890192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2fdc102760 0x7f2fdc0fffe0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 >> 172.21.15.8:0/1322890192 conn(0x7f2fdc0fb610 msgr2=0x7f2fdc105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 shutdown_connections 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.468+0000 7f2fe2f68700 1 -- 172.21.15.8:0/1322890192 wait complete. 2024-04-12T06:02:33.465 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:02:33.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 -- 172.21.15.8:0/2508754217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea30105bd0 msgr2=0x7fea30105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 --2- 172.21.15.8:0/2508754217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea30105bd0 0x7fea30105fb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea20009910 tx=0x7fea20009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:33.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 -- 172.21.15.8:0/2508754217 shutdown_connections 2024-04-12T06:02:33.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 --2- 172.21.15.8:0/2508754217 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea300ffc20 0x7fea30100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 --2- 172.21.15.8:0/2508754217 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea30105bd0 0x7fea30105fb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 -- 172.21.15.8:0/2508754217 >> 172.21.15.8:0/2508754217 conn(0x7fea30075670 msgr2=0x7fea30075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 -- 172.21.15.8:0/2508754217 shutdown_connections 2024-04-12T06:02:33.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.582+0000 7fea34af6700 1 -- 172.21.15.8:0/2508754217 wait complete. 2024-04-12T06:02:33.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 Processor -- start 2024-04-12T06:02:33.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 -- start start 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30105bd0 0x7fea30101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea301022c0 con 0x7fea300ffc20 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea34af6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30102400 con 0x7fea30105bd0 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea2e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea2e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50196/0 (socket says 172.21.15.8:50196) 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea2e59c700 1 -- 172.21.15.8:0/2432615285 learned_addr learned my addr 172.21.15.8:0/2432615285 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.583+0000 7fea27fff700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30105bd0 0x7fea30101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea2e59c700 1 -- 172.21.15.8:0/2432615285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30105bd0 msgr2=0x7fea30101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea2e59c700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30105bd0 0x7fea30101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea2e59c700 1 -- 172.21.15.8:0/2432615285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea200095f0 con 0x7fea300ffc20 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea2e59c700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea20000c00 tx=0x7fea2000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea2001d070 con 0x7fea300ffc20 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea20005dc0 con 0x7fea300ffc20 2024-04-12T06:02:33.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea20003940 con 0x7fea300ffc20 2024-04-12T06:02:33.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea30102680 con 0x7fea300ffc20 2024-04-12T06:02:33.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.584+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea301a2c60 con 0x7fea300ffc20 2024-04-12T06:02:33.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.585+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3004f2e0 con 0x7fea300ffc20 2024-04-12T06:02:33.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.585+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fea20004ae0 con 0x7fea300ffc20 2024-04-12T06:02:33.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.586+0000 7fea277fe700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.586+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fea2008dae0 con 0x7fea300ffc20 2024-04-12T06:02:33.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.586+0000 7fea27fff700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.582 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.586+0000 7fea27fff700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 0x7fea1c06f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea30103030 tx=0x7fea18009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.585 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.590+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fea20057c00 con 0x7fea300ffc20 2024-04-12T06:02:33.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.813+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fea30068db0 con 0x7fea1c06cce0 2024-04-12T06:02:33.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.814+0000 7fea277fe700 1 -- 172.21.15.8:0/2432615285 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fea30068db0 con 0x7fea1c06cce0 2024-04-12T06:02:33.809 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:02:33.809 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:02:33.809 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "9/23 daemons upgraded", 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:02:33.810 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:02:33.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 msgr2=0x7fea1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 0x7fea1c06f1a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea30103030 tx=0x7fea18009260 comp rx=0 tx=0).stop 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 msgr2=0x7fea30101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea20000c00 tx=0x7fea2000bde0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 shutdown_connections 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fea1c06cce0 0x7fea1c06f1a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fea300ffc20 0x7fea30101840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 --2- 172.21.15.8:0/2432615285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fea30105bd0 0x7fea30101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 >> 172.21.15.8:0/2432615285 conn(0x7fea30075670 msgr2=0x7fea300686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 shutdown_connections 2024-04-12T06:02:33.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.816+0000 7fea34af6700 1 -- 172.21.15.8:0/2432615285 wait complete. 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: Upgrade: Updating osd.3 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: Deploying daemon osd.3 on smithi194 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2301418684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:33.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:33 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1322890192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: Upgrade: Updating osd.3 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:33.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: Deploying daemon osd.3 on smithi194 2024-04-12T06:02:33.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:33.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2301418684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:33.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:33 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1322890192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:02:33.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 -- 172.21.15.8:0/2299828378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0106cd0 msgr2=0x7f82c01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2299828378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0106cd0 0x7f82c01070b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009910 tx=0x7f82b0009c20 comp rx=0 tx=0).stop 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 -- 172.21.15.8:0/2299828378 shutdown_connections 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2299828378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0068580 0x7f82c00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2299828378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0106cd0 0x7f82c01070b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 -- 172.21.15.8:0/2299828378 >> 172.21.15.8:0/2299828378 conn(0x7f82c0075670 msgr2=0x7f82c0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 -- 172.21.15.8:0/2299828378 shutdown_connections 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.922+0000 7f82c65cd700 1 -- 172.21.15.8:0/2299828378 wait complete. 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 Processor -- start 2024-04-12T06:02:33.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 -- start start 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 0x7f82c019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c019d130 con 0x7f82c0068580 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82c65cd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0196630 con 0x7f82c0106cd0 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50220/0 (socket says 172.21.15.8:50220) 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82bffff700 1 -- 172.21.15.8:0/2711699366 learned_addr learned my addr 172.21.15.8:0/2711699366 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.923+0000 7f82bf7fe700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 0x7f82c019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bffff700 1 -- 172.21.15.8:0/2711699366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 msgr2=0x7f82c019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bffff700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 0x7f82c019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:33.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bffff700 1 -- 172.21.15.8:0/2711699366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00095f0 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bf7fe700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 0x7f82c019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bffff700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b001d070 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f82b00044c0 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c01968b0 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b00039d0 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.924+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c0196d40 con 0x7f82c0068580 2024-04-12T06:02:33.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.925+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c01091f0 con 0x7f82c0068580 2024-04-12T06:02:33.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.925+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f82b00048e0 con 0x7f82c0068580 2024-04-12T06:02:33.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.926+0000 7f82bd7fa700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 0x7f82ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:02:33.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.926+0000 7f82bf7fe700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 0x7f82ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:02:33.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.926+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f82b008db50 con 0x7f82c0068580 2024-04-12T06:02:33.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.926+0000 7f82bf7fe700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 0x7f82ac06f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82c00fe390 tx=0x7f82a8009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:02:33.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:33.931+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f82b005b700 con 0x7f82c0068580 2024-04-12T06:02:34.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.216+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f82c004f2e0 con 0x7f82c0068580 2024-04-12T06:02:34.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.217+0000 7f82bd7fa700 1 -- 172.21.15.8:0/2711699366 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f82b0004be0 con 0x7f82c0068580 2024-04-12T06:02:34.213 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:02:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 msgr2=0x7f82ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 0x7f82ac06f1a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82c00fe390 tx=0x7f82a8009260 comp rx=0 tx=0).stop 2024-04-12T06:02:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 msgr2=0x7f82c019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:02:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b000be90 comp rx=0 tx=0).stop 2024-04-12T06:02:34.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 shutdown_connections 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f82ac06cce0 0x7f82ac06f1a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f82c0068580 0x7f82c019c560 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 --2- 172.21.15.8:0/2711699366 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f82c0106cd0 0x7f82c019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 >> 172.21.15.8:0/2711699366 conn(0x7f82c0075670 msgr2=0x7f82c00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 shutdown_connections 2024-04-12T06:02:34.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:02:34.219+0000 7f82c65cd700 1 -- 172.21.15.8:0/2711699366 wait complete. 2024-04-12T06:02:34.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:34 smithi194 ceph-mon[106982]: pgmap v206: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:02:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:34 smithi194 ceph-mon[106982]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:34.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:34 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2711699366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:02:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:34 smithi008 ceph-mon[146084]: pgmap v206: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:02:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:34 smithi008 ceph-mon[146084]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:02:34.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:34 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2711699366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:02:35.820 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:35 smithi194 systemd[1]: Stopping Ceph osd.3 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:02:35.820 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[56703]: 2024-04-12T06:02:35.785+0000 7fe11320d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:02:35.820 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[56703]: 2024-04-12T06:02:35.785+0000 7fe11320d700 -1 osd.3 63 *** Got signal Terminated *** 2024-04-12T06:02:35.821 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[56703]: 2024-04-12T06:02:35.785+0000 7fe11320d700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:02:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:36 smithi194 ceph-mon[106982]: pgmap v207: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:02:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:36 smithi194 ceph-mon[106982]: osd.3 marked itself down and dead 2024-04-12T06:02:36.834 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:36 smithi008 ceph-mon[146084]: pgmap v207: 65 pgs: 65 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:02:36.834 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:36 smithi008 ceph-mon[146084]: osd.3 marked itself down and dead 2024-04-12T06:02:37.569 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:37 smithi194 bash[118368]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3 2024-04-12T06:02:37.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:37 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:02:37.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:37 smithi194 ceph-mon[106982]: osdmap e64: 6 total, 5 up, 6 in 2024-04-12T06:02:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:37 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:02:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:37 smithi008 ceph-mon[146084]: osdmap e64: 6 total, 5 up, 6 in 2024-04-12T06:02:38.750 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:38 smithi194 ceph-mon[106982]: pgmap v209: 65 pgs: 16 stale+active+clean, 49 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:02:38.751 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:38 smithi194 ceph-mon[106982]: osdmap e65: 6 total, 5 up, 6 in 2024-04-12T06:02:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:38 smithi008 ceph-mon[146084]: pgmap v209: 65 pgs: 16 stale+active+clean, 49 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:02:38.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:38 smithi008 ceph-mon[146084]: osdmap e65: 6 total, 5 up, 6 in 2024-04-12T06:02:39.320 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:39 smithi194 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.3.service: Succeeded. 2024-04-12T06:02:39.320 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:39 smithi194 systemd[1]: Stopped Ceph osd.3 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:02:39.320 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:39 smithi194 systemd[1]: Starting Ceph osd.3 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:02:39.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:39 smithi194 ceph-mon[106982]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-12T06:02:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:39 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 40/333 objects degraded (12.012%), 9 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:39.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:39 smithi008 ceph-mon[146084]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-12T06:02:39.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:39 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 40/333 objects degraded (12.012%), 9 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-12T06:02:40.507 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:40 smithi194 bash[118644]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-12T06:02:40.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:40 smithi194 ceph-mon[106982]: pgmap v211: 65 pgs: 9 active+undersized+degraded, 7 peering, 3 stale+active+clean, 9 active+undersized, 37 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/333 objects degraded (12.012%) 2024-04-12T06:02:40.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:40 smithi008 ceph-mon[146084]: pgmap v211: 65 pgs: 9 active+undersized+degraded, 7 peering, 3 stale+active+clean, 9 active+undersized, 37 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 40/333 objects degraded (12.012%) 2024-04-12T06:02:41.569 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:41 smithi194 ceph-osd[118959]: -- 172.21.15.194:0/230159074 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c0334551e0 con 0x55c033424c00 2024-04-12T06:02:41.570 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:41 smithi194 bash[118921]: 95a8d9917f81ad60a8a0791cceabdd57862c6afe4a41e27f15ad41e17f2fccd4 2024-04-12T06:02:41.570 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:41 smithi194 systemd[1]: Started Ceph osd.3 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:02:42.069 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:41 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:02:41.790+0000 7fb5ca0547c0 -1 Falling back to public interface 2024-04-12T06:02:42.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:42 smithi008 ceph-mon[146084]: pgmap v212: 65 pgs: 16 active+undersized+degraded, 7 peering, 15 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/333 objects degraded (18.619%) 2024-04-12T06:02:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:42.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:42.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:42 smithi194 ceph-mon[106982]: pgmap v212: 65 pgs: 16 active+undersized+degraded, 7 peering, 15 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 62/333 objects degraded (18.619%) 2024-04-12T06:02:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:43 smithi194 ceph-mon[106982]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-12T06:02:43.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:43 smithi008 ceph-mon[146084]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-12T06:02:44.819 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:44 smithi194 ceph-mon[106982]: pgmap v213: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:44.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:44 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:44.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:44 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:44 smithi008 ceph-mon[146084]: pgmap v213: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:44 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:44.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:44 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:46.320 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:46 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:02:46.116+0000 7fb5ca0547c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-12T06:02:46.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:45 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:46.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:45 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:46.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:45 smithi194 ceph-mon[106982]: pgmap v214: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:46.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:45 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:45 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:46.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:45 smithi008 ceph-mon[146084]: pgmap v214: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:46.799 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:46 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:02:46.661+0000 7fb5ca0547c0 -1 osd.3 63 log_to_monitors true 2024-04-12T06:02:47.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:47 smithi194 ceph-mon[106982]: from='osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T06:02:47.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:47 smithi194 ceph-mon[106982]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T06:02:47.070 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:02:47 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:02:47.042+0000 7fb5bb4ae700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:02:47.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:47 smithi008 ceph-mon[146084]: from='osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T06:02:47.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:47 smithi008 ceph-mon[146084]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-12T06:02:48.023 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T06:02:48.023 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: osdmap e66: 6 total, 5 up, 6 in 2024-04-12T06:02:48.024 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: from='osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:02:48.024 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:02:48.024 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: pgmap v216: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:48.024 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:48.024 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-12T06:02:48.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: osdmap e66: 6 total, 5 up, 6 in 2024-04-12T06:02:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: from='osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:02:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:02:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: pgmap v216: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-04-12T06:02:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:48.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463] boot 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: osdmap e67: 6 total, 6 up, 6 in 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 69/333 objects degraded (20.721%), 19 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-12T06:02:49.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:49 smithi194 ceph-mon[106982]: osdmap e68: 6 total, 6 up, 6 in 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: osd.3 [v2:172.21.15.194:6800/1524342463,v1:172.21.15.194:6801/1524342463] boot 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: osdmap e67: 6 total, 6 up, 6 in 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 69/333 objects degraded (20.721%), 19 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:02:49.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-12T06:02:49.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:49 smithi008 ceph-mon[146084]: osdmap e68: 6 total, 6 up, 6 in 2024-04-12T06:02:50.319 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:50 smithi194 ceph-mon[106982]: pgmap v219: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 59/333 objects degraded (17.718%) 2024-04-12T06:02:50.403 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:50 smithi008 ceph-mon[146084]: pgmap v219: 65 pgs: 8 peering, 16 active+undersized+degraded, 14 active+undersized, 27 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 59/333 objects degraded (17.718%) 2024-04-12T06:02:52.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:52 smithi194 ceph-mon[106982]: pgmap v220: 65 pgs: 8 peering, 11 active+undersized+degraded, 9 active+undersized, 37 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 44/333 objects degraded (13.213%) 2024-04-12T06:02:52.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:52 smithi008 ceph-mon[146084]: pgmap v220: 65 pgs: 8 peering, 11 active+undersized+degraded, 9 active+undersized, 37 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 44/333 objects degraded (13.213%) 2024-04-12T06:02:53.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:53 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 44/333 objects degraded (13.213%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:53.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:53 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 44/333 objects degraded (13.213%), 11 pgs degraded (PG_DEGRADED) 2024-04-12T06:02:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:54 smithi194 ceph-mon[106982]: pgmap v221: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-04-12T06:02:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:54 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/333 objects degraded (13.213%), 11 pgs degraded) 2024-04-12T06:02:54.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:54 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:02:54.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:54 smithi008 ceph-mon[146084]: pgmap v221: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 3 op/s 2024-04-12T06:02:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:54 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/333 objects degraded (13.213%), 11 pgs degraded) 2024-04-12T06:02:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:54 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:02:56.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:56 smithi194 ceph-mon[106982]: pgmap v222: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-04-12T06:02:56.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:56 smithi008 ceph-mon[146084]: pgmap v222: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-04-12T06:02:58.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:02:58 smithi194 ceph-mon[106982]: pgmap v223: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-12T06:02:58.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:02:58 smithi008 ceph-mon[146084]: pgmap v223: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 3 op/s 2024-04-12T06:03:00.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:00 smithi194 ceph-mon[106982]: pgmap v224: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:03:00.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:00 smithi008 ceph-mon[146084]: pgmap v224: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:03:02.778 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:02 smithi194 ceph-mon[106982]: pgmap v225: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-04-12T06:03:02.778 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:02.778 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:02 smithi008 ceph-mon[146084]: pgmap v225: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-04-12T06:03:02.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:02.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:03.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:03.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 -- 172.21.15.8:0/1688749054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc100e90 msgr2=0x7f72cc101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 --2- 172.21.15.8:0/1688749054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc100e90 0x7f72cc101270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009910 tx=0x7f72bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 -- 172.21.15.8:0/1688749054 shutdown_connections 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 --2- 172.21.15.8:0/1688749054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc101840 0x7f72cc105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 --2- 172.21.15.8:0/1688749054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc100e90 0x7f72cc101270 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 -- 172.21.15.8:0/1688749054 >> 172.21.15.8:0/1688749054 conn(0x7f72cc0fc740 msgr2=0x7f72cc0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 -- 172.21.15.8:0/1688749054 shutdown_connections 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 -- 172.21.15.8:0/1688749054 wait complete. 2024-04-12T06:03:04.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.341+0000 7f72d18da700 1 Processor -- start 2024-04-12T06:03:04.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72d18da700 1 -- start start 2024-04-12T06:03:04.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72d18da700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc100e90 0x7f72cc19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72d18da700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72d18da700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc19d190 con 0x7f72cc101840 2024-04-12T06:03:04.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72d18da700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc196640 con 0x7f72cc100e90 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50322/0 (socket says 172.21.15.8:50322) 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 -- 172.21.15.8:0/1021671913 learned_addr learned my addr 172.21.15.8:0/1021671913 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72caffd700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc100e90 0x7f72cc19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 -- 172.21.15.8:0/1021671913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc100e90 msgr2=0x7f72cc19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc100e90 0x7f72cc19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.342+0000 7f72ca7fc700 1 -- 172.21.15.8:0/1021671913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0095f0 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72ca7fc700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d900 tx=0x7f72c000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c0009900 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72c0010460 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c000f5f0 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc196920 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.343+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc196e70 con 0x7f72cc101840 2024-04-12T06:03:04.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.344+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc109230 con 0x7f72cc101840 2024-04-12T06:03:04.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.345+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f72c00105d0 con 0x7f72cc101840 2024-04-12T06:03:04.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.345+0000 7f72d08d8700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 0x7f72b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.346+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f72c008c090 con 0x7f72cc101840 2024-04-12T06:03:04.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.346+0000 7f72caffd700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 0x7f72b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.346+0000 7f72caffd700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 0x7f72b406f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00b5c0 tx=0x7f72bc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:04.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.350+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f72c0059ca0 con 0x7f72cc101840 2024-04-12T06:03:04.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.568+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72cc1970e0 con 0x7f72b406cce0 2024-04-12T06:03:04.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.570+0000 7f72d08d8700 1 -- 172.21.15.8:0/1021671913 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f72cc1970e0 con 0x7f72b406cce0 2024-04-12T06:03:04.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 msgr2=0x7f72b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 0x7f72b406f1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00b5c0 tx=0x7f72bc01a040 comp rx=0 tx=0).stop 2024-04-12T06:03:04.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 msgr2=0x7f72cc19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f72c000d900 tx=0x7f72c000dc10 comp rx=0 tx=0).stop 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 shutdown_connections 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72b406cce0 0x7f72b406f1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72cc100e90 0x7f72cc19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 --2- 172.21.15.8:0/1021671913 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72cc101840 0x7f72cc19cb00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 >> 172.21.15.8:0/1021671913 conn(0x7f72cc0fc740 msgr2=0x7f72cc0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 shutdown_connections 2024-04-12T06:03:04.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.572+0000 7f72d18da700 1 -- 172.21.15.8:0/1021671913 wait complete. 2024-04-12T06:03:04.578 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:03:04.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 -- 172.21.15.8:0/1319256722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc105bf0 msgr2=0x7fc7bc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/1319256722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc105fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009910 tx=0x7fc7b0009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 -- 172.21.15.8:0/1319256722 shutdown_connections 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/1319256722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/1319256722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc105fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 -- 172.21.15.8:0/1319256722 >> 172.21.15.8:0/1319256722 conn(0x7fc7bc0fb610 msgr2=0x7fc7bc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 -- 172.21.15.8:0/1319256722 shutdown_connections 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.679+0000 7fc7c2f29700 1 -- 172.21.15.8:0/1319256722 wait complete. 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 Processor -- start 2024-04-12T06:03:04.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 -- start start 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc198110 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c2f29700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc198250 con 0x7fc7bc105bf0 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c0cc5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c0cc5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50340/0 (socket says 172.21.15.8:50340) 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.680+0000 7fc7c0cc5700 1 -- 172.21.15.8:0/4093745664 learned_addr learned my addr 172.21.15.8:0/4093745664 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7bbfff700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c0cc5700 1 -- 172.21.15.8:0/4093745664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc105bf0 msgr2=0x7fc7bc197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c0cc5700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c0cc5700 1 -- 172.21.15.8:0/4093745664 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b00095f0 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c0cc5700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00092e0 tx=0x7fc7b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b001d070 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7b0005e00 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc1a0620 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7b000fe90 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.681+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc1a0a30 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.682+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc108020 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.683+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc7b000f6d0 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.683+0000 7fc7b9ffb700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 0x7fc7ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:04.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.684+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc7b008d7a0 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.684+0000 7fc7bbfff700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 0x7fc7ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:04.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.684+0000 7fc7bbfff700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 0x7fc7ac06f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0696c0 tx=0x7fc7a800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:04.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.688+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc7b0057820 con 0x7fc7bc0ffb40 2024-04-12T06:03:04.766 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:04 smithi194 ceph-mon[106982]: pgmap v226: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s 2024-04-12T06:03:04.766 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:04 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:03:04.766 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:04 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:03:04.766 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:04 smithi194 ceph-mon[106982]: Upgrade: osd.4 is safe to restart 2024-04-12T06:03:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:04 smithi008 ceph-mon[146084]: pgmap v226: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s 2024-04-12T06:03:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:04 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:03:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:04 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-12T06:03:04.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:04 smithi008 ceph-mon[146084]: Upgrade: osd.4 is safe to restart 2024-04-12T06:03:04.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.909+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7bc02cce0 con 0x7fc7ac06cce0 2024-04-12T06:03:04.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.910+0000 7fc7b9ffb700 1 -- 172.21.15.8:0/4093745664 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc7bc02cce0 con 0x7fc7ac06cce0 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 msgr2=0x7fc7ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 0x7fc7ac06f1a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc0696c0 tx=0x7fc7a800b410 comp rx=0 tx=0).stop 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 msgr2=0x7fc7bc1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00092e0 tx=0x7fc7b000bd70 comp rx=0 tx=0).stop 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 shutdown_connections 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc7ac06cce0 0x7fc7ac06f1a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7bc0ffb40 0x7fc7bc1975b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 --2- 172.21.15.8:0/4093745664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7bc105bf0 0x7fc7bc197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 >> 172.21.15.8:0/4093745664 conn(0x7fc7bc0fb610 msgr2=0x7fc7bc0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 shutdown_connections 2024-04-12T06:03:04.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:04.912+0000 7fc7c2f29700 1 -- 172.21.15.8:0/4093745664 wait complete. 2024-04-12T06:03:05.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.018+0000 7f2e477b7700 1 -- 172.21.15.8:0/3898275979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 msgr2=0x7f2e400ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.018+0000 7f2e477b7700 1 --2- 172.21.15.8:0/3898275979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e400ff980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009910 tx=0x7f2e3c009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:05.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 -- 172.21.15.8:0/3898275979 shutdown_connections 2024-04-12T06:03:05.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 --2- 172.21.15.8:0/3898275979 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 0x7f2e40105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 --2- 172.21.15.8:0/3898275979 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e400ff980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 -- 172.21.15.8:0/3898275979 >> 172.21.15.8:0/3898275979 conn(0x7f2e40074e30 msgr2=0x7f2e40075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:05.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 -- 172.21.15.8:0/3898275979 shutdown_connections 2024-04-12T06:03:05.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 -- 172.21.15.8:0/3898275979 wait complete. 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 Processor -- start 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 -- start start 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.019+0000 7f2e477b7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 0x7f2e4019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e477b7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4019f360 con 0x7f2e400ff5a0 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e477b7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40198810 con 0x7f2e400fff50 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50362/0 (socket says 172.21.15.8:50362) 2024-04-12T06:03:05.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 -- 172.21.15.8:0/1205598097 learned_addr learned my addr 172.21.15.8:0/1205598097 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e44d52700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 0x7f2e4019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 -- 172.21.15.8:0/1205598097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 msgr2=0x7f2e4019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 0x7f2e4019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 -- 172.21.15.8:0/1205598097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c0095f0 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e45553700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3c01d070 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e3c005710 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e40198a90 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3c0039d0 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.020+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e40198f20 con 0x7f2e400ff5a0 2024-04-12T06:03:05.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.021+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e40104fc0 con 0x7f2e400ff5a0 2024-04-12T06:03:05.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.022+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2e3c003c00 con 0x7f2e400ff5a0 2024-04-12T06:03:05.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.023+0000 7f2e367fc700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 0x7f2e2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.023+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2e3c08e140 con 0x7f2e400ff5a0 2024-04-12T06:03:05.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.023+0000 7f2e44d52700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 0x7f2e2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.023+0000 7f2e44d52700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 0x7f2e2c06f1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e40107700 tx=0x7f2e30009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.027+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2e3c05be10 con 0x7f2e400ff5a0 2024-04-12T06:03:05.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.241+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e401991f0 con 0x7f2e2c06cce0 2024-04-12T06:03:05.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.252+0000 7f2e367fc700 1 -- 172.21.15.8:0/1205598097 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2e401991f0 con 0x7f2e2c06cce0 2024-04-12T06:03:05.247 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:03:05.247 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (4m) 54s ago 16m 27.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:03:05.247 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (16m) 54s ago 16m 8262k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:03:05.247 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (15m) 20s ago 15m 8266k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (2m) 54s ago 16m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (2m) 20s ago 15m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (4m) 54s ago 16m 80.8M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (10m) 54s ago 10m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (10m) 54s ago 10m 71.2M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (10m) 20s ago 10m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (10m) 20s ago 10m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (7m) 54s ago 17m 510M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (6m) 20s ago 15m 451M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (3m) 54s ago 18m 47.3M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (2m) 20s ago 15m 38.7M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (5m) 54s ago 16m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (5m) 20s ago 15m 18.1M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:03:05.248 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (2m) 54s ago 14m 95.6M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (96s) 54s ago 13m 103M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (60s) 54s ago 13m 11.2M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (24s) 20s ago 12m 11.2M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (12m) 20s ago 12m 232M 2251M 18.2.1 5be31c24972a 4c0ac37b9b35 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (11m) 20s ago 11m 210M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:03:05.249 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (5m) 54s ago 15m 75.0M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 msgr2=0x7f2e2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 0x7f2e2c06f1a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e40107700 tx=0x7f2e30009260 comp rx=0 tx=0).stop 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 msgr2=0x7f2e4019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c000c00 tx=0x7f2e3c00bde0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 shutdown_connections 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e2c06cce0 0x7f2e2c06f1a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e400ff5a0 0x7f2e4019e790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 --2- 172.21.15.8:0/1205598097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e400fff50 0x7f2e4019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 >> 172.21.15.8:0/1205598097 conn(0x7f2e40074e30 msgr2=0x7f2e400fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 shutdown_connections 2024-04-12T06:03:05.250 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.254+0000 7f2e477b7700 1 -- 172.21.15.8:0/1205598097 wait complete. 2024-04-12T06:03:05.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 -- 172.21.15.8:0/3757477984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 msgr2=0x7f2208105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 --2- 172.21.15.8:0/3757477984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208105fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2204009910 tx=0x7f2204009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:05.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 -- 172.21.15.8:0/3757477984 shutdown_connections 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 --2- 172.21.15.8:0/3757477984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 0x7f2208100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 --2- 172.21.15.8:0/3757477984 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208105fb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 -- 172.21.15.8:0/3757477984 >> 172.21.15.8:0/3757477984 conn(0x7f2208075670 msgr2=0x7f2208075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 -- 172.21.15.8:0/3757477984 shutdown_connections 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.359+0000 7f22103b3700 1 -- 172.21.15.8:0/3757477984 wait complete. 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 Processor -- start 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 -- start start 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 0x7f22081975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2208198140 con 0x7f2208105bd0 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f22103b3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22081982b0 con 0x7f22080ffc20 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50376/0 (socket says 172.21.15.8:50376) 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 -- 172.21.15.8:0/3069624653 learned_addr learned my addr 172.21.15.8:0/3069624653 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220e14f700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 0x7f22081975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 -- 172.21.15.8:0/3069624653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 msgr2=0x7f22081975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 0x7f22081975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 -- 172.21.15.8:0/3069624653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040095f0 con 0x7f2208105bd0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.360+0000 7f220d94e700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d8d0 tx=0x7f21f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.361+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f80098a0 con 0x7f2208105bd0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.361+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f21f8010460 con 0x7f2208105bd0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.361+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22081a0650 con 0x7f2208105bd0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.361+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21f8009ea0 con 0x7f2208105bd0 2024-04-12T06:03:05.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.361+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22081a0b40 con 0x7f2208105bd0 2024-04-12T06:03:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.362+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22081080d0 con 0x7f2208105bd0 2024-04-12T06:03:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.362+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f21f80105d0 con 0x7f2208105bd0 2024-04-12T06:03:05.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.363+0000 7f21ff7fe700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 0x7f21f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.363+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f21f808c260 con 0x7f2208105bd0 2024-04-12T06:03:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.363+0000 7f220e14f700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 0x7f21f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.363+0000 7f220e14f700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 0x7f21f406f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f220401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.367+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f21f8056550 con 0x7f2208105bd0 2024-04-12T06:03:05.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.653+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f220804f2e0 con 0x7f2208105bd0 2024-04-12T06:03:05.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.654+0000 7f21ff7fe700 1 -- 172.21.15.8:0/3069624653 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f21f8010880 con 0x7f2208105bd0 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:03:05.650 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 8 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:03:05.651 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 msgr2=0x7f21f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 0x7f21f406f1a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f220401a040 comp rx=0 tx=0).stop 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 msgr2=0x7f2208197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21f800d8d0 tx=0x7f21f800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 shutdown_connections 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f21f406cce0 0x7f21f406f1a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22080ffc20 0x7f22081975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 --2- 172.21.15.8:0/3069624653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2208105bd0 0x7f2208197b20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 >> 172.21.15.8:0/3069624653 conn(0x7f2208075670 msgr2=0x7f22080fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 shutdown_connections 2024-04-12T06:03:05.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.656+0000 7f22103b3700 1 -- 172.21.15.8:0/3069624653 wait complete. 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 -- 172.21.15.8:0/1150244978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 msgr2=0x7efcfc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 --2- 172.21.15.8:0/1150244978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc0689e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efcec009910 tx=0x7efcec009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 -- 172.21.15.8:0/1150244978 shutdown_connections 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 --2- 172.21.15.8:0/1150244978 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc0689e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 --2- 172.21.15.8:0/1150244978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 0x7efcfc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 -- 172.21.15.8:0/1150244978 >> 172.21.15.8:0/1150244978 conn(0x7efcfc075670 msgr2=0x7efcfc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:05.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 -- 172.21.15.8:0/1150244978 shutdown_connections 2024-04-12T06:03:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.763+0000 7efd03fdb700 1 -- 172.21.15.8:0/1150244978 wait complete. 2024-04-12T06:03:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.764+0000 7efd03fdb700 1 Processor -- start 2024-04-12T06:03:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.764+0000 7efd03fdb700 1 -- start start 2024-04-12T06:03:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 0x7efcfc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19d110 con 0x7efcfc068580 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1965c0 con 0x7efcfc106cd0 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50394/0 (socket says 172.21.15.8:50394) 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 -- 172.21.15.8:0/2967907696 learned_addr learned my addr 172.21.15.8:0/2967907696 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01576700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 0x7efcfc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 -- 172.21.15.8:0/2967907696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 msgr2=0x7efcfc19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 0x7efcfc19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 -- 172.21.15.8:0/2967907696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec0095f0 con 0x7efcfc068580 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd01d77700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e920 tx=0x7efcf800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf800c9d0 con 0x7efcfc068580 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efcf8010430 con 0x7efcfc068580 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1968a0 con 0x7efcfc068580 2024-04-12T06:03:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efcf8018650 con 0x7efcfc068580 2024-04-12T06:03:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.765+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc196df0 con 0x7efcfc068580 2024-04-12T06:03:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.766+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc1091d0 con 0x7efcfc068580 2024-04-12T06:03:05.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.767+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7efcf80105a0 con 0x7efcfc068580 2024-04-12T06:03:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.767+0000 7efcf2ffd700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 0x7efce806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.767+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efcf8014070 con 0x7efcfc068580 2024-04-12T06:03:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.767+0000 7efd01576700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 0x7efce806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:05.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.768+0000 7efd01576700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 0x7efce806f1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0fe260 tx=0x7efcec005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:05.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:05.772+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7efcf8052ab0 con 0x7efcfc068580 2024-04-12T06:03:05.792 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:05 smithi194 ceph-mon[106982]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:05.792 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:05 smithi194 ceph-mon[106982]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:05.792 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:05 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:05.792 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:05 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T06:03:05.792 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:05 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:05.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:05 smithi008 ceph-mon[146084]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:05.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:05 smithi008 ceph-mon[146084]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:05.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:05 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:05.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:05 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-12T06:03:05.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:05 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:06.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.018+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efcfc04f2e0 con 0x7efcfc068580 2024-04-12T06:03:06.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.019+0000 7efcf2ffd700 1 -- 172.21.15.8:0/2967907696 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7efcf805a350 con 0x7efcfc068580 2024-04-12T06:03:06.014 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:03:06.015 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:03:06.016 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:06.017 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:06.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 msgr2=0x7efce806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 0x7efce806f1a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0fe260 tx=0x7efcec005940 comp rx=0 tx=0).stop 2024-04-12T06:03:06.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 msgr2=0x7efcfc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efcf800e920 tx=0x7efcf800ec30 comp rx=0 tx=0).stop 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 shutdown_connections 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efce806cce0 0x7efce806f1a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efcfc068580 0x7efcfc19c540 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 --2- 172.21.15.8:0/2967907696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efcfc106cd0 0x7efcfc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 >> 172.21.15.8:0/2967907696 conn(0x7efcfc075670 msgr2=0x7efcfc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 shutdown_connections 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.021+0000 7efd03fdb700 1 -- 172.21.15.8:0/2967907696 wait complete. 2024-04-12T06:03:06.019 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 -- 172.21.15.8:0/3695534219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980101de0 msgr2=0x7f09801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 --2- 172.21.15.8:0/3695534219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980101de0 0x7f09801021c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0968009910 tx=0x7f0968009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 -- 172.21.15.8:0/3695534219 shutdown_connections 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 --2- 172.21.15.8:0/3695534219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980102700 0x7f098010abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 --2- 172.21.15.8:0/3695534219 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980101de0 0x7f09801021c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 -- 172.21.15.8:0/3695534219 >> 172.21.15.8:0/3695534219 conn(0x7f09800fb610 msgr2=0x7f09800fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 -- 172.21.15.8:0/3695534219 shutdown_connections 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 -- 172.21.15.8:0/3695534219 wait complete. 2024-04-12T06:03:06.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.129+0000 7f0985e6f700 1 Processor -- start 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f0985e6f700 1 -- start start 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f0985e6f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980101de0 0x7f098019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f0985e6f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f0985e6f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019d190 con 0x7f0980102700 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f0985e6f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980196640 con 0x7f0980101de0 2024-04-12T06:03:06.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50420/0 (socket says 172.21.15.8:50420) 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 -- 172.21.15.8:0/4216475735 learned_addr learned my addr 172.21.15.8:0/4216475735 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097f7fe700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980101de0 0x7f098019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 -- 172.21.15.8:0/4216475735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980101de0 msgr2=0x7f098019c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980101de0 0x7f098019c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.130+0000 7f097effd700 1 -- 172.21.15.8:0/4216475735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680095f0 con 0x7f0980102700 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f097effd700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f097000e940 tx=0x7f097000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09700096e0 con 0x7f0980102700 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f097000c7c0 con 0x7f0980102700 2024-04-12T06:03:06.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0980196920 con 0x7f0980102700 2024-04-12T06:03:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0970005420 con 0x7f0980102700 2024-04-12T06:03:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.131+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0980196e70 con 0x7f0980102700 2024-04-12T06:03:06.129 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.132+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09801082f0 con 0x7f0980102700 2024-04-12T06:03:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.132+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f097000c930 con 0x7f0980102700 2024-04-12T06:03:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.133+0000 7f097cff9700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 0x7f096c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.133+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0970014070 con 0x7f0980102700 2024-04-12T06:03:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.133+0000 7f097f7fe700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 0x7f096c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.133+0000 7f097f7fe700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 0x7f096c06f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:06.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.137+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f097005a280 con 0x7f0980102700 2024-04-12T06:03:06.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.356+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0980197c30 con 0x7f096c06cce0 2024-04-12T06:03:06.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.357+0000 7f097cff9700 1 -- 172.21.15.8:0/4216475735 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0980197c30 con 0x7f096c06cce0 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "10/23 daemons upgraded", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:03:06.353 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:03:06.354 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 msgr2=0x7f096c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 0x7f096c06f1a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0968009fd0 tx=0x7f0968005940 comp rx=0 tx=0).stop 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 msgr2=0x7f098019cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f097000e940 tx=0x7f097000ec50 comp rx=0 tx=0).stop 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 shutdown_connections 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096c06cce0 0x7f096c06f1a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980101de0 0x7f098019c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 --2- 172.21.15.8:0/4216475735 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0980102700 0x7f098019cb00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 >> 172.21.15.8:0/4216475735 conn(0x7f09800fb610 msgr2=0x7f0980105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:06.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 shutdown_connections 2024-04-12T06:03:06.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.359+0000 7f0985e6f700 1 -- 172.21.15.8:0/4216475735 wait complete. 2024-04-12T06:03:06.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 -- 172.21.15.8:0/3751675052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 msgr2=0x7f69380fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 --2- 172.21.15.8:0/3751675052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69380fffc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6928009910 tx=0x7f6928009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 -- 172.21.15.8:0/3751675052 shutdown_connections 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 --2- 172.21.15.8:0/3751675052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69380fffc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 --2- 172.21.15.8:0/3751675052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 0x7f6938105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.465+0000 7f693fbbb700 1 -- 172.21.15.8:0/3751675052 >> 172.21.15.8:0/3751675052 conn(0x7f69380fb610 msgr2=0x7f69380fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 -- 172.21.15.8:0/3751675052 shutdown_connections 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 -- 172.21.15.8:0/3751675052 wait complete. 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 Processor -- start 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 -- start start 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 0x7f6938101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6938102230 con 0x7f69380ffb40 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.466+0000 7f693fbbb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6938102370 con 0x7f6938105bf0 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:50444/0 (socket says 172.21.15.8:50444) 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 -- 172.21.15.8:0/2942619902 learned_addr learned my addr 172.21.15.8:0/2942619902 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d156700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 0x7f6938101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 -- 172.21.15.8:0/2942619902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 msgr2=0x7f6938101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 0x7f6938101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 -- 172.21.15.8:0/2942619902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280095f0 con 0x7f69380ffb40 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693d957700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f693400b4b0 tx=0x7f693400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6934011800 con 0x7f69380ffb40 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6934011e00 con 0x7f69380ffb40 2024-04-12T06:03:06.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69381a2a20 con 0x7f69380ffb40 2024-04-12T06:03:06.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6934010e60 con 0x7f69380ffb40 2024-04-12T06:03:06.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.467+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69381a2e30 con 0x7f69380ffb40 2024-04-12T06:03:06.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.468+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693804f2e0 con 0x7f69380ffb40 2024-04-12T06:03:06.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.469+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f6934011960 con 0x7f69380ffb40 2024-04-12T06:03:06.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.469+0000 7f692effd700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 0x7f692406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:06.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.469+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f693408bff0 con 0x7f69380ffb40 2024-04-12T06:03:06.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.469+0000 7f693d156700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 0x7f692406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:06.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.470+0000 7f693d156700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 0x7f692406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6938102fa0 tx=0x7f692801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:06.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.474+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6934056110 con 0x7f69380ffb40 2024-04-12T06:03:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.764+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6938102ce0 con 0x7f69380ffb40 2024-04-12T06:03:06.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.764+0000 7f692effd700 1 -- 172.21.15.8:0/2942619902 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6934059730 con 0x7f69380ffb40 2024-04-12T06:03:06.760 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:03:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 msgr2=0x7f692406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 0x7f692406f1a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6938102fa0 tx=0x7f692801a040 comp rx=0 tx=0).stop 2024-04-12T06:03:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 msgr2=0x7f69381017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:06.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f693400b4b0 tx=0x7f693400b7c0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 shutdown_connections 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f692406cce0 0x7f692406f1a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f69380ffb40 0x7f69381017b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 --2- 172.21.15.8:0/2942619902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6938105bf0 0x7f6938101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.766+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 >> 172.21.15.8:0/2942619902 conn(0x7f69380fb610 msgr2=0x7f6938069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.767+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 shutdown_connections 2024-04-12T06:03:06.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:06.767+0000 7f693fbbb700 1 -- 172.21.15.8:0/2942619902 wait complete. 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: Upgrade: Updating osd.4 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: Deploying daemon osd.4 on smithi194 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: pgmap v227: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3069624653' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:06.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:06 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2967907696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: Upgrade: Updating osd.4 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: Deploying daemon osd.4 on smithi194 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: pgmap v227: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3069624653' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:06 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2967907696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:03:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:07 smithi194 ceph-mon[106982]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:07.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:07 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2942619902' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:03:07.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:07 smithi008 ceph-mon[146084]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:07.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:07 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2942619902' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:03:08.632 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:08 smithi194 ceph-mon[106982]: pgmap v228: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:03:08.633 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:08 smithi194 systemd[1]: Stopping Ceph osd.4 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:03:08.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:08 smithi008 ceph-mon[146084]: pgmap v228: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:03:09.070 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:08 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[64703]: 2024-04-12T06:03:08.635+0000 7f6b3cd4b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:03:09.070 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:08 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[64703]: 2024-04-12T06:03:08.635+0000 7f6b3cd4b700 -1 osd.4 68 *** Got signal Terminated *** 2024-04-12T06:03:09.070 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:08 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[64703]: 2024-04-12T06:03:08.635+0000 7f6b3cd4b700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:03:09.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:09 smithi194 ceph-mon[106982]: osd.4 marked itself down and dead 2024-04-12T06:03:09.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:09 smithi008 ceph-mon[146084]: osd.4 marked itself down and dead 2024-04-12T06:03:10.570 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:10 smithi194 bash[124698]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4 2024-04-12T06:03:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:10 smithi194 ceph-mon[106982]: pgmap v229: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:03:10.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:10 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:03:10.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:10 smithi194 ceph-mon[106982]: osdmap e69: 6 total, 5 up, 6 in 2024-04-12T06:03:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:10 smithi008 ceph-mon[146084]: pgmap v229: 65 pgs: 65 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:03:10.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:10 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:03:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:10 smithi008 ceph-mon[146084]: osdmap e69: 6 total, 5 up, 6 in 2024-04-12T06:03:11.791 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:11 smithi194 ceph-mon[106982]: osdmap e70: 6 total, 5 up, 6 in 2024-04-12T06:03:11.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:11 smithi008 ceph-mon[146084]: osdmap e70: 6 total, 5 up, 6 in 2024-04-12T06:03:12.070 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:12 smithi194 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.4.service: Succeeded. 2024-04-12T06:03:12.070 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:12 smithi194 systemd[1]: Stopped Ceph osd.4 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:03:12.490 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:12 smithi194 systemd[1]: Starting Ceph osd.4 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:03:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:12 smithi194 ceph-mon[106982]: pgmap v232: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:03:12.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:12 smithi194 ceph-mon[106982]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T06:03:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:12 smithi008 ceph-mon[146084]: pgmap v232: 65 pgs: 8 peering, 4 stale+active+clean, 2 active+undersized, 51 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-12T06:03:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:12 smithi008 ceph-mon[146084]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-12T06:03:13.542 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-12T06:03:13.543 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:13 smithi194 bash[124971]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-12T06:03:13.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:13 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 44/333 objects degraded (13.213%), 12 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:13.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:13 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 44/333 objects degraded (13.213%), 12 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:14.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-mon[106982]: pgmap v233: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/333 objects degraded (13.213%) 2024-04-12T06:03:14.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:14.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:14.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:14.822 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-osd[125292]: -- 172.21.15.194:0/2428803792 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b29cf6b1e0 con 0x55b29cf3b000 2024-04-12T06:03:14.822 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:14 smithi194 bash[125255]: 8f6f5d666aebabc4eb9c3b78f8fd839b13bb3c59a2462cd75c88f76f56b15aeb 2024-04-12T06:03:14.823 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:14 smithi194 systemd[1]: Started Ceph osd.4 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:03:14.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:14 smithi008 ceph-mon[146084]: pgmap v233: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 44/333 objects degraded (13.213%) 2024-04-12T06:03:14.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:14.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:14.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:15.239 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:14 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:03:14.891+0000 7f2253e657c0 -1 Falling back to public interface 2024-04-12T06:03:15.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:15 smithi008 ceph-mon[146084]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T06:03:15.953 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:15 smithi194 ceph-mon[106982]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-12T06:03:16.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:16 smithi194 ceph-mon[106982]: pgmap v234: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:16.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:16 smithi008 ceph-mon[146084]: pgmap v234: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: pgmap v235: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.653 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: pgmap v235: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:18.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:19.525 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:19 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:03:19.450+0000 7f2253e657c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-12T06:03:20.320 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:19 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:03:19.967+0000 7f2253e657c0 -1 osd.4 68 log_to_monitors true 2024-04-12T06:03:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:20 smithi194 ceph-mon[106982]: pgmap v236: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:20 smithi194 ceph-mon[106982]: from='osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T06:03:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:20 smithi194 ceph-mon[106982]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T06:03:20.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:20 smithi008 ceph-mon[146084]: pgmap v236: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:20.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:20 smithi008 ceph-mon[146084]: from='osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T06:03:20.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:20 smithi008 ceph-mon[146084]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-12T06:03:21.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-mon[106982]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T06:03:21.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-mon[106982]: osdmap e71: 6 total, 5 up, 6 in 2024-04-12T06:03:21.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-mon[106982]: from='osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:21.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-mon[106982]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:21.548 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-mon[106982]: from='osd.4 ' entity='osd.4' 2024-04-12T06:03:21.549 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:03:21 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:03:21.307+0000 7f22412b7700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:03:21.782 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:21 smithi008 ceph-mon[146084]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-12T06:03:21.782 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:21 smithi008 ceph-mon[146084]: osdmap e71: 6 total, 5 up, 6 in 2024-04-12T06:03:21.782 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:21 smithi008 ceph-mon[146084]: from='osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:21.782 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:21 smithi008 ceph-mon[146084]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:21.782 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:21 smithi008 ceph-mon[146084]: from='osd.4 ' entity='osd.4' 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: pgmap v238: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604] boot 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: osdmap e72: 6 total, 6 up, 6 in 2024-04-12T06:03:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T06:03:22.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: pgmap v238: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 55/333 objects degraded (16.517%) 2024-04-12T06:03:22.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:22.982 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: osd.4 [v2:172.21.15.194:6808/3298239604,v1:172.21.15.194:6809/3298239604] boot 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: osdmap e72: 6 total, 6 up, 6 in 2024-04-12T06:03:22.983 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-12T06:03:23.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:23 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 55/333 objects degraded (16.517%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:23.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:23 smithi008 ceph-mon[146084]: osdmap e73: 6 total, 6 up, 6 in 2024-04-12T06:03:24.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:23 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 55/333 objects degraded (16.517%), 16 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:24.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:23 smithi194 ceph-mon[106982]: osdmap e73: 6 total, 6 up, 6 in 2024-04-12T06:03:24.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:24 smithi008 ceph-mon[146084]: pgmap v241: 65 pgs: 6 peering, 12 active+undersized+degraded, 12 active+undersized, 35 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 41/333 objects degraded (12.312%) 2024-04-12T06:03:25.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:24 smithi194 ceph-mon[106982]: pgmap v241: 65 pgs: 6 peering, 12 active+undersized+degraded, 12 active+undersized, 35 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 41/333 objects degraded (12.312%) 2024-04-12T06:03:26.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:26 smithi008 ceph-mon[146084]: pgmap v242: 65 pgs: 6 peering, 8 active+undersized+degraded, 9 active+undersized, 42 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 30/333 objects degraded (9.009%) 2024-04-12T06:03:27.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:26 smithi194 ceph-mon[106982]: pgmap v242: 65 pgs: 6 peering, 8 active+undersized+degraded, 9 active+undersized, 42 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 30/333 objects degraded (9.009%) 2024-04-12T06:03:28.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:27 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/333 objects degraded (9.009%), 8 pgs degraded) 2024-04-12T06:03:28.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:27 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:03:28.036 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:27 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 30/333 objects degraded (9.009%), 8 pgs degraded) 2024-04-12T06:03:28.037 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:27 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:03:29.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:28 smithi194 ceph-mon[106982]: pgmap v243: 65 pgs: 6 peering, 59 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-12T06:03:29.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:28 smithi008 ceph-mon[146084]: pgmap v243: 65 pgs: 6 peering, 59 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-04-12T06:03:31.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:30 smithi194 ceph-mon[106982]: pgmap v244: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-04-12T06:03:31.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:30 smithi008 ceph-mon[146084]: pgmap v244: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 2 op/s 2024-04-12T06:03:33.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:32 smithi194 ceph-mon[106982]: pgmap v245: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-12T06:03:33.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:33.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:32 smithi008 ceph-mon[146084]: pgmap v245: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-04-12T06:03:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:33.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:34.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:34 smithi008 ceph-mon[146084]: pgmap v246: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-04-12T06:03:34.570 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:34 smithi194 ceph-mon[106982]: pgmap v246: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 2 op/s 2024-04-12T06:03:36.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:36 smithi194 ceph-mon[106982]: pgmap v247: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 -- 172.21.15.8:0/2556290384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 msgr2=0x7f14c01070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 --2- 172.21.15.8:0/2556290384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c01070b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009910 tx=0x7f14bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 -- 172.21.15.8:0/2556290384 shutdown_connections 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 --2- 172.21.15.8:0/2556290384 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c00689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 --2- 172.21.15.8:0/2556290384 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c01070b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:36.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 -- 172.21.15.8:0/2556290384 >> 172.21.15.8:0/2556290384 conn(0x7f14c0075670 msgr2=0x7f14c0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:36.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 -- 172.21.15.8:0/2556290384 shutdown_connections 2024-04-12T06:03:36.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.889+0000 7f14c7a19700 1 -- 172.21.15.8:0/2556290384 wait complete. 2024-04-12T06:03:36.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 Processor -- start 2024-04-12T06:03:36.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 -- start start 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c019d130 con 0x7f14c0106cd0 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c7a19700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0196630 con 0x7f14c0068580 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c57b5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c4fb4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c57b5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:39824/0 (socket says 172.21.15.8:39824) 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c57b5700 1 -- 172.21.15.8:0/3538861284 learned_addr learned my addr 172.21.15.8:0/3538861284 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.890+0000 7f14c4fb4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38702/0 (socket says 172.21.15.8:38702) 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c57b5700 1 -- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 msgr2=0x7f14c019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c57b5700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:36.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c57b5700 1 -- 172.21.15.8:0/3538861284 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0095f0 con 0x7f14c0068580 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c4fb4700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c57b5700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000c00 tx=0x7f14bc00bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14bc01d070 con 0x7f14c0068580 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c01968b0 con 0x7f14c0068580 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f14bc005710 con 0x7f14c0068580 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14bc0039d0 con 0x7f14c0068580 2024-04-12T06:03:36.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.891+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c0196d40 con 0x7f14c0068580 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.892+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c01091f0 con 0x7f14c0068580 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.893+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f14bc003c50 con 0x7f14c0068580 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.893+0000 7f14b67fc700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 0x7f14ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.893+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f14bc08db00 con 0x7f14c0068580 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.893+0000 7f14c4fb4700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 0x7f14ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:36.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.894+0000 7f14c4fb4700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 0x7f14ac06f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f14c00fe390 tx=0x7f14b0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:36.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:36.898+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f14bc05b610 con 0x7f14c0068580 2024-04-12T06:03:36.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:36 smithi008 ceph-mon[146084]: pgmap v247: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-04-12T06:03:37.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.122+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c0196fc0 con 0x7f14ac06cce0 2024-04-12T06:03:37.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.123+0000 7f14b67fc700 1 -- 172.21.15.8:0/3538861284 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f14c0196fc0 con 0x7f14ac06cce0 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 msgr2=0x7f14ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 0x7f14ac06f1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f14c00fe390 tx=0x7f14b0009260 comp rx=0 tx=0).stop 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 msgr2=0x7f14c019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14bc000c00 tx=0x7f14bc00bde0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 shutdown_connections 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f14ac06cce0 0x7f14ac06f1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f14c0068580 0x7f14c019c560 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 --2- 172.21.15.8:0/3538861284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f14c0106cd0 0x7f14c019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 >> 172.21.15.8:0/3538861284 conn(0x7f14c0075670 msgr2=0x7f14c00fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 shutdown_connections 2024-04-12T06:03:37.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.125+0000 7f14c7a19700 1 -- 172.21.15.8:0/3538861284 wait complete. 2024-04-12T06:03:37.131 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 -- 172.21.15.8:0/1195650086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90200ffc20 msgr2=0x7f9020100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 --2- 172.21.15.8:0/1195650086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90200ffc20 0x7f9020100080 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9010009910 tx=0x7f9010009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 -- 172.21.15.8:0/1195650086 shutdown_connections 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 --2- 172.21.15.8:0/1195650086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90200ffc20 0x7f9020100080 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 --2- 172.21.15.8:0/1195650086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9020105bd0 0x7f9020105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 -- 172.21.15.8:0/1195650086 >> 172.21.15.8:0/1195650086 conn(0x7f9020075670 msgr2=0x7f9020075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.232+0000 7f9026ab9700 1 -- 172.21.15.8:0/1195650086 shutdown_connections 2024-04-12T06:03:37.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 -- 172.21.15.8:0/1195650086 wait complete. 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 Processor -- start 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 -- start start 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90200ffc20 0x7f9020101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020102350 con 0x7f9020105bd0 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9026ab9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90201024c0 con 0x7f90200ffc20 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9017fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9017fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38714/0 (socket says 172.21.15.8:38714) 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.233+0000 7f9017fff700 1 -- 172.21.15.8:0/2500332554 learned_addr learned my addr 172.21.15.8:0/2500332554 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9024855700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90200ffc20 0x7f9020101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9017fff700 1 -- 172.21.15.8:0/2500332554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90200ffc20 msgr2=0x7f9020101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9017fff700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90200ffc20 0x7f9020101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9017fff700 1 -- 172.21.15.8:0/2500332554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100095f0 con 0x7f9020105bd0 2024-04-12T06:03:37.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9017fff700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f901000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901001d070 con 0x7f9020105bd0 2024-04-12T06:03:37.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90201a29a0 con 0x7f9020105bd0 2024-04-12T06:03:37.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9010005dc0 con 0x7f9020105bd0 2024-04-12T06:03:37.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9010003940 con 0x7f9020105bd0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.234+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90201a2d00 con 0x7f9020105bd0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.235+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902004f2e0 con 0x7f9020105bd0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.236+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9010004ae0 con 0x7f9020105bd0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.236+0000 7f9015ffb700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 0x7f900806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.236+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f901008db50 con 0x7f9020105bd0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.236+0000 7f9024855700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 0x7f900806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.237+0000 7f9024855700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 0x7f900806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f901c009c60 tx=0x7f901c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.241+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9010057bd0 con 0x7f9020105bd0 2024-04-12T06:03:37.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.461+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9020104990 con 0x7f900806cce0 2024-04-12T06:03:37.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.462+0000 7f9015ffb700 1 -- 172.21.15.8:0/2500332554 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9020104990 con 0x7f900806cce0 2024-04-12T06:03:37.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.464+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 msgr2=0x7f900806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.464+0000 7f9026ab9700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 0x7f900806f1a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f901c009c60 tx=0x7f901c009260 comp rx=0 tx=0).stop 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.464+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 msgr2=0x7f9020101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f901000bde0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 shutdown_connections 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f900806cce0 0x7f900806f1a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90200ffc20 0x7f9020101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 --2- 172.21.15.8:0/2500332554 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9020105bd0 0x7f9020101d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 >> 172.21.15.8:0/2500332554 conn(0x7f9020075670 msgr2=0x7f9020068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 shutdown_connections 2024-04-12T06:03:37.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.465+0000 7f9026ab9700 1 -- 172.21.15.8:0/2500332554 wait complete. 2024-04-12T06:03:37.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:37 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:37.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:37 smithi194 ceph-mon[106982]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:37.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:37 smithi194 ceph-mon[106982]: Upgrade: osd.5 is safe to restart 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 -- 172.21.15.8:0/2631297521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90900ffc20 msgr2=0x7f9090100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 --2- 172.21.15.8:0/2631297521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90900ffc20 0x7f9090100080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9084009910 tx=0x7f9084009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 -- 172.21.15.8:0/2631297521 shutdown_connections 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 --2- 172.21.15.8:0/2631297521 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f90900ffc20 0x7f9090100080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 --2- 172.21.15.8:0/2631297521 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9090105bd0 0x7f9090105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 -- 172.21.15.8:0/2631297521 >> 172.21.15.8:0/2631297521 conn(0x7f9090075670 msgr2=0x7f9090075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 -- 172.21.15.8:0/2631297521 shutdown_connections 2024-04-12T06:03:37.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.576+0000 7f90987e5700 1 -- 172.21.15.8:0/2631297521 wait complete. 2024-04-12T06:03:37.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 Processor -- start 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 -- start start 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90900ffc20 0x7f90901976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090198200 con 0x7f9090105bd0 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f90987e5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090198370 con 0x7f90900ffc20 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f9095d80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f9095d80700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38738/0 (socket says 172.21.15.8:38738) 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f9095d80700 1 -- 172.21.15.8:0/1018675230 learned_addr learned my addr 172.21.15.8:0/1018675230 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.577+0000 7f9096581700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90900ffc20 0x7f90901976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f9095d80700 1 -- 172.21.15.8:0/1018675230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90900ffc20 msgr2=0x7f90901976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f9095d80700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90900ffc20 0x7f90901976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f9095d80700 1 -- 172.21.15.8:0/1018675230 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90840095f0 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f9095d80700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9084000c00 tx=0x7f908400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f908401d070 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901a0710 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9084005dc0 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9084003940 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.578+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901a0b20 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.579+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90901080d0 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.580+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9084004ae0 con 0x7f9090105bd0 2024-04-12T06:03:37.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.580+0000 7f90837fe700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 0x7f907c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.580+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f908408dab0 con 0x7f9090105bd0 2024-04-12T06:03:37.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.580+0000 7f9096581700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 0x7f907c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.580+0000 7f9096581700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 0x7f907c06f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f908c009c60 tx=0x7f908c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.585+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9084057bd0 con 0x7f9090105bd0 2024-04-12T06:03:37.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.797+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9090068ca0 con 0x7f907c06cce0 2024-04-12T06:03:37.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.808+0000 7f90837fe700 1 -- 172.21.15.8:0/1018675230 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9090068ca0 con 0x7f907c06cce0 2024-04-12T06:03:37.814 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:03:37.814 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (5m) 86s ago 16m 27.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:03:37.814 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (17m) 86s ago 17m 8262k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (15m) 20s ago 15m 8312k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (3m) 86s ago 17m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (3m) 20s ago 15m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (5m) 86s ago 16m 80.8M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (10m) 86s ago 10m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (10m) 86s ago 10m 71.2M - 18.2.1 5be31c24972a 43992e61c6f4 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (10m) 20s ago 10m 55.5M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (10m) 20s ago 10m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (7m) 86s ago 18m 510M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (6m) 20s ago 15m 451M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (3m) 86s ago 18m 47.3M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (3m) 20s ago 15m 38.5M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (6m) 86s ago 17m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:03:37.815 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (6m) 20s ago 15m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (2m) 86s ago 14m 95.6M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (2m) 86s ago 14m 103M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (92s) 86s ago 13m 11.2M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (56s) 20s ago 13m 103M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (23s) 20s ago 12m 11.4M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (12m) 20s ago 12m 211M 2251M 18.2.1 5be31c24972a 57ed99cbb9e7 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (5m) 86s ago 16m 75.0M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 msgr2=0x7f907c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 0x7f907c06f1a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f908c009c60 tx=0x7f908c009260 comp rx=0 tx=0).stop 2024-04-12T06:03:37.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 msgr2=0x7f9090197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9084000c00 tx=0x7f908400bde0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 shutdown_connections 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f907c06cce0 0x7f907c06f1a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90900ffc20 0x7f90901976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 --2- 172.21.15.8:0/1018675230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9090105bd0 0x7f9090197be0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 >> 172.21.15.8:0/1018675230 conn(0x7f9090075670 msgr2=0x7f90900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 shutdown_connections 2024-04-12T06:03:37.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.810+0000 7f90987e5700 1 -- 172.21.15.8:0/1018675230 wait complete. 2024-04-12T06:03:37.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:37 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:37.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:37 smithi008 ceph-mon[146084]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-12T06:03:37.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:37 smithi008 ceph-mon[146084]: Upgrade: osd.5 is safe to restart 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 -- 172.21.15.8:0/1435479457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 msgr2=0x7f4c341070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 --2- 172.21.15.8:0/1435479457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c341070b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009910 tx=0x7f4c24009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 -- 172.21.15.8:0/1435479457 shutdown_connections 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 --2- 172.21.15.8:0/1435479457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 0x7f4c340689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 --2- 172.21.15.8:0/1435479457 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c341070b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.915+0000 7f4c39431700 1 -- 172.21.15.8:0/1435479457 >> 172.21.15.8:0/1435479457 conn(0x7f4c34075670 msgr2=0x7f4c34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.916+0000 7f4c39431700 1 -- 172.21.15.8:0/1435479457 shutdown_connections 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.916+0000 7f4c39431700 1 -- 172.21.15.8:0/1435479457 wait complete. 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.916+0000 7f4c39431700 1 Processor -- start 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.916+0000 7f4c39431700 1 -- start start 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 0x7f4c3419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3419d050 con 0x7f4c34106cd0 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34196630 con 0x7f4c34068580 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38770/0 (socket says 172.21.15.8:38770) 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 -- 172.21.15.8:0/2216980159 learned_addr learned my addr 172.21.15.8:0/2216980159 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c32ffd700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 0x7f4c3419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 -- 172.21.15.8:0/2216980159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 msgr2=0x7f4c3419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:37.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 0x7f4c3419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 -- 172.21.15.8:0/2216980159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c240095f0 con 0x7f4c34106cd0 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c327fc700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e920 tx=0x7f4c2800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2800c9d0 con 0x7f4c34106cd0 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c28010430 con 0x7f4c34106cd0 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34196910 con 0x7f4c34106cd0 2024-04-12T06:03:37.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c28018650 con 0x7f4c34106cd0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.917+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34196e60 con 0x7f4c34106cd0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.918+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c341091d0 con 0x7f4c34106cd0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.919+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4c280105a0 con 0x7f4c34106cd0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.919+0000 7f4c1bfff700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 0x7f4c1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.919+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4c28014070 con 0x7f4c34106cd0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.920+0000 7f4c32ffd700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 0x7f4c1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:37.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.920+0000 7f4c32ffd700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 0x7f4c1c06f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b6e0 tx=0x7f4c24005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:37.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:37.924+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4c28052ab0 con 0x7f4c34106cd0 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.212+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c3404f2e0 con 0x7f4c34106cd0 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.213+0000 7f4c1bfff700 1 -- 172.21.15.8:0/2216980159 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f4c2805a360 con 0x7f4c34106cd0 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:03:38.209 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 5 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-04-12T06:03:38.210 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 9 2024-04-12T06:03:38.211 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:03:38.211 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 msgr2=0x7f4c1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 0x7f4c1c06f1a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b6e0 tx=0x7f4c24005a90 comp rx=0 tx=0).stop 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 msgr2=0x7f4c3419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800e920 tx=0x7f4c2800ec30 comp rx=0 tx=0).stop 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 shutdown_connections 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c1c06cce0 0x7f4c1c06f1a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c34068580 0x7f4c3419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 --2- 172.21.15.8:0/2216980159 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c34106cd0 0x7f4c3419ca30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.215+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 >> 172.21.15.8:0/2216980159 conn(0x7f4c34075670 msgr2=0x7f4c340fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.216+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 shutdown_connections 2024-04-12T06:03:38.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.216+0000 7f4c39431700 1 -- 172.21.15.8:0/2216980159 wait complete. 2024-04-12T06:03:38.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 -- 172.21.15.8:0/1158741030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d40ffc20 msgr2=0x7f46d4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 --2- 172.21.15.8:0/1158741030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d40ffc20 0x7f46d4100080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009910 tx=0x7f46bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 -- 172.21.15.8:0/1158741030 shutdown_connections 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 --2- 172.21.15.8:0/1158741030 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d40ffc20 0x7f46d4100080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 --2- 172.21.15.8:0/1158741030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d4105bd0 0x7f46d4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 -- 172.21.15.8:0/1158741030 >> 172.21.15.8:0/1158741030 conn(0x7f46d4075670 msgr2=0x7f46d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 -- 172.21.15.8:0/1158741030 shutdown_connections 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.323+0000 7f46da2cc700 1 -- 172.21.15.8:0/1158741030 wait complete. 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.324+0000 7f46da2cc700 1 Processor -- start 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.324+0000 7f46da2cc700 1 -- start start 2024-04-12T06:03:38.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.324+0000 7f46da2cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d40ffc20 0x7f46d41976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.324+0000 7f46da2cc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.324+0000 7f46da2cc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4198220 con 0x7f46d4105bd0 2024-04-12T06:03:38.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46da2cc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d4198360 con 0x7f46d40ffc20 2024-04-12T06:03:38.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38778/0 (socket says 172.21.15.8:38778) 2024-04-12T06:03:38.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 -- 172.21.15.8:0/52650737 learned_addr learned my addr 172.21.15.8:0/52650737 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:38.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46d3fff700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d40ffc20 0x7f46d41976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 -- 172.21.15.8:0/52650737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d40ffc20 msgr2=0x7f46d41976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d40ffc20 0x7f46d41976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 -- 172.21.15.8:0/52650737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc0095f0 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46cbfff700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc01d070 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d41a0730 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f46bc005710 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc00fe20 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.325+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d41a0b40 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.326+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d41080d0 con 0x7f46d4105bd0 2024-04-12T06:03:38.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.327+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f46bc003a60 con 0x7f46d4105bd0 2024-04-12T06:03:38.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.327+0000 7f46d1ffb700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 0x7f46c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.327+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f46bc08d5f0 con 0x7f46d4105bd0 2024-04-12T06:03:38.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.327+0000 7f46d3fff700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 0x7f46c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.328+0000 7f46d3fff700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 0x7f46c006f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005950 tx=0x7f46c400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.332+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f46bc057710 con 0x7f46d4105bd0 2024-04-12T06:03:38.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.579+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46d404f2e0 con 0x7f46d4105bd0 2024-04-12T06:03:38.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.580+0000 7f46d1ffb700 1 -- 172.21.15.8:0/52650737 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1866 (secure 0 0 0) 0x7f46bc05ad30 con 0x7f46d4105bd0 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:e25 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 25 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:03:38.576 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:00:14.502437+0000 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 46 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:03:38.577 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=14476,1=24277} 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{0:14476} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.8:6828/2135087803,v1:172.21.15.8:6829/2135087803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:14488} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/534481461,v1:172.21.15.8:6827/534481461] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{1:24277} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.194:6824/1464587758,v1:172.21.15.194:6825/1464587758] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:38.578 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.194:6826/3772682938,v1:172.21.15.194:6827/3772682938] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:03:38.579 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:38.579 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:03:38.579 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 msgr2=0x7f46c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 0x7f46c006f1a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f46c4005950 tx=0x7f46c400b410 comp rx=0 tx=0).stop 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 msgr2=0x7f46d4197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc00bd70 comp rx=0 tx=0).stop 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 shutdown_connections 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f46c006cce0 0x7f46c006f1a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f46d40ffc20 0x7f46d41976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 --2- 172.21.15.8:0/52650737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f46d4105bd0 0x7f46d4197c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 >> 172.21.15.8:0/52650737 conn(0x7f46d4075670 msgr2=0x7f46d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 shutdown_connections 2024-04-12T06:03:38.580 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.582+0000 7f46da2cc700 1 -- 172.21.15.8:0/52650737 wait complete. 2024-04-12T06:03:38.581 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:03:38.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 -- 172.21.15.8:0/2934168622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 msgr2=0x7fe668105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 --2- 172.21.15.8:0/2934168622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668105fb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe658009910 tx=0x7fe658009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 -- 172.21.15.8:0/2934168622 shutdown_connections 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 --2- 172.21.15.8:0/2934168622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 0x7fe668100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 --2- 172.21.15.8:0/2934168622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668105fb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 -- 172.21.15.8:0/2934168622 >> 172.21.15.8:0/2934168622 conn(0x7fe668075670 msgr2=0x7fe668075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 -- 172.21.15.8:0/2934168622 shutdown_connections 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.690+0000 7fe66e350700 1 -- 172.21.15.8:0/2934168622 wait complete. 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 Processor -- start 2024-04-12T06:03:38.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 -- start start 2024-04-12T06:03:38.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 0x7fe668195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe668196260 con 0x7fe668105bd0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe66e350700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe668198f90 con 0x7fe6680ffc20 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.691+0000 7fe6677fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38794/0 (socket says 172.21.15.8:38794) 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 -- 172.21.15.8:0/1576361870 learned_addr learned my addr 172.21.15.8:0/1576361870 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe667fff700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 0x7fe668195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 -- 172.21.15.8:0/1576361870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 msgr2=0x7fe668195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 0x7fe668195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 -- 172.21.15.8:0/1576361870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6580095f0 con 0x7fe668105bd0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6677fe700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe650009970 tx=0x7fe650009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe650012070 con 0x7fe668105bd0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe650007ad0 con 0x7fe668105bd0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe668199210 con 0x7fe668105bd0 2024-04-12T06:03:38.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe650017bb0 con 0x7fe668105bd0 2024-04-12T06:03:38.690 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.692+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe668199760 con 0x7fe668105bd0 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.693+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66804f2e0 con 0x7fe668105bd0 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.694+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe650007c40 con 0x7fe668105bd0 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.694+0000 7fe6657fa700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 0x7fe65406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.694+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe65008d8f0 con 0x7fe668105bd0 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.694+0000 7fe667fff700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 0x7fe65406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:38.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.695+0000 7fe667fff700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 0x7fe65406f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe658006010 tx=0x7fe65801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:38.694 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.699+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe650057b40 con 0x7fe668105bd0 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='client.44259 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: pgmap v248: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.732 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2216980159' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:38.733 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:38.733 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T06:03:38.733 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='client.44259 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: pgmap v248: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 2 op/s 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2216980159' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-12T06:03:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:38.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.918+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe668199aa0 con 0x7fe65406cce0 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.919+0000 7fe6657fa700 1 -- 172.21.15.8:0/1576361870 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe668199aa0 con 0x7fe65406cce0 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:03:38.915 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "11/23 daemons upgraded", 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading osd daemons", 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:03:38.916 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:03:38.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 msgr2=0x7fe65406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 0x7fe65406f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe658006010 tx=0x7fe65801a040 comp rx=0 tx=0).stop 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 msgr2=0x7fe668195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe650009970 tx=0x7fe650009c80 comp rx=0 tx=0).stop 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 shutdown_connections 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe65406cce0 0x7fe65406f1a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6680ffc20 0x7fe668195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 --2- 172.21.15.8:0/1576361870 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe668105bd0 0x7fe668195bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 >> 172.21.15.8:0/1576361870 conn(0x7fe668075670 msgr2=0x7fe6680fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 shutdown_connections 2024-04-12T06:03:38.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:38.922+0000 7fe66e350700 1 -- 172.21.15.8:0/1576361870 wait complete. 2024-04-12T06:03:39.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/3497000962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e78105bd0 msgr2=0x7f6e78105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/3497000962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e78105bd0 0x7f6e78105fb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e68009c20 comp rx=0 tx=0).stop 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/3497000962 shutdown_connections 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/3497000962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e780ffc20 0x7f6e78100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/3497000962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e78105bd0 0x7f6e78105fb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.028+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/3497000962 >> 172.21.15.8:0/3497000962 conn(0x7f6e78075670 msgr2=0x7f6e78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.029+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/3497000962 shutdown_connections 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.029+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/3497000962 wait complete. 2024-04-12T06:03:39.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.029+0000 7f6e7dfbf700 1 Processor -- start 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.029+0000 7f6e7dfbf700 1 -- start start 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.029+0000 7f6e7dfbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e7dfbf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e78105bd0 0x7f6e78198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e7dfbf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78199560 con 0x7f6e780ffc20 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e7dfbf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7819d370 con 0x7f6e78105bd0 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38812/0 (socket says 172.21.15.8:38812) 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 -- 172.21.15.8:0/2307941947 learned_addr learned my addr 172.21.15.8:0/2307941947 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e76ffd700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e78105bd0 0x7f6e78198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 -- 172.21.15.8:0/2307941947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e78105bd0 msgr2=0x7f6e78198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e78105bd0 0x7f6e78198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 -- 172.21.15.8:0/2307941947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680095f0 con 0x7f6e780ffc20 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e777fe700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c00 tx=0x7f6e6800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6801d070 con 0x7f6e780ffc20 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7819d5f0 con 0x7f6e780ffc20 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6e68005710 con 0x7f6e780ffc20 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e680039d0 con 0x7f6e780ffc20 2024-04-12T06:03:39.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.030+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7819da80 con 0x7f6e780ffc20 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.031+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7804f2e0 con 0x7f6e780ffc20 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.032+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f6e68003c50 con 0x7f6e780ffc20 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.032+0000 7f6e74ff9700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.032+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6e6808db20 con 0x7f6e780ffc20 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.032+0000 7f6e76ffd700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:03:39.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.033+0000 7f6e76ffd700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 0x7f6e6006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e78199fb0 tx=0x7f6e6c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:03:39.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.037+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6e6805b630 con 0x7f6e780ffc20 2024-04-12T06:03:39.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.322+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6e7819dd50 con 0x7f6e780ffc20 2024-04-12T06:03:39.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.323+0000 7f6e74ff9700 1 -- 172.21.15.8:0/2307941947 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6e6805b1c0 con 0x7f6e780ffc20 2024-04-12T06:03:39.319 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 msgr2=0x7f6e6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 0x7f6e6006f1a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6e78199fb0 tx=0x7f6e6c009260 comp rx=0 tx=0).stop 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 msgr2=0x7f6e78198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c00 tx=0x7f6e6800bde0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 shutdown_connections 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6e6006cce0 0x7f6e6006f1a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6e780ffc20 0x7f6e78198990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 --2- 172.21.15.8:0/2307941947 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6e78105bd0 0x7f6e78198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:03:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 >> 172.21.15.8:0/2307941947 conn(0x7f6e78075670 msgr2=0x7f6e780686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:03:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 shutdown_connections 2024-04-12T06:03:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:03:39.325+0000 7f6e7dfbf700 1 -- 172.21.15.8:0/2307941947 wait complete. 2024-04-12T06:03:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:39 smithi194 ceph-mon[106982]: Upgrade: Updating osd.5 2024-04-12T06:03:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:39 smithi194 ceph-mon[106982]: Deploying daemon osd.5 on smithi194 2024-04-12T06:03:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/52650737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:03:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:39 smithi194 ceph-mon[106982]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:39.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2307941947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:03:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:39 smithi008 ceph-mon[146084]: Upgrade: Updating osd.5 2024-04-12T06:03:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:39 smithi008 ceph-mon[146084]: Deploying daemon osd.5 on smithi194 2024-04-12T06:03:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/52650737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:03:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:39 smithi008 ceph-mon[146084]: from='client.34302 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:03:39.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2307941947' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:03:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:40 smithi194 ceph-mon[106982]: pgmap v249: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:03:40.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:40 smithi008 ceph-mon[146084]: pgmap v249: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:03:41.820 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:41 smithi194 systemd[1]: Stopping Ceph osd.5 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:03:41.820 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:41 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[72935]: 2024-04-12T06:03:41.701+0000 7f099e177700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:03:41.820 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:41 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[72935]: 2024-04-12T06:03:41.701+0000 7f099e177700 -1 osd.5 73 *** Got signal Terminated *** 2024-04-12T06:03:41.821 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:41 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[72935]: 2024-04-12T06:03:41.701+0000 7f099e177700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:03:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:42 smithi194 ceph-mon[106982]: pgmap v250: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:03:42.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:42 smithi194 ceph-mon[106982]: osd.5 marked itself down and dead 2024-04-12T06:03:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:42 smithi008 ceph-mon[146084]: pgmap v250: 65 pgs: 65 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:03:42.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:42 smithi008 ceph-mon[146084]: osd.5 marked itself down and dead 2024-04-12T06:03:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:43 smithi194 ceph-mon[106982]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:03:43.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:43 smithi194 ceph-mon[106982]: osdmap e74: 6 total, 5 up, 6 in 2024-04-12T06:03:43.821 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:43 smithi194 bash[130869]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5 2024-04-12T06:03:43.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:43 smithi008 ceph-mon[146084]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-12T06:03:43.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:43 smithi008 ceph-mon[146084]: osdmap e74: 6 total, 5 up, 6 in 2024-04-12T06:03:44.759 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:44 smithi194 ceph-mon[106982]: pgmap v252: 65 pgs: 13 stale+active+clean, 52 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-12T06:03:44.759 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:44 smithi194 ceph-mon[106982]: osdmap e75: 6 total, 5 up, 6 in 2024-04-12T06:03:44.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:44 smithi008 ceph-mon[146084]: pgmap v252: 65 pgs: 13 stale+active+clean, 52 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-12T06:03:44.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:44 smithi008 ceph-mon[146084]: osdmap e75: 6 total, 5 up, 6 in 2024-04-12T06:03:45.798 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:45 smithi194 ceph-mon[106982]: Health check failed: Degraded data redundancy: 7/333 objects degraded (2.102%), 2 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:45.798 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:45 smithi194 systemd[1]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.5.service: Succeeded. 2024-04-12T06:03:45.799 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:45 smithi194 systemd[1]: Stopped Ceph osd.5 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:03:45.799 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:45 smithi194 systemd[1]: Starting Ceph osd.5 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:03:45.903 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:45 smithi008 ceph-mon[146084]: Health check failed: Degraded data redundancy: 7/333 objects degraded (2.102%), 2 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:46.729 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:46 smithi194 ceph-mon[106982]: pgmap v254: 65 pgs: 2 active+undersized+degraded, 12 stale+active+clean, 1 active+undersized, 50 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 7/333 objects degraded (2.102%) 2024-04-12T06:03:46.729 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-12T06:03:46.730 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:46 smithi194 bash[131149]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-12T06:03:46.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:46 smithi008 ceph-mon[146084]: pgmap v254: 65 pgs: 2 active+undersized+degraded, 12 stale+active+clean, 1 active+undersized, 50 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 7/333 objects degraded (2.102%) 2024-04-12T06:03:47.820 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:47 smithi194 ceph-osd[131446]: -- 172.21.15.194:0/1242770059 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5651d1a931e0 con 0x5651d1a63000 2024-04-12T06:03:47.820 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:47 smithi194 bash[131408]: 860f425d081e207b6bb848c83fc94dd364570cb3844e0c82f8917311d70f4296 2024-04-12T06:03:47.821 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:47 smithi194 systemd[1]: Started Ceph osd.5 for b9cc93fe-f88f-11ee-bc8c-c7b262605968. 2024-04-12T06:03:48.320 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:03:48.028+0000 7fef466917c0 -1 Falling back to public interface 2024-04-12T06:03:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: pgmap v255: 65 pgs: 14 active+undersized+degraded, 2 stale+active+clean, 11 active+undersized, 38 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: pgmap v255: 65 pgs: 14 active+undersized+degraded, 2 stale+active+clean, 11 active+undersized, 38 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:50.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:50 smithi194 ceph-mon[106982]: pgmap v256: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:50.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:50 smithi008 ceph-mon[146084]: pgmap v256: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:51.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:51 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:51.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:51 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:51.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:51 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:51.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:51 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:52.026 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:51 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:52.026 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:51 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:52.027 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:51 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:52.027 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:51 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:52.320 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:52 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:03:52.270+0000 7fef466917c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-12T06:03:52.713 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:52 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:03:52.663+0000 7fef466917c0 -1 osd.5 73 log_to_monitors true 2024-04-12T06:03:52.714 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:52 smithi194 ceph-mon[106982]: pgmap v257: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:52.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:52 smithi008 ceph-mon[146084]: pgmap v257: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:53.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:53 smithi008 ceph-mon[146084]: from='osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T06:03:53.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:53 smithi008 ceph-mon[146084]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T06:03:53.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:53 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:54.070 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:03:53 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:03:53.665+0000 7fef37aeb700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-12T06:03:54.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:53 smithi194 ceph-mon[106982]: from='osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T06:03:54.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:53 smithi194 ceph-mon[106982]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-12T06:03:54.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:53 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:54.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:54 smithi194 ceph-mon[106982]: pgmap v258: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:54.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:54 smithi194 ceph-mon[106982]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T06:03:54.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:54 smithi194 ceph-mon[106982]: osdmap e76: 6 total, 5 up, 6 in 2024-04-12T06:03:54.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:54 smithi194 ceph-mon[106982]: from='osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:54.762 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:54 smithi194 ceph-mon[106982]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:54 smithi008 ceph-mon[146084]: pgmap v258: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 273 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-04-12T06:03:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:54 smithi008 ceph-mon[146084]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-12T06:03:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:54 smithi008 ceph-mon[146084]: osdmap e76: 6 total, 5 up, 6 in 2024-04-12T06:03:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:54 smithi008 ceph-mon[146084]: from='osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:54.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:54 smithi008 ceph-mon[146084]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-04-12T06:03:56.038 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628] boot 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: osdmap e77: 6 total, 6 up, 6 in 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.039 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all osd 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-12T06:03:56.040 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-12T06:03:56.041 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-12T06:03:56.041 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.041 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-12T06:03:56.041 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:55 smithi194 ceph-mon[106982]: osdmap e78: 6 total, 6 up, 6 in 2024-04-12T06:03:56.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-12T06:03:56.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: osd.5 [v2:172.21.15.194:6816/2816217628,v1:172.21.15.194:6817/2816217628] boot 2024-04-12T06:03:56.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: osdmap e77: 6 total, 6 up, 6 in 2024-04-12T06:03:56.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all osd 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-12T06:03:56.047 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:55 smithi008 ceph-mon[146084]: osdmap e78: 6 total, 6 up, 6 in 2024-04-12T06:03:56.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:03:56.165+0000 7f39f1f15700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T06:03:56.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-12T06:03:56.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: pgmap v262: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 47/333 objects degraded (14.114%) 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-12T06:03:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: osdmap e79: 6 total, 6 up, 6 in 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.194:6826/1928513879,v1:172.21.15.194:6827/1928513879] up:boot 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/885712375,v1:172.21.15.8:6827/885712375] up:boot 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:03:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:56 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:03:57.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-12T06:03:57.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: pgmap v262: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 47/333 objects degraded (14.114%) 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.lmvepz=up:active,1=cephfs.smithi194.uzfpyu=up:active} 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:03:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: osdmap e79: 6 total, 6 up, 6 in 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.194:6826/1928513879,v1:172.21.15.194:6827/1928513879] up:boot 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/885712375,v1:172.21.15.8:6827/885712375] up:boot 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:03:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:56 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:03:58.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-12T06:03:58.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.194:6824/2084066297,v1:172.21.15.194:6825/2084066297] up:boot 2024-04-12T06:03:58.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2959889136,v1:172.21.15.8:6829/2959889136] up:boot 2024-04-12T06:03:58.023 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:03:58.024 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:03:58.024 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:03:58.024 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:58.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-12T06:03:58.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.194:6824/2084066297,v1:172.21.15.194:6825/2084066297] up:boot 2024-04-12T06:03:58.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2959889136,v1:172.21.15.8:6829/2959889136] up:boot 2024-04-12T06:03:58.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:03:58.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:03:58.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:03:58.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: pgmap v264: 65 pgs: 2 peering, 5 active+undersized+degraded, 9 active+undersized, 49 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 15/333 objects degraded (4.505%) 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: Upgrade: Updating mds.cephfs.smithi008.rhdhsr 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: Deploying daemon mds.cephfs.smithi008.rhdhsr on smithi008 2024-04-12T06:03:59.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:58 smithi008 ceph-mon[146084]: Health check update: Degraded data redundancy: 47/333 objects degraded (14.114%), 13 pgs degraded (PG_DEGRADED) 2024-04-12T06:03:59.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: pgmap v264: 65 pgs: 2 peering, 5 active+undersized+degraded, 9 active+undersized, 49 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 2 op/s; 15/333 objects degraded (4.505%) 2024-04-12T06:03:59.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: Upgrade: Updating mds.cephfs.smithi008.rhdhsr 2024-04-12T06:03:59.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:03:59.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.rhdhsr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:03:59.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:03:59.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: Deploying daemon mds.cephfs.smithi008.rhdhsr on smithi008 2024-04-12T06:03:59.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:58 smithi194 ceph-mon[106982]: Health check update: Degraded data redundancy: 47/333 objects degraded (14.114%), 13 pgs degraded (PG_DEGRADED) 2024-04-12T06:04:00.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:03:59 smithi008 ceph-mon[146084]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/333 objects degraded (4.505%), 5 pgs degraded) 2024-04-12T06:04:00.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:03:59 smithi194 ceph-mon[106982]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/333 objects degraded (4.505%), 5 pgs degraded) 2024-04-12T06:04:01.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:00 smithi008 ceph-mon[146084]: pgmap v265: 65 pgs: 2 peering, 63 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:01.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:00 smithi008 ceph-mon[146084]: osdmap e80: 6 total, 6 up, 6 in 2024-04-12T06:04:01.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:00 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:00 smithi194 ceph-mon[106982]: pgmap v265: 65 pgs: 2 peering, 63 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:00 smithi194 ceph-mon[106982]: osdmap e80: 6 total, 6 up, 6 in 2024-04-12T06:04:01.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:00 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:02.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:01 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:04:02.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:01 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:04:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:02 smithi008 ceph-mon[146084]: pgmap v267: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:03.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:02 smithi194 ceph-mon[106982]: pgmap v267: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:03.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:05.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:04 smithi008 ceph-mon[146084]: pgmap v268: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:05.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:04 smithi194 ceph-mon[106982]: pgmap v268: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: pgmap v269: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:boot 2024-04-12T06:04:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:06 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:04:07.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: pgmap v269: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:boot 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:07.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:06 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:04:08.770 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:08 smithi008 ceph-mon[146084]: pgmap v270: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:08.770 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:08 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:08.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:08 smithi194 ceph-mon[106982]: pgmap v270: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:08.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:08 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:09.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 -- 172.21.15.8:0/83104648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 msgr2=0x7f3224100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 --2- 172.21.15.8:0/83104648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f3224100080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3214009910 tx=0x7f3214009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:09.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 -- 172.21.15.8:0/83104648 shutdown_connections 2024-04-12T06:04:09.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 --2- 172.21.15.8:0/83104648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f3224100080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 --2- 172.21.15.8:0/83104648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f3224105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 -- 172.21.15.8:0/83104648 >> 172.21.15.8:0/83104648 conn(0x7f3224075670 msgr2=0x7f3224075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 -- 172.21.15.8:0/83104648 shutdown_connections 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.436+0000 7f322b60e700 1 -- 172.21.15.8:0/83104648 wait complete. 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 Processor -- start 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 -- start start 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f322419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419d0c0 con 0x7f32240ffc20 2024-04-12T06:04:09.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f322b60e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3224196600 con 0x7f3224105bd0 2024-04-12T06:04:09.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51520/0 (socket says 172.21.15.8:51520) 2024-04-12T06:04:09.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 -- 172.21.15.8:0/3901406957 learned_addr learned my addr 172.21.15.8:0/3901406957 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:09.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f3228ba9700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f322419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 -- 172.21.15.8:0/3901406957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 msgr2=0x7f322419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f322419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 -- 172.21.15.8:0/3901406957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140095f0 con 0x7f32240ffc20 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f3228ba9700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f322419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.437+0000 7f32293aa700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f322000c930 tx=0x7f322000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3220007ad0 con 0x7f32240ffc20 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3220007c30 con 0x7f32240ffc20 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32241968e0 con 0x7f32240ffc20 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3220018630 con 0x7f32240ffc20 2024-04-12T06:04:09.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3224196e30 con 0x7f32240ffc20 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.438+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322404f2e0 con 0x7f32240ffc20 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.439+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f322000f450 con 0x7f32240ffc20 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.439+0000 7f321a7fc700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 0x7f321006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.439+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f322008c6a0 con 0x7f32240ffc20 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.439+0000 7f3228ba9700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 0x7f321006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.440+0000 7f3228ba9700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 0x7f321006f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f321400b5c0 tx=0x7f3214005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:09.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.444+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f322005a2d0 con 0x7f32240ffc20 2024-04-12T06:04:09.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.720+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3224197180 con 0x7f321006cce0 2024-04-12T06:04:09.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.722+0000 7f321a7fc700 1 -- 172.21.15.8:0/3901406957 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3224197180 con 0x7f321006cce0 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 msgr2=0x7f321006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 0x7f321006f1a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f321400b5c0 tx=0x7f3214005960 comp rx=0 tx=0).stop 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 msgr2=0x7f322419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f322000c930 tx=0x7f322000ccf0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 shutdown_connections 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f321006cce0 0x7f321006f1a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32240ffc20 0x7f322419c580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 --2- 172.21.15.8:0/3901406957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3224105bd0 0x7f322419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 >> 172.21.15.8:0/3901406957 conn(0x7f3224075670 msgr2=0x7f3224068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:09.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 shutdown_connections 2024-04-12T06:04:09.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.724+0000 7f322b60e700 1 -- 172.21.15.8:0/3901406957 wait complete. 2024-04-12T06:04:09.731 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 -- 172.21.15.8:0/1037687259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 msgr2=0x7f436c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 --2- 172.21.15.8:0/1037687259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c0fffc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4360009910 tx=0x7f4360009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 -- 172.21.15.8:0/1037687259 shutdown_connections 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 --2- 172.21.15.8:0/1037687259 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c0fffc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 --2- 172.21.15.8:0/1037687259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 -- 172.21.15.8:0/1037687259 >> 172.21.15.8:0/1037687259 conn(0x7f436c0fb610 msgr2=0x7f436c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 -- 172.21.15.8:0/1037687259 shutdown_connections 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.837+0000 7f4371357700 1 -- 172.21.15.8:0/1037687259 wait complete. 2024-04-12T06:04:09.834 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 Processor -- start 2024-04-12T06:04:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 -- start start 2024-04-12T06:04:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c196240 con 0x7f436c0ffb40 2024-04-12T06:04:09.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.838+0000 7f4371357700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c199fd0 con 0x7f436c105bf0 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51542/0 (socket says 172.21.15.8:51542) 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 -- 172.21.15.8:0/1828596734 learned_addr learned my addr 172.21.15.8:0/1828596734 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436a7fc700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 -- 172.21.15.8:0/1828596734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 msgr2=0x7f436c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 -- 172.21.15.8:0/1828596734 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43600095f0 con 0x7f436c0ffb40 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436a7fc700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f436affd700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f435c007910 tx=0x7f435c007cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435c012070 con 0x7f436c0ffb40 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c19a250 con 0x7f436c0ffb40 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f435c00d810 con 0x7f436c0ffb40 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f435c0053d0 con 0x7f436c0ffb40 2024-04-12T06:04:09.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.839+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c19a7a0 con 0x7f436c0ffb40 2024-04-12T06:04:09.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.840+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c04f2e0 con 0x7f436c0ffb40 2024-04-12T06:04:09.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.841+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f435c016070 con 0x7f436c0ffb40 2024-04-12T06:04:09.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.841+0000 7f4353fff700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 0x7f435406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:09.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.841+0000 7f436a7fc700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 0x7f435406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.841+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f435c08d750 con 0x7f436c0ffb40 2024-04-12T06:04:09.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.842+0000 7f436a7fc700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 0x7f435406f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4360006010 tx=0x7f436000b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:09.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:09.846+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f435c057840 con 0x7f436c0ffb40 2024-04-12T06:04:10.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.076+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f436c19acb0 con 0x7f435406cce0 2024-04-12T06:04:10.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.078+0000 7f4353fff700 1 -- 172.21.15.8:0/1828596734 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f436c19acb0 con 0x7f435406cce0 2024-04-12T06:04:10.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 msgr2=0x7f435406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 0x7f435406f1a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4360006010 tx=0x7f436000b540 comp rx=0 tx=0).stop 2024-04-12T06:04:10.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 msgr2=0x7f436c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f435c007910 tx=0x7f435c007cd0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 shutdown_connections 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f435406cce0 0x7f435406f1a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f436c0ffb40 0x7f436c195620 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 --2- 172.21.15.8:0/1828596734 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f436c105bf0 0x7f436c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 >> 172.21.15.8:0/1828596734 conn(0x7f436c0fb610 msgr2=0x7f436c0fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 shutdown_connections 2024-04-12T06:04:10.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.080+0000 7f4371357700 1 -- 172.21.15.8:0/1828596734 wait complete. 2024-04-12T06:04:10.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 -- 172.21.15.8:0/2455444308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 msgr2=0x7fda940ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 --2- 172.21.15.8:0/2455444308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda940ff7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009910 tx=0x7fda7c009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 -- 172.21.15.8:0/2455444308 shutdown_connections 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 --2- 172.21.15.8:0/2455444308 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda940ff7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 --2- 172.21.15.8:0/2455444308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 0x7fda94105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 -- 172.21.15.8:0/2455444308 >> 172.21.15.8:0/2455444308 conn(0x7fda94074e30 msgr2=0x7fda94075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 -- 172.21.15.8:0/2455444308 shutdown_connections 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 -- 172.21.15.8:0/2455444308 wait complete. 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.198+0000 7fda993b9700 1 Processor -- start 2024-04-12T06:04:10.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda993b9700 1 -- start start 2024-04-12T06:04:10.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda993b9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda993b9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 0x7fda94198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda993b9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda94199570 con 0x7fda940ff340 2024-04-12T06:04:10.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda993b9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9419d380 con 0x7fda94105350 2024-04-12T06:04:10.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51554/0 (socket says 172.21.15.8:51554) 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 -- 172.21.15.8:0/309711323 learned_addr learned my addr 172.21.15.8:0/309711323 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda927fc700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 0x7fda94198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 -- 172.21.15.8:0/309711323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 msgr2=0x7fda94198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 0x7fda94198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 -- 172.21.15.8:0/309711323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c0095f0 con 0x7fda940ff340 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.199+0000 7fda92ffd700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d8d0 tx=0x7fda8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.200+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8c009940 con 0x7fda940ff340 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.200+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fda8c010460 con 0x7fda940ff340 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.200+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9419d660 con 0x7fda940ff340 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.200+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8c00f5f0 con 0x7fda940ff340 2024-04-12T06:04:10.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.200+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9419dbb0 con 0x7fda940ff340 2024-04-12T06:04:10.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.201+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fda8c0105d0 con 0x7fda940ff340 2024-04-12T06:04:10.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.201+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9404f2e0 con 0x7fda940ff340 2024-04-12T06:04:10.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.202+0000 7fda8bfff700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 0x7fda8006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.202+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fda8c08bee0 con 0x7fda940ff340 2024-04-12T06:04:10.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.202+0000 7fda927fc700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 0x7fda8006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.202+0000 7fda927fc700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 0x7fda8006f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda94199fc0 tx=0x7fda7c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.206+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fda8c059a60 con 0x7fda940ff340 2024-04-12T06:04:10.433 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.437+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fda9419de60 con 0x7fda8006cce0 2024-04-12T06:04:10.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.448+0000 7fda8bfff700 1 -- 172.21.15.8:0/309711323 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fda9419de60 con 0x7fda8006cce0 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (6m) 119s ago 17m 27.9M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (17m) 119s ago 17m 8262k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (16m) 19s ago 16m 8350k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (3m) 119s ago 17m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (3m) 19s ago 16m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (5m) 119s ago 17m 80.8M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (11m) 119s ago 11m 121M - 18.2.1 5be31c24972a 80466e37ec5d 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 starting - - - - 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (11m) 19s ago 11m 55.2M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:04:10.444 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (11m) 19s ago 11m 117M - 18.2.1 5be31c24972a c145ce249691 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (8m) 119s ago 19m 510M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (7m) 19s ago 16m 451M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (4m) 119s ago 19m 47.3M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (4m) 19s ago 16m 38.8M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (6m) 119s ago 17m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (6m) 19s ago 16m 18.6M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (3m) 119s ago 15m 95.6M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (2m) 119s ago 14m 103M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (2m) 119s ago 14m 11.2M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (89s) 19s ago 13m 104M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (56s) 19s ago 13m 108M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (23s) 19s ago 12m 11.0M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:04:10.445 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (6m) 119s ago 16m 75.0M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 msgr2=0x7fda8006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 0x7fda8006f1a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda94199fc0 tx=0x7fda7c01a040 comp rx=0 tx=0).stop 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 msgr2=0x7fda941989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d8d0 tx=0x7fda8c00dc90 comp rx=0 tx=0).stop 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 shutdown_connections 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fda8006cce0 0x7fda8006f1a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fda940ff340 0x7fda941989a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 --2- 172.21.15.8:0/309711323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fda94105350 0x7fda94198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 >> 172.21.15.8:0/309711323 conn(0x7fda94074e30 msgr2=0x7fda94102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 shutdown_connections 2024-04-12T06:04:10.447 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.450+0000 7fda993b9700 1 -- 172.21.15.8:0/309711323 wait complete. 2024-04-12T06:04:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/1115486052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 msgr2=0x7fbd281070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/1115486052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd281070b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009910 tx=0x7fbd1c009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:10.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/1115486052 shutdown_connections 2024-04-12T06:04:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/1115486052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 0x7fbd280689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/1115486052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd281070b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/1115486052 >> 172.21.15.8:0/1115486052 conn(0x7fbd28075670 msgr2=0x7fbd28075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/1115486052 shutdown_connections 2024-04-12T06:04:10.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.557+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/1115486052 wait complete. 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 Processor -- start 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 -- start start 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 0x7fbd2819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2819d130 con 0x7fbd28106cd0 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2cdbd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28196630 con 0x7fbd28068580 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd25d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd25d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51574/0 (socket says 172.21.15.8:51574) 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd25d9b700 1 -- 172.21.15.8:0/363939137 learned_addr learned my addr 172.21.15.8:0/363939137 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.558+0000 7fbd2659c700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 0x7fbd2819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd25d9b700 1 -- 172.21.15.8:0/363939137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 msgr2=0x7fbd2819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd25d9b700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 0x7fbd2819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd25d9b700 1 -- 172.21.15.8:0/363939137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c0095f0 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd25d9b700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800d900 tx=0x7fbd1800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd180041f0 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd18004cd0 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd28196910 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd18005660 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.559+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd28196e60 con 0x7fbd28106cd0 2024-04-12T06:04:10.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.560+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd281091d0 con 0x7fbd28106cd0 2024-04-12T06:04:10.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.560+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbd18004e40 con 0x7fbd28106cd0 2024-04-12T06:04:10.558 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.561+0000 7fbd177fe700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 0x7fbd1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.561+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd1808c2f0 con 0x7fbd28106cd0 2024-04-12T06:04:10.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.561+0000 7fbd2659c700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 0x7fbd1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.561+0000 7fbd2659c700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 0x7fbd1006f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b5c0 tx=0x7fbd1c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.566+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbd18059e70 con 0x7fbd28106cd0 2024-04-12T06:04:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:10 smithi194 ceph-mon[106982]: pgmap v271: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:10.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:10 smithi194 ceph-mon[106982]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:10.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.856+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd2804f2e0 con 0x7fbd28106cd0 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.857+0000 7fbd177fe700 1 -- 172.21.15.8:0/363939137 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fbd18059a00 con 0x7fbd28106cd0 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:10.853 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 1 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:10.854 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:04:10.855 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-04-12T06:04:10.855 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 11 2024-04-12T06:04:10.855 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:04:10.855 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:04:10.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 msgr2=0x7fbd1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 0x7fbd1006f1a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b5c0 tx=0x7fbd1c01a040 comp rx=0 tx=0).stop 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 msgr2=0x7fbd2819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800d900 tx=0x7fbd1800dc10 comp rx=0 tx=0).stop 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 shutdown_connections 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1006cce0 0x7fbd1006f1a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28068580 0x7fbd2819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 --2- 172.21.15.8:0/363939137 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106cd0 0x7fbd2819caa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 >> 172.21.15.8:0/363939137 conn(0x7fbd28075670 msgr2=0x7fbd280fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 shutdown_connections 2024-04-12T06:04:10.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.859+0000 7fbd2cdbd700 1 -- 172.21.15.8:0/363939137 wait complete. 2024-04-12T06:04:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:10 smithi008 ceph-mon[146084]: pgmap v271: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:10.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:10 smithi008 ceph-mon[146084]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:10.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 -- 172.21.15.8:0/2810990831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce0100510 msgr2=0x7fbce01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 --2- 172.21.15.8:0/2810990831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce0100510 0x7fbce01049e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009910 tx=0x7fbcd4009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:10.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 -- 172.21.15.8:0/2810990831 shutdown_connections 2024-04-12T06:04:10.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 --2- 172.21.15.8:0/2810990831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce0100510 0x7fbce01049e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 --2- 172.21.15.8:0/2810990831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce00ffbf0 0x7fbce00fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.978+0000 7fbce5642700 1 -- 172.21.15.8:0/2810990831 >> 172.21.15.8:0/2810990831 conn(0x7fbce00fb650 msgr2=0x7fbce00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.979+0000 7fbce5642700 1 -- 172.21.15.8:0/2810990831 shutdown_connections 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.979+0000 7fbce5642700 1 -- 172.21.15.8:0/2810990831 wait complete. 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.979+0000 7fbce5642700 1 Processor -- start 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.979+0000 7fbce5642700 1 -- start start 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbce5642700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbce5642700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce0100510 0x7fbce019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbce5642700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce019da70 con 0x7fbce00ffbf0 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbce5642700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce019dbb0 con 0x7fbce0100510 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51590/0 (socket says 172.21.15.8:51590) 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 -- 172.21.15.8:0/72646274 learned_addr learned my addr 172.21.15.8:0/72646274 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcde7fc700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce0100510 0x7fbce019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 -- 172.21.15.8:0/72646274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce0100510 msgr2=0x7fbce019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce0100510 0x7fbce019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 -- 172.21.15.8:0/72646274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd40095f0 con 0x7fbce00ffbf0 2024-04-12T06:04:10.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.980+0000 7fbcdeffd700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c930 tx=0x7fbcd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.981+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd0007ad0 con 0x7fbce00ffbf0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.981+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce01a1a00 con 0x7fbce00ffbf0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.981+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbcd000ce80 con 0x7fbce00ffbf0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.981+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcd0018630 con 0x7fbce00ffbf0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.981+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce01a1f50 con 0x7fbce00ffbf0 2024-04-12T06:04:10.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.982+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce0102090 con 0x7fbce00ffbf0 2024-04-12T06:04:10.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.982+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbcd0007c30 con 0x7fbce00ffbf0 2024-04-12T06:04:10.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.983+0000 7fbcc7fff700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 0x7fbcc806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:10.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.983+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbcd008c610 con 0x7fbce00ffbf0 2024-04-12T06:04:10.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.983+0000 7fbcde7fc700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 0x7fbcc806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:10.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.983+0000 7fbcde7fc700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 0x7fbcc806f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbce019e500 tx=0x7fbcd401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:10.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:10.987+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbcd00565e0 con 0x7fbce00ffbf0 2024-04-12T06:04:11.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.257+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbce0101a70 con 0x7fbce00ffbf0 2024-04-12T06:04:11.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.258+0000 7fbcc7fff700 1 -- 172.21.15.8:0/72646274 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1837 (secure 0 0 0) 0x7fbcd0059c00 con 0x7fbce00ffbf0 2024-04-12T06:04:11.254 INFO:teuthology.orchestra.run.smithi008.stdout:e30 2024-04-12T06:04:11.254 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:04:11.254 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:04:11.255 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:04:11.256 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34326} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2959889136,v1:172.21.15.8:6829/2959889136] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:11.257 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/1928513879,v1:172.21.15.194:6827/1928513879] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:11.258 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44299} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/2084066297,v1:172.21.15.194:6825/2084066297] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 msgr2=0x7fbcc806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 0x7fbcc806f1a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbce019e500 tx=0x7fbcd401a040 comp rx=0 tx=0).stop 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 msgr2=0x7fbce019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000c930 tx=0x7fbcd000cc40 comp rx=0 tx=0).stop 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 shutdown_connections 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbcc806cce0 0x7fbcc806f1a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbce00ffbf0 0x7fbce019cee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 --2- 172.21.15.8:0/72646274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbce0100510 0x7fbce019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 >> 172.21.15.8:0/72646274 conn(0x7fbce00fb650 msgr2=0x7fbce0111060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 shutdown_connections 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.260+0000 7fbce5642700 1 -- 172.21.15.8:0/72646274 wait complete. 2024-04-12T06:04:11.259 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-04-12T06:04:11.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 -- 172.21.15.8:0/979529053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e4100750 msgr2=0x7f66e41047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 --2- 172.21.15.8:0/979529053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e4100750 0x7f66e41047a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009910 tx=0x7f66d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:11.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 -- 172.21.15.8:0/979529053 shutdown_connections 2024-04-12T06:04:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 --2- 172.21.15.8:0/979529053 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e4100750 0x7f66e41047a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 --2- 172.21.15.8:0/979529053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e40ffda0 0x7f66e4100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 -- 172.21.15.8:0/979529053 >> 172.21.15.8:0/979529053 conn(0x7f66e40fb650 msgr2=0x7f66e40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 -- 172.21.15.8:0/979529053 shutdown_connections 2024-04-12T06:04:11.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.373+0000 7f66eac6d700 1 -- 172.21.15.8:0/979529053 wait complete. 2024-04-12T06:04:11.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 Processor -- start 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 -- start start 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e4100750 0x7f66e419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419c150 con 0x7f66e40ffda0 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66eac6d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419c310 con 0x7f66e4100750 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66e8a09700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66e8a09700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51616/0 (socket says 172.21.15.8:51616) 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66e8a09700 1 -- 172.21.15.8:0/1300933150 learned_addr learned my addr 172.21.15.8:0/1300933150 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.374+0000 7f66e3fff700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e4100750 0x7f66e419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e8a09700 1 -- 172.21.15.8:0/1300933150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e4100750 msgr2=0x7f66e419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.372 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e8a09700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e4100750 0x7f66e419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e8a09700 1 -- 172.21.15.8:0/1300933150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d00095f0 con 0x7f66e40ffda0 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e8a09700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66d800d900 tx=0x7f66d800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d8009900 con 0x7f66e40ffda0 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66d800b6e0 con 0x7f66e40ffda0 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.375+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e419c570 con 0x7f66e40ffda0 2024-04-12T06:04:11.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.376+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e419ca90 con 0x7f66e40ffda0 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.376+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d800f3c0 con 0x7f66e40ffda0 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.376+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8005320 con 0x7f66e40ffda0 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.377+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f66d800f5c0 con 0x7f66e40ffda0 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.377+0000 7f66e1ffb700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 0x7f66d406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.377+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f66d808b720 con 0x7f66e40ffda0 2024-04-12T06:04:11.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.378+0000 7f66e3fff700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 0x7f66d406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.378+0000 7f66e3fff700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 0x7f66d406f1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66e4069c70 tx=0x7f66d001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:11.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.382+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f66d8090060 con 0x7f66e40ffda0 2024-04-12T06:04:11.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.611+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66c8000bf0 con 0x7f66d406cd30 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.613+0000 7f66e1ffb700 1 -- 172.21.15.8:0/1300933150 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f66c8000bf0 con 0x7f66d406cd30 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:04:11.609 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "13/23 daemons upgraded", 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:04:11.610 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:04:11.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 msgr2=0x7f66d406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 0x7f66d406f1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f66e4069c70 tx=0x7f66d001a040 comp rx=0 tx=0).stop 2024-04-12T06:04:11.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 msgr2=0x7f66e419b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66d800d900 tx=0x7f66d800dc10 comp rx=0 tx=0).stop 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 shutdown_connections 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cd30 0x7f66d406f1f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e40ffda0 0x7f66e419b580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 --2- 172.21.15.8:0/1300933150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e4100750 0x7f66e419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 >> 172.21.15.8:0/1300933150 conn(0x7f66e40fb650 msgr2=0x7f66e40fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 shutdown_connections 2024-04-12T06:04:11.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.615+0000 7f66eac6d700 1 -- 172.21.15.8:0/1300933150 wait complete. 2024-04-12T06:04:11.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 -- 172.21.15.8:0/2545774236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 msgr2=0x7ff6801070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 --2- 172.21.15.8:0/2545774236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff6801070b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff670009910 tx=0x7ff670009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:11.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 -- 172.21.15.8:0/2545774236 shutdown_connections 2024-04-12T06:04:11.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 --2- 172.21.15.8:0/2545774236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 0x7ff6800689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 --2- 172.21.15.8:0/2545774236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff6801070b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 -- 172.21.15.8:0/2545774236 >> 172.21.15.8:0/2545774236 conn(0x7ff680075670 msgr2=0x7ff680075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:11.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 -- 172.21.15.8:0/2545774236 shutdown_connections 2024-04-12T06:04:11.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.730+0000 7ff684bb5700 1 -- 172.21.15.8:0/2545774236 wait complete. 2024-04-12T06:04:11.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 Processor -- start 2024-04-12T06:04:11.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 -- start start 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 0x7ff68019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68019d130 con 0x7ff680106cd0 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff684bb5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff680196630 con 0x7ff680068580 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff67dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff67dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:51626/0 (socket says 172.21.15.8:51626) 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff67dd9b700 1 -- 172.21.15.8:0/681576765 learned_addr learned my addr 172.21.15.8:0/681576765 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.731+0000 7ff67e59c700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 0x7ff68019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff67dd9b700 1 -- 172.21.15.8:0/681576765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 msgr2=0x7ff68019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff67dd9b700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 0x7ff68019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:11.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff67dd9b700 1 -- 172.21.15.8:0/681576765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700095f0 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff67dd9b700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff67400d8d0 tx=0x7ff67400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff674009940 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff680196910 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff674010460 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff67400f5f0 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.732+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff680196e60 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.733+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6801091d0 con 0x7ff680106cd0 2024-04-12T06:04:11.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.733+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff67400f870 con 0x7ff680106cd0 2024-04-12T06:04:11.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.734+0000 7ff66f7fe700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 0x7ff66806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:11.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.734+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff67408c5c0 con 0x7ff680106cd0 2024-04-12T06:04:11.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.734+0000 7ff67e59c700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 0x7ff66806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:11.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.734+0000 7ff67e59c700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 0x7ff66806f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff670006010 tx=0x7ff67001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:11.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:11.738+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff67405a0c0 con 0x7ff680106cd0 2024-04-12T06:04:11.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/363939137' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:11.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:11 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/72646274' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/363939137' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:11.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:11 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/72646274' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:04:12.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.043+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff68004f2e0 con 0x7ff680106cd0 2024-04-12T06:04:12.039 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.043+0000 7ff66f7fe700 1 -- 172.21.15.8:0/681576765 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff674059c50 con 0x7ff680106cd0 2024-04-12T06:04:12.040 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:04:12.040 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:04:12.040 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:04:12.040 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:04:12.040 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:04:12.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 msgr2=0x7ff66806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:12.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 0x7ff66806f1a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff670006010 tx=0x7ff67001a040 comp rx=0 tx=0).stop 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 msgr2=0x7ff68019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff67400d8d0 tx=0x7ff67400dc90 comp rx=0 tx=0).stop 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 shutdown_connections 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff66806cce0 0x7ff66806f1a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff680068580 0x7ff68019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 --2- 172.21.15.8:0/681576765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff680106cd0 0x7ff68019caa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 >> 172.21.15.8:0/681576765 conn(0x7ff680075670 msgr2=0x7ff6800fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 shutdown_connections 2024-04-12T06:04:12.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:12.047+0000 7ff684bb5700 1 -- 172.21.15.8:0/681576765 wait complete. 2024-04-12T06:04:13.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:12 smithi008 ceph-mon[146084]: pgmap v272: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:13.045 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:12 smithi008 ceph-mon[146084]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:13.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:13.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:13.046 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:12 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/681576765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:04:13.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:12 smithi194 ceph-mon[106982]: pgmap v272: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s 2024-04-12T06:04:13.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:12 smithi194 ceph-mon[106982]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:13.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:13.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:13.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:12 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/681576765' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:04:15.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:14 smithi008 ceph-mon[146084]: pgmap v273: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:15.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:14 smithi194 ceph-mon[106982]: pgmap v273: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: pgmap v274: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.053 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:16 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: pgmap v274: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:16 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:18.037 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:18.037 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:18.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:18.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:18.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:18.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:18.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi008.lmvepz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:18.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:18.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:18 smithi194 ceph-mon[106982]: pgmap v275: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:18.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:18 smithi194 ceph-mon[106982]: Upgrade: Updating mds.cephfs.smithi008.lmvepz 2024-04-12T06:04:18.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:18 smithi194 ceph-mon[106982]: Deploying daemon mds.cephfs.smithi008.lmvepz on smithi008 2024-04-12T06:04:19.051 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:18 smithi008 ceph-mon[146084]: pgmap v275: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:19.051 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:18 smithi008 ceph-mon[146084]: Upgrade: Updating mds.cephfs.smithi008.lmvepz 2024-04-12T06:04:19.051 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:18 smithi008 ceph-mon[146084]: Deploying daemon mds.cephfs.smithi008.lmvepz on smithi008 2024-04-12T06:04:20.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:20 smithi194 ceph-mon[106982]: pgmap v276: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:20 smithi194 ceph-mon[106982]: osdmap e81: 6 total, 6 up, 6 in 2024-04-12T06:04:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:20 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:20 smithi008 ceph-mon[146084]: pgmap v276: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:20 smithi008 ceph-mon[146084]: osdmap e81: 6 total, 6 up, 6 in 2024-04-12T06:04:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:20 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:22.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:22 smithi194 ceph-mon[106982]: pgmap v278: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:22.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:04:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:22 smithi008 ceph-mon[146084]: pgmap v278: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:22.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:04:24.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:24 smithi194 ceph-mon[106982]: pgmap v279: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:24.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:24 smithi008 ceph-mon[146084]: pgmap v279: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: pgmap v280: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:boot 2024-04-12T06:04:26.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:26.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:26 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:04:26.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: pgmap v280: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:boot 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:26 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:04:28.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:28 smithi194 ceph-mon[106982]: pgmap v281: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:28 smithi008 ceph-mon[146084]: pgmap v281: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:30.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:30 smithi008 ceph-mon[146084]: pgmap v282: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:30.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:30 smithi194 ceph-mon[106982]: pgmap v282: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:32.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:33.138 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:32 smithi008 ceph-mon[146084]: pgmap v283: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:33.139 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:33.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:32 smithi194 ceph-mon[106982]: pgmap v283: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:33.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:35.065 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:34 smithi008 ceph-mon[146084]: pgmap v284: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:35.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:34 smithi194 ceph-mon[106982]: pgmap v284: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:36.684 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: pgmap v285: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:36.684 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.684 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.684 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:36.684 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.685 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:36 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: pgmap v285: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:36 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:38.320 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: Upgrade: Updating mds.cephfs.smithi194.uzfpyu 2024-04-12T06:04:38.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:38.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:38.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:38.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: Deploying daemon mds.cephfs.smithi194.uzfpyu on smithi194 2024-04-12T06:04:38.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:38 smithi194 ceph-mon[106982]: pgmap v286: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:38.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: Upgrade: Updating mds.cephfs.smithi194.uzfpyu 2024-04-12T06:04:38.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:38.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.uzfpyu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:38.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:38.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: Deploying daemon mds.cephfs.smithi194.uzfpyu on smithi194 2024-04-12T06:04:38.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:38 smithi008 ceph-mon[146084]: pgmap v286: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:40.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:39 smithi194 ceph-mon[106982]: osdmap e82: 6 total, 6 up, 6 in 2024-04-12T06:04:40.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:39 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:40.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:39 smithi008 ceph-mon[146084]: osdmap e82: 6 total, 6 up, 6 in 2024-04-12T06:04:40.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:39 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:40.820 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:40 smithi194 ceph-mon[106982]: pgmap v288: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:41.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:40 smithi008 ceph-mon[146084]: pgmap v288: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:42.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/99378648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38102720 msgr2=0x7f9f3810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/99378648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38102720 0x7f9f3810ac10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009910 tx=0x7f9f20009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:42.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/99378648 shutdown_connections 2024-04-12T06:04:42.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/99378648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38102720 0x7f9f3810ac10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.168 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/99378648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38101e00 0x7f9f381021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/99378648 >> 172.21.15.8:0/99378648 conn(0x7f9f380fb650 msgr2=0x7f9f380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:42.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.171+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/99378648 shutdown_connections 2024-04-12T06:04:42.169 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.172+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/99378648 wait complete. 2024-04-12T06:04:42.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.172+0000 7f9f3ccd9700 1 Processor -- start 2024-04-12T06:04:42.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.172+0000 7f9f3ccd9700 1 -- start start 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3ccd9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3ccd9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38102720 0x7f9f380fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3ccd9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f381004e0 con 0x7f9f38101e00 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3ccd9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38100620 con 0x7f9f38102720 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37230/0 (socket says 172.21.15.8:37230) 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 -- 172.21.15.8:0/1257870248 learned_addr learned my addr 172.21.15.8:0/1257870248 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f35d9b700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38102720 0x7f9f380fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 -- 172.21.15.8:0/1257870248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38102720 msgr2=0x7f9f380fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38102720 0x7f9f380fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 -- 172.21.15.8:0/1257870248 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f200095f0 con 0x7f9f38101e00 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f3659c700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c930 tx=0x7f9f2800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.173+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f28007ad0 con 0x7f9f38101e00 2024-04-12T06:04:42.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.174+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9f2800ce80 con 0x7f9f38101e00 2024-04-12T06:04:42.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.174+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f381a2a50 con 0x7f9f38101e00 2024-04-12T06:04:42.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.174+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f28018630 con 0x7f9f38101e00 2024-04-12T06:04:42.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.174+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f381a2db0 con 0x7f9f38101e00 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.175+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f38108310 con 0x7f9f38101e00 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.175+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9f28007c30 con 0x7f9f38101e00 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.175+0000 7f9f2f7fe700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 0x7f9f2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.176+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9f2808c490 con 0x7f9f38101e00 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.176+0000 7f9f35d9b700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 0x7f9f2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.176+0000 7f9f35d9b700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 0x7f9f2406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006010 tx=0x7f9f2001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.180+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9f28056580 con 0x7f9f38101e00 2024-04-12T06:04:42.396 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.400+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f380611d0 con 0x7f9f2406cce0 2024-04-12T06:04:42.397 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.401+0000 7f9f2f7fe700 1 -- 172.21.15.8:0/1257870248 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f380611d0 con 0x7f9f2406cce0 2024-04-12T06:04:42.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 msgr2=0x7f9f2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 0x7f9f2406f1a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9f20006010 tx=0x7f9f2001a040 comp rx=0 tx=0).stop 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 msgr2=0x7f9f380ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800c930 tx=0x7f9f2800cc40 comp rx=0 tx=0).stop 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 shutdown_connections 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9f2406cce0 0x7f9f2406f1a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9f38101e00 0x7f9f380ffa60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 --2- 172.21.15.8:0/1257870248 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f38102720 0x7f9f380fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 >> 172.21.15.8:0/1257870248 conn(0x7f9f380fb650 msgr2=0x7f9f38105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 shutdown_connections 2024-04-12T06:04:42.400 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.403+0000 7f9f3ccd9700 1 -- 172.21.15.8:0/1257870248 wait complete. 2024-04-12T06:04:42.410 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:04:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 -- 172.21.15.8:0/1484092479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 msgr2=0x7fe17c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 --2- 172.21.15.8:0/1484092479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c105fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009910 tx=0x7fe16c009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 -- 172.21.15.8:0/1484092479 shutdown_connections 2024-04-12T06:04:42.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 --2- 172.21.15.8:0/1484092479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 --2- 172.21.15.8:0/1484092479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c105fb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 -- 172.21.15.8:0/1484092479 >> 172.21.15.8:0/1484092479 conn(0x7fe17c075670 msgr2=0x7fe17c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 -- 172.21.15.8:0/1484092479 shutdown_connections 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.511+0000 7fe183ed4700 1 -- 172.21.15.8:0/1484092479 wait complete. 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 Processor -- start 2024-04-12T06:04:42.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 -- start start 2024-04-12T06:04:42.510 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c19f370 con 0x7fe17c105bd0 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe183ed4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c198820 con 0x7fe17c0ffc20 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe18146f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe18146f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37236/0 (socket says 172.21.15.8:37236) 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe18146f700 1 -- 172.21.15.8:0/490460901 learned_addr learned my addr 172.21.15.8:0/490460901 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.512+0000 7fe181c70700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe18146f700 1 -- 172.21.15.8:0/490460901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 msgr2=0x7fe17c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe18146f700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe18146f700 1 -- 172.21.15.8:0/490460901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0095f0 con 0x7fe17c105bd0 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe181c70700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe18146f700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d8d0 tx=0x7fe17800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1780098a0 con 0x7fe17c105bd0 2024-04-12T06:04:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe178010460 con 0x7fe17c105bd0 2024-04-12T06:04:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe178009ea0 con 0x7fe17c105bd0 2024-04-12T06:04:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c198b00 con 0x7fe17c105bd0 2024-04-12T06:04:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.513+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c199050 con 0x7fe17c105bd0 2024-04-12T06:04:42.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.514+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c04f2e0 con 0x7fe17c105bd0 2024-04-12T06:04:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.515+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe1780105d0 con 0x7fe17c105bd0 2024-04-12T06:04:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.515+0000 7fe172ffd700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 0x7fe16806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.515+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe17808c370 con 0x7fe17c105bd0 2024-04-12T06:04:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.515+0000 7fe181c70700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 0x7fe16806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.516+0000 7fe181c70700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 0x7fe16806f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe16c006010 tx=0x7fe16c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.520+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe178059ef0 con 0x7fe17c105bd0 2024-04-12T06:04:42.739 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.743+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe17c199330 con 0x7fe16806cce0 2024-04-12T06:04:42.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.744+0000 7fe172ffd700 1 -- 172.21.15.8:0/490460901 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe17c199330 con 0x7fe16806cce0 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 msgr2=0x7fe16806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 0x7fe16806f1a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe16c006010 tx=0x7fe16c01a040 comp rx=0 tx=0).stop 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 msgr2=0x7fe17c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d8d0 tx=0x7fe17800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 shutdown_connections 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe16806cce0 0x7fe16806f1a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe17c0ffc20 0x7fe17c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 --2- 172.21.15.8:0/490460901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe17c105bd0 0x7fe17c19ece0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 >> 172.21.15.8:0/490460901 conn(0x7fe17c075670 msgr2=0x7fe17c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 shutdown_connections 2024-04-12T06:04:42.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.746+0000 7fe183ed4700 1 -- 172.21.15.8:0/490460901 wait complete. 2024-04-12T06:04:42.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 -- 172.21.15.8:0/2604092546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c0ffc20 msgr2=0x7fb61c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 --2- 172.21.15.8:0/2604092546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c0ffc20 0x7fb61c100080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb60c009910 tx=0x7fb60c009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 -- 172.21.15.8:0/2604092546 shutdown_connections 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 --2- 172.21.15.8:0/2604092546 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c0ffc20 0x7fb61c100080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 --2- 172.21.15.8:0/2604092546 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c105bd0 0x7fb61c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 -- 172.21.15.8:0/2604092546 >> 172.21.15.8:0/2604092546 conn(0x7fb61c075670 msgr2=0x7fb61c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 -- 172.21.15.8:0/2604092546 shutdown_connections 2024-04-12T06:04:42.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.853+0000 7fb6236fc700 1 -- 172.21.15.8:0/2604092546 wait complete. 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 Processor -- start 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 -- start start 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c0ffc20 0x7fb61c195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c195f10 con 0x7fb61c105bd0 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb6236fc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c196080 con 0x7fb61c0ffc20 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb620c97700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb620c97700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37256/0 (socket says 172.21.15.8:37256) 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb620c97700 1 -- 172.21.15.8:0/1517269443 learned_addr learned my addr 172.21.15.8:0/1517269443 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:42.852 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.854+0000 7fb621498700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c0ffc20 0x7fb61c195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb620c97700 1 -- 172.21.15.8:0/1517269443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c0ffc20 msgr2=0x7fb61c195490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb620c97700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c0ffc20 0x7fb61c195490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb620c97700 1 -- 172.21.15.8:0/1517269443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c0095f0 con 0x7fb61c105bd0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb620c97700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb60c000c00 tx=0x7fb60c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb60c01d070 con 0x7fb61c105bd0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb60c005dc0 con 0x7fb61c105bd0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c196300 con 0x7fb61c105bd0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb60c003940 con 0x7fb61c105bd0 2024-04-12T06:04:42.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.855+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c1a2d20 con 0x7fb61c105bd0 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.856+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c1080d0 con 0x7fb61c105bd0 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.857+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb60c004ae0 con 0x7fb61c105bd0 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.857+0000 7fb6127fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 0x7fb60806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.857+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb60c08d910 con 0x7fb61c105bd0 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.857+0000 7fb621498700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 0x7fb60806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:42.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.858+0000 7fb621498700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 0x7fb60806f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb618009c60 tx=0x7fb618009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:42.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:42.862+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb60c05b410 con 0x7fb61c105bd0 2024-04-12T06:04:43.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.074+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61c1962d0 con 0x7fb60806cce0 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: pgmap v289: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] up:boot 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:42 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:04:43.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.085+0000 7fb6127fc700 1 -- 172.21.15.8:0/1517269443 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb61c1962d0 con 0x7fb60806cce0 2024-04-12T06:04:43.081 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:04:43.081 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (6m) 12s ago 17m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:04:43.081 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (18m) 12s ago 18m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (17m) 52s ago 17m 8350k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (4m) 12s ago 18m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (4m) 52s ago 16m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (6m) 12s ago 17m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (18s) 12s ago 11m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (38s) 12s ago 11m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (11m) 52s ago 11m 55.2M - 18.2.1 5be31c24972a 7b78d5a5da1c 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 starting - - - - 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (8m) 12s ago 19m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (7m) 52s ago 16m 451M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (4m) 12s ago 19m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (4m) 52s ago 16m 38.8M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (7m) 12s ago 18m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:04:43.082 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (7m) 52s ago 16m 18.6M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (3m) 12s ago 15m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (3m) 12s ago 15m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (2m) 12s ago 14m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (2m) 52s ago 14m 104M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (89s) 52s ago 14m 108M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (55s) 52s ago 13m 11.0M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:04:43.083 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (6m) 12s ago 17m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 msgr2=0x7fb60806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 0x7fb60806f1a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb618009c60 tx=0x7fb618009260 comp rx=0 tx=0).stop 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 msgr2=0x7fb61c1959d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb60c000c00 tx=0x7fb60c00bde0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 shutdown_connections 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb60806cce0 0x7fb60806f1a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb61c0ffc20 0x7fb61c195490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 --2- 172.21.15.8:0/1517269443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb61c105bd0 0x7fb61c1959d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 >> 172.21.15.8:0/1517269443 conn(0x7fb61c075670 msgr2=0x7fb61c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 shutdown_connections 2024-04-12T06:04:43.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.087+0000 7fb6236fc700 1 -- 172.21.15.8:0/1517269443 wait complete. 2024-04-12T06:04:43.112 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: pgmap v289: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] up:boot 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:43.113 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:42 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 -- 172.21.15.8:0/2589535572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc105bf0 msgr2=0x7f8cfc105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 --2- 172.21.15.8:0/2589535572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc105fd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009910 tx=0x7f8cec009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 -- 172.21.15.8:0/2589535572 shutdown_connections 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 --2- 172.21.15.8:0/2589535572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 --2- 172.21.15.8:0/2589535572 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc105fd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 -- 172.21.15.8:0/2589535572 >> 172.21.15.8:0/2589535572 conn(0x7f8cfc0fb610 msgr2=0x7f8cfc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 -- 172.21.15.8:0/2589535572 shutdown_connections 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 -- 172.21.15.8:0/2589535572 wait complete. 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.194+0000 7f8d01f57700 1 Processor -- start 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8d01f57700 1 -- start start 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8d01f57700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8d01f57700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8d01f57700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc19d0c0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8d01f57700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1965c0 con 0x7f8cfc105bf0 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37270/0 (socket says 172.21.15.8:37270) 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 -- 172.21.15.8:0/3614059794 learned_addr learned my addr 172.21.15.8:0/3614059794 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfaffd700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 -- 172.21.15.8:0/3614059794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc105bf0 msgr2=0x7f8cfc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 -- 172.21.15.8:0/3614059794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0095f0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cfb7fe700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.195+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cec01d070 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.196+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8cec005dc0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.196+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc196840 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.196+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cec003940 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.196+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc196cd0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.197+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc04f2e0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.197+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8cec004ae0 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.198+0000 7f8cf8ff9700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 0x7f8ce406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.198+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8cec08e270 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.198+0000 7f8cfaffd700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 0x7f8ce406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.198+0000 7f8cfaffd700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 0x7f8ce406f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0feb50 tx=0x7f8cf0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.202+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8cec05bd70 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.490+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8cfc197a80 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.491+0000 7f8cf8ff9700 1 -- 172.21.15.8:0/3614059794 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8cec05b900 con 0x7f8cfc0ffb40 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:04:43.487 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 3 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 13 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:04:43.488 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 msgr2=0x7f8ce406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 0x7f8ce406f1a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc0feb50 tx=0x7f8cf0009260 comp rx=0 tx=0).stop 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 msgr2=0x7f8cfc19c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8cec000c00 tx=0x7f8cec00bde0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 shutdown_connections 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8ce406cce0 0x7f8ce406f1a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8cfc0ffb40 0x7f8cfc19c4f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 --2- 172.21.15.8:0/3614059794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8cfc105bf0 0x7f8cfc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 >> 172.21.15.8:0/3614059794 conn(0x7f8cfc0fb610 msgr2=0x7f8cfc068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 shutdown_connections 2024-04-12T06:04:43.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.493+0000 7f8d01f57700 1 -- 172.21.15.8:0/3614059794 wait complete. 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.600+0000 7f827c8fd700 1 -- 172.21.15.8:0/3098879282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 msgr2=0x7f82781070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.600+0000 7f827c8fd700 1 --2- 172.21.15.8:0/3098879282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f82781070b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8268009910 tx=0x7f8268009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 -- 172.21.15.8:0/3098879282 shutdown_connections 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 --2- 172.21.15.8:0/3098879282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 0x7f82780689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 --2- 172.21.15.8:0/3098879282 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f82781070b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 -- 172.21.15.8:0/3098879282 >> 172.21.15.8:0/3098879282 conn(0x7f8278075670 msgr2=0x7f8278075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 -- 172.21.15.8:0/3098879282 shutdown_connections 2024-04-12T06:04:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 -- 172.21.15.8:0/3098879282 wait complete. 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 Processor -- start 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.601+0000 7f827c8fd700 1 -- start start 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 0x7f82780ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278100770 con 0x7f8278106cd0 2024-04-12T06:04:43.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82781008b0 con 0x7f8278068580 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36942/0 (socket says 172.21.15.8:36942) 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 -- 172.21.15.8:0/1755082815 learned_addr learned my addr 172.21.15.8:0/1755082815 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827659c700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 0x7f82780ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 -- 172.21.15.8:0/1755082815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 msgr2=0x7f82780ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 0x7f82780ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 -- 172.21.15.8:0/1755082815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680095f0 con 0x7f8278106cd0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f8275d9b700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e920 tx=0x7f826c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c00c9d0 con 0x7f8278106cd0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f826c010430 con 0x7f8278106cd0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82781a29b0 con 0x7f8278106cd0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826c018650 con 0x7f8278106cd0 2024-04-12T06:04:43.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.602+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82781a2d60 con 0x7f8278106cd0 2024-04-12T06:04:43.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.604+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82781091d0 con 0x7f8278106cd0 2024-04-12T06:04:43.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.604+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f826c0105a0 con 0x7f8278106cd0 2024-04-12T06:04:43.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.604+0000 7f82677fe700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 0x7f826006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.604+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f826c014070 con 0x7f8278106cd0 2024-04-12T06:04:43.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.605+0000 7f827659c700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 0x7f826006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.605+0000 7f827659c700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 0x7f826006f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f826800b6e0 tx=0x7f8268005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.605 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.609+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f826c052ab0 con 0x7f8278106cd0 2024-04-12T06:04:43.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.859+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8278100b60 con 0x7f8278106cd0 2024-04-12T06:04:43.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.859+0000 7f82677fe700 1 -- 172.21.15.8:0/1755082815 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f8278100b60 con 0x7f8278106cd0 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:e34 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:04:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:04:43.857 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/1928513879,v1:172.21.15.194:6827/1928513879] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:43.858 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:04:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 msgr2=0x7f826006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 0x7f826006f1a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f826800b6e0 tx=0x7f8268005a90 comp rx=0 tx=0).stop 2024-04-12T06:04:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 msgr2=0x7f8278100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f826c00e920 tx=0x7f826c00ec30 comp rx=0 tx=0).stop 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 shutdown_connections 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f826006cce0 0x7f826006f1a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8278068580 0x7f82780ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 --2- 172.21.15.8:0/1755082815 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8278106cd0 0x7f8278100230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 >> 172.21.15.8:0/1755082815 conn(0x7f8278075670 msgr2=0x7f82780fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.861+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 shutdown_connections 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.862+0000 7f827c8fd700 1 -- 172.21.15.8:0/1755082815 wait complete. 2024-04-12T06:04:43.860 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 34 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 -- 172.21.15.8:0/1389635052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160068580 msgr2=0x7f21600689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 --2- 172.21.15.8:0/1389635052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160068580 0x7f21600689e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2150009910 tx=0x7f2150009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 -- 172.21.15.8:0/1389635052 shutdown_connections 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 --2- 172.21.15.8:0/1389635052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160068580 0x7f21600689e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 --2- 172.21.15.8:0/1389635052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160106cd0 0x7f21601070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 -- 172.21.15.8:0/1389635052 >> 172.21.15.8:0/1389635052 conn(0x7f2160075670 msgr2=0x7f2160075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 -- 172.21.15.8:0/1389635052 shutdown_connections 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.969+0000 7f21687b5700 1 -- 172.21.15.8:0/1389635052 wait complete. 2024-04-12T06:04:43.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 Processor -- start 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 -- start start 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160068580 0x7f216019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216019d110 con 0x7f2160106cd0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f21687b5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601965c0 con 0x7f2160068580 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36956/0 (socket says 172.21.15.8:36956) 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 -- 172.21.15.8:0/3101753302 learned_addr learned my addr 172.21.15.8:0/3101753302 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2166551700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160068580 0x7f216019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 -- 172.21.15.8:0/3101753302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160068580 msgr2=0x7f216019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160068580 0x7f216019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 -- 172.21.15.8:0/3101753302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500095f0 con 0x7f2160106cd0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.970+0000 7f2165d50700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2150006010 tx=0x7f215000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.971+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215001d070 con 0x7f2160106cd0 2024-04-12T06:04:43.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.971+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2150005710 con 0x7f2160106cd0 2024-04-12T06:04:43.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.971+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f215000fe20 con 0x7f2160106cd0 2024-04-12T06:04:43.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.971+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2160196840 con 0x7f2160106cd0 2024-04-12T06:04:43.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.971+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2160196d30 con 0x7f2160106cd0 2024-04-12T06:04:43.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.972+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21601091d0 con 0x7f2160106cd0 2024-04-12T06:04:43.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.972+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2150003a60 con 0x7f2160106cd0 2024-04-12T06:04:43.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.973+0000 7f21577fe700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 0x7f214c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:43.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.973+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f215008da90 con 0x7f2160106cd0 2024-04-12T06:04:43.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.973+0000 7f2166551700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 0x7f214c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:43.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.973+0000 7f2166551700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 0x7f214c06f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f215c005950 tx=0x7f215c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:43.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:43.977+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f215005b610 con 0x7f2160106cd0 2024-04-12T06:04:44.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:43 smithi194 ceph-mon[106982]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:44.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:43 smithi194 ceph-mon[106982]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:44.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:43 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3614059794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:43 smithi008 ceph-mon[146084]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:43 smithi008 ceph-mon[146084]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:44.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:43 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3614059794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:44.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.196+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2160196f50 con 0x7f214c06cce0 2024-04-12T06:04:44.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.197+0000 7f21577fe700 1 -- 172.21.15.8:0/3101753302 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2160196f50 con 0x7f214c06cce0 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "crash", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "mon" 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "14/23 daemons upgraded", 2024-04-12T06:04:44.194 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:04:44.195 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:04:44.195 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 msgr2=0x7f214c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 0x7f214c06f1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f215c005950 tx=0x7f215c00b410 comp rx=0 tx=0).stop 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 msgr2=0x7f216019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2150006010 tx=0x7f215000bd70 comp rx=0 tx=0).stop 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 shutdown_connections 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f214c06cce0 0x7f214c06f1a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2160068580 0x7f216019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 --2- 172.21.15.8:0/3101753302 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2160106cd0 0x7f216019ca80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.199+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 >> 172.21.15.8:0/3101753302 conn(0x7f2160075670 msgr2=0x7f21600fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:44.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.200+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 shutdown_connections 2024-04-12T06:04:44.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.200+0000 7f21687b5700 1 -- 172.21.15.8:0/3101753302 wait complete. 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 -- 172.21.15.8:0/408385230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3940ffda0 msgr2=0x7fd394100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 --2- 172.21.15.8:0/408385230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3940ffda0 0x7fd394100180 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009910 tx=0x7fd37c009c20 comp rx=0 tx=0).stop 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 -- 172.21.15.8:0/408385230 shutdown_connections 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 --2- 172.21.15.8:0/408385230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd394100750 0x7fd3941047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 --2- 172.21.15.8:0/408385230 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd3940ffda0 0x7fd394100180 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 -- 172.21.15.8:0/408385230 >> 172.21.15.8:0/408385230 conn(0x7fd3940fb650 msgr2=0x7fd3940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 -- 172.21.15.8:0/408385230 shutdown_connections 2024-04-12T06:04:44.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.306+0000 7fd3996c9700 1 -- 172.21.15.8:0/408385230 wait complete. 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 Processor -- start 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 -- start start 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3940ffda0 0x7fd3941956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394196290 con 0x7fd394100750 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3996c9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd394198fc0 con 0x7fd3940ffda0 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.307+0000 7fd3927fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36974/0 (socket says 172.21.15.8:36974) 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 -- 172.21.15.8:0/3303339724 learned_addr learned my addr 172.21.15.8:0/3303339724 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:04:44.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd392ffd700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3940ffda0 0x7fd3941956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 -- 172.21.15.8:0/3303339724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3940ffda0 msgr2=0x7fd3941956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3940ffda0 0x7fd3941956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 -- 172.21.15.8:0/3303339724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0095f0 con 0x7fd394100750 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3927fc700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd38400e8f0 tx=0x7fd38400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd384009790 con 0x7fd394100750 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3941992a0 con 0x7fd394100750 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd38400c810 con 0x7fd394100750 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd384005520 con 0x7fd394100750 2024-04-12T06:04:44.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.308+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3941997f0 con 0x7fd394100750 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.309+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd394108140 con 0x7fd394100750 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.310+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd38400c980 con 0x7fd394100750 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.310+0000 7fd38bfff700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 0x7fd38006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.310+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd384014070 con 0x7fd394100750 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.310+0000 7fd392ffd700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 0x7fd38006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:04:44.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.311+0000 7fd392ffd700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 0x7fd38006f1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009fd0 tx=0x7fd37c005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:04:44.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.315+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd384056890 con 0x7fd394100750 2024-04-12T06:04:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.602+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd39404f2e0 con 0x7fd394100750 2024-04-12T06:04:44.599 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.603+0000 7fd38bfff700 1 -- 172.21.15.8:0/3303339724 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd38400cc30 con 0x7fd394100750 2024-04-12T06:04:44.599 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:04:44.599 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:04:44.599 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:04:44.600 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:04:44.600 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:04:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 msgr2=0x7fd38006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 0x7fd38006f1a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd37c009fd0 tx=0x7fd37c005940 comp rx=0 tx=0).stop 2024-04-12T06:04:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 msgr2=0x7fd394195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:04:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd38400e8f0 tx=0x7fd38400ec00 comp rx=0 tx=0).stop 2024-04-12T06:04:44.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 shutdown_connections 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd38006cce0 0x7fd38006f1a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd3940ffda0 0x7fd3941956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 --2- 172.21.15.8:0/3303339724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd394100750 0x7fd394195c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 >> 172.21.15.8:0/3303339724 conn(0x7fd3940fb650 msgr2=0x7fd3940fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 shutdown_connections 2024-04-12T06:04:44.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:04:44.605+0000 7fd3996c9700 1 -- 172.21.15.8:0/3303339724 wait complete. 2024-04-12T06:04:45.070 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:45.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: pgmap v290: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:45.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1755082815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:04:45.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:45.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:45.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:44 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3303339724' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:04:45.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:45.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: pgmap v290: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:45.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1755082815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:04:45.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:45.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:45.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:44 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3303339724' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:04:46.027 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:45 smithi194 ceph-mon[106982]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:46.027 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:45 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:46.027 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:45 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:46.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:45 smithi008 ceph-mon[146084]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:04:46.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:45 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:46.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:45 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:47.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:46 smithi194 ceph-mon[106982]: pgmap v291: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:47.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:46 smithi008 ceph-mon[146084]: pgmap v291: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:48.024 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:48.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: pgmap v292: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: pgmap v292: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:49.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:04:51.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:50 smithi194 ceph-mon[106982]: pgmap v293: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:51.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:50 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:51.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:50 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:51.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:50 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:51.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:50 smithi008 ceph-mon[146084]: pgmap v293: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:51.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:50 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:51.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:50 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi194.gbfdvc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-12T06:04:51.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:50 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:04:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:51 smithi194 ceph-mon[106982]: Upgrade: Updating mds.cephfs.smithi194.gbfdvc 2024-04-12T06:04:51.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:51 smithi194 ceph-mon[106982]: Deploying daemon mds.cephfs.smithi194.gbfdvc on smithi194 2024-04-12T06:04:52.154 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:51 smithi008 ceph-mon[146084]: Upgrade: Updating mds.cephfs.smithi194.gbfdvc 2024-04-12T06:04:52.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:51 smithi008 ceph-mon[146084]: Deploying daemon mds.cephfs.smithi194.gbfdvc on smithi194 2024-04-12T06:04:53.202 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:52 smithi008 ceph-mon[146084]: pgmap v294: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:53.202 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:52 smithi008 ceph-mon[146084]: osdmap e83: 6 total, 6 up, 6 in 2024-04-12T06:04:53.202 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:52 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:53.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:52 smithi194 ceph-mon[106982]: pgmap v294: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:53.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:52 smithi194 ceph-mon[106982]: osdmap e83: 6 total, 6 up, 6 in 2024-04-12T06:04:53.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:52 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-12T06:04:55.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:54 smithi194 ceph-mon[106982]: pgmap v296: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:55.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:54 smithi008 ceph-mon[146084]: pgmap v296: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:57.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:56 smithi194 ceph-mon[106982]: pgmap v297: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:57.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:56 smithi008 ceph-mon[146084]: pgmap v297: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: pgmap v298: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] up:boot 2024-04-12T06:04:58.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:58.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:04:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: pgmap v298: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] up:boot 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-12T06:04:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:04:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:05:00.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:00 smithi194 ceph-mon[106982]: pgmap v299: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:00.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:00 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:00.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:00 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:00.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:00 smithi008 ceph-mon[146084]: pgmap v299: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:00.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:00 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:00.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:00 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:02.198 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:02.198 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:02.198 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:02 smithi194 ceph-mon[106982]: pgmap v300: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:02.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:05:02.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:02 smithi008 ceph-mon[146084]: pgmap v300: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:03.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:03.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:04.321 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:04 smithi194 ceph-mon[106982]: pgmap v301: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:04.404 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:04 smithi008 ceph-mon[146084]: pgmap v301: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:06.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:06 smithi194 ceph-mon[106982]: pgmap v302: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:06.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:06 smithi008 ceph-mon[146084]: pgmap v302: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:08.654 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:08 smithi008 ceph-mon[146084]: pgmap v303: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:08.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:08 smithi194 ceph-mon[106982]: pgmap v303: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:10.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:10 smithi194 ceph-mon[106982]: pgmap v304: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:10.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:10 smithi008 ceph-mon[146084]: pgmap v304: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:12.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:12 smithi194 ceph-mon[106982]: pgmap v305: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:12.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:12 smithi008 ceph-mon[146084]: pgmap v305: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 -- 172.21.15.8:0/1643960935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4068580 msgr2=0x7f8ba40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/1643960935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4068580 0x7f8ba40689e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009910 tx=0x7f8b8c009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 -- 172.21.15.8:0/1643960935 shutdown_connections 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/1643960935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4068580 0x7f8ba40689e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/1643960935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4106cd0 0x7f8ba41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 -- 172.21.15.8:0/1643960935 >> 172.21.15.8:0/1643960935 conn(0x7f8ba4075670 msgr2=0x7f8ba4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 -- 172.21.15.8:0/1643960935 shutdown_connections 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.727+0000 7f8ba9a06700 1 -- 172.21.15.8:0/1643960935 wait complete. 2024-04-12T06:05:14.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 Processor -- start 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 -- start start 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4068580 0x7f8ba419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba419d080 con 0x7f8ba4106cd0 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8ba9a06700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba41965c0 con 0x7f8ba4068580 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8b9a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8b9a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38294/0 (socket says 172.21.15.8:38294) 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.728+0000 7f8b9a7fc700 1 -- 172.21.15.8:0/2981241101 learned_addr learned my addr 172.21.15.8:0/2981241101 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba2ffd700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4068580 0x7f8ba419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8b9a7fc700 1 -- 172.21.15.8:0/2981241101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4068580 msgr2=0x7f8ba419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8b9a7fc700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4068580 0x7f8ba419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8b9a7fc700 1 -- 172.21.15.8:0/2981241101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0095f0 con 0x7f8ba4106cd0 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8b9a7fc700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b5c0 tx=0x7f8b8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:14.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8c01d070 con 0x7f8ba4106cd0 2024-04-12T06:05:14.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba4196840 con 0x7f8ba4106cd0 2024-04-12T06:05:14.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b8c00f460 con 0x7f8ba4106cd0 2024-04-12T06:05:14.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8c021640 con 0x7f8ba4106cd0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.729+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba4196d30 con 0x7f8ba4106cd0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.730+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba41091d0 con 0x7f8ba4106cd0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.731+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8b8c00fa20 con 0x7f8ba4106cd0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.731+0000 7f8ba8a04700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 0x7f8b9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.731+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8b8c08d7b0 con 0x7f8ba4106cd0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.731+0000 7f8ba2ffd700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 0x7f8b9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:14.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.732+0000 7f8ba2ffd700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 0x7f8b9006f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400ac50 tx=0x7f8b9400a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:14.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.736+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8b8c05b2b0 con 0x7f8ba4106cd0 2024-04-12T06:05:14.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:14 smithi194 ceph-mon[106982]: pgmap v306: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:14.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:14 smithi008 ceph-mon[146084]: pgmap v306: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:14.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.958+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ba4197b80 con 0x7f8b9006cce0 2024-04-12T06:05:14.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.959+0000 7f8ba8a04700 1 -- 172.21.15.8:0/2981241101 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8ba4197b80 con 0x7f8b9006cce0 2024-04-12T06:05:14.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.961+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 msgr2=0x7f8b9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:14.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.961+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 0x7f8b9006f1a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400ac50 tx=0x7f8b9400a380 comp rx=0 tx=0).stop 2024-04-12T06:05:14.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 msgr2=0x7f8ba419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:14.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b5c0 tx=0x7f8b8c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 shutdown_connections 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b9006cce0 0x7f8b9006f1a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8ba4068580 0x7f8ba419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 --2- 172.21.15.8:0/2981241101 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8ba4106cd0 0x7f8ba419ca80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 >> 172.21.15.8:0/2981241101 conn(0x7f8ba4075670 msgr2=0x7f8ba40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 shutdown_connections 2024-04-12T06:05:14.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:14.962+0000 7f8ba9a06700 1 -- 172.21.15.8:0/2981241101 wait complete. 2024-04-12T06:05:14.969 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:05:15.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 -- 172.21.15.8:0/2737100280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c106cd0 msgr2=0x7f984c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 --2- 172.21.15.8:0/2737100280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c106cd0 0x7f984c1070b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f983c009910 tx=0x7f983c009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 -- 172.21.15.8:0/2737100280 shutdown_connections 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 --2- 172.21.15.8:0/2737100280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c068580 0x7f984c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 --2- 172.21.15.8:0/2737100280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c106cd0 0x7f984c1070b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 -- 172.21.15.8:0/2737100280 >> 172.21.15.8:0/2737100280 conn(0x7f984c075670 msgr2=0x7f984c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 -- 172.21.15.8:0/2737100280 shutdown_connections 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.071+0000 7f9851df8700 1 -- 172.21.15.8:0/2737100280 wait complete. 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 Processor -- start 2024-04-12T06:05:15.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 -- start start 2024-04-12T06:05:15.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c106cd0 0x7f984c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c19d0a0 con 0x7f984c068580 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f9851df8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c196630 con 0x7f984c106cd0 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f984b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f984b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38308/0 (socket says 172.21.15.8:38308) 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.072+0000 7f984b7fe700 1 -- 172.21.15.8:0/1321638382 learned_addr learned my addr 172.21.15.8:0/1321638382 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f984affd700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c106cd0 0x7f984c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f984b7fe700 1 -- 172.21.15.8:0/1321638382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c106cd0 msgr2=0x7f984c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f984b7fe700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c106cd0 0x7f984c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f984b7fe700 1 -- 172.21.15.8:0/1321638382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0095f0 con 0x7f984c068580 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f984b7fe700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f983c00b5c0 tx=0x7f983c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983c01d070 con 0x7f984c068580 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f983c00f460 con 0x7f984c068580 2024-04-12T06:05:15.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c1968b0 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f983c021640 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.073+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c196da0 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.074+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c1091f0 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.075+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f983c00f9b0 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.075+0000 7f9848ff9700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 0x7f983806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.075+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f983c08d7a0 con 0x7f984c068580 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.075+0000 7f984affd700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 0x7f983806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.075+0000 7f984affd700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 0x7f983806f1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9834005ea0 tx=0x7f9834005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.079+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f983c05b2a0 con 0x7f984c068580 2024-04-12T06:05:15.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.301+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f984c197c10 con 0x7f983806cce0 2024-04-12T06:05:15.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.303+0000 7f9848ff9700 1 -- 172.21.15.8:0/1321638382 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f983400a890 con 0x7f983806cce0 2024-04-12T06:05:15.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 msgr2=0x7f983806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 0x7f983806f1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9834005ea0 tx=0x7f9834005e30 comp rx=0 tx=0).stop 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 msgr2=0x7f984c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f983c00b5c0 tx=0x7f983c00be20 comp rx=0 tx=0).stop 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 shutdown_connections 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f983806cce0 0x7f983806f1a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f984c068580 0x7f984c19c560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 --2- 172.21.15.8:0/1321638382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f984c106cd0 0x7f984c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 >> 172.21.15.8:0/1321638382 conn(0x7f984c075670 msgr2=0x7f984c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:15.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 shutdown_connections 2024-04-12T06:05:15.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.305+0000 7f9851df8700 1 -- 172.21.15.8:0/1321638382 wait complete. 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 -- 172.21.15.8:0/1976065198 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 msgr2=0x7f1b54105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 --2- 172.21.15.8:0/1976065198 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54105fd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009910 tx=0x7f1b44009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 -- 172.21.15.8:0/1976065198 shutdown_connections 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 --2- 172.21.15.8:0/1976065198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 0x7f1b540fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 --2- 172.21.15.8:0/1976065198 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54105fd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 -- 172.21.15.8:0/1976065198 >> 172.21.15.8:0/1976065198 conn(0x7f1b540fb610 msgr2=0x7f1b540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 -- 172.21.15.8:0/1976065198 shutdown_connections 2024-04-12T06:05:15.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.413+0000 7f1b5c457700 1 -- 172.21.15.8:0/1976065198 wait complete. 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 Processor -- start 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 -- start start 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 0x7f1b541975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54198110 con 0x7f1b54105bf0 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5c457700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54198250 con 0x7f1b540ffb40 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b599f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b599f2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38328/0 (socket says 172.21.15.8:38328) 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b599f2700 1 -- 172.21.15.8:0/2366465749 learned_addr learned my addr 172.21.15.8:0/2366465749 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.414+0000 7f1b5a1f3700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 0x7f1b541975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b599f2700 1 -- 172.21.15.8:0/2366465749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 msgr2=0x7f1b541975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b599f2700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 0x7f1b541975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b599f2700 1 -- 172.21.15.8:0/2366465749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440095f0 con 0x7f1b54105bf0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b599f2700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d8d0 tx=0x7f1b5000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b500098a0 con 0x7f1b54105bf0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b541a0620 con 0x7f1b54105bf0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b50010460 con 0x7f1b54105bf0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50009ea0 con 0x7f1b54105bf0 2024-04-12T06:05:15.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.415+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b541a0ae0 con 0x7f1b54105bf0 2024-04-12T06:05:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.416+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b54108020 con 0x7f1b54105bf0 2024-04-12T06:05:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.416+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1b500105d0 con 0x7f1b54105bf0 2024-04-12T06:05:15.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.417+0000 7f1b4b7fe700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 0x7f1b4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.417+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1b5008c440 con 0x7f1b54105bf0 2024-04-12T06:05:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.417+0000 7f1b5a1f3700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 0x7f1b4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.417+0000 7f1b5a1f3700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 0x7f1b4006f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400b5c0 tx=0x7f1b4401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.421+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1b500564b0 con 0x7f1b54105bf0 2024-04-12T06:05:15.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.633+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b540611d0 con 0x7f1b4006cce0 2024-04-12T06:05:15.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.644+0000 7f1b4b7fe700 1 -- 172.21.15.8:0/2366465749 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1b540611d0 con 0x7f1b4006cce0 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (7m) 44s ago 18m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (18m) 44s ago 18m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (17m) 15s ago 17m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (4m) 44s ago 18m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (4m) 15s ago 17m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (6m) 44s ago 18m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (50s) 44s ago 12m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (70s) 44s ago 12m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (18s) 15s ago 12m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (34s) 15s ago 12m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (9m) 44s ago 20m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (8m) 15s ago 17m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:05:15.641 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (5m) 44s ago 20m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (5m) 15s ago 17m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (8m) 44s ago 18m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (7m) 15s ago 17m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (4m) 44s ago 16m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (3m) 44s ago 16m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (3m) 44s ago 15m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (2m) 15s ago 15m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (2m) 15s ago 14m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (88s) 15s ago 14m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:05:15.642 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (7m) 44s ago 17m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 msgr2=0x7f1b4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 0x7f1b4006f1a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400b5c0 tx=0x7f1b4401a040 comp rx=0 tx=0).stop 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 msgr2=0x7f1b54197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000d8d0 tx=0x7f1b5000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 shutdown_connections 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1b4006cce0 0x7f1b4006f1a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b540ffb40 0x7f1b541975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 --2- 172.21.15.8:0/2366465749 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1b54105bf0 0x7f1b54197af0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 >> 172.21.15.8:0/2366465749 conn(0x7f1b540fb610 msgr2=0x7f1b540fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:15.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 shutdown_connections 2024-04-12T06:05:15.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.646+0000 7f1b5c457700 1 -- 172.21.15.8:0/2366465749 wait complete. 2024-04-12T06:05:15.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 -- 172.21.15.8:0/1414625077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 msgr2=0x7f122c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1414625077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c105fb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1228009910 tx=0x7f1228009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:15.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 -- 172.21.15.8:0/1414625077 shutdown_connections 2024-04-12T06:05:15.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1414625077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1414625077 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c105fb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 -- 172.21.15.8:0/1414625077 >> 172.21.15.8:0/1414625077 conn(0x7f122c075670 msgr2=0x7f122c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:15.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 -- 172.21.15.8:0/1414625077 shutdown_connections 2024-04-12T06:05:15.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.751+0000 7f1233b8d700 1 -- 172.21.15.8:0/1414625077 wait complete. 2024-04-12T06:05:15.750 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.752+0000 7f1233b8d700 1 Processor -- start 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.752+0000 7f1233b8d700 1 -- start start 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.752+0000 7f1233b8d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1233b8d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1233b8d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c19d160 con 0x7f122c105bd0 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1233b8d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c196610 con 0x7f122c0ffc20 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38344/0 (socket says 172.21.15.8:38344) 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 -- 172.21.15.8:0/1675252304 learned_addr learned my addr 172.21.15.8:0/1675252304 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231929700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 -- 172.21.15.8:0/1675252304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 msgr2=0x7f122c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:15.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 -- 172.21.15.8:0/1675252304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280095f0 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231929700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1231128700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f122000d8d0 tx=0x7f122000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1220009940 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1968f0 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1220010460 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f122000f5f0 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.753+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c196e40 con 0x7f122c105bd0 2024-04-12T06:05:15.752 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.754+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c04f2e0 con 0x7f122c105bd0 2024-04-12T06:05:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.755+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f12200105d0 con 0x7f122c105bd0 2024-04-12T06:05:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.755+0000 7f121effd700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 0x7f121806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.755+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f122008c5d0 con 0x7f122c105bd0 2024-04-12T06:05:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.755+0000 7f1231929700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 0x7f121806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:15.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.756+0000 7f1231929700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 0x7f121806f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1228006010 tx=0x7f122801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:15.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:15.760+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f122005a0d0 con 0x7f122c105bd0 2024-04-12T06:05:15.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:15 smithi194 ceph-mon[106982]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:15.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:15 smithi008 ceph-mon[146084]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:16.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.046+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f122c197b20 con 0x7f122c105bd0 2024-04-12T06:05:16.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.047+0000 7f121effd700 1 -- 172.21.15.8:0/1675252304 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1220010880 con 0x7f122c105bd0 2024-04-12T06:05:16.043 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:05:16.043 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:05:16.044 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:05:16.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 msgr2=0x7f121806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 0x7f121806f1a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1228006010 tx=0x7f122801a040 comp rx=0 tx=0).stop 2024-04-12T06:05:16.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 msgr2=0x7f122c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f122000d8d0 tx=0x7f122000dc90 comp rx=0 tx=0).stop 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 shutdown_connections 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121806cce0 0x7f121806f1a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f122c0ffc20 0x7f122c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 --2- 172.21.15.8:0/1675252304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f122c105bd0 0x7f122c19cad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 >> 172.21.15.8:0/1675252304 conn(0x7f122c075670 msgr2=0x7f122c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 shutdown_connections 2024-04-12T06:05:16.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.049+0000 7f1233b8d700 1 -- 172.21.15.8:0/1675252304 wait complete. 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 -- 172.21.15.8:0/3026673390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 msgr2=0x7f5fec100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/3026673390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec100080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009910 tx=0x7f5fe0009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 -- 172.21.15.8:0/3026673390 shutdown_connections 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/3026673390 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec100080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/3026673390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 0x7f5fec105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 -- 172.21.15.8:0/3026673390 >> 172.21.15.8:0/3026673390 conn(0x7f5fec075670 msgr2=0x7f5fec075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 -- 172.21.15.8:0/3026673390 shutdown_connections 2024-04-12T06:05:16.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.155+0000 7f5ff1a72700 1 -- 172.21.15.8:0/3026673390 wait complete. 2024-04-12T06:05:16.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 Processor -- start 2024-04-12T06:05:16.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 -- start start 2024-04-12T06:05:16.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 0x7f5fec195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec195ed0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5ff1a72700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec196040 con 0x7f5fec105bd0 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38364/0 (socket says 172.21.15.8:38364) 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 -- 172.21.15.8:0/1542805770 learned_addr learned my addr 172.21.15.8:0/1542805770 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feaffd700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 0x7f5fec195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 -- 172.21.15.8:0/1542805770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 msgr2=0x7f5fec195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 0x7f5fec195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.156+0000 7f5feb7fe700 1 -- 172.21.15.8:0/1542805770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe00095f0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5feb7fe700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c930 tx=0x7f5fdc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc007ad0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec1a2ac0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5fdc007c30 con 0x7f5fec0ffc20 2024-04-12T06:05:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc018630 con 0x7f5fec0ffc20 2024-04-12T06:05:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.157+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec1a2e20 con 0x7f5fec0ffc20 2024-04-12T06:05:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.158+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec1080d0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.158+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f5fdc00f450 con 0x7f5fec0ffc20 2024-04-12T06:05:16.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.159+0000 7f5fe8ff9700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 0x7f5fd406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.159+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5fdc08c680 con 0x7f5fec0ffc20 2024-04-12T06:05:16.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.159+0000 7f5feaffd700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 0x7f5fd406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.159+0000 7f5feaffd700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 0x7f5fd406f1a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fec068f40 tx=0x7f5fe001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.163+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5fdc05a230 con 0x7f5fec0ffc20 2024-04-12T06:05:16.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.412+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5fec04f2e0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.412+0000 7f5fe8ff9700 1 -- 172.21.15.8:0/1542805770 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f5fdc059dc0 con 0x7f5fec0ffc20 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:05:16.409 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:05:16.410 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:16.411 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:05:16.412 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:16.412 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:16.412 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:16.412 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:16.412 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:16.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 msgr2=0x7f5fd406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 0x7f5fd406f1a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5fec068f40 tx=0x7f5fe001a040 comp rx=0 tx=0).stop 2024-04-12T06:05:16.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 msgr2=0x7f5fec195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c930 tx=0x7f5fdc00ccf0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 shutdown_connections 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5fd406cce0 0x7f5fd406f1a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5fec0ffc20 0x7f5fec195450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 --2- 172.21.15.8:0/1542805770 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5fec105bd0 0x7f5fec195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 >> 172.21.15.8:0/1542805770 conn(0x7f5fec075670 msgr2=0x7f5fec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 shutdown_connections 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.414+0000 7f5ff1a72700 1 -- 172.21.15.8:0/1542805770 wait complete. 2024-04-12T06:05:16.414 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:05:16.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 -- 172.21.15.8:0/1921333408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7738100750 msgr2=0x7f77381047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 --2- 172.21.15.8:0/1921333408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7738100750 0x7f77381047a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7720009910 tx=0x7f7720009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 -- 172.21.15.8:0/1921333408 shutdown_connections 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 --2- 172.21.15.8:0/1921333408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7738100750 0x7f77381047a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 --2- 172.21.15.8:0/1921333408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77380ffda0 0x7f7738100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 -- 172.21.15.8:0/1921333408 >> 172.21.15.8:0/1921333408 conn(0x7f77380fb650 msgr2=0x7f77380fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 -- 172.21.15.8:0/1921333408 shutdown_connections 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.522+0000 7f773d7e0700 1 -- 172.21.15.8:0/1921333408 wait complete. 2024-04-12T06:05:16.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 Processor -- start 2024-04-12T06:05:16.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 -- start start 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7738100750 0x7f7738197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738198260 con 0x7f77380ffda0 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f773d7e0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381983a0 con 0x7f7738100750 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f7736ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.523+0000 7f7736ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38380/0 (socket says 172.21.15.8:38380) 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f7736ffd700 1 -- 172.21.15.8:0/2758894990 learned_addr learned my addr 172.21.15.8:0/2758894990 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f77367fc700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7738100750 0x7f7738197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f7736ffd700 1 -- 172.21.15.8:0/2758894990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7738100750 msgr2=0x7f7738197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f7736ffd700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7738100750 0x7f7738197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f7736ffd700 1 -- 172.21.15.8:0/2758894990 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200095f0 con 0x7f77380ffda0 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f7736ffd700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f772800d8d0 tx=0x7f772800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77280098a0 con 0x7f77380ffda0 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7728010460 con 0x7f77380ffda0 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77381a0770 con 0x7f77380ffda0 2024-04-12T06:05:16.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7728009ea0 con 0x7f77380ffda0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.524+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77381a0c30 con 0x7f77380ffda0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.525+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77381081a0 con 0x7f77380ffda0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.526+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f77280105d0 con 0x7f77380ffda0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.526+0000 7f772ffff700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 0x7f772406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.526+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f772808c440 con 0x7f77380ffda0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.526+0000 7f77367fc700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 0x7f772406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.526+0000 7f77367fc700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 0x7f772406f1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7720009fd0 tx=0x7f772001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.531+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f77280564b0 con 0x7f77380ffda0 2024-04-12T06:05:16.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.751+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77380611d0 con 0x7f772406cce0 2024-04-12T06:05:16.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.752+0000 7f772ffff700 1 -- 172.21.15.8:0/2758894990 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f77380611d0 con 0x7f772406cce0 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:05:16.749 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:05:16.750 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:05:16.750 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 msgr2=0x7f772406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 0x7f772406f1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7720009fd0 tx=0x7f772001a040 comp rx=0 tx=0).stop 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 msgr2=0x7f7738197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f772800d8d0 tx=0x7f772800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 shutdown_connections 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f772406cce0 0x7f772406f1a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77380ffda0 0x7f7738197700 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 --2- 172.21.15.8:0/2758894990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7738100750 0x7f7738197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 >> 172.21.15.8:0/2758894990 conn(0x7f77380fb650 msgr2=0x7f77380fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 shutdown_connections 2024-04-12T06:05:16.751 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.754+0000 7f773d7e0700 1 -- 172.21.15.8:0/2758894990 wait complete. 2024-04-12T06:05:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:16 smithi194 ceph-mon[106982]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:16 smithi194 ceph-mon[106982]: pgmap v307: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:16 smithi194 ceph-mon[106982]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:16 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1675252304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:05:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:16 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1542805770' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:05:16.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 -- 172.21.15.8:0/3748030138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec101850 msgr2=0x7f26ec1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 --2- 172.21.15.8:0/3748030138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec101850 0x7f26ec1058a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009910 tx=0x7f26e0009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:16.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 -- 172.21.15.8:0/3748030138 shutdown_connections 2024-04-12T06:05:16.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 --2- 172.21.15.8:0/3748030138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec101850 0x7f26ec1058a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 --2- 172.21.15.8:0/3748030138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec100ea0 0x7f26ec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 -- 172.21.15.8:0/3748030138 >> 172.21.15.8:0/3748030138 conn(0x7f26ec0fc750 msgr2=0x7f26ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 -- 172.21.15.8:0/3748030138 shutdown_connections 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.861+0000 7f26f235c700 1 -- 172.21.15.8:0/3748030138 wait complete. 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 Processor -- start 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 -- start start 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec101850 0x7f26ec19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec19d1f0 con 0x7f26ec100ea0 2024-04-12T06:05:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26f235c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ec1966a0 con 0x7f26ec101850 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26ebfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26ebfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38388/0 (socket says 172.21.15.8:38388) 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26ebfff700 1 -- 172.21.15.8:0/4145716535 learned_addr learned my addr 172.21.15.8:0/4145716535 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.862+0000 7f26eb7fe700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec101850 0x7f26ec19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26ebfff700 1 -- 172.21.15.8:0/4145716535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec101850 msgr2=0x7f26ec19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26ebfff700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec101850 0x7f26ec19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26ebfff700 1 -- 172.21.15.8:0/4145716535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e00095f0 con 0x7f26ec100ea0 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26ebfff700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00d8d0 tx=0x7f26dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26dc0098a0 con 0x7f26ec100ea0 2024-04-12T06:05:16.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec196980 con 0x7f26ec100ea0 2024-04-12T06:05:16.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f26dc010460 con 0x7f26ec100ea0 2024-04-12T06:05:16.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26dc009ea0 con 0x7f26ec100ea0 2024-04-12T06:05:16.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.863+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec196ea0 con 0x7f26ec100ea0 2024-04-12T06:05:16.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.864+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec1092a0 con 0x7f26ec100ea0 2024-04-12T06:05:16.866 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.864+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f26dc0105d0 con 0x7f26ec100ea0 2024-04-12T06:05:16.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.865+0000 7f26e97fa700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 0x7f26d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:16.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.865+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f26dc08c440 con 0x7f26ec100ea0 2024-04-12T06:05:16.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.865+0000 7f26eb7fe700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 0x7f26d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:16.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.865+0000 7f26eb7fe700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 0x7f26d406f1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006010 tx=0x7f26e001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:16.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:16.870+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f26dc059f40 con 0x7f26ec100ea0 2024-04-12T06:05:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:16 smithi008 ceph-mon[146084]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:16.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:16 smithi008 ceph-mon[146084]: pgmap v307: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:16 smithi008 ceph-mon[146084]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:16 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1675252304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:05:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:16 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1542805770' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:05:17.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.156+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f26ec04f2e0 con 0x7f26ec100ea0 2024-04-12T06:05:17.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.157+0000 7f26e97fa700 1 -- 172.21.15.8:0/4145716535 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f26dc010880 con 0x7f26ec100ea0 2024-04-12T06:05:17.153 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:05:17.153 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:05:17.154 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:05:17.154 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:05:17.154 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:05:17.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 msgr2=0x7f26d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:17.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 0x7f26d406f1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f26e0006010 tx=0x7f26e001a040 comp rx=0 tx=0).stop 2024-04-12T06:05:17.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 msgr2=0x7f26ec19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:17.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00d8d0 tx=0x7f26dc00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:05:17.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 shutdown_connections 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f26d406cce0 0x7f26d406f1a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f26ec100ea0 0x7f26ec19c620 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 --2- 172.21.15.8:0/4145716535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f26ec101850 0x7f26ec19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 >> 172.21.15.8:0/4145716535 conn(0x7f26ec0fc750 msgr2=0x7f26ec0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 shutdown_connections 2024-04-12T06:05:17.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:17.159+0000 7f26f235c700 1 -- 172.21.15.8:0/4145716535 wait complete. 2024-04-12T06:05:17.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:17 smithi194 ceph-mon[106982]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:17.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:17 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4145716535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:05:17.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:17 smithi008 ceph-mon[146084]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:17 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4145716535' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:05:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:18.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:18 smithi194 ceph-mon[106982]: pgmap v308: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:18.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:18 smithi008 ceph-mon[146084]: pgmap v308: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:20 smithi194 ceph-mon[106982]: pgmap v309: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:20.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:20 smithi008 ceph-mon[146084]: pgmap v309: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:22.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:22 smithi194 ceph-mon[106982]: pgmap v310: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:22 smithi008 ceph-mon[146084]: pgmap v310: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:24.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:24 smithi194 ceph-mon[106982]: pgmap v311: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:24.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:24 smithi008 ceph-mon[146084]: pgmap v311: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:26.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:26 smithi194 ceph-mon[106982]: pgmap v312: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:26.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:26 smithi008 ceph-mon[146084]: pgmap v312: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:28.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:28 smithi194 ceph-mon[106982]: pgmap v313: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:28.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:28 smithi008 ceph-mon[146084]: pgmap v313: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:30.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:30 smithi194 ceph-mon[106982]: pgmap v314: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:30.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:30 smithi008 ceph-mon[146084]: pgmap v314: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:32.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:32 smithi194 ceph-mon[106982]: pgmap v315: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:32.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:32.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:32 smithi008 ceph-mon[146084]: pgmap v315: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:32.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:34.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:34 smithi194 ceph-mon[106982]: pgmap v316: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:34.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:34 smithi008 ceph-mon[146084]: pgmap v316: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:36.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:36 smithi194 ceph-mon[106982]: pgmap v317: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:36 smithi008 ceph-mon[146084]: pgmap v317: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:38.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:38 smithi194 ceph-mon[106982]: pgmap v318: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:38.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:38 smithi008 ceph-mon[146084]: pgmap v318: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:40.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:40 smithi194 ceph-mon[106982]: pgmap v319: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:40 smithi008 ceph-mon[146084]: pgmap v319: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:42.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:42 smithi194 ceph-mon[106982]: pgmap v320: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:42.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:42 smithi008 ceph-mon[146084]: pgmap v320: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:44.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:44 smithi194 ceph-mon[106982]: pgmap v321: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:44.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:44 smithi008 ceph-mon[146084]: pgmap v321: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:46.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:46 smithi194 ceph-mon[106982]: pgmap v322: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:46.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:46 smithi008 ceph-mon[146084]: pgmap v322: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3184093823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb54105bd0 msgr2=0x7feb54105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3184093823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb54105bd0 0x7feb54105fb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feb44009910 tx=0x7feb44009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3184093823 shutdown_connections 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3184093823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb540ffc20 0x7feb54100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3184093823 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb54105bd0 0x7feb54105fb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3184093823 >> 172.21.15.8:0/3184093823 conn(0x7feb54075670 msgr2=0x7feb54075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3184093823 shutdown_connections 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.280+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3184093823 wait complete. 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 Processor -- start 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 -- start start 2024-04-12T06:05:47.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 0x7feb5419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb5419d080 con 0x7feb540ffc20 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb5b1cc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54196610 con 0x7feb54105bd0 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.281+0000 7feb58f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55606/0 (socket says 172.21.15.8:55606) 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 -- 172.21.15.8:0/3207262900 learned_addr learned my addr 172.21.15.8:0/3207262900 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb53fff700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 0x7feb5419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 -- 172.21.15.8:0/3207262900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 msgr2=0x7feb5419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 0x7feb5419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 -- 172.21.15.8:0/3207262900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440095f0 con 0x7feb540ffc20 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb53fff700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 0x7feb5419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb58f68700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb44006010 tx=0x7feb4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4401d070 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb54196890 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb4400f460 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.282+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb44021620 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.283+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb54196d80 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.284+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb5404f2e0 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.284+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7feb4402b440 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.284+0000 7feb51ffb700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 0x7feb3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.284+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb4408d780 con 0x7feb540ffc20 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.284+0000 7feb53fff700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 0x7feb3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.285+0000 7feb53fff700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 0x7feb3c06f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb48005fd0 tx=0x7feb48005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.289+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7feb4405b280 con 0x7feb540ffc20 2024-04-12T06:05:47.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.514+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb54197080 con 0x7feb3c06cce0 2024-04-12T06:05:47.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.515+0000 7feb51ffb700 1 -- 172.21.15.8:0/3207262900 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feb54197080 con 0x7feb3c06cce0 2024-04-12T06:05:47.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 msgr2=0x7feb3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 0x7feb3c06f1a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb48005fd0 tx=0x7feb48005ee0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 msgr2=0x7feb5419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb44006010 tx=0x7feb4400be20 comp rx=0 tx=0).stop 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 shutdown_connections 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb3c06cce0 0x7feb3c06f1a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feb540ffc20 0x7feb5419c520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 --2- 172.21.15.8:0/3207262900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb54105bd0 0x7feb5419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 >> 172.21.15.8:0/3207262900 conn(0x7feb54075670 msgr2=0x7feb54068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 shutdown_connections 2024-04-12T06:05:47.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.517+0000 7feb5b1cc700 1 -- 172.21.15.8:0/3207262900 wait complete. 2024-04-12T06:05:47.525 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- 172.21.15.8:0/3314300906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068580 msgr2=0x7f620c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 --2- 172.21.15.8:0/3314300906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068580 0x7f620c068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009910 tx=0x7f61fc009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- 172.21.15.8:0/3314300906 shutdown_connections 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 --2- 172.21.15.8:0/3314300906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068ea0 0x7f620c10d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 --2- 172.21.15.8:0/3314300906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068580 0x7f620c068960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- 172.21.15.8:0/3314300906 >> 172.21.15.8:0/3314300906 conn(0x7f620c075670 msgr2=0x7f620c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:47.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- 172.21.15.8:0/3314300906 shutdown_connections 2024-04-12T06:05:47.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- 172.21.15.8:0/3314300906 wait complete. 2024-04-12T06:05:47.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 Processor -- start 2024-04-12T06:05:47.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.626+0000 7f6212672700 1 -- start start 2024-04-12T06:05:47.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f6212672700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068580 0x7f620c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f6212672700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f6212672700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c199550 con 0x7f620c068ea0 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f6212672700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c19d360 con 0x7f620c068580 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55618/0 (socket says 172.21.15.8:55618) 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 -- 172.21.15.8:0/3998950995 learned_addr learned my addr 172.21.15.8:0/3998950995 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620bfff700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068580 0x7f620c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 -- 172.21.15.8:0/3998950995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068580 msgr2=0x7f620c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068580 0x7f620c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 -- 172.21.15.8:0/3998950995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0095f0 con 0x7f620c068ea0 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.627+0000 7f620b7fe700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f620000e920 tx=0x7f620000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.628+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6200009790 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.628+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f620000cba0 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.628+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c19d640 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.628+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6200018650 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.628+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c19db90 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.629+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c10aa00 con 0x7f620c068ea0 2024-04-12T06:05:47.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.629+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f620000cd10 con 0x7f620c068ea0 2024-04-12T06:05:47.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.629+0000 7f62097fa700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 0x7f61f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.629+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6200014070 con 0x7f620c068ea0 2024-04-12T06:05:47.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.630+0000 7f620bfff700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 0x7f61f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.630+0000 7f620bfff700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 0x7f61f406f1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009fd0 tx=0x7f61fc005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.630 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.634+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6200052ab0 con 0x7f620c068ea0 2024-04-12T06:05:47.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:47.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.859+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f620c19de00 con 0x7f61f406cce0 2024-04-12T06:05:47.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.860+0000 7f62097fa700 1 -- 172.21.15.8:0/3998950995 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f620c19de00 con 0x7f61f406cce0 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 msgr2=0x7f61f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 0x7f61f406f1a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009fd0 tx=0x7f61fc005940 comp rx=0 tx=0).stop 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 msgr2=0x7f620c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f620000e920 tx=0x7f620000ece0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 shutdown_connections 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f61f406cce0 0x7f61f406f1a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f620c068580 0x7f620c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 --2- 172.21.15.8:0/3998950995 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f620c068ea0 0x7f620c198ec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 >> 172.21.15.8:0/3998950995 conn(0x7f620c075670 msgr2=0x7f620c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:47.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 shutdown_connections 2024-04-12T06:05:47.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.862+0000 7f6212672700 1 -- 172.21.15.8:0/3998950995 wait complete. 2024-04-12T06:05:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:05:47.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 -- 172.21.15.8:0/1267964782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 msgr2=0x7f2e14105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/1267964782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e14105890 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009910 tx=0x7f2e10009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 -- 172.21.15.8:0/1267964782 shutdown_connections 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/1267964782 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e14105890 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/1267964782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 -- 172.21.15.8:0/1267964782 >> 172.21.15.8:0/1267964782 conn(0x7f2e140fc740 msgr2=0x7f2e140feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 -- 172.21.15.8:0/1267964782 shutdown_connections 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 -- 172.21.15.8:0/1267964782 wait complete. 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.969+0000 7f2e1b55a700 1 Processor -- start 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 -- start start 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14073b10 con 0x7f2e14101840 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e14073c50 con 0x7f2e14100e90 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55634/0 (socket says 172.21.15.8:55634) 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 -- 172.21.15.8:0/3891661027 learned_addr learned my addr 172.21.15.8:0/3891661027 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e192f6700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 -- 172.21.15.8:0/3891661027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 msgr2=0x7f2e14073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 -- 172.21.15.8:0/3891661027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100095f0 con 0x7f2e14101840 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e192f6700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e18af5700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000bda0 tx=0x7f2e1000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1001d070 con 0x7f2e14101840 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e1000f460 con 0x7f2e14101840 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.970+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e14073ed0 con 0x7f2e14101840 2024-04-12T06:05:47.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.971+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e10021640 con 0x7f2e14101840 2024-04-12T06:05:47.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.971+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e141a2d60 con 0x7f2e14101840 2024-04-12T06:05:47.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.972+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e14109290 con 0x7f2e14101840 2024-04-12T06:05:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.972+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2e1000f9d0 con 0x7f2e14101840 2024-04-12T06:05:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.972+0000 7f2e0a7fc700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 0x7f2e0006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.973+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2e1008d760 con 0x7f2e14101840 2024-04-12T06:05:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.973+0000 7f2e192f6700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 0x7f2e0006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:47.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.973+0000 7f2e192f6700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 0x7f2e0006f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2e04005fd0 tx=0x7f2e04005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:47.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:47.977+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2e10057850 con 0x7f2e14101840 2024-04-12T06:05:48.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.189+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e140746d0 con 0x7f2e0006cce0 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.200+0000 7f2e0a7fc700 1 -- 172.21.15.8:0/3891661027 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f2e140746d0 con 0x7f2e0006cce0 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (7m) 77s ago 19m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (19m) 77s ago 19m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (18m) 47s ago 18m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (5m) 77s ago 19m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (5m) 47s ago 18m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (7m) 77s ago 18m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (83s) 77s ago 12m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:05:48.197 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (103s) 77s ago 13m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (51s) 47s ago 12m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (67s) 47s ago 12m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (9m) 77s ago 20m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (8m) 47s ago 17m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (5m) 77s ago 20m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (5m) 47s ago 17m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (8m) 77s ago 19m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (8m) 47s ago 17m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (4m) 77s ago 17m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (4m) 77s ago 16m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (3m) 77s ago 16m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (3m) 47s ago 15m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:05:48.198 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (2m) 47s ago 15m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (2m) 47s ago 14m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (7m) 77s ago 18m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 msgr2=0x7f2e0006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 0x7f2e0006f1a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2e04005fd0 tx=0x7f2e04005dc0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 msgr2=0x7f2e140735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000bda0 tx=0x7f2e1000be80 comp rx=0 tx=0).stop 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 shutdown_connections 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2e0006cce0 0x7f2e0006f1a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2e14100e90 0x7f2e14073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 --2- 172.21.15.8:0/3891661027 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2e14101840 0x7f2e140735d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 >> 172.21.15.8:0/3891661027 conn(0x7f2e140fc740 msgr2=0x7f2e140fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 shutdown_connections 2024-04-12T06:05:48.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.202+0000 7f2e1b55a700 1 -- 172.21.15.8:0/3891661027 wait complete. 2024-04-12T06:05:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3621628347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 msgr2=0x7f1db8105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3621628347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db8105840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009910 tx=0x7f1dac009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:48.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3621628347 shutdown_connections 2024-04-12T06:05:48.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3621628347 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db8105840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3621628347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 0x7f1db80ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3621628347 >> 172.21.15.8:0/3621628347 conn(0x7f1db8074e30 msgr2=0x7f1db8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:48.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.307+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3621628347 shutdown_connections 2024-04-12T06:05:48.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.308+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3621628347 wait complete. 2024-04-12T06:05:48.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.308+0000 7f1dbe1ac700 1 Processor -- start 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1dbe1ac700 1 -- start start 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1dbe1ac700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 0x7f1db819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1dbe1ac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1dbe1ac700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db819d260 con 0x7f1db80fff50 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1dbe1ac700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8196710 con 0x7f1db80ff5a0 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55654/0 (socket says 172.21.15.8:55654) 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 -- 172.21.15.8:0/3112879288 learned_addr learned my addr 172.21.15.8:0/3112879288 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db77fe700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 0x7f1db819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 -- 172.21.15.8:0/3112879288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 msgr2=0x7f1db819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 0x7f1db819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 -- 172.21.15.8:0/3112879288 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac0095f0 con 0x7f1db80fff50 2024-04-12T06:05:48.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.309+0000 7f1db6ffd700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0092e0 tx=0x7f1dac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.310+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dac01d070 con 0x7f1db80fff50 2024-04-12T06:05:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.310+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1dac005940 con 0x7f1db80fff50 2024-04-12T06:05:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.310+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db8196990 con 0x7f1db80fff50 2024-04-12T06:05:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.310+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dac00fe50 con 0x7f1db80fff50 2024-04-12T06:05:48.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.310+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db8196e80 con 0x7f1db80fff50 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.311+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db8109200 con 0x7f1db80fff50 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.311+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1dac003a60 con 0x7f1db80fff50 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.311+0000 7f1db4ff9700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 0x7f1da006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.312+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1dac08d7d0 con 0x7f1db80fff50 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.312+0000 7f1db77fe700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 0x7f1da006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.312+0000 7f1db77fe700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 0x7f1da006f1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005950 tx=0x7f1da80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:48.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.316+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1dac05b2d0 con 0x7f1db80fff50 2024-04-12T06:05:48.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.603+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1db804f2e0 con 0x7f1db80fff50 2024-04-12T06:05:48.600 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.604+0000 7f1db4ff9700 1 -- 172.21.15.8:0/3112879288 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1dac003d10 con 0x7f1db80fff50 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:05:48.601 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:05:48.602 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 msgr2=0x7f1da006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 0x7f1da006f1a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005950 tx=0x7f1da80058e0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 msgr2=0x7f1db819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0092e0 tx=0x7f1dac00be20 comp rx=0 tx=0).stop 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 shutdown_connections 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1da006cce0 0x7f1da006f1a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1db80ff5a0 0x7f1db819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 --2- 172.21.15.8:0/3112879288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1db80fff50 0x7f1db819cbd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 >> 172.21.15.8:0/3112879288 conn(0x7f1db8074e30 msgr2=0x7f1db80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 shutdown_connections 2024-04-12T06:05:48.603 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.606+0000 7f1dbe1ac700 1 -- 172.21.15.8:0/3112879288 wait complete. 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 -- 172.21.15.8:0/2573259312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 msgr2=0x7f7624101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 --2- 172.21.15.8:0/2573259312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f7624101280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7614009910 tx=0x7f7614009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 -- 172.21.15.8:0/2573259312 shutdown_connections 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 --2- 172.21.15.8:0/2573259312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 0x7f76241058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 --2- 172.21.15.8:0/2573259312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f7624101280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 -- 172.21.15.8:0/2573259312 >> 172.21.15.8:0/2573259312 conn(0x7f76240fc750 msgr2=0x7f76240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 -- 172.21.15.8:0/2573259312 shutdown_connections 2024-04-12T06:05:48.709 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.712+0000 7f762a44c700 1 -- 172.21.15.8:0/2573259312 wait complete. 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 Processor -- start 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 -- start start 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 0x7f762419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762419d130 con 0x7f7624100ea0 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f762a44c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241965e0 con 0x7f7624101850 2024-04-12T06:05:48.711 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55670/0 (socket says 172.21.15.8:55670) 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 -- 172.21.15.8:0/1004733871 learned_addr learned my addr 172.21.15.8:0/1004733871 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f76237fe700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 0x7f762419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 -- 172.21.15.8:0/1004733871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 msgr2=0x7f762419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 0x7f762419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.713+0000 7f7623fff700 1 -- 172.21.15.8:0/1004733871 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140095f0 con 0x7f7624100ea0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f7623fff700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f761400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f761401d070 con 0x7f7624100ea0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7614005dc0 con 0x7f7624100ea0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7624196860 con 0x7f7624100ea0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7614003940 con 0x7f7624100ea0 2024-04-12T06:05:48.712 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.714+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7624196cf0 con 0x7f7624100ea0 2024-04-12T06:05:48.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.715+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7624109240 con 0x7f7624100ea0 2024-04-12T06:05:48.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.715+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7614004ae0 con 0x7f7624100ea0 2024-04-12T06:05:48.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.716+0000 7f76217fa700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 0x7f760c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:48.713 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.716+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f761408dac0 con 0x7f7624100ea0 2024-04-12T06:05:48.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.716+0000 7f76237fe700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 0x7f760c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:48.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.716+0000 7f76237fe700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 0x7f760c06f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7624105b20 tx=0x7f7618009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:48.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.720+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f761405b5c0 con 0x7f7624100ea0 2024-04-12T06:05:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:48 smithi194 ceph-mon[106982]: pgmap v323: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:48 smithi194 ceph-mon[106982]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:48.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:48 smithi194 ceph-mon[106982]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:48.904 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:48 smithi008 ceph-mon[146084]: pgmap v323: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:48.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:48 smithi008 ceph-mon[146084]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:48.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:48 smithi008 ceph-mon[146084]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:48.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.965+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f762404f2e0 con 0x7f7624100ea0 2024-04-12T06:05:48.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.965+0000 7f76217fa700 1 -- 172.21.15.8:0/1004733871 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f7614026740 con 0x7f7624100ea0 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:05:48.963 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:05:48.964 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:48.965 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:05:48.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.967+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 msgr2=0x7f760c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.967+0000 7f762a44c700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 0x7f760c06f1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7624105b20 tx=0x7f7618009260 comp rx=0 tx=0).stop 2024-04-12T06:05:48.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.967+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 msgr2=0x7f762419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.967+0000 7f762a44c700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7614000c00 tx=0x7f761400bde0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 shutdown_connections 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f760c06cce0 0x7f760c06f1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7624100ea0 0x7f762419c560 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 --2- 172.21.15.8:0/1004733871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7624101850 0x7f762419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 >> 172.21.15.8:0/1004733871 conn(0x7f76240fc750 msgr2=0x7f76240fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 shutdown_connections 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:48.968+0000 7f762a44c700 1 -- 172.21.15.8:0/1004733871 wait complete. 2024-04-12T06:05:48.967 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:05:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.075+0000 7f3039435700 1 -- 172.21.15.8:0/1080034375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 msgr2=0x7f3034105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.075+0000 7f3039435700 1 --2- 172.21.15.8:0/1080034375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034105840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f301c009910 tx=0x7f301c009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:49.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- 172.21.15.8:0/1080034375 shutdown_connections 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 --2- 172.21.15.8:0/1080034375 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034105840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 --2- 172.21.15.8:0/1080034375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f30340ff980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- 172.21.15.8:0/1080034375 >> 172.21.15.8:0/1080034375 conn(0x7f3034074e30 msgr2=0x7f3034075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- 172.21.15.8:0/1080034375 shutdown_connections 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- 172.21.15.8:0/1080034375 wait complete. 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 Processor -- start 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- start start 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f3034100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034100fa0 con 0x7f30340fff50 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.076+0000 7f3039435700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30341010e0 con 0x7f30340ff5a0 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55690/0 (socket says 172.21.15.8:55690) 2024-04-12T06:05:49.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 -- 172.21.15.8:0/2990104052 learned_addr learned my addr 172.21.15.8:0/2990104052 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3032ffd700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f3034100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 -- 172.21.15.8:0/2990104052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 msgr2=0x7f3034100520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f3034100520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 -- 172.21.15.8:0/2990104052 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0095f0 con 0x7f30340fff50 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3032ffd700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f3034100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f302bfff700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f301c00bd70 tx=0x7f301c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:49.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c01d070 con 0x7f30340fff50 2024-04-12T06:05:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034101360 con 0x7f30340fff50 2024-04-12T06:05:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f301c00f460 con 0x7f30340fff50 2024-04-12T06:05:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c021640 con 0x7f30340fff50 2024-04-12T06:05:49.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.077+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341a2cc0 con 0x7f30340fff50 2024-04-12T06:05:49.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.078+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034109200 con 0x7f30340fff50 2024-04-12T06:05:49.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.079+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f301c00f9e0 con 0x7f30340fff50 2024-04-12T06:05:49.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.079+0000 7f3030ff9700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 0x7f302006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.079+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f301c08d780 con 0x7f30340fff50 2024-04-12T06:05:49.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.079+0000 7f3032ffd700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 0x7f302006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.080+0000 7f3032ffd700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 0x7f302006f1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3024005950 tx=0x7f30240058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:49.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.084+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f301c0577f0 con 0x7f30340fff50 2024-04-12T06:05:49.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.305+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30340611d0 con 0x7f302006cce0 2024-04-12T06:05:49.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.307+0000 7f3030ff9700 1 -- 172.21.15.8:0/2990104052 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f30340611d0 con 0x7f302006cce0 2024-04-12T06:05:49.303 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:05:49.303 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:05:49.304 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:05:49.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 msgr2=0x7f302006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 0x7f302006f1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3024005950 tx=0x7f30240058e0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 msgr2=0x7f3034100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f301c00bd70 tx=0x7f301c00be50 comp rx=0 tx=0).stop 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 shutdown_connections 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f302006cce0 0x7f302006f1a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30340ff5a0 0x7f3034100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 --2- 172.21.15.8:0/2990104052 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30340fff50 0x7f3034100a60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 >> 172.21.15.8:0/2990104052 conn(0x7f3034074e30 msgr2=0x7f30340fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 shutdown_connections 2024-04-12T06:05:49.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.309+0000 7f3039435700 1 -- 172.21.15.8:0/2990104052 wait complete. 2024-04-12T06:05:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 -- 172.21.15.8:0/2970839531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca300ffc20 msgr2=0x7fca30100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 --2- 172.21.15.8:0/2970839531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca300ffc20 0x7fca30100080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009910 tx=0x7fca1c009c20 comp rx=0 tx=0).stop 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 -- 172.21.15.8:0/2970839531 shutdown_connections 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 --2- 172.21.15.8:0/2970839531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca300ffc20 0x7fca30100080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 --2- 172.21.15.8:0/2970839531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca30105bd0 0x7fca30105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.415+0000 7fca36bc2700 1 -- 172.21.15.8:0/2970839531 >> 172.21.15.8:0/2970839531 conn(0x7fca30075670 msgr2=0x7fca30075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 -- 172.21.15.8:0/2970839531 shutdown_connections 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 -- 172.21.15.8:0/2970839531 wait complete. 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 Processor -- start 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 -- start start 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca300ffc20 0x7fca3019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019d150 con 0x7fca30105bd0 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca36bc2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca30196600 con 0x7fca300ffc20 2024-04-12T06:05:49.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.416+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55712/0 (socket says 172.21.15.8:55712) 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 -- 172.21.15.8:0/3745598976 learned_addr learned my addr 172.21.15.8:0/3745598976 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca3495e700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca300ffc20 0x7fca3019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 -- 172.21.15.8:0/3745598976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca300ffc20 msgr2=0x7fca3019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca300ffc20 0x7fca3019c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 -- 172.21.15.8:0/3745598976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c0095f0 con 0x7fca30105bd0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2ffff700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1c01d070 con 0x7fca30105bd0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca30196880 con 0x7fca30105bd0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca1c005710 con 0x7fca30105bd0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1c0039d0 con 0x7fca30105bd0 2024-04-12T06:05:49.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.417+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca30196d10 con 0x7fca30105bd0 2024-04-12T06:05:49.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.418+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3004f2e0 con 0x7fca30105bd0 2024-04-12T06:05:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.419+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fca1c003c50 con 0x7fca30105bd0 2024-04-12T06:05:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.419+0000 7fca2dffb700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 0x7fca2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:05:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.419+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca1c08dbb0 con 0x7fca30105bd0 2024-04-12T06:05:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.419+0000 7fca3495e700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 0x7fca2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:05:49.418 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.420+0000 7fca3495e700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 0x7fca2006f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca24009c60 tx=0x7fca24009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:05:49.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.424+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fca1c05b610 con 0x7fca30105bd0 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.720+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca30196fc0 con 0x7fca30105bd0 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.720+0000 7fca2dffb700 1 -- 172.21.15.8:0/3745598976 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fca1c05b1a0 con 0x7fca30105bd0 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:05:49.717 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:05:49.718 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 msgr2=0x7fca2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 0x7fca2006f1a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca24009c60 tx=0x7fca24009260 comp rx=0 tx=0).stop 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 msgr2=0x7fca3019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000c00 tx=0x7fca1c00bde0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 shutdown_connections 2024-04-12T06:05:49.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2006cce0 0x7fca2006f1a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca300ffc20 0x7fca3019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 --2- 172.21.15.8:0/3745598976 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca30105bd0 0x7fca3019cac0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:05:49.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 >> 172.21.15.8:0/3745598976 conn(0x7fca30075670 msgr2=0x7fca300686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:05:49.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 shutdown_connections 2024-04-12T06:05:49.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:05:49.722+0000 7fca36bc2700 1 -- 172.21.15.8:0/3745598976 wait complete. 2024-04-12T06:05:49.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:49 smithi194 ceph-mon[106982]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:49.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:49 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3112879288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:05:49.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:49 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1004733871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:05:49.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:49 smithi008 ceph-mon[146084]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:49.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:49 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3112879288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:05:49.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:49 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1004733871' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:05:50.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:50 smithi194 ceph-mon[106982]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:50.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:50 smithi194 ceph-mon[106982]: pgmap v324: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:50.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:50 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3745598976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:05:50.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:50 smithi008 ceph-mon[146084]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:05:50.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:50 smithi008 ceph-mon[146084]: pgmap v324: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:50.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:50 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3745598976' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:05:52.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:52 smithi194 ceph-mon[106982]: pgmap v325: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:52.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:52 smithi008 ceph-mon[146084]: pgmap v325: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:54.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:54 smithi194 ceph-mon[106982]: pgmap v326: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:54.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:54 smithi008 ceph-mon[146084]: pgmap v326: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:56.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:56 smithi194 ceph-mon[106982]: pgmap v327: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:56 smithi008 ceph-mon[146084]: pgmap v327: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:58.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:05:58 smithi194 ceph-mon[106982]: pgmap v328: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:05:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:05:58 smithi008 ceph-mon[146084]: pgmap v328: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:00.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:00 smithi194 ceph-mon[106982]: pgmap v329: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:00.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:00 smithi008 ceph-mon[146084]: pgmap v329: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:02.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:02 smithi194 ceph-mon[106982]: pgmap v330: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:02.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:02.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:02 smithi008 ceph-mon[146084]: pgmap v330: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:02.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:04.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:04 smithi194 ceph-mon[106982]: pgmap v331: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:04.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:04 smithi008 ceph-mon[146084]: pgmap v331: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:06.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:06 smithi194 ceph-mon[106982]: pgmap v332: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:06 smithi008 ceph-mon[146084]: pgmap v332: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:08.896 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:08 smithi008 ceph-mon[146084]: pgmap v333: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:09.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:08 smithi194 ceph-mon[106982]: pgmap v333: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:10.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:10 smithi008 ceph-mon[146084]: pgmap v334: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:11.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:10 smithi194 ceph-mon[106982]: pgmap v334: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:12.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:12 smithi008 ceph-mon[146084]: pgmap v335: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:13.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:12 smithi194 ceph-mon[106982]: pgmap v335: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:14.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:14 smithi008 ceph-mon[146084]: pgmap v336: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:15.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:14 smithi194 ceph-mon[106982]: pgmap v336: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:16 smithi008 ceph-mon[146084]: pgmap v337: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:17.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:16 smithi194 ceph-mon[106982]: pgmap v337: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:18.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:18.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:18 smithi008 ceph-mon[146084]: pgmap v338: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:19.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:18 smithi194 ceph-mon[106982]: pgmap v338: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:19.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.843+0000 7f2f8a080700 1 -- 172.21.15.8:0/2162486225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 msgr2=0x7f2f841070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:19.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.843+0000 7f2f8a080700 1 --2- 172.21.15.8:0/2162486225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f841070b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009910 tx=0x7f2f74009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:19.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 -- 172.21.15.8:0/2162486225 shutdown_connections 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 --2- 172.21.15.8:0/2162486225 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 0x7f2f840689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 --2- 172.21.15.8:0/2162486225 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f841070b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 -- 172.21.15.8:0/2162486225 >> 172.21.15.8:0/2162486225 conn(0x7f2f84075670 msgr2=0x7f2f84075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 -- 172.21.15.8:0/2162486225 shutdown_connections 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 -- 172.21.15.8:0/2162486225 wait complete. 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.844+0000 7f2f8a080700 1 Processor -- start 2024-04-12T06:06:19.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f8a080700 1 -- start start 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f8a080700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 0x7f2f8419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f8a080700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f8a080700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8419d130 con 0x7f2f84106cd0 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f8a080700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84196630 con 0x7f2f84068580 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55206/0 (socket says 172.21.15.8:55206) 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 -- 172.21.15.8:0/3112490401 learned_addr learned my addr 172.21.15.8:0/3112490401 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f837fe700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 0x7f2f8419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 -- 172.21.15.8:0/3112490401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 msgr2=0x7f2f8419c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 0x7f2f8419c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 -- 172.21.15.8:0/3112490401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f740095f0 con 0x7f2f84106cd0 2024-04-12T06:06:19.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.845+0000 7f2f82ffd700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d8d0 tx=0x7f2f7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.846+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f78009940 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.846+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f78010460 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.846+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84196910 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.846+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f7800f5f0 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.846+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f84196e60 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.847+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f841091d0 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.847+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2f780105d0 con 0x7f2f84106cd0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.848+0000 7f2f80ff9700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 0x7f2f6c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.848+0000 7f2f837fe700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 0x7f2f6c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:19.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.848+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2f7808c5d0 con 0x7f2f84106cd0 2024-04-12T06:06:19.847 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.848+0000 7f2f837fe700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 0x7f2f6c06f1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f74006010 tx=0x7f2f7401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:19.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:19.853+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2f7805a0d0 con 0x7f2f84106cd0 2024-04-12T06:06:20.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.076+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f84197080 con 0x7f2f6c06cce0 2024-04-12T06:06:20.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.077+0000 7f2f80ff9700 1 -- 172.21.15.8:0/3112490401 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2f84197080 con 0x7f2f6c06cce0 2024-04-12T06:06:20.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 msgr2=0x7f2f6c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 0x7f2f6c06f1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f74006010 tx=0x7f2f7401a040 comp rx=0 tx=0).stop 2024-04-12T06:06:20.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 msgr2=0x7f2f8419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d8d0 tx=0x7f2f7800dc90 comp rx=0 tx=0).stop 2024-04-12T06:06:20.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 shutdown_connections 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f6c06cce0 0x7f2f6c06f1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f84068580 0x7f2f8419c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 --2- 172.21.15.8:0/3112490401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f84106cd0 0x7f2f8419caa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 >> 172.21.15.8:0/3112490401 conn(0x7f2f84075670 msgr2=0x7f2f840fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 shutdown_connections 2024-04-12T06:06:20.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.079+0000 7f2f8a080700 1 -- 172.21.15.8:0/3112490401 wait complete. 2024-04-12T06:06:20.087 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:06:20.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 -- 172.21.15.8:0/1512614424 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380100ea0 msgr2=0x7f7380101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 --2- 172.21.15.8:0/1512614424 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380100ea0 0x7f7380101280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7374009910 tx=0x7f7374009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 -- 172.21.15.8:0/1512614424 shutdown_connections 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 --2- 172.21.15.8:0/1512614424 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380101850 0x7f73801058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 --2- 172.21.15.8:0/1512614424 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380100ea0 0x7f7380101280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 -- 172.21.15.8:0/1512614424 >> 172.21.15.8:0/1512614424 conn(0x7f73800fc750 msgr2=0x7f73800feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 -- 172.21.15.8:0/1512614424 shutdown_connections 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.187+0000 7f7386dcb700 1 -- 172.21.15.8:0/1512614424 wait complete. 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 Processor -- start 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 -- start start 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 0x7f7380073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380073b10 con 0x7f7380101850 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7386dcb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7380073c50 con 0x7f7380100ea0 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55214/0 (socket says 172.21.15.8:55214) 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 -- 172.21.15.8:0/4062429312 learned_addr learned my addr 172.21.15.8:0/4062429312 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7384b67700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 0x7f7380073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 -- 172.21.15.8:0/4062429312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 msgr2=0x7f7380073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 0x7f7380073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 -- 172.21.15.8:0/4062429312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740095f0 con 0x7f7380101850 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f7384b67700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 0x7f7380073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.188+0000 7f737ffff700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e8f0 tx=0x7f736c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.189+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c009790 con 0x7f7380101850 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.189+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f736c004cd0 con 0x7f7380101850 2024-04-12T06:06:20.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.189+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73801a2960 con 0x7f7380101850 2024-04-12T06:06:20.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.189+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736c005580 con 0x7f7380101850 2024-04-12T06:06:20.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.189+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73801a2d70 con 0x7f7380101850 2024-04-12T06:06:20.188 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.190+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7380109240 con 0x7f7380101850 2024-04-12T06:06:20.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.190+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f736c004750 con 0x7f7380101850 2024-04-12T06:06:20.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.190+0000 7f737dffb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 0x7f737006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.190+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f736c014070 con 0x7f7380101850 2024-04-12T06:06:20.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.191+0000 7f7384b67700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 0x7f737006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.191+0000 7f7384b67700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 0x7f737006f1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f737400b6e0 tx=0x7f7374009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.195+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f736c056990 con 0x7f7380101850 2024-04-12T06:06:20.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.416+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73800611d0 con 0x7f737006cce0 2024-04-12T06:06:20.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.417+0000 7f737dffb700 1 -- 172.21.15.8:0/4062429312 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f73800611d0 con 0x7f737006cce0 2024-04-12T06:06:20.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 msgr2=0x7f737006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 0x7f737006f1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f737400b6e0 tx=0x7f7374009f70 comp rx=0 tx=0).stop 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 msgr2=0x7f73800735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f736c00e8f0 tx=0x7f736c00ecb0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 shutdown_connections 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f737006cce0 0x7f737006f1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7380100ea0 0x7f7380073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 --2- 172.21.15.8:0/4062429312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7380101850 0x7f73800735d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 >> 172.21.15.8:0/4062429312 conn(0x7f73800fc750 msgr2=0x7f73800fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.419+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 shutdown_connections 2024-04-12T06:06:20.417 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.420+0000 7f7386dcb700 1 -- 172.21.15.8:0/4062429312 wait complete. 2024-04-12T06:06:20.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 -- 172.21.15.8:0/2209071485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240068580 msgr2=0x7f72400689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 --2- 172.21.15.8:0/2209071485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240068580 0x7f72400689e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7228009910 tx=0x7f7228009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:20.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 -- 172.21.15.8:0/2209071485 shutdown_connections 2024-04-12T06:06:20.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 --2- 172.21.15.8:0/2209071485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240068580 0x7f72400689e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 --2- 172.21.15.8:0/2209071485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240106cd0 0x7f72401070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 -- 172.21.15.8:0/2209071485 >> 172.21.15.8:0/2209071485 conn(0x7f7240075670 msgr2=0x7f7240075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 -- 172.21.15.8:0/2209071485 shutdown_connections 2024-04-12T06:06:20.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.528+0000 7f7244aee700 1 -- 172.21.15.8:0/2209071485 wait complete. 2024-04-12T06:06:20.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 Processor -- start 2024-04-12T06:06:20.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 -- start start 2024-04-12T06:06:20.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240068580 0x7f724019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724019d110 con 0x7f7240106cd0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7244aee700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72401965c0 con 0x7f7240068580 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7237fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7237fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55222/0 (socket says 172.21.15.8:55222) 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f7237fff700 1 -- 172.21.15.8:0/1245707075 learned_addr learned my addr 172.21.15.8:0/1245707075 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.529+0000 7f723e59c700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240068580 0x7f724019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7237fff700 1 -- 172.21.15.8:0/1245707075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240068580 msgr2=0x7f724019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7237fff700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240068580 0x7f724019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7237fff700 1 -- 172.21.15.8:0/1245707075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72280095f0 con 0x7f7240106cd0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7237fff700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72280092e0 tx=0x7f722800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722801d070 con 0x7f7240106cd0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7240196840 con 0x7f7240106cd0 2024-04-12T06:06:20.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7228005710 con 0x7f7240106cd0 2024-04-12T06:06:20.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72280039d0 con 0x7f7240106cd0 2024-04-12T06:06:20.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.530+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7240196cd0 con 0x7f7240106cd0 2024-04-12T06:06:20.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.531+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72401091d0 con 0x7f7240106cd0 2024-04-12T06:06:20.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.531+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7228004b70 con 0x7f7240106cd0 2024-04-12T06:06:20.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.532+0000 7f72377fe700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 0x7f722c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.532+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f722808db50 con 0x7f7240106cd0 2024-04-12T06:06:20.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.532+0000 7f723e59c700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 0x7f722c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.532+0000 7f723e59c700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 0x7f722c06f1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7230009c60 tx=0x7f7230009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.537+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f722805b650 con 0x7f7240106cd0 2024-04-12T06:06:20.747 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.750+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7240196f50 con 0x7f722c06cce0 2024-04-12T06:06:20.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.761+0000 7f72377fe700 1 -- 172.21.15.8:0/1245707075 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7240196f50 con 0x7f722c06cce0 2024-04-12T06:06:20.758 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:06:20.758 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (8m) 109s ago 19m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:06:20.758 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (19m) 109s ago 19m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (18m) 80s ago 18m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (5m) 109s ago 19m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (5m) 80s ago 18m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (7m) 109s ago 19m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (116s) 109s ago 13m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (2m) 109s ago 13m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (83s) 80s ago 13m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (99s) 80s ago 13m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (10m) 109s ago 21m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (9m) 80s ago 18m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (6m) 109s ago 21m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (6m) 80s ago 18m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (9m) 109s ago 19m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:06:20.759 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (8m) 80s ago 18m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (5m) 109s ago 17m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (4m) 109s ago 17m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (4m) 109s ago 16m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (3m) 80s ago 16m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (3m) 80s ago 15m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (2m) 80s ago 15m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:06:20.760 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (8m) 109s ago 19m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 msgr2=0x7f722c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 0x7f722c06f1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7230009c60 tx=0x7f7230009260 comp rx=0 tx=0).stop 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 msgr2=0x7f724019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f72280092e0 tx=0x7f722800bde0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 shutdown_connections 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f722c06cce0 0x7f722c06f1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7240068580 0x7f724019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 --2- 172.21.15.8:0/1245707075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7240106cd0 0x7f724019ca80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.763+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 >> 172.21.15.8:0/1245707075 conn(0x7f7240075670 msgr2=0x7f72400fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.764+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 shutdown_connections 2024-04-12T06:06:20.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.764+0000 7f7244aee700 1 -- 172.21.15.8:0/1245707075 wait complete. 2024-04-12T06:06:20.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 -- 172.21.15.8:0/643095765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba80ffb40 msgr2=0x7feba80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 --2- 172.21.15.8:0/643095765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba80ffb40 0x7feba80fffc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009910 tx=0x7feb9c009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:20.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 -- 172.21.15.8:0/643095765 shutdown_connections 2024-04-12T06:06:20.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 --2- 172.21.15.8:0/643095765 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba80ffb40 0x7feba80fffc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 --2- 172.21.15.8:0/643095765 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba8105bf0 0x7feba8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 -- 172.21.15.8:0/643095765 >> 172.21.15.8:0/643095765 conn(0x7feba80fb610 msgr2=0x7feba80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:20.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 -- 172.21.15.8:0/643095765 shutdown_connections 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.869+0000 7febb01c7700 1 -- 172.21.15.8:0/643095765 wait complete. 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 Processor -- start 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 -- start start 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 0x7feba819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba819d0c0 con 0x7feba8105bf0 2024-04-12T06:06:20.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febb01c7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8196570 con 0x7feba80ffb40 2024-04-12T06:06:20.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55242/0 (socket says 172.21.15.8:55242) 2024-04-12T06:06:20.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 -- 172.21.15.8:0/437819793 learned_addr learned my addr 172.21.15.8:0/437819793 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:20.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febadf63700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 0x7feba819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 -- 172.21.15.8:0/437819793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 msgr2=0x7feba819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 0x7feba819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febad762700 1 -- 172.21.15.8:0/437819793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0095f0 con 0x7feba8105bf0 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.870+0000 7febadf63700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 0x7feba819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7febad762700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0092e0 tx=0x7feb9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9c01d070 con 0x7feba8105bf0 2024-04-12T06:06:20.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feb9c005e00 con 0x7feba8105bf0 2024-04-12T06:06:20.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba81967f0 con 0x7feba8105bf0 2024-04-12T06:06:20.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9c00fe60 con 0x7feba8105bf0 2024-04-12T06:06:20.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.871+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba8196ce0 con 0x7feba8105bf0 2024-04-12T06:06:20.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.872+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba804f2e0 con 0x7feba8105bf0 2024-04-12T06:06:20.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.873+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7feb9c00f6d0 con 0x7feba8105bf0 2024-04-12T06:06:20.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.873+0000 7feb9affd700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 0x7feb9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:20.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.873+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb9c08db50 con 0x7feba8105bf0 2024-04-12T06:06:20.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.873+0000 7febadf63700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 0x7feb9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:20.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.874+0000 7febadf63700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 0x7feb9406f1a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feba400bc10 tx=0x7feba400b5f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:20.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:20.877+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7feb9c08de60 con 0x7feba8105bf0 2024-04-12T06:06:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:20 smithi008 ceph-mon[146084]: pgmap v339: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:21.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:20 smithi194 ceph-mon[106982]: pgmap v339: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.165+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feba8197a80 con 0x7feba8105bf0 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.165+0000 7feb9affd700 1 -- 172.21.15.8:0/437819793 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7feb9c05b6d0 con 0x7feba8105bf0 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:06:21.163 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:06:21.164 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:06:21.165 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:06:21.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.167+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 msgr2=0x7feb9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.165 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.167+0000 7febb01c7700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 0x7feb9406f1a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feba400bc10 tx=0x7feba400b5f0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.167+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 msgr2=0x7feba819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.167+0000 7febb01c7700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0092e0 tx=0x7feb9c00be20 comp rx=0 tx=0).stop 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 shutdown_connections 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feb9406cce0 0x7feb9406f1a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feba80ffb40 0x7feba819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 --2- 172.21.15.8:0/437819793 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feba8105bf0 0x7feba819ca30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 >> 172.21.15.8:0/437819793 conn(0x7feba80fb610 msgr2=0x7feba8068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 shutdown_connections 2024-04-12T06:06:21.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.168+0000 7febb01c7700 1 -- 172.21.15.8:0/437819793 wait complete. 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/3868929361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 msgr2=0x7f4e68100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/3868929361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e68100180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009910 tx=0x7f4e5c009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/3868929361 shutdown_connections 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/3868929361 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e681047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/3868929361 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e68100180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/3868929361 >> 172.21.15.8:0/3868929361 conn(0x7f4e680fb650 msgr2=0x7f4e680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/3868929361 shutdown_connections 2024-04-12T06:06:21.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/3868929361 wait complete. 2024-04-12T06:06:21.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 Processor -- start 2024-04-12T06:06:21.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.274+0000 7f4e6d7d7700 1 -- start start 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e6d7d7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e6d7d7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e6819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e6d7d7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6819d140 con 0x7f4e680ffda0 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e6d7d7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e681965f0 con 0x7f4e68100750 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55262/0 (socket says 172.21.15.8:55262) 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 -- 172.21.15.8:0/2268967831 learned_addr learned my addr 172.21.15.8:0/2268967831 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 -- 172.21.15.8:0/2268967831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 msgr2=0x7f4e6819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e667fc700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e6819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e6819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 -- 172.21.15.8:0/2268967831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c0095f0 con 0x7f4e680ffda0 2024-04-12T06:06:21.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e667fc700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e6819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.275+0000 7f4e66ffd700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006010 tx=0x7f4e5c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.276+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5c01d070 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.276+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4e5c00f460 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.276+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e68196870 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.276+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5c021620 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.276+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e68196d60 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.277+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6804f2e0 con 0x7f4e680ffda0 2024-04-12T06:06:21.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.277+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4e5c02b440 con 0x7f4e680ffda0 2024-04-12T06:06:21.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.277+0000 7f4e4ffff700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 0x7f4e5006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.277+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4e5c08d790 con 0x7f4e680ffda0 2024-04-12T06:06:21.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.277+0000 7f4e667fc700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 0x7f4e5006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.278+0000 7f4e667fc700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 0x7f4e5006f1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e58005fd0 tx=0x7f4e58005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.282+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4e5c05b300 con 0x7f4e680ffda0 2024-04-12T06:06:21.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.532+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4e68197020 con 0x7f4e680ffda0 2024-04-12T06:06:21.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.532+0000 7f4e4ffff700 1 -- 172.21.15.8:0/2268967831 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4e5c026740 con 0x7f4e680ffda0 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:06:21.530 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:06:21.531 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:21.532 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 msgr2=0x7f4e5006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 0x7f4e5006f1a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4e58005fd0 tx=0x7f4e58005ee0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 msgr2=0x7f4e6819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006010 tx=0x7f4e5c00be20 comp rx=0 tx=0).stop 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 shutdown_connections 2024-04-12T06:06:21.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4e5006cce0 0x7f4e5006f1a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4e680ffda0 0x7f4e6819c570 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.534+0000 7f4e6d7d7700 1 --2- 172.21.15.8:0/2268967831 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e68100750 0x7f4e6819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.535+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 >> 172.21.15.8:0/2268967831 conn(0x7f4e680fb650 msgr2=0x7f4e68103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.535+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 shutdown_connections 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.535+0000 7f4e6d7d7700 1 -- 172.21.15.8:0/2268967831 wait complete. 2024-04-12T06:06:21.534 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:06:21.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/2325664720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8068580 msgr2=0x7fd2e80689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/2325664720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8068580 0x7fd2e80689e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc009910 tx=0x7fd2dc009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/2325664720 shutdown_connections 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/2325664720 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8068580 0x7fd2e80689e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/2325664720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8106cd0 0x7fd2e81070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/2325664720 >> 172.21.15.8:0/2325664720 conn(0x7fd2e8075670 msgr2=0x7fd2e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/2325664720 shutdown_connections 2024-04-12T06:06:21.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.642+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/2325664720 wait complete. 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.643+0000 7fd2ed1b1700 1 Processor -- start 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.643+0000 7fd2ed1b1700 1 -- start start 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2ed1b1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8068580 0x7fd2e819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2ed1b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2ed1b1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e819d110 con 0x7fd2e8106cd0 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2ed1b1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e81965c0 con 0x7fd2e8068580 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55284/0 (socket says 172.21.15.8:55284) 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 -- 172.21.15.8:0/117760466 learned_addr learned my addr 172.21.15.8:0/117760466 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e6d9d700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8068580 0x7fd2e819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 -- 172.21.15.8:0/117760466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8068580 msgr2=0x7fd2e819c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8068580 0x7fd2e819c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 -- 172.21.15.8:0/117760466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0095f0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2e659c700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00bde0 tx=0x7fd2dc00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2dc01d070 con 0x7fd2e8106cd0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd2dc0044c0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.644+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e8196840 con 0x7fd2e8106cd0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.645+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2dc003ab0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.645+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e8196cd0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.646+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e81091d0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.646+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd2dc0048e0 con 0x7fd2e8106cd0 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.646+0000 7fd2cffff700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 0x7fd2d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.647+0000 7fd2e6d9d700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 0x7fd2d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.647+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd2dc08dc40 con 0x7fd2e8106cd0 2024-04-12T06:06:21.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.647+0000 7fd2e6d9d700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 0x7fd2d006f1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009bf0 tx=0x7fd2d8009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.651+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd2dc05b800 con 0x7fd2e8106cd0 2024-04-12T06:06:21.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.872+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2e8196f50 con 0x7fd2d006cce0 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.873+0000 7fd2cffff700 1 -- 172.21.15.8:0/117760466 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd2e8196f50 con 0x7fd2d006cce0 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:06:21.870 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:06:21.871 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:06:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 msgr2=0x7fd2d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 0x7fd2d006f1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8009bf0 tx=0x7fd2d8009260 comp rx=0 tx=0).stop 2024-04-12T06:06:21.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 msgr2=0x7fd2e819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00bde0 tx=0x7fd2dc00bec0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 shutdown_connections 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd2d006cce0 0x7fd2d006f1a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd2e8068580 0x7fd2e819c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 --2- 172.21.15.8:0/117760466 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd2e8106cd0 0x7fd2e819ca80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 >> 172.21.15.8:0/117760466 conn(0x7fd2e8075670 msgr2=0x7fd2e80fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 shutdown_connections 2024-04-12T06:06:21.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.875+0000 7fd2ed1b1700 1 -- 172.21.15.8:0/117760466 wait complete. 2024-04-12T06:06:21.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:21 smithi008 ceph-mon[146084]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:21.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:21 smithi008 ceph-mon[146084]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:21.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:21 smithi008 ceph-mon[146084]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:21.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:21 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/437819793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:06:21.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:21 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2268967831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 -- 172.21.15.8:0/3842983433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45780ffc20 msgr2=0x7f4578100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3842983433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45780ffc20 0x7f4578100080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f456c009990 tx=0x7f456c009ca0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 -- 172.21.15.8:0/3842983433 shutdown_connections 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3842983433 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f45780ffc20 0x7f4578100080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3842983433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4578105bd0 0x7f4578105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 -- 172.21.15.8:0/3842983433 >> 172.21.15.8:0/3842983433 conn(0x7f4578075670 msgr2=0x7f4578075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:21.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 -- 172.21.15.8:0/3842983433 shutdown_connections 2024-04-12T06:06:21.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.981+0000 7f457d1d4700 1 -- 172.21.15.8:0/3842983433 wait complete. 2024-04-12T06:06:21.980 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.982+0000 7f457d1d4700 1 Processor -- start 2024-04-12T06:06:21.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.982+0000 7f457d1d4700 1 -- start start 2024-04-12T06:06:21.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 0x7f457819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457819f400 con 0x7f4578105bd0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45781988b0 con 0x7f45780ffc20 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55308/0 (socket says 172.21.15.8:55308) 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 -- 172.21.15.8:0/3229096818 learned_addr learned my addr 172.21.15.8:0/3229096818 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f4576d9d700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 0x7f457819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 -- 172.21.15.8:0/3229096818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 msgr2=0x7f457819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 0x7f457819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 -- 172.21.15.8:0/3229096818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c0095f0 con 0x7f4578105bd0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f4576d9d700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 0x7f457819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457659c700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f456c018040 tx=0x7f456c005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c0041f0 con 0x7f4578105bd0 2024-04-12T06:06:21.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4578198b30 con 0x7f4578105bd0 2024-04-12T06:06:21.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f456c004d80 con 0x7f4578105bd0 2024-04-12T06:06:21.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f456c003d40 con 0x7f4578105bd0 2024-04-12T06:06:21.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.983+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4578199080 con 0x7f4578105bd0 2024-04-12T06:06:21.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.984+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45781080d0 con 0x7f4578105bd0 2024-04-12T06:06:21.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.985+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f456c0048e0 con 0x7f4578105bd0 2024-04-12T06:06:21.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.985+0000 7f455ffff700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 0x7f456006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:21.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.986+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f456c08ea20 con 0x7f4578105bd0 2024-04-12T06:06:21.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.986+0000 7f4576d9d700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 0x7f456006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:21.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.986+0000 7f4576d9d700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 0x7f456006f1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4568005950 tx=0x7f45680058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:21.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:21.990+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f456c05c520 con 0x7f4578105bd0 2024-04-12T06:06:22.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:21 smithi194 ceph-mon[106982]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:22.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:21 smithi194 ceph-mon[106982]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:22.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:21 smithi194 ceph-mon[106982]: from='client.34462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:22.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:21 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/437819793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:06:22.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:21 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2268967831' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:06:22.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.277+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f457804f2e0 con 0x7f4578105bd0 2024-04-12T06:06:22.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.277+0000 7f455ffff700 1 -- 172.21.15.8:0/3229096818 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f456c027e90 con 0x7f4578105bd0 2024-04-12T06:06:22.275 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:06:22.275 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:06:22.275 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:06:22.275 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:06:22.275 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:06:22.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.279+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 msgr2=0x7f456006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:22.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.279+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 0x7f456006f1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4568005950 tx=0x7f45680058e0 comp rx=0 tx=0).stop 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.279+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 msgr2=0x7f457819ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.279+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f456c018040 tx=0x7f456c005590 comp rx=0 tx=0).stop 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 shutdown_connections 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f456006cce0 0x7f456006f1a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45780ffc20 0x7f457819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 --2- 172.21.15.8:0/3229096818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4578105bd0 0x7f457819ed70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 >> 172.21.15.8:0/3229096818 conn(0x7f4578075670 msgr2=0x7f45780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 shutdown_connections 2024-04-12T06:06:22.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:22.280+0000 7f457d1d4700 1 -- 172.21.15.8:0/3229096818 wait complete. 2024-04-12T06:06:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:22 smithi008 ceph-mon[146084]: pgmap v340: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:22 smithi008 ceph-mon[146084]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:22.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:22 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3229096818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:06:23.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:22 smithi194 ceph-mon[106982]: pgmap v340: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:23.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:22 smithi194 ceph-mon[106982]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:23.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:22 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3229096818' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:06:24.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:24 smithi008 ceph-mon[146084]: pgmap v341: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:25.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:24 smithi194 ceph-mon[106982]: pgmap v341: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:26.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:26 smithi008 ceph-mon[146084]: pgmap v342: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:27.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:26 smithi194 ceph-mon[106982]: pgmap v342: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:28 smithi008 ceph-mon[146084]: pgmap v343: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:29.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:28 smithi194 ceph-mon[106982]: pgmap v343: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:30.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:30 smithi008 ceph-mon[146084]: pgmap v344: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:31.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:30 smithi194 ceph-mon[106982]: pgmap v344: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:32.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:32 smithi008 ceph-mon[146084]: pgmap v345: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:32.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:33.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:32 smithi194 ceph-mon[106982]: pgmap v345: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:33.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:34 smithi008 ceph-mon[146084]: pgmap v346: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:35.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:34 smithi194 ceph-mon[106982]: pgmap v346: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:36 smithi008 ceph-mon[146084]: pgmap v347: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:37.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:36 smithi194 ceph-mon[106982]: pgmap v347: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:38.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:38 smithi008 ceph-mon[146084]: pgmap v348: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:39.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:38 smithi194 ceph-mon[106982]: pgmap v348: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:40 smithi008 ceph-mon[146084]: pgmap v349: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:41.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:40 smithi194 ceph-mon[106982]: pgmap v349: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:42.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:42 smithi008 ceph-mon[146084]: pgmap v350: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:43.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:42 smithi194 ceph-mon[106982]: pgmap v350: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:44.893 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:44 smithi194 ceph-mon[106982]: pgmap v351: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:44.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:44 smithi008 ceph-mon[146084]: pgmap v351: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:46.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:46 smithi008 ceph-mon[146084]: pgmap v352: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:47.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:46 smithi194 ceph-mon[106982]: pgmap v352: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:48.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:06:48.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:48 smithi008 ceph-mon[146084]: pgmap v353: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:49.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:48 smithi194 ceph-mon[106982]: pgmap v353: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:50.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:50 smithi008 ceph-mon[146084]: pgmap v354: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:51.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:50 smithi194 ceph-mon[106982]: pgmap v354: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:52.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3238676313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 msgr2=0x7fb0bc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.398 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3238676313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc101280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009910 tx=0x7fb0ac009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3238676313 shutdown_connections 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3238676313 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 0x7fb0bc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3238676313 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc101280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3238676313 >> 172.21.15.8:0/3238676313 conn(0x7fb0bc0fc750 msgr2=0x7fb0bc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3238676313 shutdown_connections 2024-04-12T06:06:52.399 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.401+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3238676313 wait complete. 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 Processor -- start 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 -- start start 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 0x7fb0bc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc073b10 con 0x7fb0bc100ea0 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0c0b74700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc073c50 con 0x7fb0bc101850 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35176/0 (socket says 172.21.15.8:35176) 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 -- 172.21.15.8:0/3084120123 learned_addr learned my addr 172.21.15.8:0/3084120123 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0b9d9b700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 0x7fb0bc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 -- 172.21.15.8:0/3084120123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 msgr2=0x7fb0bc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 0x7fb0bc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.402+0000 7fb0ba59c700 1 -- 172.21.15.8:0/3084120123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0095f0 con 0x7fb0bc100ea0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0ba59c700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0092e0 tx=0x7fb0ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ac01d070 con 0x7fb0bc100ea0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc073ed0 con 0x7fb0bc100ea0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb0ac005e00 con 0x7fb0bc100ea0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0ac00fe90 con 0x7fb0bc100ea0 2024-04-12T06:06:52.402 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.403+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc1a2c90 con 0x7fb0bc100ea0 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.404+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc109240 con 0x7fb0bc100ea0 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.405+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb0ac00f6d0 con 0x7fb0bc100ea0 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.405+0000 7fb0ab7fe700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 0x7fb0a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.405+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb0ac08d780 con 0x7fb0bc100ea0 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.405+0000 7fb0b9d9b700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 0x7fb0a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.406+0000 7fb0b9d9b700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 0x7fb0a406f1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc074880 tx=0x7fb0b000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:52.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.410+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb0ac0577f0 con 0x7fb0bc100ea0 2024-04-12T06:06:52.625 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.628+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0bc0611d0 con 0x7fb0a406cce0 2024-04-12T06:06:52.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.630+0000 7fb0ab7fe700 1 -- 172.21.15.8:0/3084120123 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb0bc0611d0 con 0x7fb0a406cce0 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.631+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 msgr2=0x7fb0a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.631+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 0x7fb0a406f1a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc074880 tx=0x7fb0b000b410 comp rx=0 tx=0).stop 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.631+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 msgr2=0x7fb0bc073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.631+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0092e0 tx=0x7fb0ac00bd70 comp rx=0 tx=0).stop 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 shutdown_connections 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb0a406cce0 0x7fb0a406f1a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0bc100ea0 0x7fb0bc073090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 --2- 172.21.15.8:0/3084120123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0bc101850 0x7fb0bc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 >> 172.21.15.8:0/3084120123 conn(0x7fb0bc0fc750 msgr2=0x7fb0bc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 shutdown_connections 2024-04-12T06:06:52.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.632+0000 7fb0c0b74700 1 -- 172.21.15.8:0/3084120123 wait complete. 2024-04-12T06:06:52.639 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:06:52.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 -- 172.21.15.8:0/293036973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 msgr2=0x7f4b3c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/293036973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c100080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009910 tx=0x7f4b30009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 -- 172.21.15.8:0/293036973 shutdown_connections 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/293036973 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c100080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/293036973 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 0x7f4b3c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 -- 172.21.15.8:0/293036973 >> 172.21.15.8:0/293036973 conn(0x7f4b3c075670 msgr2=0x7f4b3c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 -- 172.21.15.8:0/293036973 shutdown_connections 2024-04-12T06:06:52.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.740+0000 7f4b43a4c700 1 -- 172.21.15.8:0/293036973 wait complete. 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 Processor -- start 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 -- start start 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 0x7f4b3c194b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c195270 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b43a4c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c199000 con 0x7f4b3c105bd0 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b417e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b417e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35192/0 (socket says 172.21.15.8:35192) 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b417e8700 1 -- 172.21.15.8:0/509301629 learned_addr learned my addr 172.21.15.8:0/509301629 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b40fe7700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 0x7f4b3c194b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b417e8700 1 -- 172.21.15.8:0/509301629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 msgr2=0x7f4b3c194b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.741+0000 7f4b417e8700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 0x7f4b3c194b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b417e8700 1 -- 172.21.15.8:0/509301629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b417e8700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e920 tx=0x7f4b3800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3800c9d0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c1992e0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b38010430 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b38018650 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.742+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c199830 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.743+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4b380105a0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.743+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c1080d0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.744+0000 7f4b2e7fc700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 0x7f4b2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.744+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b38014070 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.744+0000 7f4b40fe7700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 0x7f4b2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:52.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.744+0000 7f4b40fe7700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 0x7f4b2806f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005170 tx=0x7f4b30005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:52.745 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.748+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4b38052ab0 con 0x7f4b3c0ffc20 2024-04-12T06:06:52.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:52 smithi008 ceph-mon[146084]: pgmap v355: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:52.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.969+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b3c199a50 con 0x7f4b2806cce0 2024-04-12T06:06:52.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.970+0000 7f4b2e7fc700 1 -- 172.21.15.8:0/509301629 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4b3c199a50 con 0x7f4b2806cce0 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 msgr2=0x7f4b2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 0x7f4b2806f1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b30005170 tx=0x7f4b30005fd0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 msgr2=0x7f4b3c194650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800e920 tx=0x7f4b3800ec30 comp rx=0 tx=0).stop 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 shutdown_connections 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b2806cce0 0x7f4b2806f1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b3c0ffc20 0x7f4b3c194650 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 --2- 172.21.15.8:0/509301629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b3c105bd0 0x7f4b3c194b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:52.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 >> 172.21.15.8:0/509301629 conn(0x7f4b3c075670 msgr2=0x7f4b3c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:52.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 shutdown_connections 2024-04-12T06:06:52.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:52.973+0000 7f4b43a4c700 1 -- 172.21.15.8:0/509301629 wait complete. 2024-04-12T06:06:53.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:52 smithi194 ceph-mon[106982]: pgmap v355: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:53.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 -- 172.21.15.8:0/4046702066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 msgr2=0x7f9c24101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/4046702066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24101270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009910 tx=0x7f9c18009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 -- 172.21.15.8:0/4046702066 shutdown_connections 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/4046702066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 0x7f9c24105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/4046702066 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24101270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 -- 172.21.15.8:0/4046702066 >> 172.21.15.8:0/4046702066 conn(0x7f9c240fc740 msgr2=0x7f9c240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 -- 172.21.15.8:0/4046702066 shutdown_connections 2024-04-12T06:06:53.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.079+0000 7f9c2adcb700 1 -- 172.21.15.8:0/4046702066 wait complete. 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 Processor -- start 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 -- start start 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 0x7f9c240735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24073b10 con 0x7f9c24100e90 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c2adcb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24073c50 con 0x7f9c24101840 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c28b67700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c28b67700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:35202/0 (socket says 172.21.15.8:35202) 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.080+0000 7f9c28b67700 1 -- 172.21.15.8:0/2973868279 learned_addr learned my addr 172.21.15.8:0/2973868279 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c23fff700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 0x7f9c240735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c28b67700 1 -- 172.21.15.8:0/2973868279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 msgr2=0x7f9c240735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c28b67700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 0x7f9c240735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c28b67700 1 -- 172.21.15.8:0/2973868279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180095f0 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c28b67700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c1801d070 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c1800f460 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24073ed0 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c18021640 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.081+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c241a2cf0 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.082+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c24109230 con 0x7f9c24100e90 2024-04-12T06:06:53.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.083+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9c1800fa60 con 0x7f9c24100e90 2024-04-12T06:06:53.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.083+0000 7f9c21ffb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 0x7f9c1406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.083+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c1808d850 con 0x7f9c24100e90 2024-04-12T06:06:53.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.083+0000 7f9c23fff700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 0x7f9c1406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.084+0000 7f9c23fff700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 0x7f9c1406f1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005fd0 tx=0x7f9c1000a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.088+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9c180578c0 con 0x7f9c24100e90 2024-04-12T06:06:53.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.300+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c240611d0 con 0x7f9c1406cce0 2024-04-12T06:06:53.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.311+0000 7f9c21ffb700 1 -- 172.21.15.8:0/2973868279 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9c240611d0 con 0x7f9c1406cce0 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (8m) 2m ago 20m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (20m) 2m ago 20m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (19m) 112s ago 19m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (6m) 2m ago 20m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (6m) 112s ago 19m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (8m) 2m ago 19m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (2m) 2m ago 13m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (2m) 2m ago 14m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (116s) 112s ago 13m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (2m) 112s ago 14m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (10m) 2m ago 21m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:06:53.309 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (9m) 112s ago 19m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (7m) 2m ago 21m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (6m) 112s ago 18m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (9m) 2m ago 20m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (9m) 112s ago 19m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (6m) 2m ago 18m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (5m) 2m ago 17m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (4m) 2m ago 17m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (4m) 112s ago 16m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (3m) 112s ago 16m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (3m) 112s ago 15m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:06:53.310 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (9m) 2m ago 19m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 msgr2=0x7f9c1406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 0x7f9c1406f1a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005fd0 tx=0x7f9c1000a560 comp rx=0 tx=0).stop 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 msgr2=0x7f9c24073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c1800be20 comp rx=0 tx=0).stop 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 shutdown_connections 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c1406cce0 0x7f9c1406f1a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9c24100e90 0x7f9c24073090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 --2- 172.21.15.8:0/2973868279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c24101840 0x7f9c240735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.313+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 >> 172.21.15.8:0/2973868279 conn(0x7f9c240fc740 msgr2=0x7f9c240fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:53.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.314+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 shutdown_connections 2024-04-12T06:06:53.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.314+0000 7f9c2adcb700 1 -- 172.21.15.8:0/2973868279 wait complete. 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 -- 172.21.15.8:0/172201868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 msgr2=0x7f3fa8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/172201868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8101280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009910 tx=0x7f3f98009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 -- 172.21.15.8:0/172201868 shutdown_connections 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/172201868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/172201868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8101280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.418+0000 7f3fad9d5700 1 -- 172.21.15.8:0/172201868 >> 172.21.15.8:0/172201868 conn(0x7f3fa80fc750 msgr2=0x7f3fa80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 -- 172.21.15.8:0/172201868 shutdown_connections 2024-04-12T06:06:53.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 -- 172.21.15.8:0/172201868 wait complete. 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 Processor -- start 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 -- start start 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.419+0000 7f3fad9d5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fad9d5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8073b10 con 0x7f3fa8100ea0 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fad9d5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8073c50 con 0x7f3fa8101850 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43538/0 (socket says 172.21.15.8:43538) 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 -- 172.21.15.8:0/278866642 learned_addr learned my addr 172.21.15.8:0/278866642 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa67fc700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 -- 172.21.15.8:0/278866642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 msgr2=0x7f3fa80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 -- 172.21.15.8:0/278866642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980095f0 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa67fc700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fa6ffd700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006010 tx=0x7f3f9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9801d070 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa8073ed0 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f9800f460 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f98021640 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.420+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa81a2c90 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.422+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa8109240 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.422+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3f9800fa20 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.422+0000 7f3fac9d3700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 0x7f3f9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.422+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3f9808d7b0 con 0x7f3fa8100ea0 2024-04-12T06:06:53.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.422+0000 7f3fa67fc700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 0x7f3f9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.423+0000 7f3fa67fc700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 0x7f3f9006f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005d90 tx=0x7f3f9c005d00 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.427+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3f98057820 con 0x7f3fa8100ea0 2024-04-12T06:06:53.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.718+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3fa8108c20 con 0x7f3fa8100ea0 2024-04-12T06:06:53.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.718+0000 7f3fac9d3700 1 -- 172.21.15.8:0/278866642 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3f9805ae40 con 0x7f3fa8100ea0 2024-04-12T06:06:53.715 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:06:53.716 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:06:53.717 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:06:53.717 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:06:53.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 msgr2=0x7f3f9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 0x7f3f9006f1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005d90 tx=0x7f3f9c005d00 comp rx=0 tx=0).stop 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 msgr2=0x7f3fa8073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3f98006010 tx=0x7f3f9800be20 comp rx=0 tx=0).stop 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 shutdown_connections 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3f9006cce0 0x7f3f9006f1a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fa8100ea0 0x7f3fa8073090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 --2- 172.21.15.8:0/278866642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fa8101850 0x7f3fa80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.720+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 >> 172.21.15.8:0/278866642 conn(0x7f3fa80fc750 msgr2=0x7f3fa80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.721+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 shutdown_connections 2024-04-12T06:06:53.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.721+0000 7f3fad9d5700 1 -- 172.21.15.8:0/278866642 wait complete. 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 -- 172.21.15.8:0/4063022522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a80ffda0 msgr2=0x7fd0a8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 --2- 172.21.15.8:0/4063022522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a80ffda0 0x7fd0a8100180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd098009910 tx=0x7fd098009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 -- 172.21.15.8:0/4063022522 shutdown_connections 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 --2- 172.21.15.8:0/4063022522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a8100750 0x7fd0a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 --2- 172.21.15.8:0/4063022522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a80ffda0 0x7fd0a8100180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 -- 172.21.15.8:0/4063022522 >> 172.21.15.8:0/4063022522 conn(0x7fd0a80fb650 msgr2=0x7fd0a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.826+0000 7fd0ad473700 1 -- 172.21.15.8:0/4063022522 shutdown_connections 2024-04-12T06:06:53.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 -- 172.21.15.8:0/4063022522 wait complete. 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 Processor -- start 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 -- start start 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a80ffda0 0x7fd0a819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a819f2e0 con 0x7fd0a8100750 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.827+0000 7fd0ad473700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8198820 con 0x7fd0a80ffda0 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43562/0 (socket says 172.21.15.8:43562) 2024-04-12T06:06:53.826 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 -- 172.21.15.8:0/3338629512 learned_addr learned my addr 172.21.15.8:0/3338629512 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a6ffd700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a80ffda0 0x7fd0a819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 -- 172.21.15.8:0/3338629512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a80ffda0 msgr2=0x7fd0a819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a80ffda0 0x7fd0a819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 -- 172.21.15.8:0/3338629512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980095f0 con 0x7fd0a8100750 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0a67fc700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c960 tx=0x7fd09c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c007a70 con 0x7fd0a8100750 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd09c007bd0 con 0x7fd0a8100750 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09c018630 con 0x7fd0a8100750 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a8198b00 con 0x7fd0a8100750 2024-04-12T06:06:53.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.828+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a8199050 con 0x7fd0a8100750 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.829+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a804f2e0 con 0x7fd0a8100750 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.830+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd09c00f9f0 con 0x7fd0a8100750 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.830+0000 7fd08ffff700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 0x7fd09006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.830+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd09c08c520 con 0x7fd0a8100750 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.830+0000 7fd0a6ffd700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 0x7fd09006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:53.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.831+0000 7fd0a6ffd700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 0x7fd09006f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b5c0 tx=0x7fd098005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:53.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:53.835+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd09c059e10 con 0x7fd0a8100750 2024-04-12T06:06:53.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:53 smithi008 ceph-mon[146084]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:53.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:53 smithi008 ceph-mon[146084]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:54.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:53 smithi194 ceph-mon[106982]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:54.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:53 smithi194 ceph-mon[106982]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:54.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.082+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd0a8199320 con 0x7fd0a8100750 2024-04-12T06:06:54.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.082+0000 7fd08ffff700 1 -- 172.21.15.8:0/3338629512 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fd09c059c30 con 0x7fd0a8100750 2024-04-12T06:06:54.080 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:06:54.080 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:06:54.080 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:06:54.080 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:06:54.081 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:06:54.082 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:54.083 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.084+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 msgr2=0x7fd09006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.084+0000 7fd0ad473700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 0x7fd09006f1a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b5c0 tx=0x7fd098005fb0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.084+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 msgr2=0x7fd0a819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.084+0000 7fd0ad473700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00c960 tx=0x7fd09c00cd20 comp rx=0 tx=0).stop 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 shutdown_connections 2024-04-12T06:06:54.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd09006cce0 0x7fd09006f1a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd0a80ffda0 0x7fd0a819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 --2- 172.21.15.8:0/3338629512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd0a8100750 0x7fd0a819ece0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 >> 172.21.15.8:0/3338629512 conn(0x7fd0a80fb650 msgr2=0x7fd0a8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 shutdown_connections 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.085+0000 7fd0ad473700 1 -- 172.21.15.8:0/3338629512 wait complete. 2024-04-12T06:06:54.085 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 -- 172.21.15.8:0/2511053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 msgr2=0x7f1f7c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 --2- 172.21.15.8:0/2511053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c100080 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009910 tx=0x7f1f6c009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 -- 172.21.15.8:0/2511053448 shutdown_connections 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 --2- 172.21.15.8:0/2511053448 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c100080 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 --2- 172.21.15.8:0/2511053448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 0x7f1f7c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 -- 172.21.15.8:0/2511053448 >> 172.21.15.8:0/2511053448 conn(0x7f1f7c075670 msgr2=0x7f1f7c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 -- 172.21.15.8:0/2511053448 shutdown_connections 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.192+0000 7f1f82271700 1 -- 172.21.15.8:0/2511053448 wait complete. 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 Processor -- start 2024-04-12T06:06:54.190 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 -- start start 2024-04-12T06:06:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 0x7f1f7c197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c198220 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f82271700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c198360 con 0x7f1f7c105bd0 2024-04-12T06:06:54.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43574/0 (socket says 172.21.15.8:43574) 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 -- 172.21.15.8:0/3048841906 learned_addr learned my addr 172.21.15.8:0/3048841906 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7b7fe700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 0x7f1f7c197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 -- 172.21.15.8:0/3048841906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 msgr2=0x7f1f7c197c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 0x7f1f7c197c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.193+0000 7f1f7bfff700 1 -- 172.21.15.8:0/3048841906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0095f0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f7bfff700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d8d0 tx=0x7f1f6400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f640098a0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f64010460 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c1a0730 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f64009ea0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.194+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c1a0bf0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.195+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c1080d0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.195+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1f640105d0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.196+0000 7f1f797fa700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 0x7f1f6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.196+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1f6408c440 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.196+0000 7f1f7b7fe700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 0x7f1f6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.196+0000 7f1f7b7fe700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 0x7f1f6806f1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006010 tx=0x7f1f6c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:54.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.200+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1f640564b0 con 0x7f1f7c0ffc20 2024-04-12T06:06:54.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.428+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f7c0611d0 con 0x7f1f6806cce0 2024-04-12T06:06:54.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.429+0000 7f1f797fa700 1 -- 172.21.15.8:0/3048841906 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1f7c0611d0 con 0x7f1f6806cce0 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:06:54.427 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 msgr2=0x7f1f6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 0x7f1f6806f1a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c006010 tx=0x7f1f6c01a040 comp rx=0 tx=0).stop 2024-04-12T06:06:54.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 msgr2=0x7f1f7c1976c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400d8d0 tx=0x7f1f6400dbe0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 shutdown_connections 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f6806cce0 0x7f1f6806f1a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f7c0ffc20 0x7f1f7c1976c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 --2- 172.21.15.8:0/3048841906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c105bd0 0x7f1f7c197c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 >> 172.21.15.8:0/3048841906 conn(0x7f1f7c075670 msgr2=0x7f1f7c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 shutdown_connections 2024-04-12T06:06:54.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.431+0000 7f1f82271700 1 -- 172.21.15.8:0/3048841906 wait complete. 2024-04-12T06:06:54.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.537+0000 7f2c660a2700 1 -- 172.21.15.8:0/3035948919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c600ffc20 msgr2=0x7f2c60100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.537+0000 7f2c660a2700 1 --2- 172.21.15.8:0/3035948919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c600ffc20 0x7f2c60100080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009910 tx=0x7f2c54009c20 comp rx=0 tx=0).stop 2024-04-12T06:06:54.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 -- 172.21.15.8:0/3035948919 shutdown_connections 2024-04-12T06:06:54.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 --2- 172.21.15.8:0/3035948919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c600ffc20 0x7f2c60100080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 --2- 172.21.15.8:0/3035948919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c60105bd0 0x7f2c60105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 -- 172.21.15.8:0/3035948919 >> 172.21.15.8:0/3035948919 conn(0x7f2c60075670 msgr2=0x7f2c60075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:54.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 -- 172.21.15.8:0/3035948919 shutdown_connections 2024-04-12T06:06:54.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 -- 172.21.15.8:0/3035948919 wait complete. 2024-04-12T06:06:54.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 Processor -- start 2024-04-12T06:06:54.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.538+0000 7f2c660a2700 1 -- start start 2024-04-12T06:06:54.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c660a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c600ffc20 0x7f2c601989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c660a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c660a2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c60199570 con 0x7f2c60105bd0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c660a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6019d380 con 0x7f2c600ffc20 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:43588/0 (socket says 172.21.15.8:43588) 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 -- 172.21.15.8:0/2188227174 learned_addr learned my addr 172.21.15.8:0/2188227174 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5f7fe700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c600ffc20 0x7f2c601989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 -- 172.21.15.8:0/2188227174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c600ffc20 msgr2=0x7f2c601989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c600ffc20 0x7f2c601989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 -- 172.21.15.8:0/2188227174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540095f0 con 0x7f2c60105bd0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5effd700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400bde0 tx=0x7f2c5400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c5401d070 con 0x7f2c60105bd0 2024-04-12T06:06:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c540044c0 con 0x7f2c60105bd0 2024-04-12T06:06:54.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.539+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c540039d0 con 0x7f2c60105bd0 2024-04-12T06:06:54.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.540+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6019d600 con 0x7f2c60105bd0 2024-04-12T06:06:54.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.540+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6019da90 con 0x7f2c60105bd0 2024-04-12T06:06:54.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.540+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6004f2e0 con 0x7f2c60105bd0 2024-04-12T06:06:54.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.541+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2c540048e0 con 0x7f2c60105bd0 2024-04-12T06:06:54.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.541+0000 7f2c5cff9700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 0x7f2c4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:06:54.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.541+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2c5408dc30 con 0x7f2c60105bd0 2024-04-12T06:06:54.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.541+0000 7f2c5f7fe700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 0x7f2c4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:06:54.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.542+0000 7f2c5f7fe700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 0x7f2c4806f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c60 tx=0x7f2c50009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:06:54.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.546+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2c5405b730 con 0x7f2c60105bd0 2024-04-12T06:06:54.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.830+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2c6019dd10 con 0x7f2c60105bd0 2024-04-12T06:06:54.827 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.830+0000 7f2c5cff9700 1 -- 172.21.15.8:0/2188227174 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2c5405b2c0 con 0x7f2c60105bd0 2024-04-12T06:06:54.828 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:06:54.828 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:06:54.828 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:06:54.828 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:06:54.828 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 msgr2=0x7f2c4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 0x7f2c4806f1a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009c60 tx=0x7f2c50009260 comp rx=0 tx=0).stop 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 msgr2=0x7f2c60198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400bde0 tx=0x7f2c5400be10 comp rx=0 tx=0).stop 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 shutdown_connections 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c4806cce0 0x7f2c4806f1a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c600ffc20 0x7f2c601989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 --2- 172.21.15.8:0/2188227174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c60105bd0 0x7f2c60198ee0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.832+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 >> 172.21.15.8:0/2188227174 conn(0x7f2c60075670 msgr2=0x7f2c600687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.833+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 shutdown_connections 2024-04-12T06:06:54.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:06:54.833+0000 7f2c660a2700 1 -- 172.21.15.8:0/2188227174 wait complete. 2024-04-12T06:06:54.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:54 smithi008 ceph-mon[146084]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:54.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:54 smithi008 ceph-mon[146084]: pgmap v356: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:54.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:54 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/278866642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:06:54.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:54 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3338629512' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:06:54.986 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:54 smithi194 ceph-mon[106982]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:54.986 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:54 smithi194 ceph-mon[106982]: pgmap v356: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:54.986 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:54 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/278866642' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:06:54.986 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:54 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3338629512' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:06:56.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:55 smithi194 ceph-mon[106982]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:56.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:55 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2188227174' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:06:56.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:55 smithi008 ceph-mon[146084]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:06:56.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:55 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2188227174' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:06:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:56 smithi194 ceph-mon[106982]: pgmap v357: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:57.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:56 smithi008 ceph-mon[146084]: pgmap v357: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:59.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:06:58 smithi194 ceph-mon[106982]: pgmap v358: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:06:59.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:06:58 smithi008 ceph-mon[146084]: pgmap v358: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:01.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:00 smithi194 ceph-mon[106982]: pgmap v359: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:01.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:00 smithi008 ceph-mon[146084]: pgmap v359: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:03.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:02 smithi194 ceph-mon[106982]: pgmap v360: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:03.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:03.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:02 smithi008 ceph-mon[146084]: pgmap v360: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:03.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:04.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:04 smithi008 ceph-mon[146084]: pgmap v361: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:04.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:04 smithi194 ceph-mon[106982]: pgmap v361: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:06.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:06 smithi194 ceph-mon[106982]: pgmap v362: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:06.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:06 smithi008 ceph-mon[146084]: pgmap v362: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:08.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:08 smithi194 ceph-mon[106982]: pgmap v363: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:08.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:08 smithi008 ceph-mon[146084]: pgmap v363: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:10.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:10 smithi194 ceph-mon[106982]: pgmap v364: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:10.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:10 smithi008 ceph-mon[146084]: pgmap v364: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:12.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:12 smithi194 ceph-mon[106982]: pgmap v365: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:12.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:12 smithi008 ceph-mon[146084]: pgmap v365: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:14.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:14 smithi194 ceph-mon[106982]: pgmap v366: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:14.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:14 smithi008 ceph-mon[146084]: pgmap v366: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:16.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:16 smithi194 ceph-mon[106982]: pgmap v367: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:16 smithi008 ceph-mon[146084]: pgmap v367: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:17.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:18.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:18 smithi194 ceph-mon[106982]: pgmap v368: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:18.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:18 smithi008 ceph-mon[146084]: pgmap v368: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:20.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:20 smithi194 ceph-mon[106982]: pgmap v369: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:20.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:20 smithi008 ceph-mon[146084]: pgmap v369: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:22.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:22 smithi194 ceph-mon[106982]: pgmap v370: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:22.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:22 smithi008 ceph-mon[146084]: pgmap v370: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:24.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:24 smithi194 ceph-mon[106982]: pgmap v371: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:24.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:24 smithi008 ceph-mon[146084]: pgmap v371: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:24.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 -- 172.21.15.8:0/2557535258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c4105bd0 msgr2=0x7f18c4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:24.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 --2- 172.21.15.8:0/2557535258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c4105bd0 0x7f18c4105fb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009910 tx=0x7f18c0009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:24.959 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 -- 172.21.15.8:0/2557535258 shutdown_connections 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 --2- 172.21.15.8:0/2557535258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c40ffc20 0x7f18c4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 --2- 172.21.15.8:0/2557535258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c4105bd0 0x7f18c4105fb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 -- 172.21.15.8:0/2557535258 >> 172.21.15.8:0/2557535258 conn(0x7f18c4075670 msgr2=0x7f18c4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 -- 172.21.15.8:0/2557535258 shutdown_connections 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.954+0000 7f18cc62f700 1 -- 172.21.15.8:0/2557535258 wait complete. 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 Processor -- start 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 -- start start 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 0x7f18c4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c41981c0 con 0x7f18c40ffc20 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18cc62f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4198300 con 0x7f18c4105bd0 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:24.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48118/0 (socket says 172.21.15.8:48118) 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 -- 172.21.15.8:0/1701399844 learned_addr learned my addr 172.21.15.8:0/1701399844 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18c9bca700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 0x7f18c4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 -- 172.21.15.8:0/1701399844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 msgr2=0x7f18c4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 0x7f18c4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18ca3cb700 1 -- 172.21.15.8:0/1701399844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c00095f0 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.955+0000 7f18c9bca700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 0x7f18c4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18ca3cb700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f18c0006010 tx=0x7f18c000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c001d070 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c41a0610 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18c000f460 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18c0021620 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.956+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c41a0a80 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.957+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c41080f0 con 0x7f18c40ffc20 2024-04-12T06:07:24.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.957+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f18c000fa40 con 0x7f18c40ffc20 2024-04-12T06:07:24.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.958+0000 7f18bb7fe700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 0x7f18b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:24.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.958+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f18c008d780 con 0x7f18c40ffc20 2024-04-12T06:07:24.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.958+0000 7f18c9bca700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 0x7f18b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:24.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.958+0000 7f18c9bca700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 0x7f18b006f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005950 tx=0x7f18b40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:24.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:24.963+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f18c00577f0 con 0x7f18c40ffc20 2024-04-12T06:07:25.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.186+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18c40611d0 con 0x7f18b006cce0 2024-04-12T06:07:25.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.187+0000 7f18bb7fe700 1 -- 172.21.15.8:0/1701399844 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f18c40611d0 con 0x7f18b006cce0 2024-04-12T06:07:25.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 msgr2=0x7f18b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 0x7f18b006f1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f18b4005950 tx=0x7f18b40058e0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 msgr2=0x7f18c41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f18c0006010 tx=0x7f18c000be20 comp rx=0 tx=0).stop 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 shutdown_connections 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f18b006cce0 0x7f18b006f1a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18c40ffc20 0x7f18c41975a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 --2- 172.21.15.8:0/1701399844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18c4105bd0 0x7f18c4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 >> 172.21.15.8:0/1701399844 conn(0x7f18c4075670 msgr2=0x7f18c40fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 shutdown_connections 2024-04-12T06:07:25.187 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.189+0000 7f18cc62f700 1 -- 172.21.15.8:0/1701399844 wait complete. 2024-04-12T06:07:25.197 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 -- 172.21.15.8:0/2434903392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728106cd0 msgr2=0x7f97281070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 --2- 172.21.15.8:0/2434903392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728106cd0 0x7f97281070b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f971c009910 tx=0x7f971c009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 -- 172.21.15.8:0/2434903392 shutdown_connections 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 --2- 172.21.15.8:0/2434903392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728068580 0x7f97280689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 --2- 172.21.15.8:0/2434903392 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728106cd0 0x7f97281070b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 -- 172.21.15.8:0/2434903392 >> 172.21.15.8:0/2434903392 conn(0x7f9728075670 msgr2=0x7f9728075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 -- 172.21.15.8:0/2434903392 shutdown_connections 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.296+0000 7f972ebc3700 1 -- 172.21.15.8:0/2434903392 wait complete. 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 Processor -- start 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 -- start start 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728106cd0 0x7f9728100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9728100770 con 0x7f9728068580 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972ebc3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281008b0 con 0x7f9728106cd0 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972c95f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972c95f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48134/0 (socket says 172.21.15.8:48134) 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f972c95f700 1 -- 172.21.15.8:0/976797162 learned_addr learned my addr 172.21.15.8:0/976797162 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.297+0000 7f9727fff700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728106cd0 0x7f9728100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972c95f700 1 -- 172.21.15.8:0/976797162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728106cd0 msgr2=0x7f9728100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972c95f700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728106cd0 0x7f9728100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972c95f700 1 -- 172.21.15.8:0/976797162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0095f0 con 0x7f9728068580 2024-04-12T06:07:25.297 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972c95f700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f971c0092e0 tx=0x7f971c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f971c01d070 con 0x7f9728068580 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f971c005e00 con 0x7f9728068580 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728100b30 con 0x7f9728068580 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f971c00fe90 con 0x7f9728068580 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.298+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281a2c90 con 0x7f9728068580 2024-04-12T06:07:25.298 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.299+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97281091f0 con 0x7f9728068580 2024-04-12T06:07:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.300+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f971c00f6d0 con 0x7f9728068580 2024-04-12T06:07:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.300+0000 7f9725ffb700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 0x7f971806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.300+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f971c08d780 con 0x7f9728068580 2024-04-12T06:07:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.300+0000 7f9727fff700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 0x7f971806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.301+0000 7f9727fff700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 0x7f971806f1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9728101490 tx=0x7f971400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.302 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.305+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f971c0577f0 con 0x7f9728068580 2024-04-12T06:07:25.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.527+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97280611d0 con 0x7f971806cce0 2024-04-12T06:07:25.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.528+0000 7f9725ffb700 1 -- 172.21.15.8:0/976797162 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f97280611d0 con 0x7f971806cce0 2024-04-12T06:07:25.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 msgr2=0x7f971806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 0x7f971806f1a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9728101490 tx=0x7f971400b410 comp rx=0 tx=0).stop 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 msgr2=0x7f97280ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f971c0092e0 tx=0x7f971c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 shutdown_connections 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f971806cce0 0x7f971806f1a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9728068580 0x7f97280ffcf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 --2- 172.21.15.8:0/976797162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9728106cd0 0x7f9728100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 >> 172.21.15.8:0/976797162 conn(0x7f9728075670 msgr2=0x7f97280fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 shutdown_connections 2024-04-12T06:07:25.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.530+0000 7f972ebc3700 1 -- 172.21.15.8:0/976797162 wait complete. 2024-04-12T06:07:25.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- 172.21.15.8:0/3771698115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 msgr2=0x7faf84100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/3771698115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84100180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faf74009910 tx=0x7faf74009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:25.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- 172.21.15.8:0/3771698115 shutdown_connections 2024-04-12T06:07:25.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/3771698115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 0x7faf841047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/3771698115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84100180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- 172.21.15.8:0/3771698115 >> 172.21.15.8:0/3771698115 conn(0x7faf840fb650 msgr2=0x7faf840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- 172.21.15.8:0/3771698115 shutdown_connections 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- 172.21.15.8:0/3771698115 wait complete. 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 Processor -- start 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.637+0000 7faf8c4a7700 1 -- start start 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8c4a7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8c4a7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 0x7faf84197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8c4a7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf84198190 con 0x7faf840ffda0 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8c4a7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841982d0 con 0x7faf84100750 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48156/0 (socket says 172.21.15.8:48156) 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 -- 172.21.15.8:0/2040468491 learned_addr learned my addr 172.21.15.8:0/2040468491 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf89a42700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 0x7faf84197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 -- 172.21.15.8:0/2040468491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 msgr2=0x7faf84197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 0x7faf84197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 -- 172.21.15.8:0/2040468491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf740095f0 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf8a243700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.638+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7401d070 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.639+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf841a06a0 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.639+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf74005710 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.639+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf7400fe20 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.639+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf841a0ab0 con 0x7faf840ffda0 2024-04-12T06:07:25.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.639+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf84108140 con 0x7faf840ffda0 2024-04-12T06:07:25.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.640+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7faf74003a60 con 0x7faf840ffda0 2024-04-12T06:07:25.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.640+0000 7faf7b7fe700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 0x7faf7006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.641+0000 7faf89a42700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 0x7faf7006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.641+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faf7408d730 con 0x7faf840ffda0 2024-04-12T06:07:25.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.641+0000 7faf89a42700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 0x7faf7006f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf840696c0 tx=0x7faf8000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.645+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7faf740577a0 con 0x7faf840ffda0 2024-04-12T06:07:25.854 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.857+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faf840611d0 con 0x7faf7006cce0 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.867+0000 7faf7b7fe700 1 -- 172.21.15.8:0/2040468491 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faf840611d0 con 0x7faf7006cce0 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (9m) 2m ago 20m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (20m) 2m ago 20m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (19m) 2m ago 19m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:07:25.865 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (7m) 2m ago 20m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (6m) 2m ago 19m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (8m) 2m ago 20m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (3m) 2m ago 14m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (3m) 2m ago 14m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (2m) 2m ago 14m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (2m) 2m ago 14m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (11m) 2m ago 22m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (10m) 2m ago 19m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (7m) 2m ago 22m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (7m) 2m ago 19m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (10m) 2m ago 20m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (10m) 2m ago 19m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:07:25.866 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (6m) 2m ago 18m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (5m) 2m ago 18m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (5m) 2m ago 17m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (4m) 2m ago 17m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (4m) 2m ago 16m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (3m) 2m ago 16m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (9m) 2m ago 20m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:07:25.867 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 msgr2=0x7faf7006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 0x7faf7006f1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf840696c0 tx=0x7faf8000b410 comp rx=0 tx=0).stop 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 msgr2=0x7faf84197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faf74006010 tx=0x7faf7400bd70 comp rx=0 tx=0).stop 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 shutdown_connections 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf7006cce0 0x7faf7006f1a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf840ffda0 0x7faf84197630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 --2- 172.21.15.8:0/2040468491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf84100750 0x7faf84197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 >> 172.21.15.8:0/2040468491 conn(0x7faf840fb650 msgr2=0x7faf840fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 shutdown_connections 2024-04-12T06:07:25.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.870+0000 7faf8c4a7700 1 -- 172.21.15.8:0/2040468491 wait complete. 2024-04-12T06:07:25.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.974+0000 7f990301c700 1 -- 172.21.15.8:0/1991799111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc101ec0 msgr2=0x7f98fc1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 --2- 172.21.15.8:0/1991799111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc101ec0 0x7f98fc1022a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009910 tx=0x7f98f0009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- 172.21.15.8:0/1991799111 shutdown_connections 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 --2- 172.21.15.8:0/1991799111 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc1027e0 0x7f98fc10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 --2- 172.21.15.8:0/1991799111 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc101ec0 0x7f98fc1022a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- 172.21.15.8:0/1991799111 >> 172.21.15.8:0/1991799111 conn(0x7f98fc0fb650 msgr2=0x7f98fc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- 172.21.15.8:0/1991799111 shutdown_connections 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- 172.21.15.8:0/1991799111 wait complete. 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 Processor -- start 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- start start 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc101ec0 0x7f98fc0ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1005a0 con 0x7f98fc1027e0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.975+0000 7f990301c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc1006e0 con 0x7f98fc101ec0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48174/0 (socket says 172.21.15.8:48174) 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 -- 172.21.15.8:0/3609610602 learned_addr learned my addr 172.21.15.8:0/3609610602 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f9900db8700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc101ec0 0x7f98fc0ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 -- 172.21.15.8:0/3609610602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc101ec0 msgr2=0x7f98fc0ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc101ec0 0x7f98fc0ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 -- 172.21.15.8:0/3609610602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00095f0 con 0x7f98fc1027e0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98fbfff700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98e800d900 tx=0x7f98e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e8009900 con 0x7f98fc1027e0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f98e8010460 con 0x7f98fc1027e0 2024-04-12T06:07:25.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc1a2980 con 0x7f98fc1027e0 2024-04-12T06:07:25.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.976+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98e800f5f0 con 0x7f98fc1027e0 2024-04-12T06:07:25.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.977+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc1a2d90 con 0x7f98fc1027e0 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.977+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc1083d0 con 0x7f98fc1027e0 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.978+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f98e80105d0 con 0x7f98fc1027e0 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.978+0000 7f98f9ffb700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 0x7f98ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.978+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f98e808c2f0 con 0x7f98fc1027e0 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.978+0000 7f9900db8700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 0x7f98ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:25.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.979+0000 7f9900db8700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 0x7f98ec06f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006010 tx=0x7f98f001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:25.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:25.983+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f98e80563e0 con 0x7f98fc1027e0 2024-04-12T06:07:26.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.268+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f98fc04f2e0 con 0x7f98fc1027e0 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.268+0000 7f98f9ffb700 1 -- 172.21.15.8:0/3609610602 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f98e8059a00 con 0x7f98fc1027e0 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:07:26.266 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:07:26.267 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:07:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 msgr2=0x7f98ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 0x7f98ec06f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f98f0006010 tx=0x7f98f001a040 comp rx=0 tx=0).stop 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 msgr2=0x7f98fc100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f98e800d900 tx=0x7f98e800dcc0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 shutdown_connections 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f98ec06cce0 0x7f98ec06f1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98fc101ec0 0x7f98fc0ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 --2- 172.21.15.8:0/3609610602 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f98fc1027e0 0x7f98fc100060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.270+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 >> 172.21.15.8:0/3609610602 conn(0x7f98fc0fb650 msgr2=0x7f98fc105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.271+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 shutdown_connections 2024-04-12T06:07:26.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.271+0000 7f990301c700 1 -- 172.21.15.8:0/3609610602 wait complete. 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 -- 172.21.15.8:0/528833901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 msgr2=0x7f57240689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 --2- 172.21.15.8:0/528833901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f57240689e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5714009910 tx=0x7f5714009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 -- 172.21.15.8:0/528833901 shutdown_connections 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 --2- 172.21.15.8:0/528833901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f57240689e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 --2- 172.21.15.8:0/528833901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f57241070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.374 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.376+0000 7f572bf73700 1 -- 172.21.15.8:0/528833901 >> 172.21.15.8:0/528833901 conn(0x7f5724075670 msgr2=0x7f5724075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:26.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 -- 172.21.15.8:0/528833901 shutdown_connections 2024-04-12T06:07:26.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 -- 172.21.15.8:0/528833901 wait complete. 2024-04-12T06:07:26.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 Processor -- start 2024-04-12T06:07:26.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 -- start start 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f572419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572419d110 con 0x7f5724068580 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.377+0000 7f572bf73700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57241965c0 con 0x7f5724106cd0 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48198/0 (socket says 172.21.15.8:48198) 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 -- 172.21.15.8:0/3402563804 learned_addr learned my addr 172.21.15.8:0/3402563804 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f572950e700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f572419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 -- 172.21.15.8:0/3402563804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 msgr2=0x7f572419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f572419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 -- 172.21.15.8:0/3402563804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140095f0 con 0x7f5724068580 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f572950e700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f572419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f5729d0f700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f572000c8a0 tx=0x7f572000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:26.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572000cea0 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57241968a0 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5720004830 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5720005500 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.378+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5724196df0 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.379+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57241091d0 con 0x7f5724068580 2024-04-12T06:07:26.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.380+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f57200049a0 con 0x7f5724068580 2024-04-12T06:07:26.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.380+0000 7f571affd700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 0x7f571006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.380+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f572008c240 con 0x7f5724068580 2024-04-12T06:07:26.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.380+0000 7f572950e700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 0x7f571006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.381+0000 7f572950e700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 0x7f571006f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5714006010 tx=0x7f571400b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:26.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.385+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5720059d40 con 0x7f5724068580 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.631+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5724197070 con 0x7f5724068580 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.631+0000 7f571affd700 1 -- 172.21.15.8:0/3402563804 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f57200598d0 con 0x7f5724068580 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:26.629 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:07:26.630 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:07:26.631 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:26.632 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 msgr2=0x7f571006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 0x7f571006f1a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5714006010 tx=0x7f571400b540 comp rx=0 tx=0).stop 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 msgr2=0x7f572419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f572000c8a0 tx=0x7f572000cc60 comp rx=0 tx=0).stop 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 shutdown_connections 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f571006cce0 0x7f571006f1a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5724068580 0x7f572419c540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 --2- 172.21.15.8:0/3402563804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5724106cd0 0x7f572419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 >> 172.21.15.8:0/3402563804 conn(0x7f5724075670 msgr2=0x7f57240fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:26.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 shutdown_connections 2024-04-12T06:07:26.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.633+0000 7f572bf73700 1 -- 172.21.15.8:0/3402563804 wait complete. 2024-04-12T06:07:26.634 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 -- 172.21.15.8:0/2316717080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 msgr2=0x7f29141058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2316717080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29141058a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2908009910 tx=0x7f2908009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 -- 172.21.15.8:0/2316717080 shutdown_connections 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2316717080 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29141058a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2316717080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 0x7f2914101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 -- 172.21.15.8:0/2316717080 >> 172.21.15.8:0/2316717080 conn(0x7f29140fc750 msgr2=0x7f29140feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 -- 172.21.15.8:0/2316717080 shutdown_connections 2024-04-12T06:07:26.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.740+0000 7f291bf1a700 1 -- 172.21.15.8:0/2316717080 wait complete. 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 Processor -- start 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 -- start start 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 0x7f2914073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914073b10 con 0x7f2914101850 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f291bf1a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914073c50 con 0x7f2914100ea0 2024-04-12T06:07:26.740 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f29194b5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f29194b5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48214/0 (socket says 172.21.15.8:48214) 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f29194b5700 1 -- 172.21.15.8:0/2484479511 learned_addr learned my addr 172.21.15.8:0/2484479511 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.741+0000 7f2919cb6700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 0x7f2914073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f29194b5700 1 -- 172.21.15.8:0/2484479511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 msgr2=0x7f2914073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f29194b5700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 0x7f2914073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f29194b5700 1 -- 172.21.15.8:0/2484479511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29080095f0 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f29194b5700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f290800bda0 tx=0x7f290800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f290801d070 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2908005710 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2914073ed0 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f290800fe90 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.742+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141a2d50 con 0x7f2914101850 2024-04-12T06:07:26.741 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.743+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29141092a0 con 0x7f2914101850 2024-04-12T06:07:26.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.743+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f290800f6f0 con 0x7f2914101850 2024-04-12T06:07:26.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.744+0000 7f2906ffd700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 0x7f290006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:26.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.744+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f290808d6c0 con 0x7f2914101850 2024-04-12T06:07:26.742 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.744+0000 7f2919cb6700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 0x7f290006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:26.743 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.744+0000 7f2919cb6700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 0x7f290006f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2910005950 tx=0x7f291000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:26.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.748+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2908057730 con 0x7f2914101850 2024-04-12T06:07:26.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:26 smithi194 ceph-mon[106982]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:26 smithi194 ceph-mon[106982]: pgmap v372: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:26.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:26 smithi194 ceph-mon[106982]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:26 smithi194 ceph-mon[106982]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:26 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3609610602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:07:26.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:26 smithi008 ceph-mon[146084]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:26 smithi008 ceph-mon[146084]: pgmap v372: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:26.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:26 smithi008 ceph-mon[146084]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:26 smithi008 ceph-mon[146084]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:26.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:26 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3609610602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:07:26.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.966+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29140746d0 con 0x7f290006cce0 2024-04-12T06:07:26.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.967+0000 7f2906ffd700 1 -- 172.21.15.8:0/2484479511 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f29140746d0 con 0x7f290006cce0 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:07:26.965 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:07:26.966 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:07:26.966 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:07:26.966 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:07:26.966 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 msgr2=0x7f290006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 0x7f290006f1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2910005950 tx=0x7f291000b410 comp rx=0 tx=0).stop 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 msgr2=0x7f29140735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f290800bda0 tx=0x7f290800bdd0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 shutdown_connections 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f290006cce0 0x7f290006f1a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2914100ea0 0x7f2914073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 --2- 172.21.15.8:0/2484479511 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2914101850 0x7f29140735d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.969+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 >> 172.21.15.8:0/2484479511 conn(0x7f29140fc750 msgr2=0x7f29140fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:26.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.970+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 shutdown_connections 2024-04-12T06:07:26.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:26.970+0000 7f291bf1a700 1 -- 172.21.15.8:0/2484479511 wait complete. 2024-04-12T06:07:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 -- 172.21.15.8:0/1540753024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 msgr2=0x7fef24100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1540753024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef24100080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef20009910 tx=0x7fef20009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 -- 172.21.15.8:0/1540753024 shutdown_connections 2024-04-12T06:07:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1540753024 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef24100080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1540753024 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef24105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 -- 172.21.15.8:0/1540753024 >> 172.21.15.8:0/1540753024 conn(0x7fef24075670 msgr2=0x7fef24075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 -- 172.21.15.8:0/1540753024 shutdown_connections 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.076+0000 7fef2ba99700 1 -- 172.21.15.8:0/1540753024 wait complete. 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 Processor -- start 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 -- start start 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef2419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2419d150 con 0x7fef240ffc20 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef2ba99700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef24196600 con 0x7fef24105bd0 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:48238/0 (socket says 172.21.15.8:48238) 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 -- 172.21.15.8:0/1264566863 learned_addr learned my addr 172.21.15.8:0/1264566863 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29034700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef2419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:27.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 -- 172.21.15.8:0/1264566863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 msgr2=0x7fef2419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef2419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 -- 172.21.15.8:0/1264566863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200095f0 con 0x7fef240ffc20 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29034700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef2419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.077+0000 7fef29835700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d8d0 tx=0x7fef1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.078+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef18009940 con 0x7fef240ffc20 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.078+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef241968e0 con 0x7fef240ffc20 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.078+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef18010460 con 0x7fef240ffc20 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.078+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef1800f5f0 con 0x7fef240ffc20 2024-04-12T06:07:27.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.078+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef24196e30 con 0x7fef240ffc20 2024-04-12T06:07:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.079+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2404f2e0 con 0x7fef240ffc20 2024-04-12T06:07:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.079+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fef180105d0 con 0x7fef240ffc20 2024-04-12T06:07:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.080+0000 7fef16ffd700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 0x7fef1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.080+0000 7fef29034700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 0x7fef1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:27.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.080+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef1808c5d0 con 0x7fef240ffc20 2024-04-12T06:07:27.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.080+0000 7fef29034700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 0x7fef1006f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef24075de0 tx=0x7fef2001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:27.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.084+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fef1805a0d0 con 0x7fef240ffc20 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.370+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fef241970f0 con 0x7fef240ffc20 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.371+0000 7fef16ffd700 1 -- 172.21.15.8:0/1264566863 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fef18010880 con 0x7fef240ffc20 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:07:27.368 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:07:27.369 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:07:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 msgr2=0x7fef1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:27.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 0x7fef1006f1a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fef24075de0 tx=0x7fef2001a040 comp rx=0 tx=0).stop 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 msgr2=0x7fef2419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d8d0 tx=0x7fef1800dc90 comp rx=0 tx=0).stop 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 shutdown_connections 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef1006cce0 0x7fef1006f1a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef240ffc20 0x7fef2419c580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 --2- 172.21.15.8:0/1264566863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef24105bd0 0x7fef2419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 >> 172.21.15.8:0/1264566863 conn(0x7fef24075670 msgr2=0x7fef24068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 shutdown_connections 2024-04-12T06:07:27.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:27.373+0000 7fef2ba99700 1 -- 172.21.15.8:0/1264566863 wait complete. 2024-04-12T06:07:27.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:27 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3402563804' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:07:27.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:27 smithi194 ceph-mon[106982]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:27.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:27 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1264566863' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:07:27.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:27 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3402563804' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:07:27.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:27 smithi008 ceph-mon[146084]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:27.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:27 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1264566863' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:07:28.821 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:28 smithi194 ceph-mon[106982]: pgmap v373: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:28 smithi008 ceph-mon[146084]: pgmap v373: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:30.572 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:30 smithi194 ceph-mon[106982]: pgmap v374: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:30.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:30 smithi008 ceph-mon[146084]: pgmap v374: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:32.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:32 smithi194 ceph-mon[106982]: pgmap v375: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:32.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:32.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:32 smithi008 ceph-mon[146084]: pgmap v375: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:32.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:34.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:34 smithi194 ceph-mon[106982]: pgmap v376: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:34.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:34 smithi008 ceph-mon[146084]: pgmap v376: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:36.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:36 smithi194 ceph-mon[106982]: pgmap v377: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:36.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:36 smithi008 ceph-mon[146084]: pgmap v377: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:38.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:38 smithi194 ceph-mon[106982]: pgmap v378: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:38.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:38 smithi008 ceph-mon[146084]: pgmap v378: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:40.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:40 smithi194 ceph-mon[106982]: pgmap v379: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:40.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:40 smithi008 ceph-mon[146084]: pgmap v379: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:42.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:42 smithi194 ceph-mon[106982]: pgmap v380: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:42.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:42 smithi008 ceph-mon[146084]: pgmap v380: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:44.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:44 smithi194 ceph-mon[106982]: pgmap v381: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:44.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:44 smithi008 ceph-mon[146084]: pgmap v381: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:46.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:46 smithi194 ceph-mon[106982]: pgmap v382: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:46.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:46 smithi008 ceph-mon[146084]: pgmap v382: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:47.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:48.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:07:48.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:48 smithi008 ceph-mon[146084]: pgmap v383: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:48 smithi194 ceph-mon[106982]: pgmap v383: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:50.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:50 smithi194 ceph-mon[106982]: pgmap v384: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:50.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:50 smithi008 ceph-mon[146084]: pgmap v384: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:52.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:52 smithi008 ceph-mon[146084]: pgmap v385: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:53.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:52 smithi194 ceph-mon[106982]: pgmap v385: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:54.885 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:54 smithi008 ceph-mon[146084]: pgmap v386: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:55.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:54 smithi194 ceph-mon[106982]: pgmap v386: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:56.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:56 smithi008 ceph-mon[146084]: pgmap v387: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:56 smithi194 ceph-mon[106982]: pgmap v387: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 -- 172.21.15.8:0/2809739975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8100ea0 msgr2=0x7f47b8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 --2- 172.21.15.8:0/2809739975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8100ea0 0x7f47b8101280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009910 tx=0x7f47a8009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:57.497 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 -- 172.21.15.8:0/2809739975 shutdown_connections 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 --2- 172.21.15.8:0/2809739975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8101850 0x7f47b81058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 --2- 172.21.15.8:0/2809739975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8100ea0 0x7f47b8101280 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 -- 172.21.15.8:0/2809739975 >> 172.21.15.8:0/2809739975 conn(0x7f47b80fc750 msgr2=0x7f47b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 -- 172.21.15.8:0/2809739975 shutdown_connections 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.497+0000 7f47be36f700 1 -- 172.21.15.8:0/2809739975 wait complete. 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 Processor -- start 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 -- start start 2024-04-12T06:07:57.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8100ea0 0x7f47b8073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8073b10 con 0x7f47b8101850 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.498+0000 7f47be36f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8073c50 con 0x7f47b8100ea0 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40290/0 (socket says 172.21.15.8:40290) 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 -- 172.21.15.8:0/173313968 learned_addr learned my addr 172.21.15.8:0/173313968 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b7fff700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8100ea0 0x7f47b8073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 -- 172.21.15.8:0/173313968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8100ea0 msgr2=0x7f47b8073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8100ea0 0x7f47b8073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.499 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 -- 172.21.15.8:0/173313968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80095f0 con 0x7f47b8101850 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b77fe700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c960 tx=0x7f47ac00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac007a70 con 0x7f47b8101850 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b81a2960 con 0x7f47b8101850 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47ac007bd0 con 0x7f47b8101850 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47ac018630 con 0x7f47b8101850 2024-04-12T06:07:57.500 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.499+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b81a2cc0 con 0x7f47b8101850 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.500+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8109240 con 0x7f47b8101850 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.501+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f47ac00f450 con 0x7f47b8101850 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.501+0000 7f47b57fa700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.501+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f47ac08c6b0 con 0x7f47b8101850 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.501+0000 7f47b7fff700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.502 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.502+0000 7f47b7fff700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006010 tx=0x7f47a801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:57.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.506+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f47ac056850 con 0x7f47b8101850 2024-04-12T06:07:57.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.730+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b802ccc0 con 0x7f47a006cce0 2024-04-12T06:07:57.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.731+0000 7f47b57fa700 1 -- 172.21.15.8:0/173313968 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f47b802ccc0 con 0x7f47a006cce0 2024-04-12T06:07:57.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 msgr2=0x7f47a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47a8006010 tx=0x7f47a801a040 comp rx=0 tx=0).stop 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 msgr2=0x7f47b80735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c960 tx=0x7f47ac00cd20 comp rx=0 tx=0).stop 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 shutdown_connections 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b8100ea0 0x7f47b8073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 --2- 172.21.15.8:0/173313968 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b8101850 0x7f47b80735d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 >> 172.21.15.8:0/173313968 conn(0x7f47b80fc750 msgr2=0x7f47b80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 shutdown_connections 2024-04-12T06:07:57.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.733+0000 7f47be36f700 1 -- 172.21.15.8:0/173313968 wait complete. 2024-04-12T06:07:57.741 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 -- 172.21.15.8:0/1004416258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250106cd0 msgr2=0x7f12501070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 --2- 172.21.15.8:0/1004416258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250106cd0 0x7f12501070b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f124c009910 tx=0x7f124c009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 -- 172.21.15.8:0/1004416258 shutdown_connections 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 --2- 172.21.15.8:0/1004416258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250068580 0x7f12500689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 --2- 172.21.15.8:0/1004416258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250106cd0 0x7f12501070b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 -- 172.21.15.8:0/1004416258 >> 172.21.15.8:0/1004416258 conn(0x7f1250075670 msgr2=0x7f1250075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:57.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 -- 172.21.15.8:0/1004416258 shutdown_connections 2024-04-12T06:07:57.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.840+0000 7f125732b700 1 -- 172.21.15.8:0/1004416258 wait complete. 2024-04-12T06:07:57.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 Processor -- start 2024-04-12T06:07:57.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 -- start start 2024-04-12T06:07:57.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 0x7f125019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125019d130 con 0x7f1250068580 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f125732b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1250196630 con 0x7f1250106cd0 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f12550c7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f12550c7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40318/0 (socket says 172.21.15.8:40318) 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.841+0000 7f12550c7700 1 -- 172.21.15.8:0/1429247253 learned_addr learned my addr 172.21.15.8:0/1429247253 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12548c6700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 0x7f125019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12550c7700 1 -- 172.21.15.8:0/1429247253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 msgr2=0x7f125019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12550c7700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 0x7f125019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12550c7700 1 -- 172.21.15.8:0/1429247253 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0095f0 con 0x7f1250068580 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12548c6700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 0x7f125019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12550c7700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f124c006010 tx=0x7f124c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124c01d070 con 0x7f1250068580 2024-04-12T06:07:57.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f124c00f460 con 0x7f1250068580 2024-04-12T06:07:57.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12501968b0 con 0x7f1250068580 2024-04-12T06:07:57.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f124c021640 con 0x7f1250068580 2024-04-12T06:07:57.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.842+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1250196da0 con 0x7f1250068580 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.843+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12501091f0 con 0x7f1250068580 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.844+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f124c00fa60 con 0x7f1250068580 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.844+0000 7f12427fc700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 0x7f123c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.844+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f124c08d820 con 0x7f1250068580 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.844+0000 7f12548c6700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 0x7f123c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:57.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.845+0000 7f12548c6700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 0x7f123c06f1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1244005fd0 tx=0x7f1244005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:57.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:57.849+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f124c05b320 con 0x7f1250068580 2024-04-12T06:07:58.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.067+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1250196fc0 con 0x7f123c06cce0 2024-04-12T06:07:58.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.068+0000 7f12427fc700 1 -- 172.21.15.8:0/1429247253 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1250196fc0 con 0x7f123c06cce0 2024-04-12T06:07:58.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 msgr2=0x7f123c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 0x7f123c06f1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1244005fd0 tx=0x7f1244005dc0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 msgr2=0x7f125019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f124c006010 tx=0x7f124c00be20 comp rx=0 tx=0).stop 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 shutdown_connections 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f123c06cce0 0x7f123c06f1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1250068580 0x7f125019c560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 --2- 172.21.15.8:0/1429247253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1250106cd0 0x7f125019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 >> 172.21.15.8:0/1429247253 conn(0x7f1250075670 msgr2=0x7f12500fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 shutdown_connections 2024-04-12T06:07:58.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.071+0000 7f125732b700 1 -- 172.21.15.8:0/1429247253 wait complete. 2024-04-12T06:07:58.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 -- 172.21.15.8:0/3316939713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 msgr2=0x7f9ac8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 --2- 172.21.15.8:0/3316939713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8105fb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009910 tx=0x7f9abc009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:58.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 -- 172.21.15.8:0/3316939713 shutdown_connections 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 --2- 172.21.15.8:0/3316939713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 0x7f9ac8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 --2- 172.21.15.8:0/3316939713 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8105fb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 -- 172.21.15.8:0/3316939713 >> 172.21.15.8:0/3316939713 conn(0x7f9ac8075670 msgr2=0x7f9ac8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 -- 172.21.15.8:0/3316939713 shutdown_connections 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.177+0000 7f9acefc4700 1 -- 172.21.15.8:0/3316939713 wait complete. 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 Processor -- start 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 -- start start 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 0x7f9ac8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac81022c0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9acefc4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8102400 con 0x7f9ac80ffc20 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40326/0 (socket says 172.21.15.8:40326) 2024-04-12T06:07:58.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 -- 172.21.15.8:0/1069497257 learned_addr learned my addr 172.21.15.8:0/1069497257 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9accd60700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 0x7f9ac8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 -- 172.21.15.8:0/1069497257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 msgr2=0x7f9ac8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 0x7f9ac8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 -- 172.21.15.8:0/1069497257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc0095f0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.178+0000 7f9ac7fff700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400c930 tx=0x7f9ab400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.179+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab4007ad0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.179+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9ab4007c30 con 0x7f9ac8105bd0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.179+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac81a2990 con 0x7f9ac8105bd0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.179+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab4018630 con 0x7f9ac8105bd0 2024-04-12T06:07:58.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.179+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac81a2cf0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.180+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac804f2e0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.180+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9ab400f450 con 0x7f9ac8105bd0 2024-04-12T06:07:58.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.180+0000 7f9ac5ffb700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 0x7f9ab806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.181+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9ab408c750 con 0x7f9ac8105bd0 2024-04-12T06:07:58.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.181+0000 7f9accd60700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 0x7f9ab806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.181+0000 7f9accd60700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 0x7f9ab806f1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9abc006010 tx=0x7f9abc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.185+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9ab40568f0 con 0x7f9ac8105bd0 2024-04-12T06:07:58.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.397+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac8068db0 con 0x7f9ab806cce0 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.408+0000 7f9ac5ffb700 1 -- 172.21.15.8:0/1069497257 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9ac8068db0 con 0x7f9ab806cce0 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (9m) 3m ago 21m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (21m) 3m ago 21m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (20m) 2m ago 20m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (7m) 3m ago 21m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (7m) 2m ago 20m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:07:58.406 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (9m) 3m ago 20m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (3m) 3m ago 15m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (3m) 3m ago 15m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (3m) 2m ago 15m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (3m) 2m ago 15m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (11m) 3m ago 22m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (10m) 2m ago 20m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (8m) 3m ago 22m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (7m) 2m ago 20m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (10m) 3m ago 21m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (10m) 2m ago 20m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 3m ago 19m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (6m) 3m ago 18m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (5m) 3m ago 18m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:07:58.407 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (5m) 2m ago 17m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:07:58.408 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (4m) 2m ago 17m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:07:58.408 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (4m) 2m ago 16m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:07:58.408 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (10m) 3m ago 20m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:07:58.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 msgr2=0x7f9ab806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 0x7f9ab806f1a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9abc006010 tx=0x7f9abc01a040 comp rx=0 tx=0).stop 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 msgr2=0x7f9ac8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400c930 tx=0x7f9ab400ccf0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 shutdown_connections 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ab806cce0 0x7f9ab806f1a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ac80ffc20 0x7f9ac8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 --2- 172.21.15.8:0/1069497257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ac8105bd0 0x7f9ac8101d80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 >> 172.21.15.8:0/1069497257 conn(0x7f9ac8075670 msgr2=0x7f9ac80686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 shutdown_connections 2024-04-12T06:07:58.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.410+0000 7f9acefc4700 1 -- 172.21.15.8:0/1069497257 wait complete. 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 -- 172.21.15.8:0/115962115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 msgr2=0x7fe6540689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/115962115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540689e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe648009910 tx=0x7fe648009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 -- 172.21.15.8:0/115962115 shutdown_connections 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/115962115 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540689e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/115962115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 0x7fe6541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 -- 172.21.15.8:0/115962115 >> 172.21.15.8:0/115962115 conn(0x7fe654075670 msgr2=0x7fe654075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 -- 172.21.15.8:0/115962115 shutdown_connections 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.515+0000 7fe65a3a3700 1 -- 172.21.15.8:0/115962115 wait complete. 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 Processor -- start 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 -- start start 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 0x7fe654100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe654100770 con 0x7fe654068580 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.516+0000 7fe65a3a3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6541008b0 con 0x7fe654106cd0 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40336/0 (socket says 172.21.15.8:40336) 2024-04-12T06:07:58.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 -- 172.21.15.8:0/3245679247 learned_addr learned my addr 172.21.15.8:0/3245679247 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe6537fe700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 0x7fe654100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 -- 172.21.15.8:0/3245679247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 msgr2=0x7fe654100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 0x7fe654100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 -- 172.21.15.8:0/3245679247 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480095f0 con 0x7fe654068580 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe653fff700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d900 tx=0x7fe64400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6440041f0 con 0x7fe654068580 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe644004cd0 con 0x7fe654068580 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe644005660 con 0x7fe654068580 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6541a2940 con 0x7fe654068580 2024-04-12T06:07:58.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.517+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6541a2d50 con 0x7fe654068580 2024-04-12T06:07:58.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.518+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6541091d0 con 0x7fe654068580 2024-04-12T06:07:58.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.519+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe644004e40 con 0x7fe654068580 2024-04-12T06:07:58.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.520+0000 7fe6517fa700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 0x7fe63c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.520+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe64408c830 con 0x7fe654068580 2024-04-12T06:07:58.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.520+0000 7fe6537fe700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 0x7fe63c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.520+0000 7fe6537fe700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 0x7fe63c06f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6541014e0 tx=0x7fe64801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.524+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe644056940 con 0x7fe654068580 2024-04-12T06:07:58.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.812+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe65404f2e0 con 0x7fe654068580 2024-04-12T06:07:58.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.813+0000 7fe6517fa700 1 -- 172.21.15.8:0/3245679247 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe644016290 con 0x7fe654068580 2024-04-12T06:07:58.810 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:07:58.811 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:07:58.812 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:07:58.812 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:07:58.812 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:07:58.812 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 msgr2=0x7fe63c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 0x7fe63c06f1a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe6541014e0 tx=0x7fe64801a040 comp rx=0 tx=0).stop 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 msgr2=0x7fe6540ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe64400d900 tx=0x7fe64400dc10 comp rx=0 tx=0).stop 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 shutdown_connections 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe63c06cce0 0x7fe63c06f1a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe654068580 0x7fe6540ffcf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 --2- 172.21.15.8:0/3245679247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe654106cd0 0x7fe654100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 >> 172.21.15.8:0/3245679247 conn(0x7fe654075670 msgr2=0x7fe6540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 shutdown_connections 2024-04-12T06:07:58.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.815+0000 7fe65a3a3700 1 -- 172.21.15.8:0/3245679247 wait complete. 2024-04-12T06:07:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:58 smithi008 ceph-mon[146084]: pgmap v388: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:58.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:58 smithi008 ceph-mon[146084]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:58.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- 172.21.15.8:0/163185941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460102700 msgr2=0x7fb46010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 --2- 172.21.15.8:0/163185941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460102700 0x7fb46010abf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb450009910 tx=0x7fb450009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:58.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- 172.21.15.8:0/163185941 shutdown_connections 2024-04-12T06:07:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 --2- 172.21.15.8:0/163185941 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460102700 0x7fb46010abf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 --2- 172.21.15.8:0/163185941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460101de0 0x7fb4601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- 172.21.15.8:0/163185941 >> 172.21.15.8:0/163185941 conn(0x7fb4600fb610 msgr2=0x7fb4600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- 172.21.15.8:0/163185941 shutdown_connections 2024-04-12T06:07:58.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- 172.21.15.8:0/163185941 wait complete. 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 Processor -- start 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.921+0000 7fb4673e4700 1 -- start start 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460102700 0x7fb46019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46019d190 con 0x7fb460101de0 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb460196640 con 0x7fb460102700 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40350/0 (socket says 172.21.15.8:40350) 2024-04-12T06:07:58.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 -- 172.21.15.8:0/1656567515 learned_addr learned my addr 172.21.15.8:0/1656567515 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb46497f700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460102700 0x7fb46019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 -- 172.21.15.8:0/1656567515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460102700 msgr2=0x7fb46019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460102700 0x7fb46019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 -- 172.21.15.8:0/1656567515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4500095f0 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb465180700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d8d0 tx=0x7fb45c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb45c009940 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb45c010460 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb460196920 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb45c00f5f0 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.922+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb460196e70 con 0x7fb460101de0 2024-04-12T06:07:58.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.923+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4601082f0 con 0x7fb460101de0 2024-04-12T06:07:58.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.924+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb45c0105d0 con 0x7fb460101de0 2024-04-12T06:07:58.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.924+0000 7fb4567fc700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 0x7fb44c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:58.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.924+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb45c08c560 con 0x7fb460101de0 2024-04-12T06:07:58.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.924+0000 7fb46497f700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 0x7fb44c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:58.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.925+0000 7fb46497f700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 0x7fb44c06f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4600fcdf0 tx=0x7fb45001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:58.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:58.929+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb45c05a040 con 0x7fb460101de0 2024-04-12T06:07:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:58 smithi194 ceph-mon[106982]: pgmap v388: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:07:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:58 smithi194 ceph-mon[106982]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:59.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.175+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb4601970e0 con 0x7fb460101de0 2024-04-12T06:07:59.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.176+0000 7fb4567fc700 1 -- 172.21.15.8:0/1656567515 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fb45c010880 con 0x7fb460101de0 2024-04-12T06:07:59.173 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:07:59.173 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:07:59.173 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:07:59.174 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:07:59.175 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:59.176 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:07:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 msgr2=0x7fb44c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 0x7fb44c06f1a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb4600fcdf0 tx=0x7fb45001a040 comp rx=0 tx=0).stop 2024-04-12T06:07:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 msgr2=0x7fb46019c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00d8d0 tx=0x7fb45c00dc90 comp rx=0 tx=0).stop 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 shutdown_connections 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb44c06cce0 0x7fb44c06f1a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb460101de0 0x7fb46019c5c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 --2- 172.21.15.8:0/1656567515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460102700 0x7fb46019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 >> 172.21.15.8:0/1656567515 conn(0x7fb4600fb610 msgr2=0x7fb460105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 shutdown_connections 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.178+0000 7fb4673e4700 1 -- 172.21.15.8:0/1656567515 wait complete. 2024-04-12T06:07:59.178 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 -- 172.21.15.8:0/1818160360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 msgr2=0x7f50c81058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 --2- 172.21.15.8:0/1818160360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c81058a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009910 tx=0x7f50b4009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 -- 172.21.15.8:0/1818160360 shutdown_connections 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 --2- 172.21.15.8:0/1818160360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c81058a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 --2- 172.21.15.8:0/1818160360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 0x7f50c8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.285+0000 7f50cef09700 1 -- 172.21.15.8:0/1818160360 >> 172.21.15.8:0/1818160360 conn(0x7f50c80fc750 msgr2=0x7f50c80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.286+0000 7f50cef09700 1 -- 172.21.15.8:0/1818160360 shutdown_connections 2024-04-12T06:07:59.283 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.286+0000 7f50cef09700 1 -- 172.21.15.8:0/1818160360 wait complete. 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.286+0000 7f50cef09700 1 Processor -- start 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.286+0000 7f50cef09700 1 -- start start 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50cef09700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 0x7f50c819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50cef09700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50cef09700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c819d1f0 con 0x7f50c8101850 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50cef09700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c81966a0 con 0x7f50c8100ea0 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40364/0 (socket says 172.21.15.8:40364) 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 -- 172.21.15.8:0/662238998 learned_addr learned my addr 172.21.15.8:0/662238998 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50ccca5700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 0x7f50c819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 -- 172.21.15.8:0/662238998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 msgr2=0x7f50c819c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 0x7f50c819c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 -- 172.21.15.8:0/662238998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40095f0 con 0x7f50c8101850 2024-04-12T06:07:59.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c7fff700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b400bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:59.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b401d070 con 0x7f50c8101850 2024-04-12T06:07:59.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.287+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50b4003cf0 con 0x7f50c8101850 2024-04-12T06:07:59.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.288+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c8196920 con 0x7f50c8101850 2024-04-12T06:07:59.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.288+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b4021410 con 0x7f50c8101850 2024-04-12T06:07:59.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.288+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8196db0 con 0x7f50c8101850 2024-04-12T06:07:59.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.289+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c81092a0 con 0x7f50c8101850 2024-04-12T06:07:59.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.290+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f50b4003850 con 0x7f50c8101850 2024-04-12T06:07:59.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.290+0000 7f50c5ffb700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 0x7f50b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.290+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f50b408dc90 con 0x7f50c8101850 2024-04-12T06:07:59.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.290+0000 7f50ccca5700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 0x7f50b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.291+0000 7f50ccca5700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 0x7f50b806f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009bf0 tx=0x7f50bc009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:59.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.294+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f50b405b930 con 0x7f50c8101850 2024-04-12T06:07:59.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.516+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c8197090 con 0x7f50b806cce0 2024-04-12T06:07:59.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.517+0000 7f50c5ffb700 1 -- 172.21.15.8:0/662238998 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f50c8197090 con 0x7f50b806cce0 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:07:59.515 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:07:59.516 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:07:59.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 msgr2=0x7f50b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 0x7f50b806f1a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009bf0 tx=0x7f50bc009260 comp rx=0 tx=0).stop 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 msgr2=0x7f50c819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b400bb20 comp rx=0 tx=0).stop 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 shutdown_connections 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f50b806cce0 0x7f50b806f1a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f50c8100ea0 0x7f50c819c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 --2- 172.21.15.8:0/662238998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f50c8101850 0x7f50c819cb60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 >> 172.21.15.8:0/662238998 conn(0x7f50c80fc750 msgr2=0x7f50c80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 shutdown_connections 2024-04-12T06:07:59.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.519+0000 7f50cef09700 1 -- 172.21.15.8:0/662238998 wait complete. 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 -- 172.21.15.8:0/1315504741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8106cd0 msgr2=0x7fb3c81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/1315504741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8106cd0 0x7fb3c81070b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009910 tx=0x7fb3c4009c20 comp rx=0 tx=0).stop 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 -- 172.21.15.8:0/1315504741 shutdown_connections 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/1315504741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8068580 0x7fb3c80689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/1315504741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8106cd0 0x7fb3c81070b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 -- 172.21.15.8:0/1315504741 >> 172.21.15.8:0/1315504741 conn(0x7fb3c8075670 msgr2=0x7fb3c8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 -- 172.21.15.8:0/1315504741 shutdown_connections 2024-04-12T06:07:59.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.626+0000 7fb3cf90a700 1 -- 172.21.15.8:0/1315504741 wait complete. 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 Processor -- start 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 -- start start 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 0x7fb3c819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819d250 con 0x7fb3c8068580 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cf90a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c8196700 con 0x7fb3c8106cd0 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.626 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40382/0 (socket says 172.21.15.8:40382) 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 -- 172.21.15.8:0/2397333744 learned_addr learned my addr 172.21.15.8:0/2397333744 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3ccea5700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 0x7fb3c819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 -- 172.21.15.8:0/2397333744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 msgr2=0x7fb3c819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 0x7fb3c819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 -- 172.21.15.8:0/2397333744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c40095f0 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3ccea5700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 0x7fb3c819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.627+0000 7fb3cd6a6700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b5c0 tx=0x7fb3c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.628+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c401d070 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.628+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb3c400f460 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.628+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c8196980 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.628+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3c4021640 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.628+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c8196e70 con 0x7fb3c8068580 2024-04-12T06:07:59.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.629+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c81091f0 con 0x7fb3c8068580 2024-04-12T06:07:59.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.629+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb3c400f9b0 con 0x7fb3c8068580 2024-04-12T06:07:59.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.630+0000 7fb3be7fc700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 0x7fb3b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:07:59.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.630+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb3c408d840 con 0x7fb3c8068580 2024-04-12T06:07:59.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.630+0000 7fb3ccea5700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 0x7fb3b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:07:59.628 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.630+0000 7fb3ccea5700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 0x7fb3b406f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8197b70 tx=0x7fb3b8008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:07:59.631 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.634+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb3c405b2a0 con 0x7fb3c8068580 2024-04-12T06:07:59.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:59 smithi008 ceph-mon[146084]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:59.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:59 smithi008 ceph-mon[146084]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:07:59.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:59 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3245679247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:07:59.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:07:59 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1656567515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:07:59.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.916+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb3c8197cb0 con 0x7fb3c8068580 2024-04-12T06:07:59.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.917+0000 7fb3be7fc700 1 -- 172.21.15.8:0/2397333744 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb3c405ae30 con 0x7fb3c8068580 2024-04-12T06:07:59.914 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:07:59.915 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:07:59.915 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:07:59.915 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:07:59.915 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:07:59.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 msgr2=0x7fb3b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 0x7fb3b406f1a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8197b70 tx=0x7fb3b8008040 comp rx=0 tx=0).stop 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 msgr2=0x7fb3c819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400b5c0 tx=0x7fb3c400be20 comp rx=0 tx=0).stop 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 shutdown_connections 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb3b406cce0 0x7fb3b406f1a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb3c8068580 0x7fb3c819c680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 --2- 172.21.15.8:0/2397333744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3c8106cd0 0x7fb3c819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 >> 172.21.15.8:0/2397333744 conn(0x7fb3c8075670 msgr2=0x7fb3c80fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 shutdown_connections 2024-04-12T06:07:59.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:07:59.919+0000 7fb3cf90a700 1 -- 172.21.15.8:0/2397333744 wait complete. 2024-04-12T06:08:00.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:59 smithi194 ceph-mon[106982]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:00.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:59 smithi194 ceph-mon[106982]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:00.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:59 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3245679247' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:08:00.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:07:59 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1656567515' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:08:00.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:00 smithi008 ceph-mon[146084]: pgmap v389: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:00.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:00 smithi008 ceph-mon[146084]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:00.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:00 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2397333744' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:08:01.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:00 smithi194 ceph-mon[106982]: pgmap v389: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:01.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:00 smithi194 ceph-mon[106982]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:01.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:00 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2397333744' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:08:02.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:02 smithi008 ceph-mon[146084]: pgmap v390: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:02.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:03.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:02 smithi194 ceph-mon[106982]: pgmap v390: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:03.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:04.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:04 smithi008 ceph-mon[146084]: pgmap v391: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:05.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:04 smithi194 ceph-mon[106982]: pgmap v391: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:06.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:06 smithi008 ceph-mon[146084]: pgmap v392: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:07.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:06 smithi194 ceph-mon[106982]: pgmap v392: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:08.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:08 smithi008 ceph-mon[146084]: pgmap v393: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:09.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:08 smithi194 ceph-mon[106982]: pgmap v393: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:10.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:10 smithi008 ceph-mon[146084]: pgmap v394: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:11.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:10 smithi194 ceph-mon[106982]: pgmap v394: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:12.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:12 smithi008 ceph-mon[146084]: pgmap v395: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:13.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:12 smithi194 ceph-mon[106982]: pgmap v395: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:14.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:14 smithi008 ceph-mon[146084]: pgmap v396: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:15.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:14 smithi194 ceph-mon[106982]: pgmap v396: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:16.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:16 smithi008 ceph-mon[146084]: pgmap v397: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:17.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:16 smithi194 ceph-mon[106982]: pgmap v397: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:17.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:18.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:18.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:18 smithi008 ceph-mon[146084]: pgmap v398: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:19.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:18 smithi194 ceph-mon[106982]: pgmap v398: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:20.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:20 smithi008 ceph-mon[146084]: pgmap v399: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:21.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:20 smithi194 ceph-mon[106982]: pgmap v399: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:22.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:22 smithi008 ceph-mon[146084]: pgmap v400: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:23.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:22 smithi194 ceph-mon[106982]: pgmap v400: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:24.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:24 smithi008 ceph-mon[146084]: pgmap v401: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:25.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:24 smithi194 ceph-mon[106982]: pgmap v401: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:26.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:26 smithi008 ceph-mon[146084]: pgmap v402: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:27.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:26 smithi194 ceph-mon[106982]: pgmap v402: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:28.905 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:28 smithi008 ceph-mon[146084]: pgmap v403: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:29.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:28 smithi194 ceph-mon[106982]: pgmap v403: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:30.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 -- 172.21.15.8:0/1748855010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec106cd0 msgr2=0x7f7aec1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 --2- 172.21.15.8:0/1748855010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec106cd0 0x7f7aec1070b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009910 tx=0x7f7adc009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:30.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 -- 172.21.15.8:0/1748855010 shutdown_connections 2024-04-12T06:08:30.041 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 --2- 172.21.15.8:0/1748855010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec068580 0x7f7aec0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 --2- 172.21.15.8:0/1748855010 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec106cd0 0x7f7aec1070b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 -- 172.21.15.8:0/1748855010 >> 172.21.15.8:0/1748855010 conn(0x7f7aec075670 msgr2=0x7f7aec075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 -- 172.21.15.8:0/1748855010 shutdown_connections 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.040+0000 7f7af2504700 1 -- 172.21.15.8:0/1748855010 wait complete. 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 Processor -- start 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 -- start start 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec106cd0 0x7f7aec100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.042 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec100770 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.041+0000 7f7af2504700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec1008b0 con 0x7f7aec106cd0 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40452/0 (socket says 172.21.15.8:40452) 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 -- 172.21.15.8:0/2178001112 learned_addr learned my addr 172.21.15.8:0/2178001112 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aeb7fe700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec106cd0 0x7f7aec100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 -- 172.21.15.8:0/2178001112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec106cd0 msgr2=0x7f7aec100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec106cd0 0x7f7aec100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 -- 172.21.15.8:0/2178001112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0095f0 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7aebfff700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006010 tx=0x7f7adc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7adc01d070 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7adc00f460 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec100b30 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7adc021620 con 0x7f7aec068580 2024-04-12T06:08:30.043 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.042+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec1a2c90 con 0x7f7aec068580 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.044+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec1091f0 con 0x7f7aec068580 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.044+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7adc003a60 con 0x7f7aec068580 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.044+0000 7f7ae97fa700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 0x7f7ad406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.044+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7adc08d7d0 con 0x7f7aec068580 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.044+0000 7f7aeb7fe700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 0x7f7ad406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.045 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.045+0000 7f7aeb7fe700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 0x7f7ad406f1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005950 tx=0x7f7ae00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.046 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.049+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7adc057840 con 0x7f7aec068580 2024-04-12T06:08:30.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.275+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aec0611d0 con 0x7f7ad406cce0 2024-04-12T06:08:30.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.276+0000 7f7ae97fa700 1 -- 172.21.15.8:0/2178001112 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7aec0611d0 con 0x7f7ad406cce0 2024-04-12T06:08:30.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 msgr2=0x7f7ad406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 0x7f7ad406f1a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005950 tx=0x7f7ae00058e0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 msgr2=0x7f7aec0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006010 tx=0x7f7adc00be20 comp rx=0 tx=0).stop 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 shutdown_connections 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ad406cce0 0x7f7ad406f1a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7aec068580 0x7f7aec0ffcf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 --2- 172.21.15.8:0/2178001112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7aec106cd0 0x7f7aec100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 >> 172.21.15.8:0/2178001112 conn(0x7f7aec075670 msgr2=0x7f7aec0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 shutdown_connections 2024-04-12T06:08:30.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.279+0000 7f7af2504700 1 -- 172.21.15.8:0/2178001112 wait complete. 2024-04-12T06:08:30.287 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:08:30.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 -- 172.21.15.8:0/3396172402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0068580 msgr2=0x7fb2d00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/3396172402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0068580 0x7fb2d00689e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009910 tx=0x7fb2c0009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 -- 172.21.15.8:0/3396172402 shutdown_connections 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/3396172402 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0068580 0x7fb2d00689e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/3396172402 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0106cd0 0x7fb2d01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 -- 172.21.15.8:0/3396172402 >> 172.21.15.8:0/3396172402 conn(0x7fb2d0075670 msgr2=0x7fb2d0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 -- 172.21.15.8:0/3396172402 shutdown_connections 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.386+0000 7fb2d84a0700 1 -- 172.21.15.8:0/3396172402 wait complete. 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 Processor -- start 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 -- start start 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0068580 0x7fb2d019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d019d110 con 0x7fb2d0106cd0 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d84a0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d01965c0 con 0x7fb2d0068580 2024-04-12T06:08:30.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40458/0 (socket says 172.21.15.8:40458) 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 -- 172.21.15.8:0/1618886249 learned_addr learned my addr 172.21.15.8:0/1618886249 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d623c700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0068580 0x7fb2d019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 -- 172.21.15.8:0/1618886249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0068580 msgr2=0x7fb2d019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0068580 0x7fb2d019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.387+0000 7fb2d5a3b700 1 -- 172.21.15.8:0/1618886249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c00095f0 con 0x7fb2d0106cd0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2d5a3b700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00092e0 tx=0x7fb2c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c001d070 con 0x7fb2d0106cd0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb2c0005e00 con 0x7fb2d0106cd0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2c000fe90 con 0x7fb2d0106cd0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d0196840 con 0x7fb2d0106cd0 2024-04-12T06:08:30.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.388+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0196d30 con 0x7fb2d0106cd0 2024-04-12T06:08:30.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.389+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d01091d0 con 0x7fb2d0106cd0 2024-04-12T06:08:30.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.389+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb2c000f6d0 con 0x7fb2d0106cd0 2024-04-12T06:08:30.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.390+0000 7fb2c77fe700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 0x7fb2bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.390+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb2c008d700 con 0x7fb2d0106cd0 2024-04-12T06:08:30.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.390+0000 7fb2d623c700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 0x7fb2bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.390+0000 7fb2d623c700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 0x7fb2bc06f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc005950 tx=0x7fb2cc00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.394+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb2c005b280 con 0x7fb2d0106cd0 2024-04-12T06:08:30.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.617+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2d0196fa0 con 0x7fb2bc06cce0 2024-04-12T06:08:30.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.618+0000 7fb2c77fe700 1 -- 172.21.15.8:0/1618886249 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2d0196fa0 con 0x7fb2bc06cce0 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 msgr2=0x7fb2bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 0x7fb2bc06f1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc005950 tx=0x7fb2cc00b410 comp rx=0 tx=0).stop 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 msgr2=0x7fb2d019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00092e0 tx=0x7fb2c000bd70 comp rx=0 tx=0).stop 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 shutdown_connections 2024-04-12T06:08:30.618 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb2bc06cce0 0x7fb2bc06f1a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2d0068580 0x7fb2d019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 --2- 172.21.15.8:0/1618886249 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb2d0106cd0 0x7fb2d019ca80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.620+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 >> 172.21.15.8:0/1618886249 conn(0x7fb2d0075670 msgr2=0x7fb2d00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.621+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 shutdown_connections 2024-04-12T06:08:30.619 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.621+0000 7fb2d84a0700 1 -- 172.21.15.8:0/1618886249 wait complete. 2024-04-12T06:08:30.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 -- 172.21.15.8:0/3970472187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f4105bf0 msgr2=0x7f05f4105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/3970472187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f4105bf0 0x7f05f4105fd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009910 tx=0x7f05e4009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 -- 172.21.15.8:0/3970472187 shutdown_connections 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/3970472187 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f40ffb40 0x7f05f40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/3970472187 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f4105bf0 0x7f05f4105fd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.727+0000 7f05f9bdb700 1 -- 172.21.15.8:0/3970472187 >> 172.21.15.8:0/3970472187 conn(0x7f05f40fb610 msgr2=0x7f05f40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 -- 172.21.15.8:0/3970472187 shutdown_connections 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 -- 172.21.15.8:0/3970472187 wait complete. 2024-04-12T06:08:30.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 Processor -- start 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 -- start start 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 0x7f05f419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f419d0c0 con 0x7f05f40ffb40 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.728+0000 7f05f9bdb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f41965c0 con 0x7f05f4105bf0 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40464/0 (socket says 172.21.15.8:40464) 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 -- 172.21.15.8:0/340966280 learned_addr learned my addr 172.21.15.8:0/340966280 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f2ffd700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 0x7f05f419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 -- 172.21.15.8:0/340966280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 msgr2=0x7f05f419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 0x7f05f419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 -- 172.21.15.8:0/340966280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40095f0 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f2ffd700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 0x7f05f419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f37fe700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f05e40092e0 tx=0x7f05e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e401d070 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f4196840 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05e4005e00 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f4196d30 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.729+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05e400fe60 con 0x7f05f40ffb40 2024-04-12T06:08:30.729 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.730+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f404f2e0 con 0x7f05f40ffb40 2024-04-12T06:08:30.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.731+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f05e400f6d0 con 0x7f05f40ffb40 2024-04-12T06:08:30.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.731+0000 7f05f0ff9700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 0x7f05dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:30.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.731+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f05e408d870 con 0x7f05f40ffb40 2024-04-12T06:08:30.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.731+0000 7f05f2ffd700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 0x7f05dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:30.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.732+0000 7f05f2ffd700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 0x7f05dc06f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005fd0 tx=0x7f05e8005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:30.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.736+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f05e405b370 con 0x7f05f40ffb40 2024-04-12T06:08:30.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:30 smithi008 ceph-mon[146084]: pgmap v404: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:30.943 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.945+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f05f4197070 con 0x7f05dc06cce0 2024-04-12T06:08:30.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.956+0000 7f05f0ff9700 1 -- 172.21.15.8:0/340966280 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f05f4197070 con 0x7f05dc06cce0 2024-04-12T06:08:30.954 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:08:30.954 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (10m) 4m ago 21m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:08:30.954 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (22m) 4m ago 22m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (20m) 3m ago 20m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (8m) 4m ago 21m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (8m) 3m ago 20m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (10m) 4m ago 21m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (4m) 4m ago 15m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (4m) 4m ago 15m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (3m) 3m ago 15m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (3m) 3m ago 15m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (12m) 4m ago 23m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (11m) 3m ago 20m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (8m) 4m ago 23m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (8m) 3m ago 20m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (11m) 4m ago 21m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:08:30.955 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (11m) 3m ago 20m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (7m) 4m ago 19m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (7m) 4m ago 19m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (6m) 4m ago 18m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (5m) 3m ago 18m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (5m) 3m ago 17m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (4m) 3m ago 17m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:08:30.956 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (10m) 4m ago 21m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 msgr2=0x7f05dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 0x7f05dc06f1a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f05e8005fd0 tx=0x7f05e8005ee0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 msgr2=0x7f05f419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f05e40092e0 tx=0x7f05e400be20 comp rx=0 tx=0).stop 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 shutdown_connections 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f05dc06cce0 0x7f05dc06f1a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f05f40ffb40 0x7f05f419c4f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 --2- 172.21.15.8:0/340966280 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05f4105bf0 0x7f05f419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.958+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 >> 172.21.15.8:0/340966280 conn(0x7f05f40fb610 msgr2=0x7f05f4068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.959+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 shutdown_connections 2024-04-12T06:08:30.957 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:30.959+0000 7f05f9bdb700 1 -- 172.21.15.8:0/340966280 wait complete. 2024-04-12T06:08:31.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- 172.21.15.8:0/815067460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 msgr2=0x7f7d44102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 --2- 172.21.15.8:0/815067460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d44102240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009910 tx=0x7f7d38009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- 172.21.15.8:0/815067460 shutdown_connections 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 --2- 172.21.15.8:0/815067460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 0x7f7d4410ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 --2- 172.21.15.8:0/815067460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d44102240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- 172.21.15.8:0/815067460 >> 172.21.15.8:0/815067460 conn(0x7f7d440fb650 msgr2=0x7f7d440fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- 172.21.15.8:0/815067460 shutdown_connections 2024-04-12T06:08:31.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- 172.21.15.8:0/815067460 wait complete. 2024-04-12T06:08:31.067 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 Processor -- start 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.064+0000 7f7d497c8700 1 -- start start 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d497c8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d497c8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 0x7f7d44100000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d497c8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44100540 con 0x7f7d44101e60 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d497c8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d44100680 con 0x7f7d44102780 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40482/0 (socket says 172.21.15.8:40482) 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 -- 172.21.15.8:0/3129251748 learned_addr learned my addr 172.21.15.8:0/3129251748 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d427fc700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 0x7f7d44100000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 -- 172.21.15.8:0/3129251748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 msgr2=0x7f7d44100000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 0x7f7d44100000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.068 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 -- 172.21.15.8:0/3129251748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d42ffd700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006010 tx=0x7f7d3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7d3800f460 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d44100900 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.065+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d38021640 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.066+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d441a2dc0 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.067+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d44108340 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.067+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7d3800fa60 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.067+0000 7f7d2bfff700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 0x7f7d2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.068+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7d3808d850 con 0x7f7d44101e60 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.068+0000 7f7d427fc700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 0x7f7d2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.068+0000 7f7d427fc700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 0x7f7d2c06f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.070 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.072+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7d380578c0 con 0x7f7d44101e60 2024-04-12T06:08:31.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:30 smithi194 ceph-mon[106982]: pgmap v404: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:31.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.357+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7d441010a0 con 0x7f7d44101e60 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.357+0000 7f7d2bfff700 1 -- 172.21.15.8:0/3129251748 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7d3805aee0 con 0x7f7d44101e60 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:08:31.355 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:08:31.356 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:08:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 msgr2=0x7f7d2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 0x7f7d2c06f1a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006fd0 tx=0x7f7d34008040 comp rx=0 tx=0).stop 2024-04-12T06:08:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 msgr2=0x7f7d440ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006010 tx=0x7f7d3800be20 comp rx=0 tx=0).stop 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 shutdown_connections 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7d2c06cce0 0x7f7d2c06f1a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7d44101e60 0x7f7d440ffac0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 --2- 172.21.15.8:0/3129251748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d44102780 0x7f7d44100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 >> 172.21.15.8:0/3129251748 conn(0x7f7d440fb650 msgr2=0x7f7d44105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 shutdown_connections 2024-04-12T06:08:31.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.359+0000 7f7d497c8700 1 -- 172.21.15.8:0/3129251748 wait complete. 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 -- 172.21.15.8:0/1941108271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178100e90 msgr2=0x7fd178101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 --2- 172.21.15.8:0/1941108271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178100e90 0x7fd178101270 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd168009910 tx=0x7fd168009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 -- 172.21.15.8:0/1941108271 shutdown_connections 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 --2- 172.21.15.8:0/1941108271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178101840 0x7fd178105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 --2- 172.21.15.8:0/1941108271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178100e90 0x7fd178101270 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.465+0000 7fd17e390700 1 -- 172.21.15.8:0/1941108271 >> 172.21.15.8:0/1941108271 conn(0x7fd1780fc740 msgr2=0x7fd1780feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.466+0000 7fd17e390700 1 -- 172.21.15.8:0/1941108271 shutdown_connections 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.466+0000 7fd17e390700 1 -- 172.21.15.8:0/1941108271 wait complete. 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.466+0000 7fd17e390700 1 Processor -- start 2024-04-12T06:08:31.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.466+0000 7fd17e390700 1 -- start start 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17819d190 con 0x7fd178101840 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178196640 con 0x7fd178100e90 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40498/0 (socket says 172.21.15.8:40498) 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 -- 172.21.15.8:0/3750305888 learned_addr learned my addr 172.21.15.8:0/3750305888 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd177fff700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 -- 172.21.15.8:0/3750305888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178100e90 msgr2=0x7fd17819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 -- 172.21.15.8:0/3750305888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1680095f0 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1777fe700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e8f0 tx=0x7fd16c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16c009790 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178196920 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd16c00c810 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd16c005520 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.467+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178196e70 con 0x7fd178101840 2024-04-12T06:08:31.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.468+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178109230 con 0x7fd178101840 2024-04-12T06:08:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.469+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd16c00c980 con 0x7fd178101840 2024-04-12T06:08:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.469+0000 7fd1757fa700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 0x7fd16006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.469+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd16c014070 con 0x7fd178101840 2024-04-12T06:08:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.469+0000 7fd177fff700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 0x7fd16006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.470+0000 7fd177fff700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 0x7fd16006f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd168009fd0 tx=0x7fd168005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.474+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd16c05a350 con 0x7fd178101840 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.724+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd17804f2e0 con 0x7fd178101840 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.724+0000 7fd1757fa700 1 -- 172.21.15.8:0/3750305888 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fd16c00cc30 con 0x7fd178101840 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:08:31.722 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:08:31.723 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:08:31.724 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:08:31.725 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 msgr2=0x7fd16006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 0x7fd16006f1a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd168009fd0 tx=0x7fd168005940 comp rx=0 tx=0).stop 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 msgr2=0x7fd17819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00e8f0 tx=0x7fd16c00ec00 comp rx=0 tx=0).stop 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 shutdown_connections 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd16006cce0 0x7fd16006f1a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178100e90 0x7fd17819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 --2- 172.21.15.8:0/3750305888 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd178101840 0x7fd17819cb00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 >> 172.21.15.8:0/3750305888 conn(0x7fd1780fc740 msgr2=0x7fd1780fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.726+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 shutdown_connections 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.727+0000 7fd17e390700 1 -- 172.21.15.8:0/3750305888 wait complete. 2024-04-12T06:08:31.726 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 -- 172.21.15.8:0/3936592363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c105bf0 msgr2=0x7fbe5c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/3936592363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c105fd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009910 tx=0x7fbe50009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 -- 172.21.15.8:0/3936592363 shutdown_connections 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/3936592363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/3936592363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c105fd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.833+0000 7fbe62fa7700 1 -- 172.21.15.8:0/3936592363 >> 172.21.15.8:0/3936592363 conn(0x7fbe5c0fb610 msgr2=0x7fbe5c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 -- 172.21.15.8:0/3936592363 shutdown_connections 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 -- 172.21.15.8:0/3936592363 wait complete. 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 Processor -- start 2024-04-12T06:08:31.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 -- start start 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c198110 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.834+0000 7fbe62fa7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c198250 con 0x7fbe5c105bf0 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40506/0 (socket says 172.21.15.8:40506) 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 -- 172.21.15.8:0/990080956 learned_addr learned my addr 172.21.15.8:0/990080956 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe5bfff700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.835 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 -- 172.21.15.8:0/990080956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 msgr2=0x7fbe5c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 -- 172.21.15.8:0/990080956 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe500095f0 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe5bfff700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe60d43700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe50006010 tx=0x7fbe5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe5001d070 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbe5000f460 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c1a0620 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe50021640 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.835+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1a0a30 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.836+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c108020 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.837+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbe5000f9e0 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.837+0000 7fbe59ffb700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 0x7fbe4c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.837+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbe5008d7d0 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.836 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.837+0000 7fbe5bfff700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 0x7fbe4c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:31.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.838+0000 7fbe5bfff700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 0x7fbe4c06f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0696c0 tx=0x7fbe48005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:31.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:31.842+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbe50057840 con 0x7fbe5c0ffb40 2024-04-12T06:08:31.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:31 smithi008 ceph-mon[146084]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:31.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:31 smithi008 ceph-mon[146084]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:31.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:31 smithi008 ceph-mon[146084]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:31.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:31 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3129251748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:08:32.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.062+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe5c0611d0 con 0x7fbe4c06cce0 2024-04-12T06:08:32.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.064+0000 7fbe59ffb700 1 -- 172.21.15.8:0/990080956 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbe5c0611d0 con 0x7fbe4c06cce0 2024-04-12T06:08:32.061 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:08:32.062 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:08:32.063 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 msgr2=0x7fbe4c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 0x7fbe4c06f1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0696c0 tx=0x7fbe48005f50 comp rx=0 tx=0).stop 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 msgr2=0x7fbe5c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe50006010 tx=0x7fbe5000be20 comp rx=0 tx=0).stop 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 shutdown_connections 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbe4c06cce0 0x7fbe4c06f1a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbe5c0ffb40 0x7fbe5c1975b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 --2- 172.21.15.8:0/990080956 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbe5c105bf0 0x7fbe5c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 >> 172.21.15.8:0/990080956 conn(0x7fbe5c0fb610 msgr2=0x7fbe5c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 shutdown_connections 2024-04-12T06:08:32.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.066+0000 7fbe62fa7700 1 -- 172.21.15.8:0/990080956 wait complete. 2024-04-12T06:08:32.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:31 smithi194 ceph-mon[106982]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:32.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:31 smithi194 ceph-mon[106982]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:32.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:31 smithi194 ceph-mon[106982]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:32.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:31 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3129251748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 -- 172.21.15.8:0/3456627580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 msgr2=0x7faedc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 --2- 172.21.15.8:0/3456627580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc0689e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faed0009910 tx=0x7faed0009c20 comp rx=0 tx=0).stop 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 -- 172.21.15.8:0/3456627580 shutdown_connections 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 --2- 172.21.15.8:0/3456627580 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc0689e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 --2- 172.21.15.8:0/3456627580 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 0x7faedc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 -- 172.21.15.8:0/3456627580 >> 172.21.15.8:0/3456627580 conn(0x7faedc075670 msgr2=0x7faedc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:32.170 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 -- 172.21.15.8:0/3456627580 shutdown_connections 2024-04-12T06:08:32.171 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.172+0000 7faee27cc700 1 -- 172.21.15.8:0/3456627580 wait complete. 2024-04-12T06:08:32.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 Processor -- start 2024-04-12T06:08:32.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 -- start start 2024-04-12T06:08:32.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:32.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 0x7faedc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:32.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc19d110 con 0x7faedc068580 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faee27cc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc1965c0 con 0x7faedc106cd0 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faedbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faedbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40528/0 (socket says 172.21.15.8:40528) 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faedbfff700 1 -- 172.21.15.8:0/2061709200 learned_addr learned my addr 172.21.15.8:0/2061709200 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.173+0000 7faedb7fe700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 0x7faedc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faedbfff700 1 -- 172.21.15.8:0/2061709200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 msgr2=0x7faedc19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faedbfff700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 0x7faedc19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faedbfff700 1 -- 172.21.15.8:0/2061709200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00095f0 con 0x7faedc068580 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faedbfff700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faecc00d8d0 tx=0x7faecc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faecc0098a0 con 0x7faedc068580 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faecc010460 con 0x7faedc068580 2024-04-12T06:08:32.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc1968a0 con 0x7faedc068580 2024-04-12T06:08:32.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faecc009ea0 con 0x7faedc068580 2024-04-12T06:08:32.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.174+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc196df0 con 0x7faedc068580 2024-04-12T06:08:32.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.175+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc1091d0 con 0x7faedc068580 2024-04-12T06:08:32.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.175+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7faecc009a00 con 0x7faedc068580 2024-04-12T06:08:32.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.176+0000 7faed97fa700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 0x7faec406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:08:32.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.176+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faecc08c370 con 0x7faedc068580 2024-04-12T06:08:32.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.176+0000 7faedb7fe700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 0x7faec406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:08:32.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.176+0000 7faedb7fe700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 0x7faec406f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faedc0fe260 tx=0x7faed001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:08:32.178 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.180+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7faecc059ef0 con 0x7faedc068580 2024-04-12T06:08:32.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.466+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faedc197060 con 0x7faedc068580 2024-04-12T06:08:32.464 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.467+0000 7faed97fa700 1 -- 172.21.15.8:0/2061709200 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7faecc059a80 con 0x7faedc068580 2024-04-12T06:08:32.464 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:08:32.465 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:08:32.465 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:08:32.465 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:08:32.465 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:08:32.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 msgr2=0x7faec406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 0x7faec406f1a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faedc0fe260 tx=0x7faed001a040 comp rx=0 tx=0).stop 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 msgr2=0x7faedc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faecc00d8d0 tx=0x7faecc00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 shutdown_connections 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faec406cce0 0x7faec406f1a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faedc068580 0x7faedc19c540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 --2- 172.21.15.8:0/2061709200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faedc106cd0 0x7faedc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 >> 172.21.15.8:0/2061709200 conn(0x7faedc075670 msgr2=0x7faedc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 shutdown_connections 2024-04-12T06:08:32.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:08:32.469+0000 7faee27cc700 1 -- 172.21.15.8:0/2061709200 wait complete. 2024-04-12T06:08:32.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:32 smithi008 ceph-mon[146084]: pgmap v405: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:32.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:32 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3750305888' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:08:32.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:32.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:32 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2061709200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:08:33.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:32 smithi194 ceph-mon[106982]: pgmap v405: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:33.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:32 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3750305888' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:08:33.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:33.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:32 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2061709200' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:08:34.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:33 smithi194 ceph-mon[106982]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:34.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:33 smithi008 ceph-mon[146084]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:08:35.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:34 smithi194 ceph-mon[106982]: pgmap v406: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:35.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:34 smithi008 ceph-mon[146084]: pgmap v406: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:37.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:36 smithi194 ceph-mon[106982]: pgmap v407: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:37.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:36 smithi008 ceph-mon[146084]: pgmap v407: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:39.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:38 smithi194 ceph-mon[106982]: pgmap v408: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:39.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:38 smithi008 ceph-mon[146084]: pgmap v408: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:40.957 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:40 smithi194 ceph-mon[106982]: pgmap v409: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:41.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:40 smithi008 ceph-mon[146084]: pgmap v409: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:43.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:42 smithi194 ceph-mon[106982]: pgmap v410: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:42 smithi008 ceph-mon[146084]: pgmap v410: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:45.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:44 smithi194 ceph-mon[106982]: pgmap v411: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:45.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:44 smithi008 ceph-mon[146084]: pgmap v411: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:47.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:46 smithi194 ceph-mon[106982]: pgmap v412: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:47.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:46 smithi008 ceph-mon[146084]: pgmap v412: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:48.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:48.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:08:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:48 smithi194 ceph-mon[106982]: pgmap v413: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:49.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:48 smithi008 ceph-mon[146084]: pgmap v413: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:51.050 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:50 smithi194 ceph-mon[106982]: pgmap v414: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:51.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:50 smithi008 ceph-mon[146084]: pgmap v414: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:53.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:52 smithi194 ceph-mon[106982]: pgmap v415: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:53.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:52 smithi008 ceph-mon[146084]: pgmap v415: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:55.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:54 smithi194 ceph-mon[106982]: pgmap v416: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:55.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:54 smithi008 ceph-mon[146084]: pgmap v416: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:56 smithi194 ceph-mon[106982]: pgmap v417: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:57.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:56 smithi008 ceph-mon[146084]: pgmap v417: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:08:58 smithi194 ceph-mon[106982]: pgmap v418: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:08:59.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:08:58 smithi008 ceph-mon[146084]: pgmap v418: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:00.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:00 smithi008 ceph-mon[146084]: pgmap v419: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:01.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:00 smithi194 ceph-mon[106982]: pgmap v419: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 -- 172.21.15.8:0/2947491766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 msgr2=0x7f8d44105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2947491766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44105fb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009910 tx=0x7f8d40009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 -- 172.21.15.8:0/2947491766 shutdown_connections 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2947491766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 0x7f8d44100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2947491766 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44105fb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 -- 172.21.15.8:0/2947491766 >> 172.21.15.8:0/2947491766 conn(0x7f8d44075670 msgr2=0x7f8d44075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 -- 172.21.15.8:0/2947491766 shutdown_connections 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.590+0000 7f8d4c561700 1 -- 172.21.15.8:0/2947491766 wait complete. 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 Processor -- start 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 -- start start 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d441035f0 con 0x7f8d44105bd0 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4c561700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44103760 con 0x7f8d440ffc20 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58268/0 (socket says 172.21.15.8:58268) 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 -- 172.21.15.8:0/2633674440 learned_addr learned my addr 172.21.15.8:0/2633674440 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:02.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d4a2fd700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 -- 172.21.15.8:0/2633674440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 msgr2=0x7f8d44101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 -- 172.21.15.8:0/2633674440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400095f0 con 0x7f8d44105bd0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.591+0000 7f8d49afc700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d8d0 tx=0x7f8d3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.592+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d34009940 con 0x7f8d44105bd0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.592+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44102380 con 0x7f8d44105bd0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.592+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d34010460 con 0x7f8d44105bd0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.592+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3400f5f0 con 0x7f8d44105bd0 2024-04-12T06:09:02.591 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.592+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d441a7330 con 0x7f8d44105bd0 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.593+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4404f2e0 con 0x7f8d44105bd0 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.593+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8d340105d0 con 0x7f8d44105bd0 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.594+0000 7f8d3b7fe700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 0x7f8d3006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.594+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d3408c5d0 con 0x7f8d44105bd0 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.594+0000 7f8d4a2fd700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 0x7f8d3006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:02.592 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.594+0000 7f8d4a2fd700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 0x7f8d3006f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d4001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:02.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.599+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8d34056640 con 0x7f8d44105bd0 2024-04-12T06:09:02.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.818+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d44068e10 con 0x7f8d3006cce0 2024-04-12T06:09:02.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.820+0000 7f8d3b7fe700 1 -- 172.21.15.8:0/2633674440 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8d44068e10 con 0x7f8d3006cce0 2024-04-12T06:09:02.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.821+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 msgr2=0x7f8d3006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.821+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 0x7f8d3006f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d4001a040 comp rx=0 tx=0).stop 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.821+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 msgr2=0x7f8d44101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8d3400d8d0 tx=0x7f8d3400dc90 comp rx=0 tx=0).stop 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 shutdown_connections 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d3006cce0 0x7f8d3006f1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d440ffc20 0x7f8d44101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 --2- 172.21.15.8:0/2633674440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d44105bd0 0x7f8d44101d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 >> 172.21.15.8:0/2633674440 conn(0x7f8d44075670 msgr2=0x7f8d44068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 shutdown_connections 2024-04-12T06:09:02.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.822+0000 7f8d4c561700 1 -- 172.21.15.8:0/2633674440 wait complete. 2024-04-12T06:09:02.830 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:09:02.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 -- 172.21.15.8:0/1513121908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6900ffda0 msgr2=0x7fd690100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 --2- 172.21.15.8:0/1513121908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6900ffda0 0x7fd690100180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd680009910 tx=0x7fd680009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 -- 172.21.15.8:0/1513121908 shutdown_connections 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 --2- 172.21.15.8:0/1513121908 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd690100750 0x7fd6901047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 --2- 172.21.15.8:0/1513121908 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6900ffda0 0x7fd690100180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 -- 172.21.15.8:0/1513121908 >> 172.21.15.8:0/1513121908 conn(0x7fd6900fb650 msgr2=0x7fd6900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:02.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 -- 172.21.15.8:0/1513121908 shutdown_connections 2024-04-12T06:09:02.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.928+0000 7fd695917700 1 -- 172.21.15.8:0/1513121908 wait complete. 2024-04-12T06:09:02.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 Processor -- start 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 -- start start 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6900ffda0 0x7fd69019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69019d0d0 con 0x7fd690100750 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.929+0000 7fd695917700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd690196610 con 0x7fd6900ffda0 2024-04-12T06:09:02.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58284/0 (socket says 172.21.15.8:58284) 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 -- 172.21.15.8:0/3179614966 learned_addr learned my addr 172.21.15.8:0/3179614966 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 -- 172.21.15.8:0/3179614966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6900ffda0 msgr2=0x7fd69019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6900ffda0 0x7fd69019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 -- 172.21.15.8:0/3179614966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800095f0 con 0x7fd690100750 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd68e7fc700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c960 tx=0x7fd68400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:02.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd684007a70 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6901968f0 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd690196e40 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd684007bd0 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.930+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd684018630 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.931+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67c0052f0 con 0x7fd690100750 2024-04-12T06:09:02.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.932+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd684018790 con 0x7fd690100750 2024-04-12T06:09:02.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.932+0000 7fd694915700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 0x7fd67806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:02.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.932+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd68408c380 con 0x7fd690100750 2024-04-12T06:09:02.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.932+0000 7fd68effd700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 0x7fd67806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:02.933 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.933+0000 7fd68effd700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 0x7fd67806f1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b5c0 tx=0x7fd680005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:02.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:02.937+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd684059c70 con 0x7fd690100750 2024-04-12T06:09:03.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:02 smithi194 ceph-mon[106982]: pgmap v420: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:03.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:02 smithi008 ceph-mon[146084]: pgmap v420: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:03.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:03.157 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.159+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd67c000bc0 con 0x7fd67806cce0 2024-04-12T06:09:03.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.160+0000 7fd694915700 1 -- 172.21.15.8:0/3179614966 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd67c000bc0 con 0x7fd67806cce0 2024-04-12T06:09:03.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 msgr2=0x7fd67806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 0x7fd67806f1a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b5c0 tx=0x7fd680005960 comp rx=0 tx=0).stop 2024-04-12T06:09:03.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 msgr2=0x7fd69019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c960 tx=0x7fd68400cd20 comp rx=0 tx=0).stop 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 shutdown_connections 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd67806cce0 0x7fd67806f1a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6900ffda0 0x7fd69019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 --2- 172.21.15.8:0/3179614966 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd690100750 0x7fd69019cad0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 >> 172.21.15.8:0/3179614966 conn(0x7fd6900fb650 msgr2=0x7fd690103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 shutdown_connections 2024-04-12T06:09:03.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.162+0000 7fd695917700 1 -- 172.21.15.8:0/3179614966 wait complete. 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.268+0000 7f97e4498700 1 -- 172.21.15.8:0/176552350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 msgr2=0x7f97dc0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.268+0000 7f97e4498700 1 --2- 172.21.15.8:0/176552350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc0689e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009910 tx=0x7f97cc009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- 172.21.15.8:0/176552350 shutdown_connections 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 --2- 172.21.15.8:0/176552350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc0689e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 --2- 172.21.15.8:0/176552350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- 172.21.15.8:0/176552350 >> 172.21.15.8:0/176552350 conn(0x7f97dc075670 msgr2=0x7f97dc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- 172.21.15.8:0/176552350 shutdown_connections 2024-04-12T06:09:03.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- 172.21.15.8:0/176552350 wait complete. 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 Processor -- start 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- start start 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc19d080 con 0x7f97dc068580 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.269+0000 7f97e4498700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1965c0 con 0x7f97dc106cd0 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58296/0 (socket says 172.21.15.8:58296) 2024-04-12T06:09:03.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 -- 172.21.15.8:0/3758068589 learned_addr learned my addr 172.21.15.8:0/3758068589 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e1a33700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 -- 172.21.15.8:0/3758068589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 msgr2=0x7f97dc19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 -- 172.21.15.8:0/3758068589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0095f0 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e1a33700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e2234700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c930 tx=0x7f97d800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d8007ad0 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f97d8007c30 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc1968a0 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d8018630 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.270+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc196df0 con 0x7f97dc068580 2024-04-12T06:09:03.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.271+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc1091d0 con 0x7f97dc068580 2024-04-12T06:09:03.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.272+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f97d800f450 con 0x7f97dc068580 2024-04-12T06:09:03.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.272+0000 7f97d37fe700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.272+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f97d808c750 con 0x7f97dc068580 2024-04-12T06:09:03.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.272+0000 7f97e1a33700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.273+0000 7f97e1a33700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0fe260 tx=0x7f97cc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:03.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.277+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f97d805a380 con 0x7f97dc068580 2024-04-12T06:09:03.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.489+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f97dc197be0 con 0x7f97c806cce0 2024-04-12T06:09:03.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.500+0000 7f97d37fe700 1 -- 172.21.15.8:0/3758068589 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f97dc197be0 con 0x7f97c806cce0 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (10m) 4m ago 22m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (22m) 4m ago 22m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (21m) 4m ago 21m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (8m) 4m ago 22m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (8m) 4m ago 21m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (10m) 4m ago 22m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (4m) 4m ago 16m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (4m) 4m ago 16m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (4m) 4m ago 16m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (4m) 4m ago 16m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (13m) 4m ago 23m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (12m) 4m ago 21m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:09:03.499 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (9m) 4m ago 24m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (8m) 4m ago 21m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (11m) 4m ago 22m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (11m) 4m ago 21m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (8m) 4m ago 20m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (7m) 4m ago 19m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (6m) 4m ago 19m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (6m) 4m ago 18m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (5m) 4m ago 18m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (5m) 4m ago 17m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:09:03.500 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (11m) 4m ago 21m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 msgr2=0x7f97c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0fe260 tx=0x7f97cc01a040 comp rx=0 tx=0).stop 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 msgr2=0x7f97dc19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c930 tx=0x7f97d800ccf0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 shutdown_connections 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f97c806cce0 0x7f97c806f1a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f97dc068580 0x7f97dc19c540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 --2- 172.21.15.8:0/3758068589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97dc106cd0 0x7f97dc19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 >> 172.21.15.8:0/3758068589 conn(0x7f97dc075670 msgr2=0x7f97dc0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 shutdown_connections 2024-04-12T06:09:03.501 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.502+0000 7f97e4498700 1 -- 172.21.15.8:0/3758068589 wait complete. 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 -- 172.21.15.8:0/3215227301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 msgr2=0x7fc6cc1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 --2- 172.21.15.8:0/3215227301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc1070b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009910 tx=0x7fc6bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 -- 172.21.15.8:0/3215227301 shutdown_connections 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 --2- 172.21.15.8:0/3215227301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 --2- 172.21.15.8:0/3215227301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc1070b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.608 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.607+0000 7fc6d081f700 1 -- 172.21.15.8:0/3215227301 >> 172.21.15.8:0/3215227301 conn(0x7fc6cc075670 msgr2=0x7fc6cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.608+0000 7fc6d081f700 1 -- 172.21.15.8:0/3215227301 shutdown_connections 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.608+0000 7fc6d081f700 1 -- 172.21.15.8:0/3215227301 wait complete. 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.608+0000 7fc6d081f700 1 Processor -- start 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.608+0000 7fc6d081f700 1 -- start start 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc100770 con 0x7fc6cc106cd0 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1008b0 con 0x7fc6cc068580 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53544/0 (socket says 172.21.15.8:53544) 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 -- 172.21.15.8:0/1315735151 learned_addr learned my addr 172.21.15.8:0/1315735151 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6ca59c700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 -- 172.21.15.8:0/1315735151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 msgr2=0x7fc6cc0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.609 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 -- 172.21.15.8:0/1315735151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0095f0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6ca59c700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:09:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6c9d9b700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000b4d0 tx=0x7fc6c000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c0011800 con 0x7fc6cc106cd0 2024-04-12T06:09:03.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc1a2aa0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6c0011e00 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c000f2e0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.609+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc1a2eb0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.610+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc1091d0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.611+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc6c0010380 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.611+0000 7fc6bb7fe700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 0x7fc6b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.611+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc6c008c150 con 0x7fc6cc106cd0 2024-04-12T06:09:03.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.611+0000 7fc6ca59c700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 0x7fc6b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:03.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.612+0000 7fc6ca59c700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 0x7fc6b406f1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009fd0 tx=0x7fc6bc00b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:03.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.616+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc6c00561c0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.903+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6cc04f2e0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.902 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.903+0000 7fc6bb7fe700 1 -- 172.21.15.8:0/1315735151 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc6c00597e0 con 0x7fc6cc106cd0 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:09:03.903 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:03.904 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:09:03.904 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:09:03.904 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:09:03.904 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:09:03.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 msgr2=0x7fc6b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 0x7fc6b406f1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009fd0 tx=0x7fc6bc00b540 comp rx=0 tx=0).stop 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 msgr2=0x7fc6cc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000b4d0 tx=0x7fc6c000b890 comp rx=0 tx=0).stop 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 shutdown_connections 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6b406cce0 0x7fc6b406f1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6cc068580 0x7fc6cc0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.905+0000 7fc6d081f700 1 --2- 172.21.15.8:0/1315735151 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6cc106cd0 0x7fc6cc100230 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.906+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 >> 172.21.15.8:0/1315735151 conn(0x7fc6cc075670 msgr2=0x7fc6cc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.906+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 shutdown_connections 2024-04-12T06:09:03.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:03.906+0000 7fc6d081f700 1 -- 172.21.15.8:0/1315735151 wait complete. 2024-04-12T06:09:04.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 -- 172.21.15.8:0/926635722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c105bd0 msgr2=0x7f921c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 --2- 172.21.15.8:0/926635722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c105bd0 0x7f921c105fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f920c009910 tx=0x7f920c009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:04.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 -- 172.21.15.8:0/926635722 shutdown_connections 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 --2- 172.21.15.8:0/926635722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c0ffc20 0x7f921c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 --2- 172.21.15.8:0/926635722 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c105bd0 0x7f921c105fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 -- 172.21.15.8:0/926635722 >> 172.21.15.8:0/926635722 conn(0x7f921c075670 msgr2=0x7f921c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 -- 172.21.15.8:0/926635722 shutdown_connections 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.011+0000 7f92209f3700 1 -- 172.21.15.8:0/926635722 wait complete. 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 Processor -- start 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 -- start start 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c105bd0 0x7f921c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c19d160 con 0x7f921c0ffc20 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f92209f3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c196610 con 0x7f921c105bd0 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f921a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f921a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53570/0 (socket says 172.21.15.8:53570) 2024-04-12T06:09:04.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f921a59c700 1 -- 172.21.15.8:0/3223825278 learned_addr learned my addr 172.21.15.8:0/3223825278 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.012+0000 7f9219d9b700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c105bd0 0x7f921c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f921a59c700 1 -- 172.21.15.8:0/3223825278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c105bd0 msgr2=0x7f921c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f921a59c700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c105bd0 0x7f921c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f921a59c700 1 -- 172.21.15.8:0/3223825278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0095f0 con 0x7f921c0ffc20 2024-04-12T06:09:04.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f921a59c700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920c01d070 con 0x7f921c0ffc20 2024-04-12T06:09:04.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c196890 con 0x7f921c0ffc20 2024-04-12T06:09:04.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f920c005940 con 0x7f921c0ffc20 2024-04-12T06:09:04.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f920c00fd90 con 0x7f921c0ffc20 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.013+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c196d80 con 0x7f921c0ffc20 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.014+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c04f2e0 con 0x7f921c0ffc20 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.015+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f920c003a60 con 0x7f921c0ffc20 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.015+0000 7f920b7fe700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 0x7f920406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.015+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f920c08d7b0 con 0x7f921c0ffc20 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.015+0000 7f9219d9b700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 0x7f920406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.016+0000 7f9219d9b700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 0x7f920406f1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f921c075ca0 tx=0x7f92100073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.020+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f920c05b210 con 0x7f921c0ffc20 2024-04-12T06:09:04.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:03 smithi194 ceph-mon[106982]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:04.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:03 smithi008 ceph-mon[146084]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.268+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f921c197030 con 0x7f921c0ffc20 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.268+0000 7f920b7fe700 1 -- 172.21.15.8:0/3223825278 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f920c05ada0 con 0x7f921c0ffc20 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:04.267 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:09:04.268 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:04.269 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:04.270 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 msgr2=0x7f920406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 0x7f920406f1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f921c075ca0 tx=0x7f92100073d0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 msgr2=0x7f921c19c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 shutdown_connections 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f920406cce0 0x7f920406f1a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f921c0ffc20 0x7f921c19c590 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 --2- 172.21.15.8:0/3223825278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f921c105bd0 0x7f921c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 >> 172.21.15.8:0/3223825278 conn(0x7f921c075670 msgr2=0x7f921c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 shutdown_connections 2024-04-12T06:09:04.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.270+0000 7f92209f3700 1 -- 172.21.15.8:0/3223825278 wait complete. 2024-04-12T06:09:04.272 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:09:04.377 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.377+0000 7f9979da2700 1 -- 172.21.15.8:0/2466918901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 msgr2=0x7f9974105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.377+0000 7f9979da2700 1 --2- 172.21.15.8:0/2466918901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974105d30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f9968009910 tx=0x7f9968009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- 172.21.15.8:0/2466918901 shutdown_connections 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 --2- 172.21.15.8:0/2466918901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 0x7f99740ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 --2- 172.21.15.8:0/2466918901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974105d30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- 172.21.15.8:0/2466918901 >> 172.21.15.8:0/2466918901 conn(0x7f9974075460 msgr2=0x7f9974075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- 172.21.15.8:0/2466918901 shutdown_connections 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- 172.21.15.8:0/2466918901 wait complete. 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 Processor -- start 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- start start 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 0x7f9974196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.378 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741970f0 con 0x7f9974105950 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.378+0000 7f9979da2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974197230 con 0x7f99740ffa20 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53574/0 (socket says 172.21.15.8:53574) 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 -- 172.21.15.8:0/3238215586 learned_addr learned my addr 172.21.15.8:0/3238215586 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9978da0700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 0x7f9974196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 -- 172.21.15.8:0/3238215586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 msgr2=0x7f9974196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 0x7f9974196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 -- 172.21.15.8:0/3238215586 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680095f0 con 0x7f9974105950 2024-04-12T06:09:04.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9973fff700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f996000c930 tx=0x7f996000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9960007ad0 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f996000ce80 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997419b080 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9960018630 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.379+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997419b5d0 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.381+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99740623c0 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.381+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9960007c30 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.381+0000 7f9971ffb700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 0x7f996406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.381+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f996008c4a0 con 0x7f9974105950 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.381+0000 7f9978da0700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 0x7f996406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.382+0000 7f9978da0700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 0x7f996406f1a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9968006010 tx=0x7f996801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.386+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9960056590 con 0x7f9974105950 2024-04-12T06:09:04.606 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.608+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f997419ba10 con 0x7f996406cce0 2024-04-12T06:09:04.607 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.609+0000 7f9971ffb700 1 -- 172.21.15.8:0/3238215586 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f997419ba10 con 0x7f996406cce0 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:09:04.608 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:09:04.609 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 msgr2=0x7f996406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 0x7f996406f1a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9968006010 tx=0x7f996801a040 comp rx=0 tx=0).stop 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 msgr2=0x7f9974196aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f996000c930 tx=0x7f996000cc40 comp rx=0 tx=0).stop 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 shutdown_connections 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f996406cce0 0x7f996406f1a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f99740ffa20 0x7f9974196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 --2- 172.21.15.8:0/3238215586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9974105950 0x7f9974196aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 >> 172.21.15.8:0/3238215586 conn(0x7f9974075460 msgr2=0x7f9974068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 shutdown_connections 2024-04-12T06:09:04.610 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.611+0000 7f9979da2700 1 -- 172.21.15.8:0/3238215586 wait complete. 2024-04-12T06:09:04.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- 172.21.15.8:0/3052828609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d4105bd0 msgr2=0x7f72d4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 --2- 172.21.15.8:0/3052828609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d4105bd0 0x7f72d4105fb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009910 tx=0x7f72d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:04.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- 172.21.15.8:0/3052828609 shutdown_connections 2024-04-12T06:09:04.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 --2- 172.21.15.8:0/3052828609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d40ffc20 0x7f72d4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 --2- 172.21.15.8:0/3052828609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d4105bd0 0x7f72d4105fb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- 172.21.15.8:0/3052828609 >> 172.21.15.8:0/3052828609 conn(0x7f72d4075670 msgr2=0x7f72d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- 172.21.15.8:0/3052828609 shutdown_connections 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- 172.21.15.8:0/3052828609 wait complete. 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 Processor -- start 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.717+0000 7f72db714700 1 -- start start 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72db714700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72db714700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 0x7f72d4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72db714700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d41022c0 con 0x7f72d40ffc20 2024-04-12T06:09:04.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72db714700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4102400 con 0x7f72d4105bd0 2024-04-12T06:09:04.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:53596/0 (socket says 172.21.15.8:53596) 2024-04-12T06:09:04.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 -- 172.21.15.8:0/806552522 learned_addr learned my addr 172.21.15.8:0/806552522 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:04.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d8caf700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 0x7f72d4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 -- 172.21.15.8:0/806552522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 msgr2=0x7f72d4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 0x7f72d4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 -- 172.21.15.8:0/806552522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00095f0 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d8caf700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 0x7f72d4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72d94b0700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72d00092e0 tx=0x7f72d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d001d070 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72d0005e00 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.718+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d4102680 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.719+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72d000fe60 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.719+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d41a2c70 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.719+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d404f2e0 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.720+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f72d000f6d0 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.720+0000 7f72ca7fc700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 0x7f72c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.721+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72d008d820 con 0x7f72d40ffc20 2024-04-12T06:09:04.720 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.721+0000 7f72d8caf700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 0x7f72c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:04.721 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.721+0000 7f72d8caf700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 0x7f72c006f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72c4005950 tx=0x7f72c40058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:04.723 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:04.725+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f72d0057890 con 0x7f72d40ffc20 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.011+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f72d41a2e60 con 0x7f72d40ffc20 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.011+0000 7f72ca7fc700 1 -- 172.21.15.8:0/806552522 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f72d005aeb0 con 0x7f72d40ffc20 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:09:05.010 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:09:05.011 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:09:05.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 msgr2=0x7f72c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:05.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 0x7f72c006f1a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72c4005950 tx=0x7f72c40058e0 comp rx=0 tx=0).stop 2024-04-12T06:09:05.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 msgr2=0x7f72d4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f72d00092e0 tx=0x7f72d000be20 comp rx=0 tx=0).stop 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 shutdown_connections 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f72c006cce0 0x7f72c006f1a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f72d40ffc20 0x7f72d4101840 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 --2- 172.21.15.8:0/806552522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72d4105bd0 0x7f72d4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 >> 172.21.15.8:0/806552522 conn(0x7f72d4075670 msgr2=0x7f72d4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 shutdown_connections 2024-04-12T06:09:05.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:05.013+0000 7f72db714700 1 -- 172.21.15.8:0/806552522 wait complete. 2024-04-12T06:09:05.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:04 smithi194 ceph-mon[106982]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:05.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:04 smithi194 ceph-mon[106982]: pgmap v421: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:05.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:04 smithi194 ceph-mon[106982]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:05.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:04 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1315735151' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:09:05.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:04 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3223825278' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:09:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:04 smithi008 ceph-mon[146084]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:04 smithi008 ceph-mon[146084]: pgmap v421: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:04 smithi008 ceph-mon[146084]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:04 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1315735151' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:09:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:04 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3223825278' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:09:05.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:05 smithi008 ceph-mon[146084]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:05.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:05 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/806552522' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:09:06.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:05 smithi194 ceph-mon[106982]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:06.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:05 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/806552522' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:09:07.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:06 smithi194 ceph-mon[106982]: pgmap v422: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:07.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:06 smithi008 ceph-mon[146084]: pgmap v422: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:09.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:08 smithi194 ceph-mon[106982]: pgmap v423: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:09.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:08 smithi008 ceph-mon[146084]: pgmap v423: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:10.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:10 smithi008 ceph-mon[146084]: pgmap v424: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:11.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:10 smithi194 ceph-mon[106982]: pgmap v424: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:13.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:12 smithi194 ceph-mon[106982]: pgmap v425: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:13.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:12 smithi008 ceph-mon[146084]: pgmap v425: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:15.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:14 smithi194 ceph-mon[106982]: pgmap v426: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:15.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:14 smithi008 ceph-mon[146084]: pgmap v426: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:17.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:16 smithi194 ceph-mon[106982]: pgmap v427: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:17.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:16 smithi008 ceph-mon[146084]: pgmap v427: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:18.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:18.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:18 smithi194 ceph-mon[106982]: pgmap v428: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:19.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:18 smithi008 ceph-mon[146084]: pgmap v428: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:21.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:20 smithi194 ceph-mon[106982]: pgmap v429: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:20 smithi008 ceph-mon[146084]: pgmap v429: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:23.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:22 smithi194 ceph-mon[106982]: pgmap v430: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:23.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:22 smithi008 ceph-mon[146084]: pgmap v430: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:25.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:24 smithi194 ceph-mon[106982]: pgmap v431: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:25.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:24 smithi008 ceph-mon[146084]: pgmap v431: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:27.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:26 smithi194 ceph-mon[106982]: pgmap v432: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:27.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:26 smithi008 ceph-mon[146084]: pgmap v432: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:29.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:28 smithi194 ceph-mon[106982]: pgmap v433: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:29.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:28 smithi008 ceph-mon[146084]: pgmap v433: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:31.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:30 smithi194 ceph-mon[106982]: pgmap v434: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:31.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:30 smithi008 ceph-mon[146084]: pgmap v434: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:33.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:32 smithi194 ceph-mon[106982]: pgmap v435: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:33.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:33.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:32 smithi008 ceph-mon[146084]: pgmap v435: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:33.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:35.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:34 smithi194 ceph-mon[106982]: pgmap v436: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:35.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.135+0000 7ff4d2a75700 1 -- 172.21.15.8:0/2603704235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc105bd0 msgr2=0x7ff4cc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.135+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/2603704235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc105bd0 0x7ff4cc105fb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc009910 tx=0x7ff4bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 -- 172.21.15.8:0/2603704235 shutdown_connections 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/2603704235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/2603704235 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc105bd0 0x7ff4cc105fb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 -- 172.21.15.8:0/2603704235 >> 172.21.15.8:0/2603704235 conn(0x7ff4cc075670 msgr2=0x7ff4cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 -- 172.21.15.8:0/2603704235 shutdown_connections 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 -- 172.21.15.8:0/2603704235 wait complete. 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.136+0000 7ff4d2a75700 1 Processor -- start 2024-04-12T06:09:35.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d2a75700 1 -- start start 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d2a75700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d2a75700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4cc105bd0 0x7ff4cc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d2a75700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc198160 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d2a75700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4cc1982a0 con 0x7ff4cc105bd0 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38098/0 (socket says 172.21.15.8:38098) 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 -- 172.21.15.8:0/3414380423 learned_addr learned my addr 172.21.15.8:0/3414380423 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 -- 172.21.15.8:0/3414380423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4cc105bd0 msgr2=0x7ff4cc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4cc105bd0 0x7ff4cc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 -- 172.21.15.8:0/3414380423 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc0095f0 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.137+0000 7ff4d0811700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000c00 tx=0x7ff4bc00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.138+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc01d070 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.138+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4cc1a0670 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.138+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff4bc0044c0 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.138+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4bc0039d0 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.138+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4cc1a0a20 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.139+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4cc1080f0 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.140+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff4bc0048e0 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.140+0000 7ff4c9ffb700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 0x7ff4b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.140+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff4bc08dc00 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.140+0000 7ff4cbfff700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 0x7ff4b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.141+0000 7ff4cbfff700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 0x7ff4b406f1a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005950 tx=0x7ff4c00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.143 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.145+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff4bc093020 con 0x7ff4cc0ffc20 2024-04-12T06:09:35.155 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:34 smithi008 ceph-mon[146084]: pgmap v436: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:35.364 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.366+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4cc0611d0 con 0x7ff4b406cce0 2024-04-12T06:09:35.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.367+0000 7ff4c9ffb700 1 -- 172.21.15.8:0/3414380423 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff4cc0611d0 con 0x7ff4b406cce0 2024-04-12T06:09:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 msgr2=0x7ff4b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 0x7ff4b406f1a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0005950 tx=0x7ff4c00058e0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 msgr2=0x7ff4cc197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc000c00 tx=0x7ff4bc00be90 comp rx=0 tx=0).stop 2024-04-12T06:09:35.367 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 shutdown_connections 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff4b406cce0 0x7ff4b406f1a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff4cc0ffc20 0x7ff4cc197600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 --2- 172.21.15.8:0/3414380423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff4cc105bd0 0x7ff4cc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 >> 172.21.15.8:0/3414380423 conn(0x7ff4cc075670 msgr2=0x7ff4cc0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 shutdown_connections 2024-04-12T06:09:35.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.369+0000 7ff4d2a75700 1 -- 172.21.15.8:0/3414380423 wait complete. 2024-04-12T06:09:35.378 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- 172.21.15.8:0/4033710937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 msgr2=0x7f12240ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 --2- 172.21.15.8:0/4033710937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f12240ff980 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1218009910 tx=0x7f1218009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- 172.21.15.8:0/4033710937 shutdown_connections 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 --2- 172.21.15.8:0/4033710937 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 0x7f1224105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 --2- 172.21.15.8:0/4033710937 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f12240ff980 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- 172.21.15.8:0/4033710937 >> 172.21.15.8:0/4033710937 conn(0x7f1224074e30 msgr2=0x7f1224075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- 172.21.15.8:0/4033710937 shutdown_connections 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- 172.21.15.8:0/4033710937 wait complete. 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 Processor -- start 2024-04-12T06:09:35.477 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.477+0000 7f122ab5b700 1 -- start start 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f122ab5b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f122ab5b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 0x7f122419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f122ab5b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419d170 con 0x7f12240ff5a0 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f122ab5b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1224196620 con 0x7f12240fff50 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38118/0 (socket says 172.21.15.8:38118) 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 -- 172.21.15.8:0/1893770987 learned_addr learned my addr 172.21.15.8:0/1893770987 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f1223fff700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 0x7f122419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 -- 172.21.15.8:0/1893770987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 msgr2=0x7f122419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 0x7f122419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 -- 172.21.15.8:0/1893770987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180095f0 con 0x7f12240ff5a0 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f12288f7700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1218006010 tx=0x7f121800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.478+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f121801d070 con 0x7f12240ff5a0 2024-04-12T06:09:35.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.479+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f121800f460 con 0x7f12240ff5a0 2024-04-12T06:09:35.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.479+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1218021640 con 0x7f12240ff5a0 2024-04-12T06:09:35.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.479+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12241968a0 con 0x7f12240ff5a0 2024-04-12T06:09:35.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.479+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1224196d40 con 0x7f12240ff5a0 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.480+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1224109270 con 0x7f12240ff5a0 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.480+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f121800f9e0 con 0x7f12240ff5a0 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.481+0000 7f1221ffb700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 0x7f121406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.481+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f121808d8c0 con 0x7f12240ff5a0 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.481+0000 7f1223fff700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 0x7f121406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.481+0000 7f1223fff700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 0x7f121406f1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1224107770 tx=0x7f1210005cf0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.483 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.485+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f121805b3c0 con 0x7f12240ff5a0 2024-04-12T06:09:35.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.706+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1224197c50 con 0x7f121406cce0 2024-04-12T06:09:35.705 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.707+0000 7f1221ffb700 1 -- 172.21.15.8:0/1893770987 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f121000b670 con 0x7f121406cce0 2024-04-12T06:09:35.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.709+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 msgr2=0x7f121406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.709+0000 7f122ab5b700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 0x7f121406f1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1224107770 tx=0x7f1210005cf0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.709+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 msgr2=0x7f122419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.709+0000 7f122ab5b700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1218006010 tx=0x7f121800be20 comp rx=0 tx=0).stop 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 shutdown_connections 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f121406cce0 0x7f121406f1a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f12240ff5a0 0x7f122419c5a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 --2- 172.21.15.8:0/1893770987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f12240fff50 0x7f122419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 >> 172.21.15.8:0/1893770987 conn(0x7f1224074e30 msgr2=0x7f12240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 shutdown_connections 2024-04-12T06:09:35.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.710+0000 7f122ab5b700 1 -- 172.21.15.8:0/1893770987 wait complete. 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 -- 172.21.15.8:0/623050892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 msgr2=0x7f37d00fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 --2- 172.21.15.8:0/623050892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d00fffc0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009910 tx=0x7f37c0009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 -- 172.21.15.8:0/623050892 shutdown_connections 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 --2- 172.21.15.8:0/623050892 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d00fffc0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 --2- 172.21.15.8:0/623050892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 0x7f37d0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.816+0000 7f37d75e9700 1 -- 172.21.15.8:0/623050892 >> 172.21.15.8:0/623050892 conn(0x7f37d00fb610 msgr2=0x7f37d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 -- 172.21.15.8:0/623050892 shutdown_connections 2024-04-12T06:09:35.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 -- 172.21.15.8:0/623050892 wait complete. 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 Processor -- start 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 -- start start 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 0x7f37d019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d019d0c0 con 0x7f37d00ffb40 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.817+0000 7f37d75e9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d0196570 con 0x7f37d0105bf0 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38146/0 (socket says 172.21.15.8:38146) 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 -- 172.21.15.8:0/3183327136 learned_addr learned my addr 172.21.15.8:0/3183327136 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:35.817 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d4b84700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 0x7f37d019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 -- 172.21.15.8:0/3183327136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 msgr2=0x7f37d019ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 0x7f37d019ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 -- 172.21.15.8:0/3183327136 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c00095f0 con 0x7f37d00ffb40 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d5385700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d8d0 tx=0x7f37cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37cc0098a0 con 0x7f37d00ffb40 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d0196850 con 0x7f37d00ffb40 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37cc010460 con 0x7f37d00ffb40 2024-04-12T06:09:35.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37cc009ea0 con 0x7f37d00ffb40 2024-04-12T06:09:35.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.818+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d0196da0 con 0x7f37d00ffb40 2024-04-12T06:09:35.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.819+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d004f2e0 con 0x7f37d00ffb40 2024-04-12T06:09:35.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.820+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f37cc0105d0 con 0x7f37d00ffb40 2024-04-12T06:09:35.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.821+0000 7f37c67fc700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 0x7f37bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:35.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.821+0000 7f37d4b84700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 0x7f37bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:35.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.821+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f37cc08c480 con 0x7f37d00ffb40 2024-04-12T06:09:35.820 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.822+0000 7f37d4b84700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 0x7f37bc06f1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37d00feb60 tx=0x7f37c001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:35.823 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:35.825+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f37cc05a0b0 con 0x7f37d00ffb40 2024-04-12T06:09:36.037 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.039+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f37d0197160 con 0x7f37bc06cce0 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.050+0000 7f37c67fc700 1 -- 172.21.15.8:0/3183327136 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f37d0197160 con 0x7f37bc06cce0 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (11m) 5m ago 22m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (23m) 5m ago 23m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (21m) 4m ago 21m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:09:36.048 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (9m) 5m ago 23m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (9m) 4m ago 21m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (11m) 5m ago 22m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (5m) 5m ago 16m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (5m) 5m ago 16m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (4m) 4m ago 16m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (4m) 4m ago 16m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (13m) 5m ago 24m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (12m) 4m ago 21m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (9m) 5m ago 24m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (9m) 4m ago 21m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (12m) 5m ago 23m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (12m) 4m ago 21m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (8m) 5m ago 20m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:09:36.049 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (8m) 5m ago 20m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:09:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (7m) 5m ago 19m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:09:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (6m) 4m ago 19m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:09:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (6m) 4m ago 18m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:09:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (5m) 4m ago 18m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:09:36.050 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (11m) 5m ago 22m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 msgr2=0x7f37bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 0x7f37bc06f1a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37d00feb60 tx=0x7f37c001a040 comp rx=0 tx=0).stop 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 msgr2=0x7f37d019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00d8d0 tx=0x7f37cc00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 shutdown_connections 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f37bc06cce0 0x7f37bc06f1a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f37d00ffb40 0x7f37d019c4f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 --2- 172.21.15.8:0/3183327136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f37d0105bf0 0x7f37d019ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 >> 172.21.15.8:0/3183327136 conn(0x7f37d00fb610 msgr2=0x7f37d0068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 shutdown_connections 2024-04-12T06:09:36.051 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.052+0000 7f37d75e9700 1 -- 172.21.15.8:0/3183327136 wait complete. 2024-04-12T06:09:36.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 -- 172.21.15.8:0/518164878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 msgr2=0x7f6ecc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/518164878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc105fb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009910 tx=0x7f6ebc009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 -- 172.21.15.8:0/518164878 shutdown_connections 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/518164878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 0x7f6ecc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/518164878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc105fb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 -- 172.21.15.8:0/518164878 >> 172.21.15.8:0/518164878 conn(0x7f6ecc075670 msgr2=0x7f6ecc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 -- 172.21.15.8:0/518164878 shutdown_connections 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.157+0000 7f6ed1c86700 1 -- 172.21.15.8:0/518164878 wait complete. 2024-04-12T06:09:36.156 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 Processor -- start 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 -- start start 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 0x7f6ecc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc19d150 con 0x7f6ecc105bd0 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ed1c86700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc196600 con 0x7f6ecc0ffc20 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ecaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ecaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38166/0 (socket says 172.21.15.8:38166) 2024-04-12T06:09:36.158 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ecaffd700 1 -- 172.21.15.8:0/75777969 learned_addr learned my addr 172.21.15.8:0/75777969 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.158+0000 7f6ecb7fe700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 0x7f6ecc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ecaffd700 1 -- 172.21.15.8:0/75777969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 msgr2=0x7f6ecc19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ecaffd700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 0x7f6ecc19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ecaffd700 1 -- 172.21.15.8:0/75777969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0095f0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ecaffd700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e920 tx=0x7f6ec000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ec000c9d0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc1968e0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6ec0010430 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ec0018650 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.159+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc196e30 con 0x7f6ecc105bd0 2024-04-12T06:09:36.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.160+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc04f2e0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.161+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f6ec00105a0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.161+0000 7f6ec8ff9700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 0x7f6eb406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.161+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ec0014070 con 0x7f6ecc105bd0 2024-04-12T06:09:36.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.161+0000 7f6ecb7fe700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 0x7f6eb406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.162+0000 7f6ecb7fe700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 0x7f6eb406f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b6e0 tx=0x7f6ebc005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.166+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6ec0052ab0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.454+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ecc1970b0 con 0x7f6ecc105bd0 2024-04-12T06:09:36.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.454+0000 7f6ec8ff9700 1 -- 172.21.15.8:0/75777969 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6ec005a350 con 0x7f6ecc105bd0 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:09:36.453 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:09:36.454 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:09:36.454 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:09:36.454 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:09:36.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.456+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 msgr2=0x7f6eb406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.456+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 0x7f6eb406f1a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b6e0 tx=0x7f6ebc005a90 comp rx=0 tx=0).stop 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.456+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 msgr2=0x7f6ecc19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.456+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000e920 tx=0x7f6ec000ec30 comp rx=0 tx=0).stop 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 shutdown_connections 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6eb406cce0 0x7f6eb406f1a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ecc0ffc20 0x7f6ecc19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 --2- 172.21.15.8:0/75777969 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6ecc105bd0 0x7f6ecc19cac0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 >> 172.21.15.8:0/75777969 conn(0x7f6ecc075670 msgr2=0x7f6ecc068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 shutdown_connections 2024-04-12T06:09:36.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.457+0000 7f6ed1c86700 1 -- 172.21.15.8:0/75777969 wait complete. 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 -- 172.21.15.8:0/2495990536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 msgr2=0x7f4b28101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/2495990536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28101280 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4b18009910 tx=0x7f4b18009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 -- 172.21.15.8:0/2495990536 shutdown_connections 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/2495990536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 0x7f4b281058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/2495990536 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28101280 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 -- 172.21.15.8:0/2495990536 >> 172.21.15.8:0/2495990536 conn(0x7f4b280fc750 msgr2=0x7f4b280feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 -- 172.21.15.8:0/2495990536 shutdown_connections 2024-04-12T06:09:36.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.563+0000 7f4b2cddf700 1 -- 172.21.15.8:0/2495990536 wait complete. 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 Processor -- start 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 -- start start 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 0x7f4b280735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b28073b10 con 0x7f4b28100ea0 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2cddf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b28073c50 con 0x7f4b28101850 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38190/0 (socket says 172.21.15.8:38190) 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2659c700 1 -- 172.21.15.8:0/1752247556 learned_addr learned my addr 172.21.15.8:0/1752247556 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b25d9b700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 0x7f4b280735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.564+0000 7f4b2659c700 1 -- 172.21.15.8:0/1752247556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 msgr2=0x7f4b280735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b2659c700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 0x7f4b280735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b2659c700 1 -- 172.21.15.8:0/1752247556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b180095f0 con 0x7f4b28100ea0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b2659c700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c00 tx=0x7f4b1800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1801d070 con 0x7f4b28100ea0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b180044c0 con 0x7f4b28100ea0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b28073ed0 con 0x7f4b28100ea0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b180039d0 con 0x7f4b28100ea0 2024-04-12T06:09:36.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.565+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b281a2c30 con 0x7f4b28100ea0 2024-04-12T06:09:36.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.566+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b28109240 con 0x7f4b28100ea0 2024-04-12T06:09:36.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.566+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4b180048e0 con 0x7f4b28100ea0 2024-04-12T06:09:36.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.567+0000 7f4b177fe700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 0x7f4b1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.567+0000 7f4b25d9b700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 0x7f4b1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.567+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b1808dc20 con 0x7f4b28100ea0 2024-04-12T06:09:36.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.567+0000 7f4b25d9b700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 0x7f4b1006f1a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b28074880 tx=0x7f4b1c009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.569 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.572+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4b18057c90 con 0x7f4b28100ea0 2024-04-12T06:09:36.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.816+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b28074660 con 0x7f4b28100ea0 2024-04-12T06:09:36.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.816+0000 7f4b177fe700 1 -- 172.21.15.8:0/1752247556 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4b1805b2b0 con 0x7f4b28100ea0 2024-04-12T06:09:36.814 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:09:36.815 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:09:36.816 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:36.817 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:09:36.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 msgr2=0x7f4b1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 0x7f4b1006f1a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b28074880 tx=0x7f4b1c009260 comp rx=0 tx=0).stop 2024-04-12T06:09:36.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 msgr2=0x7f4b28073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4b18000c00 tx=0x7f4b1800be90 comp rx=0 tx=0).stop 2024-04-12T06:09:36.818 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 shutdown_connections 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1006cce0 0x7f4b1006f1a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b28100ea0 0x7f4b28073090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 --2- 172.21.15.8:0/1752247556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b28101850 0x7f4b280735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 >> 172.21.15.8:0/1752247556 conn(0x7f4b280fc750 msgr2=0x7f4b280fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 shutdown_connections 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.818+0000 7f4b2cddf700 1 -- 172.21.15.8:0/1752247556 wait complete. 2024-04-12T06:09:36.819 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:09:36.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 -- 172.21.15.8:0/3341676013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 msgr2=0x7f65e410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.924 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/3341676013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e410abf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009910 tx=0x7f65d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 -- 172.21.15.8:0/3341676013 shutdown_connections 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/3341676013 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e410abf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/3341676013 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 0x7f65e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 -- 172.21.15.8:0/3341676013 >> 172.21.15.8:0/3341676013 conn(0x7f65e40fb610 msgr2=0x7f65e40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 -- 172.21.15.8:0/3341676013 shutdown_connections 2024-04-12T06:09:36.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.926+0000 7f65eb1ab700 1 -- 172.21.15.8:0/3341676013 wait complete. 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 Processor -- start 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 -- start start 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 0x7f65e419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e419d190 con 0x7f65e4102700 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65eb1ab700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4196640 con 0x7f65e4101de0 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65e3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38206/0 (socket says 172.21.15.8:38206) 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.927+0000 7f65e3fff700 1 -- 172.21.15.8:0/970357144 learned_addr learned my addr 172.21.15.8:0/970357144 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e8f47700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 0x7f65e419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e3fff700 1 -- 172.21.15.8:0/970357144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 msgr2=0x7f65e419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e3fff700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 0x7f65e419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e3fff700 1 -- 172.21.15.8:0/970357144 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00095f0 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e3fff700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65d00092e0 tx=0x7f65d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d001d070 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65d000f460 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e41968c0 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d0005720 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.928+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e4196db0 con 0x7f65e4102700 2024-04-12T06:09:36.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.929+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e41082f0 con 0x7f65e4102700 2024-04-12T06:09:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.930+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f65d000f5d0 con 0x7f65e4102700 2024-04-12T06:09:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.930+0000 7f65e1ffb700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 0x7f65d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.930+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f65d008d760 con 0x7f65e4102700 2024-04-12T06:09:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.930+0000 7f65e8f47700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 0x7f65d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:36.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.931+0000 7f65e8f47700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 0x7f65d406f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d8005fd0 tx=0x7f65d800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:36.932 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:36.935+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f65d005b260 con 0x7f65e4102700 2024-04-12T06:09:37.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:36 smithi194 ceph-mon[106982]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:37.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:36 smithi194 ceph-mon[106982]: pgmap v437: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:37.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:36 smithi194 ceph-mon[106982]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:37.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:36 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/75777969' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:09:37.154 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.156+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f65e4197020 con 0x7f65d406cce0 2024-04-12T06:09:37.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.157+0000 7f65e1ffb700 1 -- 172.21.15.8:0/970357144 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f65e4197020 con 0x7f65d406cce0 2024-04-12T06:09:37.155 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:09:37.156 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:09:37.157 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:09:37.158 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:09:37.158 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:09:37.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:36 smithi008 ceph-mon[146084]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:37.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:36 smithi008 ceph-mon[146084]: pgmap v437: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:37.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:36 smithi008 ceph-mon[146084]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:37.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:36 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/75777969' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:09:37.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 msgr2=0x7f65d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 0x7f65d406f1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d8005fd0 tx=0x7f65d800a560 comp rx=0 tx=0).stop 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 msgr2=0x7f65e419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f65d00092e0 tx=0x7f65d000be20 comp rx=0 tx=0).stop 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 shutdown_connections 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65d406cce0 0x7f65d406f1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65e4101de0 0x7f65e419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 --2- 172.21.15.8:0/970357144 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65e4102700 0x7f65e419cb00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 >> 172.21.15.8:0/970357144 conn(0x7f65e40fb610 msgr2=0x7f65e4105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:37.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.159+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 shutdown_connections 2024-04-12T06:09:37.161 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.160+0000 7f65eb1ab700 1 -- 172.21.15.8:0/970357144 wait complete. 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.265+0000 7f9634e37700 1 -- 172.21.15.8:0/3606341648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 msgr2=0x7f96300ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.265+0000 7f9634e37700 1 --2- 172.21.15.8:0/3606341648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f96300ff7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9618009910 tx=0x7f9618009c20 comp rx=0 tx=0).stop 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 -- 172.21.15.8:0/3606341648 shutdown_connections 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 --2- 172.21.15.8:0/3606341648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f96300ff7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 --2- 172.21.15.8:0/3606341648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 0x7f9630105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 -- 172.21.15.8:0/3606341648 >> 172.21.15.8:0/3606341648 conn(0x7f9630074e30 msgr2=0x7f9630075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 -- 172.21.15.8:0/3606341648 shutdown_connections 2024-04-12T06:09:37.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 -- 172.21.15.8:0/3606341648 wait complete. 2024-04-12T06:09:37.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 Processor -- start 2024-04-12T06:09:37.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.266+0000 7f9634e37700 1 -- start start 2024-04-12T06:09:37.266 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f9634e37700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f9634e37700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 0x7f9630101500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f9634e37700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630101a40 con 0x7f96300ff340 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f9634e37700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9630101b80 con 0x7f9630105350 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38218/0 (socket says 172.21.15.8:38218) 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 -- 172.21.15.8:0/1702357439 learned_addr learned my addr 172.21.15.8:0/1702357439 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962dd9b700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 0x7f9630101500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 -- 172.21.15.8:0/1702357439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 msgr2=0x7f9630101500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 0x7f9630101500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 -- 172.21.15.8:0/1702357439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180095f0 con 0x7f96300ff340 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f962e59c700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f962000dd40 tx=0x7f962000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.267+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f962000ad10 con 0x7f96300ff340 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.268+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f962000ae70 con 0x7f96300ff340 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.268+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96301a2a50 con 0x7f96300ff340 2024-04-12T06:09:37.267 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.268+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96200168a0 con 0x7f96300ff340 2024-04-12T06:09:37.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.268+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96301a2e60 con 0x7f96300ff340 2024-04-12T06:09:37.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.269+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f962000e450 con 0x7f96300ff340 2024-04-12T06:09:37.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.269+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9630109860 con 0x7f96300ff340 2024-04-12T06:09:37.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.269+0000 7f96277fe700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 0x7f961c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:09:37.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.269+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f962008c740 con 0x7f96300ff340 2024-04-12T06:09:37.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.269+0000 7f962dd9b700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 0x7f961c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:09:37.269 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.270+0000 7f962dd9b700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 0x7f961c06f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f961800b5c0 tx=0x7f9618005fb0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:09:37.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.274+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9620056860 con 0x7f96300ff340 2024-04-12T06:09:37.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.561+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f963004f2e0 con 0x7f96300ff340 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.561+0000 7f96277fe700 1 -- 172.21.15.8:0/1702357439 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f9620059e80 con 0x7f96300ff340 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:09:37.560 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 msgr2=0x7f961c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 0x7f961c06f1a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f961800b5c0 tx=0x7f9618005fb0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 msgr2=0x7f9630100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f962000dd40 tx=0x7f962000a530 comp rx=0 tx=0).stop 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 shutdown_connections 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f961c06cce0 0x7f961c06f1a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f96300ff340 0x7f9630100fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 --2- 172.21.15.8:0/1702357439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9630105350 0x7f9630101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:09:37.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 >> 172.21.15.8:0/1702357439 conn(0x7f9630074e30 msgr2=0x7f96301046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:09:37.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 shutdown_connections 2024-04-12T06:09:37.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:09:37.563+0000 7f9634e37700 1 -- 172.21.15.8:0/1702357439 wait complete. 2024-04-12T06:09:38.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:37 smithi008 ceph-mon[146084]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:38.026 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:37 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1752247556' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:09:38.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:37 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1702357439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:09:38.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:37 smithi194 ceph-mon[106982]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:38.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:37 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1752247556' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:09:38.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:37 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1702357439' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:09:39.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:38 smithi194 ceph-mon[106982]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:39.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:38 smithi194 ceph-mon[106982]: pgmap v438: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:39.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:38 smithi008 ceph-mon[146084]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:09:39.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:38 smithi008 ceph-mon[146084]: pgmap v438: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:41.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:40 smithi194 ceph-mon[106982]: pgmap v439: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:41.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:40 smithi008 ceph-mon[146084]: pgmap v439: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:43.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:42 smithi194 ceph-mon[106982]: pgmap v440: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:42 smithi008 ceph-mon[146084]: pgmap v440: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:45.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:44 smithi194 ceph-mon[106982]: pgmap v441: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:45.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:44 smithi008 ceph-mon[146084]: pgmap v441: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:46.822 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:46 smithi194 ceph-mon[106982]: pgmap v442: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:47.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:46 smithi008 ceph-mon[146084]: pgmap v442: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:48.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:48.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:09:49.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:48 smithi194 ceph-mon[106982]: pgmap v443: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:49.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:48 smithi008 ceph-mon[146084]: pgmap v443: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:51.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:50 smithi194 ceph-mon[106982]: pgmap v444: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:51.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:50 smithi008 ceph-mon[146084]: pgmap v444: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:53.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:52 smithi194 ceph-mon[106982]: pgmap v445: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:53.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:52 smithi008 ceph-mon[146084]: pgmap v445: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:55.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:54 smithi194 ceph-mon[106982]: pgmap v446: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:55.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:54 smithi008 ceph-mon[146084]: pgmap v446: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:57.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:56 smithi194 ceph-mon[106982]: pgmap v447: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:57.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:56 smithi008 ceph-mon[146084]: pgmap v447: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:59.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:09:58 smithi194 ceph-mon[106982]: pgmap v448: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:09:59.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:09:58 smithi008 ceph-mon[146084]: pgmap v448: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:00.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:09:59.999+0000 7f39f471a700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:10:00.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:09:59.999+0000 7f39f471a700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:10:00.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:09:59.999+0000 7f39f471a700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-12T06:10:00.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:09:59.999+0000 7f39f471a700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:10:00.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:09:59.999+0000 7f39f471a700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: pgmap v449: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: fs cephfs is degraded 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:10:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:00 smithi194 ceph-mon[106982]: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: pgmap v449: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: fs cephfs is degraded 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:10:01.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:00 smithi008 ceph-mon[146084]: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:10:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:02 smithi194 ceph-mon[106982]: pgmap v450: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:03.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:02 smithi008 ceph-mon[146084]: pgmap v450: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:03.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:05.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:04 smithi194 ceph-mon[106982]: pgmap v451: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:05.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:04 smithi008 ceph-mon[146084]: pgmap v451: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:07.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:06 smithi194 ceph-mon[106982]: pgmap v452: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:07.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:06 smithi008 ceph-mon[146084]: pgmap v452: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 -- 172.21.15.8:0/2791539338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 msgr2=0x7f462c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 --2- 172.21.15.8:0/2791539338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c105fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f461c009910 tx=0x7f461c009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 -- 172.21.15.8:0/2791539338 shutdown_connections 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 --2- 172.21.15.8:0/2791539338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 0x7f462c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 --2- 172.21.15.8:0/2791539338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c105fb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.684+0000 7f4632cbd700 1 -- 172.21.15.8:0/2791539338 >> 172.21.15.8:0/2791539338 conn(0x7f462c075670 msgr2=0x7f462c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 -- 172.21.15.8:0/2791539338 shutdown_connections 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 -- 172.21.15.8:0/2791539338 wait complete. 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 Processor -- start 2024-04-12T06:10:07.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 -- start start 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 0x7f462c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c198140 con 0x7f462c105bd0 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.685+0000 7f4632cbd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c1982b0 con 0x7f462c0ffc20 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58058/0 (socket says 172.21.15.8:58058) 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 -- 172.21.15.8:0/1523381738 learned_addr learned my addr 172.21.15.8:0/1523381738 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f4630a59700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 0x7f462c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 -- 172.21.15.8:0/1523381738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 msgr2=0x7f462c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 0x7f462c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 -- 172.21.15.8:0/1523381738 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0095f0 con 0x7f462c105bd0 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f462bfff700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f462000e920 tx=0x7f462000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:07.686 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462000c9d0 con 0x7f462c105bd0 2024-04-12T06:10:07.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4620010430 con 0x7f462c105bd0 2024-04-12T06:10:07.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c1a0650 con 0x7f462c105bd0 2024-04-12T06:10:07.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.686+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4620018650 con 0x7f462c105bd0 2024-04-12T06:10:07.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.687+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c1a0b40 con 0x7f462c105bd0 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.688+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c1080d0 con 0x7f462c105bd0 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.688+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f46200105a0 con 0x7f462c105bd0 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.689+0000 7f4629ffb700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 0x7f461406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.689+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4620014070 con 0x7f462c105bd0 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.689+0000 7f4630a59700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 0x7f461406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:07.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.689+0000 7f4630a59700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 0x7f461406f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fd0 tx=0x7f461c005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:07.691 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.693+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4620052ab0 con 0x7f462c105bd0 2024-04-12T06:10:07.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.915+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f462c02ccc0 con 0x7f461406cce0 2024-04-12T06:10:07.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.916+0000 7f4629ffb700 1 -- 172.21.15.8:0/1523381738 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f462c02ccc0 con 0x7f461406cce0 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.918+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 msgr2=0x7f461406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.918+0000 7f4632cbd700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 0x7f461406f1a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f461c009fd0 tx=0x7f461c005940 comp rx=0 tx=0).stop 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.918+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 msgr2=0x7f462c197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.918+0000 7f4632cbd700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f462000e920 tx=0x7f462000ec30 comp rx=0 tx=0).stop 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 shutdown_connections 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f461406cce0 0x7f461406f1a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f462c0ffc20 0x7f462c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 --2- 172.21.15.8:0/1523381738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f462c105bd0 0x7f462c197b20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 >> 172.21.15.8:0/1523381738 conn(0x7f462c075670 msgr2=0x7f462c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 shutdown_connections 2024-04-12T06:10:07.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:07.919+0000 7f4632cbd700 1 -- 172.21.15.8:0/1523381738 wait complete. 2024-04-12T06:10:07.927 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:10:08.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- 172.21.15.8:0/1724380512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6400ffa20 msgr2=0x7ff6400ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 --2- 172.21.15.8:0/1724380512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6400ffa20 0x7ff6400ffea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009910 tx=0x7ff62c009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:08.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- 172.21.15.8:0/1724380512 shutdown_connections 2024-04-12T06:10:08.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 --2- 172.21.15.8:0/1724380512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff6400ffa20 0x7ff6400ffea0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 --2- 172.21.15.8:0/1724380512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff640105950 0x7ff640105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- 172.21.15.8:0/1724380512 >> 172.21.15.8:0/1724380512 conn(0x7ff640075460 msgr2=0x7ff640075870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- 172.21.15.8:0/1724380512 shutdown_connections 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- 172.21.15.8:0/1724380512 wait complete. 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 Processor -- start 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.026+0000 7ff6459a2700 1 -- start start 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6459a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6400ffa20 0x7ff6401951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6459a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6459a2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640195c20 con 0x7ff640105950 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6459a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640195d90 con 0x7ff6400ffa20 2024-04-12T06:10:08.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58070/0 (socket says 172.21.15.8:58070) 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 -- 172.21.15.8:0/2340285585 learned_addr learned my addr 172.21.15.8:0/2340285585 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6449a0700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6400ffa20 0x7ff6401951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 -- 172.21.15.8:0/2340285585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6400ffa20 msgr2=0x7ff6401951a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6400ffa20 0x7ff6401951a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 -- 172.21.15.8:0/2340285585 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c0095f0 con 0x7ff640105950 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63ffff700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62c01d070 con 0x7ff640105950 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff62c005dc0 con 0x7ff640105950 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640196010 con 0x7ff640105950 2024-04-12T06:10:08.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.027+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62c021410 con 0x7ff640105950 2024-04-12T06:10:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.028+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6401a2a90 con 0x7ff640105950 2024-04-12T06:10:08.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.029+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff640107e70 con 0x7ff640105950 2024-04-12T06:10:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.029+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff62c003a60 con 0x7ff640105950 2024-04-12T06:10:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.030+0000 7ff63dffb700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 0x7ff63006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.030+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff62c08d530 con 0x7ff640105950 2024-04-12T06:10:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.030+0000 7ff6449a0700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 0x7ff63006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.030 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.030+0000 7ff6449a0700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 0x7ff63006f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff634005950 tx=0x7ff63400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.034+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff62c05b030 con 0x7ff640105950 2024-04-12T06:10:08.255 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.256+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff640196010 con 0x7ff63006cce0 2024-04-12T06:10:08.256 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.258+0000 7ff63dffb700 1 -- 172.21.15.8:0/2340285585 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff640196010 con 0x7ff63006cce0 2024-04-12T06:10:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 msgr2=0x7ff63006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.258 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 0x7ff63006f1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff634005950 tx=0x7ff63400b410 comp rx=0 tx=0).stop 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 msgr2=0x7ff6401956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff62c006010 tx=0x7ff62c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 shutdown_connections 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff63006cce0 0x7ff63006f1a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff6400ffa20 0x7ff6401951a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 --2- 172.21.15.8:0/2340285585 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff640105950 0x7ff6401956e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 >> 172.21.15.8:0/2340285585 conn(0x7ff640075460 msgr2=0x7ff6400feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 shutdown_connections 2024-04-12T06:10:08.259 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.260+0000 7ff6459a2700 1 -- 172.21.15.8:0/2340285585 wait complete. 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 -- 172.21.15.8:0/1143891806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950100ea0 msgr2=0x7fd950101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 --2- 172.21.15.8:0/1143891806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950100ea0 0x7fd950101280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009910 tx=0x7fd94c009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 -- 172.21.15.8:0/1143891806 shutdown_connections 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 --2- 172.21.15.8:0/1143891806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950101850 0x7fd9501058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 --2- 172.21.15.8:0/1143891806 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950100ea0 0x7fd950101280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 -- 172.21.15.8:0/1143891806 >> 172.21.15.8:0/1143891806 conn(0x7fd9500fc750 msgr2=0x7fd9500feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 -- 172.21.15.8:0/1143891806 shutdown_connections 2024-04-12T06:10:08.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.366+0000 7fd957b12700 1 -- 172.21.15.8:0/1143891806 wait complete. 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.367+0000 7fd957b12700 1 Processor -- start 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.367+0000 7fd957b12700 1 -- start start 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 0x7fd95019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95019d130 con 0x7fd950101850 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501965e0 con 0x7fd950100ea0 2024-04-12T06:10:08.368 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58080/0 (socket says 172.21.15.8:58080) 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 -- 172.21.15.8:0/677536553 learned_addr learned my addr 172.21.15.8:0/677536553 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9558ae700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 0x7fd95019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 -- 172.21.15.8:0/677536553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 msgr2=0x7fd95019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 0x7fd95019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 -- 172.21.15.8:0/677536553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c0095f0 con 0x7fd950101850 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9558ae700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 0x7fd95019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd9550ad700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d900 tx=0x7fd94400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd944009900 con 0x7fd950101850 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd944010460 con 0x7fd950101850 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9501968c0 con 0x7fd950101850 2024-04-12T06:10:08.369 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd94400f5f0 con 0x7fd950101850 2024-04-12T06:10:08.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.368+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd950196e10 con 0x7fd950101850 2024-04-12T06:10:08.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.369+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950109240 con 0x7fd950101850 2024-04-12T06:10:08.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.370+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd9440105d0 con 0x7fd950101850 2024-04-12T06:10:08.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.370+0000 7fd942ffd700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 0x7fd93c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.370 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.370+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd94408c380 con 0x7fd950101850 2024-04-12T06:10:08.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.370+0000 7fd9558ae700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 0x7fd93c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.371 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.371+0000 7fd9558ae700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 0x7fd93c06f1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00b5c0 tx=0x7fd94c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.373 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.375+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd944059e80 con 0x7fd950101850 2024-04-12T06:10:08.584 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.586+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd950197030 con 0x7fd93c06cce0 2024-04-12T06:10:08.594 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.597+0000 7fd942ffd700 1 -- 172.21.15.8:0/677536553 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd950197030 con 0x7fd93c06cce0 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (12m) 5m ago 23m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (23m) 5m ago 23m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (22m) 5m ago 22m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (9m) 5m ago 23m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (9m) 5m ago 22m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (11m) 5m ago 23m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (5m) 5m ago 17m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (6m) 5m ago 17m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:10:08.595 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (5m) 5m ago 17m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (5m) 5m ago 17m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (14m) 5m ago 24m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (13m) 5m ago 22m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (10m) 5m ago 25m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (9m) 5m ago 22m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (12m) 5m ago 23m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (12m) 5m ago 22m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (9m) 5m ago 21m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (8m) 5m ago 20m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (8m) 5m ago 20m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (7m) 5m ago 19m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (6m) 5m ago 19m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (6m) 5m ago 18m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:10:08.596 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (12m) 5m ago 22m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:10:08.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 msgr2=0x7fd93c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 0x7fd93c06f1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00b5c0 tx=0x7fd94c005960 comp rx=0 tx=0).stop 2024-04-12T06:10:08.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 msgr2=0x7fd95019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d900 tx=0x7fd94400dcc0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 shutdown_connections 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd93c06cce0 0x7fd93c06f1a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd950100ea0 0x7fd95019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 --2- 172.21.15.8:0/677536553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd950101850 0x7fd95019caa0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 >> 172.21.15.8:0/677536553 conn(0x7fd9500fc750 msgr2=0x7fd9500fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 shutdown_connections 2024-04-12T06:10:08.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.599+0000 7fd957b12700 1 -- 172.21.15.8:0/677536553 wait complete. 2024-04-12T06:10:08.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 -- 172.21.15.8:0/2397605955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 msgr2=0x7f8d70105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 --2- 172.21.15.8:0/2397605955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d70105fb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009910 tx=0x7f8d60009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 -- 172.21.15.8:0/2397605955 shutdown_connections 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 --2- 172.21.15.8:0/2397605955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 0x7f8d70100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 --2- 172.21.15.8:0/2397605955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d70105fb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 -- 172.21.15.8:0/2397605955 >> 172.21.15.8:0/2397605955 conn(0x7f8d70075670 msgr2=0x7f8d70075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 -- 172.21.15.8:0/2397605955 shutdown_connections 2024-04-12T06:10:08.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.704+0000 7f8d74e61700 1 -- 172.21.15.8:0/2397605955 wait complete. 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 Processor -- start 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 -- start start 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 0x7f8d7019c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7019d140 con 0x7f8d70105bd0 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d74e61700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701965f0 con 0x7f8d700ffc20 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d6dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d6dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58100/0 (socket says 172.21.15.8:58100) 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.705+0000 7f8d6dd9b700 1 -- 172.21.15.8:0/3844909502 learned_addr learned my addr 172.21.15.8:0/3844909502 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d6e59c700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 0x7f8d7019c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d6dd9b700 1 -- 172.21.15.8:0/3844909502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 msgr2=0x7f8d7019c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.706 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d6dd9b700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 0x7f8d7019c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d6dd9b700 1 -- 172.21.15.8:0/3844909502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600095f0 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d6dd9b700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e920 tx=0x7f8d6400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d64009790 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8d6400cba0 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d701968d0 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d64018650 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.706+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d70196e20 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.707+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7004f2e0 con 0x7f8d70105bd0 2024-04-12T06:10:08.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.708+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8d6400cd10 con 0x7f8d70105bd0 2024-04-12T06:10:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.708+0000 7f8d5f7fe700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 0x7f8d5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.708+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d64014070 con 0x7f8d70105bd0 2024-04-12T06:10:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.708+0000 7f8d6e59c700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 0x7f8d5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:08.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.709+0000 7f8d6e59c700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 0x7f8d5806f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009fd0 tx=0x7f8d60005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:08.710 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.712+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8d64052ab0 con 0x7f8d70105bd0 2024-04-12T06:10:08.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.998+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d701970a0 con 0x7f8d70105bd0 2024-04-12T06:10:08.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:08.998+0000 7f8d5f7fe700 1 -- 172.21.15.8:0/3844909502 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f8d6405a1b0 con 0x7f8d70105bd0 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:08.997 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:10:08.998 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 msgr2=0x7f8d5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 0x7f8d5806f1a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009fd0 tx=0x7f8d60005940 comp rx=0 tx=0).stop 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 msgr2=0x7f8d7019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8d6400e920 tx=0x7f8d6400ece0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 shutdown_connections 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8d5806cce0 0x7f8d5806f1a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8d700ffc20 0x7f8d7019c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 --2- 172.21.15.8:0/3844909502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8d70105bd0 0x7f8d7019cab0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 >> 172.21.15.8:0/3844909502 conn(0x7f8d70075670 msgr2=0x7f8d70103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:08.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 shutdown_connections 2024-04-12T06:10:09.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.000+0000 7f8d74e61700 1 -- 172.21.15.8:0/3844909502 wait complete. 2024-04-12T06:10:09.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:08 smithi194 ceph-mon[106982]: pgmap v453: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:09.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:08 smithi194 ceph-mon[106982]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:09.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- 172.21.15.8:0/2951291140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40105bd0 msgr2=0x7f8a40105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 --2- 172.21.15.8:0/2951291140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40105bd0 0x7f8a40105fb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009910 tx=0x7f8a3c009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:09.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- 172.21.15.8:0/2951291140 shutdown_connections 2024-04-12T06:10:09.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 --2- 172.21.15.8:0/2951291140 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a400ffc20 0x7f8a40100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 --2- 172.21.15.8:0/2951291140 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a40105bd0 0x7f8a40105fb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- 172.21.15.8:0/2951291140 >> 172.21.15.8:0/2951291140 conn(0x7f8a40075670 msgr2=0x7f8a40075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- 172.21.15.8:0/2951291140 shutdown_connections 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- 172.21.15.8:0/2951291140 wait complete. 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 Processor -- start 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.108+0000 7f8a484a6700 1 -- start start 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 0x7f8a4019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019d0d0 con 0x7f8a400ffc20 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40196610 con 0x7f8a40105bd0 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58118/0 (socket says 172.21.15.8:58118) 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 -- 172.21.15.8:0/948779021 learned_addr learned my addr 172.21.15.8:0/948779021 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:09.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a45a41700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 0x7f8a4019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 -- 172.21.15.8:0/948779021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 msgr2=0x7f8a4019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 0x7f8a4019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 -- 172.21.15.8:0/948779021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0095f0 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a45a41700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 0x7f8a4019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a46242700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b5c0 tx=0x7f8a3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c01d070 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a3c00f460 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c021640 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a40196890 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.109+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a40196d80 con 0x7f8a400ffc20 2024-04-12T06:10:09.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.110+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4004f2e0 con 0x7f8a400ffc20 2024-04-12T06:10:09.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.111+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8a3c00f9b0 con 0x7f8a400ffc20 2024-04-12T06:10:09.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.111+0000 7f8a377fe700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 0x7f8a2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.111+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a3c08d850 con 0x7f8a400ffc20 2024-04-12T06:10:09.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.111+0000 7f8a45a41700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 0x7f8a2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.112+0000 7f8a45a41700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 0x7f8a2c06f1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005fd0 tx=0x7f8a30005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.116+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8a3c05b350 con 0x7f8a400ffc20 2024-04-12T06:10:09.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:08 smithi008 ceph-mon[146084]: pgmap v453: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:09.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:08 smithi008 ceph-mon[146084]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:09.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.362+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8a40197060 con 0x7f8a400ffc20 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.363+0000 7f8a377fe700 1 -- 172.21.15.8:0/948779021 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f8a3c05aee0 con 0x7f8a400ffc20 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:10:09.361 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:10:09.362 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:10:09.363 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:09.364 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:09.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 msgr2=0x7f8a2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 0x7f8a2c06f1a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005fd0 tx=0x7f8a30005e20 comp rx=0 tx=0).stop 2024-04-12T06:10:09.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 msgr2=0x7f8a4019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00b5c0 tx=0x7f8a3c00be20 comp rx=0 tx=0).stop 2024-04-12T06:10:09.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 shutdown_connections 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8a2c06cce0 0x7f8a2c06f1a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8a400ffc20 0x7f8a4019c590 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 --2- 172.21.15.8:0/948779021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8a40105bd0 0x7f8a4019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 >> 172.21.15.8:0/948779021 conn(0x7f8a40075670 msgr2=0x7f8a400686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 shutdown_connections 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.365+0000 7f8a484a6700 1 -- 172.21.15.8:0/948779021 wait complete. 2024-04-12T06:10:09.366 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:10:09.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 -- 172.21.15.8:0/224333587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 msgr2=0x7f170c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 --2- 172.21.15.8:0/224333587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c105fb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1708009910 tx=0x7f1708009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 -- 172.21.15.8:0/224333587 shutdown_connections 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 --2- 172.21.15.8:0/224333587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 0x7f170c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 --2- 172.21.15.8:0/224333587 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c105fb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 -- 172.21.15.8:0/224333587 >> 172.21.15.8:0/224333587 conn(0x7f170c075670 msgr2=0x7f170c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 -- 172.21.15.8:0/224333587 shutdown_connections 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.472+0000 7f171464f700 1 -- 172.21.15.8:0/224333587 wait complete. 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 Processor -- start 2024-04-12T06:10:09.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 -- start start 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 0x7f170c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c1022c0 con 0x7f170c105bd0 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f171464f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c102400 con 0x7f170c0ffc20 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58142/0 (socket says 172.21.15.8:58142) 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 -- 172.21.15.8:0/3738589356 learned_addr learned my addr 172.21.15.8:0/3738589356 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f17123eb700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 0x7f170c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 -- 172.21.15.8:0/3738589356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 msgr2=0x7f170c101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 0x7f170c101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.473 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.473+0000 7f1711bea700 1 -- 172.21.15.8:0/3738589356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17080095f0 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f1711bea700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c930 tx=0x7f16fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16fc007ad0 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f16fc00ce80 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c1a2980 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16fc018630 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.474+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c1a2ce0 con 0x7f170c105bd0 2024-04-12T06:10:09.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.475+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c04f2e0 con 0x7f170c105bd0 2024-04-12T06:10:09.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.475+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f16fc007c30 con 0x7f170c105bd0 2024-04-12T06:10:09.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.476+0000 7f17037fe700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 0x7f16f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.476+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f16fc08c4a0 con 0x7f170c105bd0 2024-04-12T06:10:09.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.476+0000 7f17123eb700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 0x7f16f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.475 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.476+0000 7f17123eb700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 0x7f16f806f1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1708006010 tx=0x7f170801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.478 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.480+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f16fc056590 con 0x7f170c105bd0 2024-04-12T06:10:09.700 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.702+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f170c068d90 con 0x7f16f806cce0 2024-04-12T06:10:09.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.703+0000 7f17037fe700 1 -- 172.21.15.8:0/3738589356 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f170c068d90 con 0x7f16f806cce0 2024-04-12T06:10:09.701 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:10:09.701 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:10:09.701 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:10:09.701 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:10:09.702 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:10:09.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 msgr2=0x7f16f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 0x7f16f806f1a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1708006010 tx=0x7f170801a040 comp rx=0 tx=0).stop 2024-04-12T06:10:09.703 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 msgr2=0x7f170c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00c930 tx=0x7f16fc00cc40 comp rx=0 tx=0).stop 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 shutdown_connections 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f16f806cce0 0x7f16f806f1a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f170c0ffc20 0x7f170c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 --2- 172.21.15.8:0/3738589356 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f170c105bd0 0x7f170c101d80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 >> 172.21.15.8:0/3738589356 conn(0x7f170c075670 msgr2=0x7f170c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 shutdown_connections 2024-04-12T06:10:09.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.705+0000 7f171464f700 1 -- 172.21.15.8:0/3738589356 wait complete. 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 -- 172.21.15.8:0/115445130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 msgr2=0x7f8514100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 --2- 172.21.15.8:0/115445130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514100080 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8504009910 tx=0x7f8504009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 -- 172.21.15.8:0/115445130 shutdown_connections 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 --2- 172.21.15.8:0/115445130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514100080 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 --2- 172.21.15.8:0/115445130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.811+0000 7f851b276700 1 -- 172.21.15.8:0/115445130 >> 172.21.15.8:0/115445130 conn(0x7f8514075670 msgr2=0x7f8514075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:09.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 -- 172.21.15.8:0/115445130 shutdown_connections 2024-04-12T06:10:09.811 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 -- 172.21.15.8:0/115445130 wait complete. 2024-04-12T06:10:09.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 Processor -- start 2024-04-12T06:10:09.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 -- start start 2024-04-12T06:10:09.812 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85141022c0 con 0x7f85140ffc20 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f851b276700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514102400 con 0x7f8514105bd0 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.812+0000 7f8519012700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58168/0 (socket says 172.21.15.8:58168) 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 -- 172.21.15.8:0/4067102485 learned_addr learned my addr 172.21.15.8:0/4067102485 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8518811700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 -- 172.21.15.8:0/4067102485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 msgr2=0x7f8514101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 -- 172.21.15.8:0/4067102485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040095f0 con 0x7f85140ffc20 2024-04-12T06:10:09.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8518811700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f8519012700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f851000d8d0 tx=0x7f851000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8510009940 con 0x7f85140ffc20 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8510010460 con 0x7f85140ffc20 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85141a2980 con 0x7f85140ffc20 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f851000f5f0 con 0x7f85140ffc20 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.813+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141a2d90 con 0x7f85140ffc20 2024-04-12T06:10:09.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.814+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851404f2e0 con 0x7f85140ffc20 2024-04-12T06:10:09.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.815+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f85100105d0 con 0x7f85140ffc20 2024-04-12T06:10:09.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.815+0000 7f850a7fc700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 0x7f850006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:09.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.815+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f851008c5d0 con 0x7f85140ffc20 2024-04-12T06:10:09.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.815+0000 7f8518811700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 0x7f850006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:09.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.816+0000 7f8518811700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 0x7f850006f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8504005b40 tx=0x7f850401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:09.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:09.821+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8510056640 con 0x7f85140ffc20 2024-04-12T06:10:10.072 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:09 smithi194 ceph-mon[106982]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:10.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:09 smithi194 ceph-mon[106982]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:10.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:09 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3844909502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:10:10.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:09 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/948779021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:10:10.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.106+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f85141a3040 con 0x7f85140ffc20 2024-04-12T06:10:10.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.106+0000 7f850a7fc700 1 -- 172.21.15.8:0/4067102485 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8510010880 con 0x7f85140ffc20 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:10:10.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 msgr2=0x7f850006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 0x7f850006f1a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8504005b40 tx=0x7f850401a040 comp rx=0 tx=0).stop 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 msgr2=0x7f8514101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f851000d8d0 tx=0x7f851000dc90 comp rx=0 tx=0).stop 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 shutdown_connections 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f850006cce0 0x7f850006f1a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f85140ffc20 0x7f8514101840 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 --2- 172.21.15.8:0/4067102485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8514105bd0 0x7f8514101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 >> 172.21.15.8:0/4067102485 conn(0x7f8514075670 msgr2=0x7f8514068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 shutdown_connections 2024-04-12T06:10:10.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:10.108+0000 7f851b276700 1 -- 172.21.15.8:0/4067102485 wait complete. 2024-04-12T06:10:10.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:09 smithi008 ceph-mon[146084]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:10.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:09 smithi008 ceph-mon[146084]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:10.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:09 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3844909502' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:10:10.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:09 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/948779021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:10:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:10 smithi194 ceph-mon[106982]: pgmap v454: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:10 smithi194 ceph-mon[106982]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:10 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4067102485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:10:11.143 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:10 smithi008 ceph-mon[146084]: pgmap v454: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:11.143 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:10 smithi008 ceph-mon[146084]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:11.143 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:10 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4067102485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:10:13.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:12 smithi008 ceph-mon[146084]: pgmap v455: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:13.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:12 smithi194 ceph-mon[106982]: pgmap v455: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:15.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:14 smithi008 ceph-mon[146084]: pgmap v456: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:14 smithi194 ceph-mon[106982]: pgmap v456: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:17.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:16 smithi008 ceph-mon[146084]: pgmap v457: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:17.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:16 smithi194 ceph-mon[106982]: pgmap v457: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:18.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:18.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:19.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:18 smithi008 ceph-mon[146084]: pgmap v458: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:19.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:18 smithi194 ceph-mon[106982]: pgmap v458: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:21.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:20 smithi008 ceph-mon[146084]: pgmap v459: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:21.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:20 smithi194 ceph-mon[106982]: pgmap v459: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:23.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:22 smithi008 ceph-mon[146084]: pgmap v460: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:23.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:22 smithi194 ceph-mon[106982]: pgmap v460: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:25.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:24 smithi008 ceph-mon[146084]: pgmap v461: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:25.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:24 smithi194 ceph-mon[106982]: pgmap v461: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:27.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:26 smithi008 ceph-mon[146084]: pgmap v462: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:27.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:26 smithi194 ceph-mon[106982]: pgmap v462: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:29.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:28 smithi008 ceph-mon[146084]: pgmap v463: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:29.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:28 smithi194 ceph-mon[106982]: pgmap v463: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:31.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:30 smithi008 ceph-mon[146084]: pgmap v464: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:31.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:30 smithi194 ceph-mon[106982]: pgmap v464: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:33.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:32 smithi008 ceph-mon[146084]: pgmap v465: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:33.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:33.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:32 smithi194 ceph-mon[106982]: pgmap v465: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:33.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:35.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:34 smithi008 ceph-mon[146084]: pgmap v466: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:35.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:34 smithi194 ceph-mon[106982]: pgmap v466: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:37.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:36 smithi008 ceph-mon[146084]: pgmap v467: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:37.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:36 smithi194 ceph-mon[106982]: pgmap v467: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:39.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:38 smithi008 ceph-mon[146084]: pgmap v468: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:39.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:38 smithi194 ceph-mon[106982]: pgmap v468: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 -- 172.21.15.8:0/3507395108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c0ffc20 msgr2=0x7f2c7c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 --2- 172.21.15.8:0/3507395108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c100080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2c64009910 tx=0x7f2c64009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 -- 172.21.15.8:0/3507395108 shutdown_connections 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 --2- 172.21.15.8:0/3507395108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c100080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 --2- 172.21.15.8:0/3507395108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 -- 172.21.15.8:0/3507395108 >> 172.21.15.8:0/3507395108 conn(0x7f2c7c075670 msgr2=0x7f2c7c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 -- 172.21.15.8:0/3507395108 shutdown_connections 2024-04-12T06:10:40.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.229+0000 7f2c812a1700 1 -- 172.21.15.8:0/3507395108 wait complete. 2024-04-12T06:10:40.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 Processor -- start 2024-04-12T06:10:40.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 -- start start 2024-04-12T06:10:40.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c19d150 con 0x7f2c7c105bd0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c812a1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c196600 con 0x7f2c7c0ffc20 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c7a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c7a7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47124/0 (socket says 172.21.15.8:47124) 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c7a7fc700 1 -- 172.21.15.8:0/2912816996 learned_addr learned my addr 172.21.15.8:0/2912816996 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.230+0000 7f2c7affd700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c7a7fc700 1 -- 172.21.15.8:0/2912816996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c0ffc20 msgr2=0x7f2c7c19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c7a7fc700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c7a7fc700 1 -- 172.21.15.8:0/2912816996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c640095f0 con 0x7f2c7c105bd0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c7a7fc700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c64006010 tx=0x7f2c6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c6401d070 con 0x7f2c7c105bd0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c64005940 con 0x7f2c7c105bd0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c196880 con 0x7f2c7c105bd0 2024-04-12T06:10:40.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c6400fd90 con 0x7f2c7c105bd0 2024-04-12T06:10:40.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.231+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c196d70 con 0x7f2c7c105bd0 2024-04-12T06:10:40.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.232+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c04f2e0 con 0x7f2c7c105bd0 2024-04-12T06:10:40.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.233+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2c64003a60 con 0x7f2c7c105bd0 2024-04-12T06:10:40.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.233+0000 7f2c73fff700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 0x7f2c6806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.233+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2c6408d710 con 0x7f2c7c105bd0 2024-04-12T06:10:40.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.233+0000 7f2c7affd700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 0x7f2c6806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.234+0000 7f2c7affd700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 0x7f2c6806f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c007be0 tx=0x7f2c6c0073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.238+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2c6405b210 con 0x7f2c7c105bd0 2024-04-12T06:10:40.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.458+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c7c197040 con 0x7f2c6806cce0 2024-04-12T06:10:40.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.459+0000 7f2c73fff700 1 -- 172.21.15.8:0/2912816996 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2c7c197040 con 0x7f2c6806cce0 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 msgr2=0x7f2c6806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 0x7f2c6806f1a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c007be0 tx=0x7f2c6c0073d0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 msgr2=0x7f2c7c19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2c64006010 tx=0x7f2c6400bd70 comp rx=0 tx=0).stop 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 shutdown_connections 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c6806cce0 0x7f2c6806f1a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c7c0ffc20 0x7f2c7c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 --2- 172.21.15.8:0/2912816996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c7c105bd0 0x7f2c7c19cac0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.461+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 >> 172.21.15.8:0/2912816996 conn(0x7f2c7c075670 msgr2=0x7f2c7c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:40.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.462+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 shutdown_connections 2024-04-12T06:10:40.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.462+0000 7f2c812a1700 1 -- 172.21.15.8:0/2912816996 wait complete. 2024-04-12T06:10:40.472 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- 172.21.15.8:0/4077226974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4106cd0 msgr2=0x7f49e41070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 --2- 172.21.15.8:0/4077226974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4106cd0 0x7f49e41070b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009910 tx=0x7f49d4009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- 172.21.15.8:0/4077226974 shutdown_connections 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 --2- 172.21.15.8:0/4077226974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4068580 0x7f49e40689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 --2- 172.21.15.8:0/4077226974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4106cd0 0x7f49e41070b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- 172.21.15.8:0/4077226974 >> 172.21.15.8:0/4077226974 conn(0x7f49e4075670 msgr2=0x7f49e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- 172.21.15.8:0/4077226974 shutdown_connections 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- 172.21.15.8:0/4077226974 wait complete. 2024-04-12T06:10:40.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 Processor -- start 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.570+0000 7f49eb74a700 1 -- start start 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49eb74a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49eb74a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4106cd0 0x7f49e419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49eb74a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e419d130 con 0x7f49e4068580 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49eb74a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4196630 con 0x7f49e4106cd0 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47142/0 (socket says 172.21.15.8:47142) 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 -- 172.21.15.8:0/85667038 learned_addr learned my addr 172.21.15.8:0/85667038 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e8ce5700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4106cd0 0x7f49e419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 -- 172.21.15.8:0/85667038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4106cd0 msgr2=0x7f49e419caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4106cd0 0x7f49e419caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 -- 172.21.15.8:0/85667038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40095f0 con 0x7f49e4068580 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49e94e6700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49d40092e0 tx=0x7f49d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d401d070 con 0x7f49e4068580 2024-04-12T06:10:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49d4005e00 con 0x7f49e4068580 2024-04-12T06:10:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e41968b0 con 0x7f49e4068580 2024-04-12T06:10:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.571+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d400fe90 con 0x7f49e4068580 2024-04-12T06:10:40.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.572+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e4196da0 con 0x7f49e4068580 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.572+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e41091f0 con 0x7f49e4068580 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.573+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f49d400f6d0 con 0x7f49e4068580 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.573+0000 7f49da7fc700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 0x7f49d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.574+0000 7f49e8ce5700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 0x7f49d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.574+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f49d408d7a0 con 0x7f49e4068580 2024-04-12T06:10:40.574 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.574+0000 7f49e8ce5700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 0x7f49d006f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49e40fe390 tx=0x7f49e000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.579+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f49d405b280 con 0x7f49e4068580 2024-04-12T06:10:40.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.801+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e4197010 con 0x7f49d006cce0 2024-04-12T06:10:40.801 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.802+0000 7f49da7fc700 1 -- 172.21.15.8:0/85667038 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f49e4197010 con 0x7f49d006cce0 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 msgr2=0x7f49d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 0x7f49d006f1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f49e40fe390 tx=0x7f49e000b410 comp rx=0 tx=0).stop 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 msgr2=0x7f49e419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f49d40092e0 tx=0x7f49d400bd70 comp rx=0 tx=0).stop 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 shutdown_connections 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49d006cce0 0x7f49d006f1a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49e4068580 0x7f49e419c560 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 --2- 172.21.15.8:0/85667038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49e4106cd0 0x7f49e419caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.804+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 >> 172.21.15.8:0/85667038 conn(0x7f49e4075670 msgr2=0x7f49e40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:40.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.805+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 shutdown_connections 2024-04-12T06:10:40.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.805+0000 7f49eb74a700 1 -- 172.21.15.8:0/85667038 wait complete. 2024-04-12T06:10:40.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 -- 172.21.15.8:0/3239079609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674101850 msgr2=0x7fb6741058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 --2- 172.21.15.8:0/3239079609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674101850 0x7fb6741058a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb660009910 tx=0x7fb660009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 -- 172.21.15.8:0/3239079609 shutdown_connections 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 --2- 172.21.15.8:0/3239079609 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674101850 0x7fb6741058a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 --2- 172.21.15.8:0/3239079609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674100ea0 0x7fb674101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 -- 172.21.15.8:0/3239079609 >> 172.21.15.8:0/3239079609 conn(0x7fb6740fc750 msgr2=0x7fb6740feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 -- 172.21.15.8:0/3239079609 shutdown_connections 2024-04-12T06:10:40.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.911+0000 7fb67ab89700 1 -- 172.21.15.8:0/3239079609 wait complete. 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 Processor -- start 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 -- start start 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 0x7fb67419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67419d1f0 con 0x7fb674100ea0 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb67ab89700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6741966a0 con 0x7fb674101850 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47162/0 (socket says 172.21.15.8:47162) 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 -- 172.21.15.8:0/1039644278 learned_addr learned my addr 172.21.15.8:0/1039644278 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:40.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb673fff700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 0x7fb67419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 -- 172.21.15.8:0/1039644278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 msgr2=0x7fb67419cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 0x7fb67419cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 -- 172.21.15.8:0/1039644278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600095f0 con 0x7fb674100ea0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb673fff700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 0x7fb67419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.912+0000 7fb678925700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb66800d900 tx=0x7fb66800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.913+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6680041f0 con 0x7fb674100ea0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.913+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb668004cd0 con 0x7fb674100ea0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.913+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb674196980 con 0x7fb674100ea0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.913+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6680056d0 con 0x7fb674100ea0 2024-04-12T06:10:40.914 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.913+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb674196ed0 con 0x7fb674100ea0 2024-04-12T06:10:40.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.914+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6741092a0 con 0x7fb674100ea0 2024-04-12T06:10:40.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.914+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb668004e40 con 0x7fb674100ea0 2024-04-12T06:10:40.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.915+0000 7fb671ffb700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 0x7fb66406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:40.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.915+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb66808c5c0 con 0x7fb674100ea0 2024-04-12T06:10:40.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.915+0000 7fb673fff700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 0x7fb66406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:40.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.915+0000 7fb673fff700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 0x7fb66406f1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6740fcfc0 tx=0x7fb66001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:40.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:40.919+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb66805a140 con 0x7fb674100ea0 2024-04-12T06:10:41.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.132+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6741970f0 con 0x7fb66406cce0 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.142+0000 7fb671ffb700 1 -- 172.21.15.8:0/1039644278 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb6741970f0 con 0x7fb66406cce0 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (12m) 6m ago 23m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (24m) 6m ago 24m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (23m) 5m ago 23m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:10:41.142 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (10m) 6m ago 24m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (10m) 5m ago 22m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (12m) 6m ago 23m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (6m) 6m ago 17m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (6m) 6m ago 17m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (5m) 5m ago 17m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (5m) 5m ago 17m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (14m) 6m ago 25m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (13m) 5m ago 22m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (10m) 6m ago 25m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (10m) 5m ago 22m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (13m) 6m ago 24m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:10:41.143 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (13m) 5m ago 22m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (9m) 6m ago 21m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (9m) 6m ago 21m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (8m) 6m ago 20m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (8m) 5m ago 20m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (7m) 5m ago 19m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (6m) 5m ago 19m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:10:41.144 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (12m) 6m ago 23m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 msgr2=0x7fb66406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 0x7fb66406f1a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6740fcfc0 tx=0x7fb66001a040 comp rx=0 tx=0).stop 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 msgr2=0x7fb67419c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb66800d900 tx=0x7fb66800dcc0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 shutdown_connections 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb66406cce0 0x7fb66406f1a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.145 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb674100ea0 0x7fb67419c620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 --2- 172.21.15.8:0/1039644278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb674101850 0x7fb67419cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 >> 172.21.15.8:0/1039644278 conn(0x7fb6740fc750 msgr2=0x7fb6740fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 shutdown_connections 2024-04-12T06:10:41.146 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.145+0000 7fb67ab89700 1 -- 172.21.15.8:0/1039644278 wait complete. 2024-04-12T06:10:41.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:40 smithi008 ceph-mon[146084]: pgmap v469: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.249+0000 7f3714122700 1 -- 172.21.15.8:0/4183607098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 msgr2=0x7f370c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.249+0000 7f3714122700 1 --2- 172.21.15.8:0/4183607098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c105fb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3708009910 tx=0x7f3708009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- 172.21.15.8:0/4183607098 shutdown_connections 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 --2- 172.21.15.8:0/4183607098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 0x7f370c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 --2- 172.21.15.8:0/4183607098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c105fb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- 172.21.15.8:0/4183607098 >> 172.21.15.8:0/4183607098 conn(0x7f370c075670 msgr2=0x7f370c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- 172.21.15.8:0/4183607098 shutdown_connections 2024-04-12T06:10:41.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- 172.21.15.8:0/4183607098 wait complete. 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 Processor -- start 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- start start 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 0x7f370c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c198160 con 0x7f370c105bd0 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.250+0000 7f3714122700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1982a0 con 0x7f370c0ffc20 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47178/0 (socket says 172.21.15.8:47178) 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 -- 172.21.15.8:0/4149970697 learned_addr learned my addr 172.21.15.8:0/4149970697 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f3711ebe700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 0x7f370c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 -- 172.21.15.8:0/4149970697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 msgr2=0x7f370c197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 0x7f370c197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 -- 172.21.15.8:0/4149970697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080095f0 con 0x7f370c105bd0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f37116bd700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f370000d8d0 tx=0x7f370000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37000098a0 con 0x7f370c105bd0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3700010460 con 0x7f370c105bd0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c1a0670 con 0x7f370c105bd0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3700009ea0 con 0x7f370c105bd0 2024-04-12T06:10:41.253 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.251+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1a0b30 con 0x7f370c105bd0 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.252+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c1080d0 con 0x7f370c105bd0 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.253+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f37000105d0 con 0x7f370c105bd0 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.253+0000 7f36feffd700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 0x7f36f806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.253+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f370008c370 con 0x7f370c105bd0 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.253+0000 7f3711ebe700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 0x7f36f806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.254 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.254+0000 7f3711ebe700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 0x7f36f806f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3708006010 tx=0x7f370801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:41.257 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.259+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3700056460 con 0x7f370c105bd0 2024-04-12T06:10:41.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:40 smithi194 ceph-mon[106982]: pgmap v469: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:41.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.544+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f370c04f2e0 con 0x7f370c105bd0 2024-04-12T06:10:41.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.544+0000 7f36feffd700 1 -- 172.21.15.8:0/4149970697 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3700010880 con 0x7f370c105bd0 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:41.544 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:10:41.545 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 msgr2=0x7f36f806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 0x7f36f806f1a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3708006010 tx=0x7f370801a040 comp rx=0 tx=0).stop 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 msgr2=0x7f370c197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f370000d8d0 tx=0x7f370000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 shutdown_connections 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f36f806cce0 0x7f36f806f1a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f370c0ffc20 0x7f370c197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 --2- 172.21.15.8:0/4149970697 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f370c105bd0 0x7f370c197b40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 >> 172.21.15.8:0/4149970697 conn(0x7f370c075670 msgr2=0x7f370c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 shutdown_connections 2024-04-12T06:10:41.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.546+0000 7f3714122700 1 -- 172.21.15.8:0/4149970697 wait complete. 2024-04-12T06:10:41.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 -- 172.21.15.8:0/3936119299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8194105bd0 msgr2=0x7f8194105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 --2- 172.21.15.8:0/3936119299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8194105bd0 0x7f8194105fb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8188009910 tx=0x7f8188009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:41.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 -- 172.21.15.8:0/3936119299 shutdown_connections 2024-04-12T06:10:41.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 --2- 172.21.15.8:0/3936119299 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81940ffc20 0x7f8194100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 --2- 172.21.15.8:0/3936119299 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8194105bd0 0x7f8194105fb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 -- 172.21.15.8:0/3936119299 >> 172.21.15.8:0/3936119299 conn(0x7f8194075670 msgr2=0x7f8194075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:41.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 -- 172.21.15.8:0/3936119299 shutdown_connections 2024-04-12T06:10:41.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.653+0000 7f819ac8a700 1 -- 172.21.15.8:0/3936119299 wait complete. 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 Processor -- start 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 -- start start 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8194105bd0 0x7f8194195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194196260 con 0x7f81940ffc20 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f819ac8a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194198f90 con 0x7f8194105bd0 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47198/0 (socket says 172.21.15.8:47198) 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 -- 172.21.15.8:0/1685567430 learned_addr learned my addr 172.21.15.8:0/1685567430 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8193fff700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8194105bd0 0x7f8194195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 -- 172.21.15.8:0/1685567430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8194105bd0 msgr2=0x7f8194195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8194105bd0 0x7f8194195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.654+0000 7f8198a26700 1 -- 172.21.15.8:0/1685567430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880095f0 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f8198a26700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81880092e0 tx=0x7f818800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818801d070 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8188005e00 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818800fe90 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8194199210 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.655+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194199700 con 0x7f81940ffc20 2024-04-12T06:10:41.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.656+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81941080f0 con 0x7f81940ffc20 2024-04-12T06:10:41.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.656+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f818800f6d0 con 0x7f81940ffc20 2024-04-12T06:10:41.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.657+0000 7f8191ffb700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:41.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.657+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f818808d7b0 con 0x7f81940ffc20 2024-04-12T06:10:41.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.657+0000 7f8193fff700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:41.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.657+0000 7f8193fff700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 0x7f818406f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8194196cb0 tx=0x7f818000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:41.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.662+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8188057820 con 0x7f81940ffc20 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.906+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8194196ac0 con 0x7f81940ffc20 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.907+0000 7f8191ffb700 1 -- 172.21.15.8:0/1685567430 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f818805ae40 con 0x7f81940ffc20 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:10:41.906 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:10:41.907 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:10:41.908 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:41.909 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 msgr2=0x7f818406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 0x7f818406f1a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8194196cb0 tx=0x7f818000b410 comp rx=0 tx=0).stop 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 msgr2=0x7f8194195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f81880092e0 tx=0x7f818800bd70 comp rx=0 tx=0).stop 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 shutdown_connections 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f818406cce0 0x7f818406f1a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81940ffc20 0x7f8194195690 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 --2- 172.21.15.8:0/1685567430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8194105bd0 0x7f8194195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:41.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 >> 172.21.15.8:0/1685567430 conn(0x7f8194075670 msgr2=0x7f81940fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:41.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 shutdown_connections 2024-04-12T06:10:41.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:41.909+0000 7f819ac8a700 1 -- 172.21.15.8:0/1685567430 wait complete. 2024-04-12T06:10:41.911 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:10:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 -- 172.21.15.8:0/698911621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 msgr2=0x7fc8340689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 --2- 172.21.15.8:0/698911621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc8340689e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc824009910 tx=0x7fc824009c20 comp rx=0 tx=0).stop 2024-04-12T06:10:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 -- 172.21.15.8:0/698911621 shutdown_connections 2024-04-12T06:10:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 --2- 172.21.15.8:0/698911621 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc8340689e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 --2- 172.21.15.8:0/698911621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 0x7fc8341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 -- 172.21.15.8:0/698911621 >> 172.21.15.8:0/698911621 conn(0x7fc834075670 msgr2=0x7fc834075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:42.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 -- 172.21.15.8:0/698911621 shutdown_connections 2024-04-12T06:10:42.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.017+0000 7fc83c3be700 1 -- 172.21.15.8:0/698911621 wait complete. 2024-04-12T06:10:42.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 Processor -- start 2024-04-12T06:10:42.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 -- start start 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 0x7fc83419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419d110 con 0x7fc834068580 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.018+0000 7fc83c3be700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8341965c0 con 0x7fc834106cd0 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47226/0 (socket says 172.21.15.8:47226) 2024-04-12T06:10:42.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 -- 172.21.15.8:0/3899728581 learned_addr learned my addr 172.21.15.8:0/3899728581 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc839959700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 0x7fc83419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 -- 172.21.15.8:0/3899728581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 msgr2=0x7fc83419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 0x7fc83419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 -- 172.21.15.8:0/3899728581 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240095f0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83a15a700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c8a0 tx=0x7fc83000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8300078e0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc83000f450 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8341968a0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83000e5e0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.019+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc834196df0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.020+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8341091d0 con 0x7fc834068580 2024-04-12T06:10:42.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.021+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc83000f5c0 con 0x7fc834068580 2024-04-12T06:10:42.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.021+0000 7fc82b7fe700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 0x7fc82006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.021+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc83008c230 con 0x7fc834068580 2024-04-12T06:10:42.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.021+0000 7fc839959700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 0x7fc82006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.022+0000 7fc839959700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 0x7fc82006f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8340fe260 tx=0x7fc82401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:42.025 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.026+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc830059c90 con 0x7fc834068580 2024-04-12T06:10:42.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:41 smithi008 ceph-mon[146084]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:42.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:41 smithi008 ceph-mon[146084]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:42.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:41 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4149970697' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:10:42.247 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.248+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc834197070 con 0x7fc82006cce0 2024-04-12T06:10:42.248 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.249+0000 7fc82b7fe700 1 -- 172.21.15.8:0/3899728581 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc834197070 con 0x7fc82006cce0 2024-04-12T06:10:42.248 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:10:42.248 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:10:42.249 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 msgr2=0x7fc82006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 0x7fc82006f1a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc8340fe260 tx=0x7fc82401a040 comp rx=0 tx=0).stop 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 msgr2=0x7fc83419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc83000c8a0 tx=0x7fc83000cbb0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 shutdown_connections 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc82006cce0 0x7fc82006f1a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc834068580 0x7fc83419c540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.251 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 --2- 172.21.15.8:0/3899728581 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc834106cd0 0x7fc83419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.251+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 >> 172.21.15.8:0/3899728581 conn(0x7fc834075670 msgr2=0x7fc8340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:42.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.252+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 shutdown_connections 2024-04-12T06:10:42.252 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.252+0000 7fc83c3be700 1 -- 172.21.15.8:0/3899728581 wait complete. 2024-04-12T06:10:42.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:41 smithi194 ceph-mon[106982]: from='client.34678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:42.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:41 smithi194 ceph-mon[106982]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:42.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:41 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4149970697' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 -- 172.21.15.8:0/3474415363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 msgr2=0x7f2f3410acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 --2- 172.21.15.8:0/3474415363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3410acd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009940 tx=0x7f2f1c009c50 comp rx=0 tx=0).stop 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 -- 172.21.15.8:0/3474415363 shutdown_connections 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 --2- 172.21.15.8:0/3474415363 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3410acd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 --2- 172.21.15.8:0/3474415363 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 0x7f2f341022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 -- 172.21.15.8:0/3474415363 >> 172.21.15.8:0/3474415363 conn(0x7f2f340fb650 msgr2=0x7f2f340fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.357+0000 7f2f39452700 1 -- 172.21.15.8:0/3474415363 shutdown_connections 2024-04-12T06:10:42.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 -- 172.21.15.8:0/3474415363 wait complete. 2024-04-12T06:10:42.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 Processor -- start 2024-04-12T06:10:42.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 -- start start 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 0x7f2f3419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3419d160 con 0x7f2f341027e0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.359+0000 7f2f39452700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34196610 con 0x7f2f34101ec0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:47240/0 (socket says 172.21.15.8:47240) 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 -- 172.21.15.8:0/1519279177 learned_addr learned my addr 172.21.15.8:0/1519279177 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f32ffd700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 0x7f2f3419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 -- 172.21.15.8:0/1519279177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 msgr2=0x7f2f3419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 0x7f2f3419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 -- 172.21.15.8:0/1519279177 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0095f0 con 0x7f2f341027e0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f327fc700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0059d0 tx=0x7f2f1c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:42.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1c01d070 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2f1c00be80 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f1c00fe90 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f34196890 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34196d80 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.360+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f34108360 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.362+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2f1c00f6d0 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.362+0000 7f2f2bfff700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 0x7f2f2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.362+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2f1c08d690 con 0x7f2f341027e0 2024-04-12T06:10:42.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.362+0000 7f2f32ffd700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 0x7f2f2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:10:42.363 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.362+0000 7f2f32ffd700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 0x7f2f2006f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f24005950 tx=0x7f2f2400b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:10:42.365 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.367+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2f1c05b190 con 0x7f2f341027e0 2024-04-12T06:10:42.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.660+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f34107d40 con 0x7f2f341027e0 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.660+0000 7f2f2bfff700 1 -- 172.21.15.8:0/1519279177 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2f1c05ad20 con 0x7f2f341027e0 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:10:42.659 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 msgr2=0x7f2f2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 0x7f2f2006f1a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f24005950 tx=0x7f2f2400b410 comp rx=0 tx=0).stop 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 msgr2=0x7f2f3419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0059d0 tx=0x7f2f1c0055f0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 shutdown_connections 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2f2006cce0 0x7f2f2006f1a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f34101ec0 0x7f2f3419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 --2- 172.21.15.8:0/1519279177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2f341027e0 0x7f2f3419cad0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 >> 172.21.15.8:0/1519279177 conn(0x7f2f340fb650 msgr2=0x7f2f341054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 shutdown_connections 2024-04-12T06:10:42.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:10:42.662+0000 7f2f39452700 1 -- 172.21.15.8:0/1519279177 wait complete. 2024-04-12T06:10:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:42 smithi008 ceph-mon[146084]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:42 smithi008 ceph-mon[146084]: pgmap v470: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:42 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1685567430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:10:43.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:42 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1519279177' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:10:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:42 smithi194 ceph-mon[106982]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:42 smithi194 ceph-mon[106982]: pgmap v470: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:42 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1685567430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:10:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:42 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1519279177' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:10:44.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:43 smithi008 ceph-mon[146084]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:44.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:43 smithi194 ceph-mon[106982]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:10:45.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:44 smithi008 ceph-mon[146084]: pgmap v471: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:45.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:44 smithi194 ceph-mon[106982]: pgmap v471: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:47.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:46 smithi008 ceph-mon[146084]: pgmap v472: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:47.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:46 smithi194 ceph-mon[106982]: pgmap v472: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:48.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:48.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:10:49.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:48 smithi008 ceph-mon[146084]: pgmap v473: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:49.322 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:48 smithi194 ceph-mon[106982]: pgmap v473: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:51.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:50 smithi008 ceph-mon[146084]: pgmap v474: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:51.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:50 smithi194 ceph-mon[106982]: pgmap v474: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:53.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:52 smithi008 ceph-mon[146084]: pgmap v475: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:53.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:52 smithi194 ceph-mon[106982]: pgmap v475: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:55.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:54 smithi008 ceph-mon[146084]: pgmap v476: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:55.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:54 smithi194 ceph-mon[106982]: pgmap v476: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:57.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:56 smithi008 ceph-mon[146084]: pgmap v477: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:57.212 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:56 smithi194 ceph-mon[106982]: pgmap v477: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:59.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:10:58 smithi008 ceph-mon[146084]: pgmap v478: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:10:59.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:10:58 smithi194 ceph-mon[106982]: pgmap v478: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:01.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:00 smithi194 ceph-mon[106982]: pgmap v479: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:01.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:00 smithi008 ceph-mon[146084]: pgmap v479: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:03.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:02 smithi194 ceph-mon[106982]: pgmap v480: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:03.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:03.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:02 smithi008 ceph-mon[146084]: pgmap v480: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:03.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:05.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:04 smithi194 ceph-mon[106982]: pgmap v481: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:05.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:04 smithi008 ceph-mon[146084]: pgmap v481: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:07.305 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:06 smithi194 ceph-mon[106982]: pgmap v482: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:07.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:06 smithi008 ceph-mon[146084]: pgmap v482: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:09.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:08 smithi194 ceph-mon[106982]: pgmap v483: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:09.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:08 smithi008 ceph-mon[146084]: pgmap v483: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:11.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:10 smithi194 ceph-mon[106982]: pgmap v484: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:11.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:10 smithi008 ceph-mon[146084]: pgmap v484: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- 172.21.15.8:0/1461847804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 msgr2=0x7f0df8105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 --2- 172.21.15.8:0/1461847804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8105fb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009910 tx=0x7f0de8009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- 172.21.15.8:0/1461847804 shutdown_connections 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 --2- 172.21.15.8:0/1461847804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 0x7f0df8100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 --2- 172.21.15.8:0/1461847804 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8105fb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- 172.21.15.8:0/1461847804 >> 172.21.15.8:0/1461847804 conn(0x7f0df8075670 msgr2=0x7f0df8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- 172.21.15.8:0/1461847804 shutdown_connections 2024-04-12T06:11:12.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- 172.21.15.8:0/1461847804 wait complete. 2024-04-12T06:11:12.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 Processor -- start 2024-04-12T06:11:12.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.785+0000 7f0dfe494700 1 -- start start 2024-04-12T06:11:12.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0dfe494700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 0x7f0df8101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:12.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0dfe494700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:12.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0dfe494700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df81022c0 con 0x7f0df8105bd0 2024-04-12T06:11:12.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0dfe494700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df8102400 con 0x7f0df80ffc20 2024-04-12T06:11:12.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34034/0 (socket says 172.21.15.8:34034) 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 -- 172.21.15.8:0/2975271081 learned_addr learned my addr 172.21.15.8:0/2975271081 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df7fff700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 0x7f0df8101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 -- 172.21.15.8:0/2975271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 msgr2=0x7f0df8101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 0x7f0df8101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 -- 172.21.15.8:0/2975271081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80095f0 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.786+0000 7f0df77fe700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e920 tx=0x7f0dec00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.787+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dec00c9d0 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.787+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df81a2990 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.787+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0dec010430 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.787+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0dec018650 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.787+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df81a2da0 con 0x7f0df8105bd0 2024-04-12T06:11:12.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.788+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df804f2e0 con 0x7f0df8105bd0 2024-04-12T06:11:12.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.788+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f0dec0105a0 con 0x7f0df8105bd0 2024-04-12T06:11:12.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.789+0000 7f0df57fa700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 0x7f0de006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:12.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.789+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0dec014070 con 0x7f0df8105bd0 2024-04-12T06:11:12.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.789+0000 7f0df7fff700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 0x7f0de006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:12.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.789+0000 7f0df7fff700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 0x7f0de006f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009fd0 tx=0x7f0de8005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:12.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:12.793+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0dec052ab0 con 0x7f0df8105bd0 2024-04-12T06:11:13.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.016+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0df8068e10 con 0x7f0de006cce0 2024-04-12T06:11:13.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.017+0000 7f0df57fa700 1 -- 172.21.15.8:0/2975271081 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0df8068e10 con 0x7f0de006cce0 2024-04-12T06:11:13.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 msgr2=0x7f0de006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 0x7f0de006f1a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009fd0 tx=0x7f0de8005940 comp rx=0 tx=0).stop 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 msgr2=0x7f0df8101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00e920 tx=0x7f0dec00ec30 comp rx=0 tx=0).stop 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 shutdown_connections 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0de006cce0 0x7f0de006f1a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0df80ffc20 0x7f0df8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 --2- 172.21.15.8:0/2975271081 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0df8105bd0 0x7f0df8101d80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 >> 172.21.15.8:0/2975271081 conn(0x7f0df8075670 msgr2=0x7f0df8068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 shutdown_connections 2024-04-12T06:11:13.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.019+0000 7f0dfe494700 1 -- 172.21.15.8:0/2975271081 wait complete. 2024-04-12T06:11:13.028 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:11:13.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 -- 172.21.15.8:0/2495792897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 msgr2=0x7fad5c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 --2- 172.21.15.8:0/2495792897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c105fb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fad4c009910 tx=0x7fad4c009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:13.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 -- 172.21.15.8:0/2495792897 shutdown_connections 2024-04-12T06:11:13.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 --2- 172.21.15.8:0/2495792897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 0x7fad5c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 --2- 172.21.15.8:0/2495792897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c105fb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 -- 172.21.15.8:0/2495792897 >> 172.21.15.8:0/2495792897 conn(0x7fad5c075670 msgr2=0x7fad5c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 -- 172.21.15.8:0/2495792897 shutdown_connections 2024-04-12T06:11:13.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.128+0000 7fad621e8700 1 -- 172.21.15.8:0/2495792897 wait complete. 2024-04-12T06:11:13.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 Processor -- start 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 -- start start 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 0x7fad5c195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c196260 con 0x7fad5c105bd0 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad621e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c198f90 con 0x7fad5c0ffc20 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad5affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.129+0000 7fad5affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:34046/0 (socket says 172.21.15.8:34046) 2024-04-12T06:11:13.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5affd700 1 -- 172.21.15.8:0/1103177729 learned_addr learned my addr 172.21.15.8:0/1103177729 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5b7fe700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 0x7fad5c195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5affd700 1 -- 172.21.15.8:0/1103177729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 msgr2=0x7fad5c195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5affd700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 0x7fad5c195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5affd700 1 -- 172.21.15.8:0/1103177729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0095f0 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad5affd700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c8a0 tx=0x7fad4400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad440078e0 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c199270 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad4400f450 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad4400e5e0 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.130+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c1997c0 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.131+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c04f2e0 con 0x7fad5c105bd0 2024-04-12T06:11:13.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.131+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fad4400f5c0 con 0x7fad5c105bd0 2024-04-12T06:11:13.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.132+0000 7fad58ff9700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 0x7fad4806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.132+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fad4408c190 con 0x7fad5c105bd0 2024-04-12T06:11:13.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.132+0000 7fad5b7fe700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 0x7fad4806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.133+0000 7fad5b7fe700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 0x7fad4806f1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005f50 tx=0x7fad4c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.137+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fad44056200 con 0x7fad5c105bd0 2024-04-12T06:11:13.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:12 smithi194 ceph-mon[106982]: pgmap v485: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:13.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.358+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad5c199ac0 con 0x7fad4806cce0 2024-04-12T06:11:13.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.359+0000 7fad58ff9700 1 -- 172.21.15.8:0/1103177729 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad5c199ac0 con 0x7fad4806cce0 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 msgr2=0x7fad4806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 0x7fad4806f1a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005f50 tx=0x7fad4c01a040 comp rx=0 tx=0).stop 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 msgr2=0x7fad5c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c8a0 tx=0x7fad4400cbb0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 shutdown_connections 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad4806cce0 0x7fad4806f1a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad5c0ffc20 0x7fad5c195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 --2- 172.21.15.8:0/1103177729 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad5c105bd0 0x7fad5c195bd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 >> 172.21.15.8:0/1103177729 conn(0x7fad5c075670 msgr2=0x7fad5c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 shutdown_connections 2024-04-12T06:11:13.360 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.361+0000 7fad621e8700 1 -- 172.21.15.8:0/1103177729 wait complete. 2024-04-12T06:11:13.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:12 smithi008 ceph-mon[146084]: pgmap v485: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.467+0000 7fe6279f6700 1 -- 172.21.15.8:0/3700002499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe620105bd0 msgr2=0x7fe620105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.467+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3700002499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe620105bd0 0x7fe620105fb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009910 tx=0x7fe61c009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 -- 172.21.15.8:0/3700002499 shutdown_connections 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3700002499 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6200ffc20 0x7fe620100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3700002499 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe620105bd0 0x7fe620105fb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 -- 172.21.15.8:0/3700002499 >> 172.21.15.8:0/3700002499 conn(0x7fe620075670 msgr2=0x7fe620075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 -- 172.21.15.8:0/3700002499 shutdown_connections 2024-04-12T06:11:13.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 -- 172.21.15.8:0/3700002499 wait complete. 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 Processor -- start 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 -- start start 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.468+0000 7fe6279f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6279f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe620105bd0 0x7fe620101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6279f6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6201022c0 con 0x7fe6200ffc20 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6279f6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe620102400 con 0x7fe620105bd0 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60226/0 (socket says 172.21.15.8:60226) 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 -- 172.21.15.8:0/3324765349 learned_addr learned my addr 172.21.15.8:0/3324765349 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:13.469 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe624f91700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe620105bd0 0x7fe620101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 -- 172.21.15.8:0/3324765349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe620105bd0 msgr2=0x7fe620101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe620105bd0 0x7fe620101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 -- 172.21.15.8:0/3324765349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0095f0 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe625792700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000c00 tx=0x7fe61c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c01d070 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe61c0044c0 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe620102680 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c0039d0 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.469+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6201a2c60 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.470+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62004f2e0 con 0x7fe6200ffc20 2024-04-12T06:11:13.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.471+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe61c0048e0 con 0x7fe6200ffc20 2024-04-12T06:11:13.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.471+0000 7fe6127fc700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 0x7fe60c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.471+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe61c08db10 con 0x7fe6200ffc20 2024-04-12T06:11:13.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.471+0000 7fe624f91700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 0x7fe60c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.471 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.472+0000 7fe624f91700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 0x7fe60c06f1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe620103030 tx=0x7fe614009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.474 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.476+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe61c057c00 con 0x7fe6200ffc20 2024-04-12T06:11:13.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.689+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe620068db0 con 0x7fe60c06cce0 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.699+0000 7fe6127fc700 1 -- 172.21.15.8:0/3324765349 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe620068db0 con 0x7fe60c06cce0 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (13m) 6m ago 24m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (24m) 6m ago 24m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (23m) 6m ago 23m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (10m) 6m ago 24m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (10m) 6m ago 23m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (12m) 6m ago 24m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:11:13.698 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (6m) 6m ago 18m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (7m) 6m ago 18m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (6m) 6m ago 18m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (6m) 6m ago 18m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (15m) 6m ago 26m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (14m) 6m ago 23m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (11m) 6m ago 26m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (11m) 6m ago 23m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (13m) 6m ago 24m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (13m) 6m ago 23m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (10m) 6m ago 22m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (9m) 6m ago 21m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (9m) 6m ago 21m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:11:13.699 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (8m) 6m ago 21m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:11:13.700 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (7m) 6m ago 20m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:11:13.700 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (7m) 6m ago 20m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:11:13.700 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (13m) 6m ago 23m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 msgr2=0x7fe60c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 0x7fe60c06f1a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe620103030 tx=0x7fe614009260 comp rx=0 tx=0).stop 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 msgr2=0x7fe620101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe61c000c00 tx=0x7fe61c00be90 comp rx=0 tx=0).stop 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 shutdown_connections 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe60c06cce0 0x7fe60c06f1a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe6200ffc20 0x7fe620101840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 --2- 172.21.15.8:0/3324765349 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe620105bd0 0x7fe620101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.701 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.701+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 >> 172.21.15.8:0/3324765349 conn(0x7fe620075670 msgr2=0x7fe6200686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.702+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 shutdown_connections 2024-04-12T06:11:13.702 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.702+0000 7fe6279f6700 1 -- 172.21.15.8:0/3324765349 wait complete. 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 -- 172.21.15.8:0/1730140183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f00ffc20 msgr2=0x7fe8f0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 --2- 172.21.15.8:0/1730140183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0100080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009910 tx=0x7fe8e0009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 -- 172.21.15.8:0/1730140183 shutdown_connections 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 --2- 172.21.15.8:0/1730140183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0100080 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 --2- 172.21.15.8:0/1730140183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.805 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.806+0000 7fe8f788b700 1 -- 172.21.15.8:0/1730140183 >> 172.21.15.8:0/1730140183 conn(0x7fe8f0075670 msgr2=0x7fe8f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:13.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 -- 172.21.15.8:0/1730140183 shutdown_connections 2024-04-12T06:11:13.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 -- 172.21.15.8:0/1730140183 wait complete. 2024-04-12T06:11:13.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 Processor -- start 2024-04-12T06:11:13.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 -- start start 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f01022c0 con 0x7fe8f0105bd0 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.807+0000 7fe8f788b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0102400 con 0x7fe8f00ffc20 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60244/0 (socket says 172.21.15.8:60244) 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 -- 172.21.15.8:0/73858818 learned_addr learned my addr 172.21.15.8:0/73858818 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f5627700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 -- 172.21.15.8:0/73858818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f00ffc20 msgr2=0x7fe8f0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 -- 172.21.15.8:0/73858818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e00095f0 con 0x7fe8f0105bd0 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f4e26700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000b5c0 tx=0x7fe8e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.808 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e001d070 con 0x7fe8f0105bd0 2024-04-12T06:11:13.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8e000f460 con 0x7fe8f0105bd0 2024-04-12T06:11:13.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f0102680 con 0x7fe8f0105bd0 2024-04-12T06:11:13.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e0021640 con 0x7fe8f0105bd0 2024-04-12T06:11:13.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.808+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f01a2c60 con 0x7fe8f0105bd0 2024-04-12T06:11:13.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.809+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f004f2e0 con 0x7fe8f0105bd0 2024-04-12T06:11:13.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.810+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe8e000fa20 con 0x7fe8f0105bd0 2024-04-12T06:11:13.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.810+0000 7fe8e67fc700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 0x7fe8dc06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:13.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.810+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe8e008d760 con 0x7fe8f0105bd0 2024-04-12T06:11:13.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.810+0000 7fe8f5627700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 0x7fe8dc06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:13.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.811+0000 7fe8f5627700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 0x7fe8dc06f1f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ac50 tx=0x7fe8ec00a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:13.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:13.815+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe8e00577b0 con 0x7fe8f0105bd0 2024-04-12T06:11:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.102+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe8f01a2f90 con 0x7fe8f0105bd0 2024-04-12T06:11:14.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.102+0000 7fe8e67fc700 1 -- 172.21.15.8:0/73858818 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fe8e005add0 con 0x7fe8f0105bd0 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:11:14.101 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:11:14.102 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:14.102 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:11:14.102 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:11:14.102 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:11:14.102 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 msgr2=0x7fe8dc06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 0x7fe8dc06f1f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00ac50 tx=0x7fe8ec00a380 comp rx=0 tx=0).stop 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 msgr2=0x7fe8f0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000b5c0 tx=0x7fe8e000bd70 comp rx=0 tx=0).stop 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 shutdown_connections 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe8dc06cd30 0x7fe8dc06f1f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe8f00ffc20 0x7fe8f0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 --2- 172.21.15.8:0/73858818 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe8f0105bd0 0x7fe8f0101d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 >> 172.21.15.8:0/73858818 conn(0x7fe8f0075670 msgr2=0x7fe8f0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 shutdown_connections 2024-04-12T06:11:14.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.104+0000 7fe8f788b700 1 -- 172.21.15.8:0/73858818 wait complete. 2024-04-12T06:11:14.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.210+0000 7f18a598c700 1 -- 172.21.15.8:0/2346822666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a0100750 msgr2=0x7f18a01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.210+0000 7f18a598c700 1 --2- 172.21.15.8:0/2346822666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a0100750 0x7f18a01047a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1890009910 tx=0x7f1890009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 -- 172.21.15.8:0/2346822666 shutdown_connections 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 --2- 172.21.15.8:0/2346822666 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a0100750 0x7f18a01047a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 --2- 172.21.15.8:0/2346822666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a00ffda0 0x7f18a0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 -- 172.21.15.8:0/2346822666 >> 172.21.15.8:0/2346822666 conn(0x7f18a00fb650 msgr2=0x7f18a00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 -- 172.21.15.8:0/2346822666 shutdown_connections 2024-04-12T06:11:14.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 -- 172.21.15.8:0/2346822666 wait complete. 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.211+0000 7f18a598c700 1 Processor -- start 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f18a598c700 1 -- start start 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f18a598c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f18a598c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a0100750 0x7f18a0197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f18a598c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01981b0 con 0x7f18a00ffda0 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f18a598c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a0198320 con 0x7f18a0100750 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60258/0 (socket says 172.21.15.8:60258) 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 -- 172.21.15.8:0/3029500038 learned_addr learned my addr 172.21.15.8:0/3029500038 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189e7fc700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a0100750 0x7f18a0197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 -- 172.21.15.8:0/3029500038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a0100750 msgr2=0x7f18a0197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a0100750 0x7f18a0197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.212+0000 7f189effd700 1 -- 172.21.15.8:0/3029500038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900095f0 con 0x7f18a00ffda0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f189effd700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f189400c930 tx=0x7f189400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1894007ad0 con 0x7f18a00ffda0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1894007c30 con 0x7f18a00ffda0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a01a0750 con 0x7f18a00ffda0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1894018630 con 0x7f18a00ffda0 2024-04-12T06:11:14.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.213+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a01a0bc0 con 0x7f18a00ffda0 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.214+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a01081a0 con 0x7f18a00ffda0 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.214+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f189400f450 con 0x7f18a00ffda0 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.215+0000 7f18a498a700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 0x7f188806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.215+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f189408c750 con 0x7f18a00ffda0 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.215+0000 7f189e7fc700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 0x7f188806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.215+0000 7f189e7fc700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 0x7f188806f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1890006010 tx=0x7f189001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.219+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f18940568f0 con 0x7f18a00ffda0 2024-04-12T06:11:14.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:13 smithi194 ceph-mon[106982]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:14.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:13 smithi008 ceph-mon[146084]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.464+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f18a0107b80 con 0x7f18a00ffda0 2024-04-12T06:11:14.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.465+0000 7f18a498a700 1 -- 172.21.15.8:0/3029500038 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f1894059f10 con 0x7f18a00ffda0 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:11:14.464 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:11:14.465 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:14.466 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 msgr2=0x7f188806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 0x7f188806f1a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1890006010 tx=0x7f189001a040 comp rx=0 tx=0).stop 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 msgr2=0x7f18a01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f189400c930 tx=0x7f189400ccf0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 shutdown_connections 2024-04-12T06:11:14.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f188806cce0 0x7f188806f1a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f18a00ffda0 0x7f18a01976e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 --2- 172.21.15.8:0/3029500038 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f18a0100750 0x7f18a0197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 >> 172.21.15.8:0/3029500038 conn(0x7f18a00fb650 msgr2=0x7f18a00fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 shutdown_connections 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.467+0000 7f18a598c700 1 -- 172.21.15.8:0/3029500038 wait complete. 2024-04-12T06:11:14.468 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:11:14.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 -- 172.21.15.8:0/3601157290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 msgr2=0x7f3fc40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3601157290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc40689e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009910 tx=0x7f3fb8009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 -- 172.21.15.8:0/3601157290 shutdown_connections 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3601157290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc40689e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3601157290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 0x7f3fc41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 -- 172.21.15.8:0/3601157290 >> 172.21.15.8:0/3601157290 conn(0x7f3fc4075670 msgr2=0x7f3fc4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 -- 172.21.15.8:0/3601157290 shutdown_connections 2024-04-12T06:11:14.573 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.574+0000 7f3fc9562700 1 -- 172.21.15.8:0/3601157290 wait complete. 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 Processor -- start 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 -- start start 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 0x7f3fc419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc419d110 con 0x7f3fc4068580 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc9562700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc41965c0 con 0x7f3fc4106cd0 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60286/0 (socket says 172.21.15.8:60286) 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc2ffd700 1 -- 172.21.15.8:0/3333585145 learned_addr learned my addr 172.21.15.8:0/3333585145 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:14.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.575+0000 7f3fc27fc700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 0x7f3fc419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc2ffd700 1 -- 172.21.15.8:0/3333585145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 msgr2=0x7f3fc419ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc2ffd700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 0x7f3fc419ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc2ffd700 1 -- 172.21.15.8:0/3333585145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80095f0 con 0x7f3fc4068580 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc2ffd700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e920 tx=0x7f3fb400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb400c9d0 con 0x7f3fc4068580 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc41968a0 con 0x7f3fc4068580 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3fb4010430 con 0x7f3fc4068580 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb4018650 con 0x7f3fc4068580 2024-04-12T06:11:14.576 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.576+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc4196df0 con 0x7f3fc4068580 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.577+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc41091d0 con 0x7f3fc4068580 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.578+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3fb40105a0 con 0x7f3fc4068580 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.578+0000 7f3fabfff700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 0x7f3fac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.578+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3fb4014070 con 0x7f3fc4068580 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.578+0000 7f3fc27fc700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 0x7f3fac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.579+0000 7f3fc27fc700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 0x7f3fac06f1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40fe260 tx=0x7f3fb8005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.581 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.583+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3fb4052ab0 con 0x7f3fc4068580 2024-04-12T06:11:14.802 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.804+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc4197010 con 0x7f3fac06cce0 2024-04-12T06:11:14.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.805+0000 7f3fabfff700 1 -- 172.21.15.8:0/3333585145 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3fc4197010 con 0x7f3fac06cce0 2024-04-12T06:11:14.803 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:11:14.804 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 msgr2=0x7f3fac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 0x7f3fac06f1a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40fe260 tx=0x7f3fb8005940 comp rx=0 tx=0).stop 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 msgr2=0x7f3fc419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400e920 tx=0x7f3fb400ec30 comp rx=0 tx=0).stop 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 shutdown_connections 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3fac06cce0 0x7f3fac06f1a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3fc4068580 0x7f3fc419c540 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 --2- 172.21.15.8:0/3333585145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3fc4106cd0 0x7f3fc419ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 >> 172.21.15.8:0/3333585145 conn(0x7f3fc4075670 msgr2=0x7f3fc40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 shutdown_connections 2024-04-12T06:11:14.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.807+0000 7f3fc9562700 1 -- 172.21.15.8:0/3333585145 wait complete. 2024-04-12T06:11:14.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 -- 172.21.15.8:0/3136748795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e80ffb40 msgr2=0x7f53e80fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 --2- 172.21.15.8:0/3136748795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e80ffb40 0x7f53e80fffc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009910 tx=0x7f53d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:14.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 -- 172.21.15.8:0/3136748795 shutdown_connections 2024-04-12T06:11:14.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 --2- 172.21.15.8:0/3136748795 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e80ffb40 0x7f53e80fffc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 --2- 172.21.15.8:0/3136748795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e8105bf0 0x7f53e8105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 -- 172.21.15.8:0/3136748795 >> 172.21.15.8:0/3136748795 conn(0x7f53e80fb610 msgr2=0x7f53e80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:14.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 -- 172.21.15.8:0/3136748795 shutdown_connections 2024-04-12T06:11:14.913 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 -- 172.21.15.8:0/3136748795 wait complete. 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.914+0000 7f53ee750700 1 Processor -- start 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53ee750700 1 -- start start 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53ee750700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e80ffb40 0x7f53e819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53ee750700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53ee750700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e819c0f0 con 0x7f53e8105bf0 2024-04-12T06:11:14.915 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53ee750700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e81955a0 con 0x7f53e80ffb40 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e77fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60308/0 (socket says 172.21.15.8:60308) 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e77fe700 1 -- 172.21.15.8:0/3964286290 learned_addr learned my addr 172.21.15.8:0/3964286290 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e7fff700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e80ffb40 0x7f53e819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e77fe700 1 -- 172.21.15.8:0/3964286290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e80ffb40 msgr2=0x7f53e819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.915+0000 7f53e77fe700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e80ffb40 0x7f53e819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53e77fe700 1 -- 172.21.15.8:0/3964286290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00095f0 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53e77fe700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d001f070 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e8195820 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f53d0019610 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53d000f670 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.916+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e8195d10 con 0x7f53e8105bf0 2024-04-12T06:11:14.916 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.917+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e804f2e0 con 0x7f53e8105bf0 2024-04-12T06:11:14.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.917+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f53d0019c10 con 0x7f53e8105bf0 2024-04-12T06:11:14.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.918+0000 7f53e57fa700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 0x7f53d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:14.917 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.918+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f53d008faa0 con 0x7f53e8105bf0 2024-04-12T06:11:14.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.918+0000 7f53e7fff700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 0x7f53d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:14.918 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.918+0000 7f53e7fff700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 0x7f53d406f1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:14.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:14.922+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f53d005d5a0 con 0x7f53e8105bf0 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.207+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f53e8196ab0 con 0x7f53e8105bf0 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.208+0000 7f53e57fa700 1 -- 172.21.15.8:0/3964286290 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f53d005d130 con 0x7f53e8105bf0 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:11:15.206 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:11:15.207 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:11:15.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.209+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 msgr2=0x7f53d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:15.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 0x7f53d406f1a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006fd0 tx=0x7f53d8008040 comp rx=0 tx=0).stop 2024-04-12T06:11:15.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 msgr2=0x7f53e819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:15.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f53d0006010 tx=0x7f53d000be20 comp rx=0 tx=0).stop 2024-04-12T06:11:15.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 shutdown_connections 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f53d406cce0 0x7f53d406f1a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f53e80ffb40 0x7f53e819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 --2- 172.21.15.8:0/3964286290 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f53e8105bf0 0x7f53e819ba60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 >> 172.21.15.8:0/3964286290 conn(0x7f53e80fb610 msgr2=0x7f53e80fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 shutdown_connections 2024-04-12T06:11:15.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:15.210+0000 7f53ee750700 1 -- 172.21.15.8:0/3964286290 wait complete. 2024-04-12T06:11:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:14 smithi194 ceph-mon[106982]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:14 smithi194 ceph-mon[106982]: pgmap v486: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:14 smithi194 ceph-mon[106982]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:14 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/73858818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:11:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:14 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3029500038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:11:15.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:14 smithi008 ceph-mon[146084]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:15.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:14 smithi008 ceph-mon[146084]: pgmap v486: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:15.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:14 smithi008 ceph-mon[146084]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:15.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:14 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/73858818' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:11:15.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:14 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3029500038' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:11:16.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:15 smithi194 ceph-mon[106982]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:16.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:15 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3964286290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:11:16.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:15 smithi008 ceph-mon[146084]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:16.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:15 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3964286290' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:11:17.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:16 smithi008 ceph-mon[146084]: pgmap v487: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:17.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:16 smithi194 ceph-mon[106982]: pgmap v487: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:18.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:18.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:19.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:18 smithi194 ceph-mon[106982]: pgmap v488: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:19.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:18 smithi008 ceph-mon[146084]: pgmap v488: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:21.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:20 smithi194 ceph-mon[106982]: pgmap v489: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:21.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:20 smithi008 ceph-mon[146084]: pgmap v489: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:23.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:22 smithi194 ceph-mon[106982]: pgmap v490: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:23.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:22 smithi008 ceph-mon[146084]: pgmap v490: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:25.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:24 smithi194 ceph-mon[106982]: pgmap v491: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:25.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:24 smithi008 ceph-mon[146084]: pgmap v491: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:27.156 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:26 smithi008 ceph-mon[146084]: pgmap v492: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:27.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:26 smithi194 ceph-mon[106982]: pgmap v492: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:29.312 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:28 smithi194 ceph-mon[106982]: pgmap v493: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:29.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:28 smithi008 ceph-mon[146084]: pgmap v493: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:31.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:30 smithi194 ceph-mon[106982]: pgmap v494: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:31.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:30 smithi008 ceph-mon[146084]: pgmap v494: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:33.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:32 smithi194 ceph-mon[106982]: pgmap v495: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:33.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:33.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:32 smithi008 ceph-mon[146084]: pgmap v495: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:33.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:35.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:34 smithi194 ceph-mon[106982]: pgmap v496: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:35.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:34 smithi008 ceph-mon[146084]: pgmap v496: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:37.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:36 smithi194 ceph-mon[106982]: pgmap v497: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:37.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:36 smithi008 ceph-mon[146084]: pgmap v497: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:39.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:38 smithi194 ceph-mon[106982]: pgmap v498: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:39.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:38 smithi008 ceph-mon[146084]: pgmap v498: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:41.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:40 smithi194 ceph-mon[106982]: pgmap v499: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:41.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:40 smithi008 ceph-mon[146084]: pgmap v499: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:42 smithi194 ceph-mon[106982]: pgmap v500: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:43.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:42 smithi008 ceph-mon[146084]: pgmap v500: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:45.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:44 smithi194 ceph-mon[106982]: pgmap v501: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:45.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 -- 172.21.15.8:0/940104062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c100750 msgr2=0x7f489c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 --2- 172.21.15.8:0/940104062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c100750 0x7f489c1047a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4884009910 tx=0x7f4884009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 -- 172.21.15.8:0/940104062 shutdown_connections 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 --2- 172.21.15.8:0/940104062 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c100750 0x7f489c1047a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 --2- 172.21.15.8:0/940104062 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c0ffda0 0x7f489c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 -- 172.21.15.8:0/940104062 >> 172.21.15.8:0/940104062 conn(0x7f489c0fb650 msgr2=0x7f489c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 -- 172.21.15.8:0/940104062 shutdown_connections 2024-04-12T06:11:45.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 -- 172.21.15.8:0/940104062 wait complete. 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.331+0000 7f48a190c700 1 Processor -- start 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f48a190c700 1 -- start start 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f48a190c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f48a190c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c100750 0x7f489c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f48a190c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c1022c0 con 0x7f489c0ffda0 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f48a190c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c102400 con 0x7f489c100750 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42710/0 (socket says 172.21.15.8:42710) 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 -- 172.21.15.8:0/232607434 learned_addr learned my addr 172.21.15.8:0/232607434 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489a7fc700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c100750 0x7f489c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 -- 172.21.15.8:0/232607434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c100750 msgr2=0x7f489c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c100750 0x7f489c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 -- 172.21.15.8:0/232607434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840095f0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.332+0000 7f489affd700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d900 tx=0x7f488c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.333+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f488c0041f0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.333+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f488c004cd0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.333+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489c1a29d0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.333+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f488c0056d0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.333+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c1a2de0 con 0x7f489c0ffda0 2024-04-12T06:11:45.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.334+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c04f2e0 con 0x7f489c0ffda0 2024-04-12T06:11:45.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.334+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f488c004e40 con 0x7f489c0ffda0 2024-04-12T06:11:45.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.335+0000 7f48a090a700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 0x7f488806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.335+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f488c08c710 con 0x7f489c0ffda0 2024-04-12T06:11:45.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.335+0000 7f489a7fc700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 0x7f488806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.336+0000 7f489a7fc700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 0x7f488806f140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f489c103030 tx=0x7f488401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:45.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.340+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f488c056780 con 0x7f489c0ffda0 2024-04-12T06:11:45.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:44 smithi008 ceph-mon[146084]: pgmap v501: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:45.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.562+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f489c0fc410 con 0x7f488806cc80 2024-04-12T06:11:45.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.564+0000 7f48a090a700 1 -- 172.21.15.8:0/232607434 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f489c0fc410 con 0x7f488806cc80 2024-04-12T06:11:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 msgr2=0x7f488806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 0x7f488806f140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f489c103030 tx=0x7f488401a040 comp rx=0 tx=0).stop 2024-04-12T06:11:45.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 msgr2=0x7f489c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d900 tx=0x7f488c00dcc0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 shutdown_connections 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f488806cc80 0x7f488806f140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f489c0ffda0 0x7f489c101840 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 --2- 172.21.15.8:0/232607434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c100750 0x7f489c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 >> 172.21.15.8:0/232607434 conn(0x7f489c0fb650 msgr2=0x7f489c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 shutdown_connections 2024-04-12T06:11:45.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.566+0000 7f48a190c700 1 -- 172.21.15.8:0/232607434 wait complete. 2024-04-12T06:11:45.575 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 -- 172.21.15.8:0/471668939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58106cd0 msgr2=0x7f3b581070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/471668939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58106cd0 0x7f3b581070b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009910 tx=0x7f3b48009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 -- 172.21.15.8:0/471668939 shutdown_connections 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/471668939 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58068580 0x7f3b580689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/471668939 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58106cd0 0x7f3b581070b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 -- 172.21.15.8:0/471668939 >> 172.21.15.8:0/471668939 conn(0x7f3b58075670 msgr2=0x7f3b58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:45.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 -- 172.21.15.8:0/471668939 shutdown_connections 2024-04-12T06:11:45.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.673+0000 7f3b5e57e700 1 -- 172.21.15.8:0/471668939 wait complete. 2024-04-12T06:11:45.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 Processor -- start 2024-04-12T06:11:45.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 -- start start 2024-04-12T06:11:45.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58106cd0 0x7f3b5819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5819d130 con 0x7f3b58068580 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b5e57e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58196630 con 0x7f3b58106cd0 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42728/0 (socket says 172.21.15.8:42728) 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 -- 172.21.15.8:0/593750431 learned_addr learned my addr 172.21.15.8:0/593750431 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b577fe700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58106cd0 0x7f3b5819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 -- 172.21.15.8:0/593750431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58106cd0 msgr2=0x7f3b5819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58106cd0 0x7f3b5819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.674+0000 7f3b57fff700 1 -- 172.21.15.8:0/593750431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480095f0 con 0x7f3b58068580 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b57fff700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b48000c00 tx=0x7f3b4800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4801d070 con 0x7f3b58068580 2024-04-12T06:11:45.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b48005dc0 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b581968b0 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b480039b0 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.675+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b58196d40 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.676+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b581091f0 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.677+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3b480048e0 con 0x7f3b58068580 2024-04-12T06:11:45.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.677+0000 7f3b557fa700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 0x7f3b4406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:45.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.677+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3b4808e310 con 0x7f3b58068580 2024-04-12T06:11:45.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.677+0000 7f3b577fe700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 0x7f3b4406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:45.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.678+0000 7f3b577fe700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 0x7f3b4406f1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b580fe390 tx=0x7f3b40009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:45.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.681+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3b4805be10 con 0x7f3b58068580 2024-04-12T06:11:45.899 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.901+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b58196fc0 con 0x7f3b4406cce0 2024-04-12T06:11:45.900 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.902+0000 7f3b557fa700 1 -- 172.21.15.8:0/593750431 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3b58196fc0 con 0x7f3b4406cce0 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 msgr2=0x7f3b4406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 0x7f3b4406f1a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3b580fe390 tx=0x7f3b40009260 comp rx=0 tx=0).stop 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 msgr2=0x7f3b5819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3b48000c00 tx=0x7f3b4800be90 comp rx=0 tx=0).stop 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 shutdown_connections 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3b4406cce0 0x7f3b4406f1a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3b58068580 0x7f3b5819c560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 --2- 172.21.15.8:0/593750431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b58106cd0 0x7f3b5819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 >> 172.21.15.8:0/593750431 conn(0x7f3b58075670 msgr2=0x7f3b580fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 shutdown_connections 2024-04-12T06:11:45.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:45.904+0000 7f3b5e57e700 1 -- 172.21.15.8:0/593750431 wait complete. 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- 172.21.15.8:0/2622275224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a00105bd0 msgr2=0x7f4a00105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 --2- 172.21.15.8:0/2622275224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a00105bd0 0x7f4a00105fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009910 tx=0x7f49f0009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- 172.21.15.8:0/2622275224 shutdown_connections 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 --2- 172.21.15.8:0/2622275224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a000ffc20 0x7f4a00100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 --2- 172.21.15.8:0/2622275224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a00105bd0 0x7f4a00105fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- 172.21.15.8:0/2622275224 >> 172.21.15.8:0/2622275224 conn(0x7f4a00075670 msgr2=0x7f4a00075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- 172.21.15.8:0/2622275224 shutdown_connections 2024-04-12T06:11:46.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- 172.21.15.8:0/2622275224 wait complete. 2024-04-12T06:11:46.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 Processor -- start 2024-04-12T06:11:46.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.012+0000 7f4a063a4700 1 -- start start 2024-04-12T06:11:46.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f4a063a4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f4a063a4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 0x7f4a00195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f4a063a4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00196260 con 0x7f4a000ffc20 2024-04-12T06:11:46.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f4a063a4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00198f90 con 0x7f4a00105bd0 2024-04-12T06:11:46.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42746/0 (socket says 172.21.15.8:42746) 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 -- 172.21.15.8:0/483904672 learned_addr learned my addr 172.21.15.8:0/483904672 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49ff7fe700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 0x7f4a00195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 -- 172.21.15.8:0/483904672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 msgr2=0x7f4a00195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 0x7f4a00195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 -- 172.21.15.8:0/483904672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f00095f0 con 0x7f4a000ffc20 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49ff7fe700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 0x7f4a00195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.013+0000 7f49fffff700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49f00092e0 tx=0x7f49f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.014+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f001d070 con 0x7f4a000ffc20 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.014+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49f000f460 con 0x7f4a000ffc20 2024-04-12T06:11:46.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.014+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a00199210 con 0x7f4a000ffc20 2024-04-12T06:11:46.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.014+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f0021640 con 0x7f4a000ffc20 2024-04-12T06:11:46.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.014+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a00199700 con 0x7f4a000ffc20 2024-04-12T06:11:46.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.015+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a001080f0 con 0x7f4a000ffc20 2024-04-12T06:11:46.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.015+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f49f000fa00 con 0x7f4a000ffc20 2024-04-12T06:11:46.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.016+0000 7f49fd7fa700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 0x7f49ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.016+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f49f008de50 con 0x7f4a000ffc20 2024-04-12T06:11:46.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.016+0000 7f49ff7fe700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 0x7f49ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.016+0000 7f49ff7fe700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 0x7f49ec06f1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e8005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.020+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f49f0057fc0 con 0x7f4a000ffc20 2024-04-12T06:11:46.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.232+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a00199920 con 0x7f49ec06cce0 2024-04-12T06:11:46.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.243+0000 7f49fd7fa700 1 -- 172.21.15.8:0/483904672 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4a00199920 con 0x7f49ec06cce0 2024-04-12T06:11:46.241 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (13m) 7m ago 24m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (25m) 7m ago 25m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (24m) 6m ago 24m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (11m) 7m ago 25m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (11m) 6m ago 24m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (13m) 7m ago 24m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (7m) 7m ago 18m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (7m) 7m ago 18m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (6m) 6m ago 18m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (7m) 6m ago 18m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (15m) 7m ago 26m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (14m) 6m ago 23m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:11:46.242 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (11m) 7m ago 26m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (11m) 6m ago 23m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (14m) 7m ago 25m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (14m) 6m ago 23m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (10m) 7m ago 23m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (10m) 7m ago 22m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (9m) 7m ago 22m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (9m) 6m ago 21m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (8m) 6m ago 21m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (7m) 6m ago 20m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:11:46.243 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (13m) 7m ago 24m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 msgr2=0x7f49ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 0x7f49ec06f1a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005fd0 tx=0x7f49e8005dc0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 msgr2=0x7f4a00195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f49f00092e0 tx=0x7f49f000be20 comp rx=0 tx=0).stop 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 shutdown_connections 2024-04-12T06:11:46.244 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49ec06cce0 0x7f49ec06f1a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4a000ffc20 0x7f4a00195690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 --2- 172.21.15.8:0/483904672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a00105bd0 0x7f4a00195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 >> 172.21.15.8:0/483904672 conn(0x7f4a00075670 msgr2=0x7f4a000fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:46.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 shutdown_connections 2024-04-12T06:11:46.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.245+0000 7f4a063a4700 1 -- 172.21.15.8:0/483904672 wait complete. 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 -- 172.21.15.8:0/437385758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 msgr2=0x7ffad01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 --2- 172.21.15.8:0/437385758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad01047a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009910 tx=0x7ffac4009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 -- 172.21.15.8:0/437385758 shutdown_connections 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 --2- 172.21.15.8:0/437385758 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad01047a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 --2- 172.21.15.8:0/437385758 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 0x7ffad0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 -- 172.21.15.8:0/437385758 >> 172.21.15.8:0/437385758 conn(0x7ffad00fb650 msgr2=0x7ffad00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:46.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 -- 172.21.15.8:0/437385758 shutdown_connections 2024-04-12T06:11:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 -- 172.21.15.8:0/437385758 wait complete. 2024-04-12T06:11:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.350+0000 7ffad5d78700 1 Processor -- start 2024-04-12T06:11:46.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffad5d78700 1 -- start start 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffad5d78700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 0x7ffad0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffad5d78700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffad5d78700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad01022c0 con 0x7ffad0100750 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffad5d78700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad0102400 con 0x7ffad00ffda0 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42754/0 (socket says 172.21.15.8:42754) 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 -- 172.21.15.8:0/3954598688 learned_addr learned my addr 172.21.15.8:0/3954598688 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffacf7fe700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 0x7ffad0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 -- 172.21.15.8:0/3954598688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 msgr2=0x7ffad0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 0x7ffad0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 -- 172.21.15.8:0/3954598688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40095f0 con 0x7ffad0100750 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.351+0000 7ffaceffd700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffac400bda0 tx=0x7ffac400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.352+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac401d070 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.352+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ffac400f460 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.352+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad0102680 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.352+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac4021640 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.352+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad01a2c60 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.353+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad01091b0 con 0x7ffad0100750 2024-04-12T06:11:46.353 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.353+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ffac400f9b0 con 0x7ffad0100750 2024-04-12T06:11:46.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.354+0000 7ffaccff9700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 0x7ffab806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.354+0000 7ffacf7fe700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 0x7ffab806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.354+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffac408d750 con 0x7ffad0100750 2024-04-12T06:11:46.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.354+0000 7ffacf7fe700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 0x7ffab806f1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005ea0 tx=0x7ffac0005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.357 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.358+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ffac4057840 con 0x7ffad0100750 2024-04-12T06:11:46.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.648+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffad004f2e0 con 0x7ffad0100750 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.648+0000 7ffaccff9700 1 -- 172.21.15.8:0/3954598688 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ffac405ae60 con 0x7ffad0100750 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:11:46.647 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:11:46.648 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.650+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 msgr2=0x7ffab806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.650+0000 7ffad5d78700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 0x7ffab806f1a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffac0005ea0 tx=0x7ffac0005e30 comp rx=0 tx=0).stop 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.650+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 msgr2=0x7ffad0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.650+0000 7ffad5d78700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffac400bda0 tx=0x7ffac400be80 comp rx=0 tx=0).stop 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 shutdown_connections 2024-04-12T06:11:46.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ffab806cce0 0x7ffab806f1a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffad00ffda0 0x7ffad0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 --2- 172.21.15.8:0/3954598688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffad0100750 0x7ffad0101d80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 >> 172.21.15.8:0/3954598688 conn(0x7ffad00fb650 msgr2=0x7ffad0069190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:46.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 shutdown_connections 2024-04-12T06:11:46.650 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.651+0000 7ffad5d78700 1 -- 172.21.15.8:0/3954598688 wait complete. 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 -- 172.21.15.8:0/1555362505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c068580 msgr2=0x7f528c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 --2- 172.21.15.8:0/1555362505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c068580 0x7f528c0689e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5280009910 tx=0x7f5280009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 -- 172.21.15.8:0/1555362505 shutdown_connections 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 --2- 172.21.15.8:0/1555362505 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c068580 0x7f528c0689e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 --2- 172.21.15.8:0/1555362505 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c106cd0 0x7f528c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 -- 172.21.15.8:0/1555362505 >> 172.21.15.8:0/1555362505 conn(0x7f528c075670 msgr2=0x7f528c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:46.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 -- 172.21.15.8:0/1555362505 shutdown_connections 2024-04-12T06:11:46.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.756+0000 7f5292924700 1 -- 172.21.15.8:0/1555362505 wait complete. 2024-04-12T06:11:46.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 Processor -- start 2024-04-12T06:11:46.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 -- start start 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c068580 0x7f528c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c100770 con 0x7f528c106cd0 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f5292924700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c1008b0 con 0x7f528c068580 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f528b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f528b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42776/0 (socket says 172.21.15.8:42776) 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f528b7fe700 1 -- 172.21.15.8:0/2452693629 learned_addr learned my addr 172.21.15.8:0/2452693629 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.757+0000 7f528bfff700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c068580 0x7f528c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f528b7fe700 1 -- 172.21.15.8:0/2452693629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c068580 msgr2=0x7f528c0ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f528b7fe700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c068580 0x7f528c0ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f528b7fe700 1 -- 172.21.15.8:0/2452693629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800095f0 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f528b7fe700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f528000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528001d070 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5280005710 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c100b30 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f528000fe20 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.758+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c1a2c70 con 0x7f528c106cd0 2024-04-12T06:11:46.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.759+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c1091d0 con 0x7f528c106cd0 2024-04-12T06:11:46.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.760+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f5280003a60 con 0x7f528c106cd0 2024-04-12T06:11:46.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.760+0000 7f52897fa700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 0x7f527406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:46.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.760+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f528008da40 con 0x7f528c106cd0 2024-04-12T06:11:46.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.760+0000 7f528bfff700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 0x7f527406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:46.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.761+0000 7f528bfff700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 0x7f527406f1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f527c005950 tx=0x7f527c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:46.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:46.764+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5280057b30 con 0x7f528c106cd0 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.012+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f528c04f2e0 con 0x7f528c106cd0 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.012+0000 7f52897fa700 1 -- 172.21.15.8:0/2452693629 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f528005b150 con 0x7f528c106cd0 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:11:47.011 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:11:47.012 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:11:47.013 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:11:47.014 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:47.014 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:47.014 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:47.014 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 msgr2=0x7f527406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 0x7f527406f1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f527c005950 tx=0x7f527c00b410 comp rx=0 tx=0).stop 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 msgr2=0x7f528c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5280006010 tx=0x7f528000bd70 comp rx=0 tx=0).stop 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 shutdown_connections 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f527406cce0 0x7f527406f1a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f528c068580 0x7f528c0ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 --2- 172.21.15.8:0/2452693629 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f528c106cd0 0x7f528c100230 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 >> 172.21.15.8:0/2452693629 conn(0x7f528c075670 msgr2=0x7f528c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 shutdown_connections 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.014+0000 7f5292924700 1 -- 172.21.15.8:0/2452693629 wait complete. 2024-04-12T06:11:47.015 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:11:47.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 -- 172.21.15.8:0/481415544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 msgr2=0x7f1aa0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 --2- 172.21.15.8:0/481415544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0101280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009910 tx=0x7f1a90009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:47.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 -- 172.21.15.8:0/481415544 shutdown_connections 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 --2- 172.21.15.8:0/481415544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 0x7f1aa01058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 --2- 172.21.15.8:0/481415544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0101280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 -- 172.21.15.8:0/481415544 >> 172.21.15.8:0/481415544 conn(0x7f1aa00fc750 msgr2=0x7f1aa00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 -- 172.21.15.8:0/481415544 shutdown_connections 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.124+0000 7f1aa6490700 1 -- 172.21.15.8:0/481415544 wait complete. 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 Processor -- start 2024-04-12T06:11:47.124 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 -- start start 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 0x7f1aa00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0073b10 con 0x7f1aa0100ea0 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1aa6490700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa0073c50 con 0x7f1aa0101850 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.125 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.125+0000 7f1a9ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42790/0 (socket says 172.21.15.8:42790) 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9ffff700 1 -- 172.21.15.8:0/882504138 learned_addr learned my addr 172.21.15.8:0/882504138 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9f7fe700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 0x7f1aa00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9ffff700 1 -- 172.21.15.8:0/882504138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 msgr2=0x7f1aa00735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9ffff700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 0x7f1aa00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9ffff700 1 -- 172.21.15.8:0/882504138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a900095f0 con 0x7f1aa0100ea0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9ffff700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000b5c0 tx=0x7f1a9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a9001d070 con 0x7f1aa0100ea0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1a9000f460 con 0x7f1aa0100ea0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa0073ed0 con 0x7f1aa0100ea0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa01a2c40 con 0x7f1aa0100ea0 2024-04-12T06:11:47.126 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.126+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a90021640 con 0x7f1aa0100ea0 2024-04-12T06:11:47.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.127+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa0109240 con 0x7f1aa0100ea0 2024-04-12T06:11:47.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.128+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1a9000f9b0 con 0x7f1aa0100ea0 2024-04-12T06:11:47.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.128+0000 7f1a9d7fa700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 0x7f1a8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.128+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a9008d8c0 con 0x7f1aa0100ea0 2024-04-12T06:11:47.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.128+0000 7f1a9f7fe700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 0x7f1a8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.128+0000 7f1a9f7fe700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 0x7f1a8806f1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0074880 tx=0x7f1a94005d50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:47.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.133+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1a90057930 con 0x7f1aa0100ea0 2024-04-12T06:11:47.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:46 smithi194 ceph-mon[106982]: pgmap v502: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:47.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:46 smithi194 ceph-mon[106982]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:47.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:46 smithi194 ceph-mon[106982]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:47.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:46 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3954598688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:11:47.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.353+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1aa00611d0 con 0x7f1a8806cce0 2024-04-12T06:11:47.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.354+0000 7f1a9d7fa700 1 -- 172.21.15.8:0/882504138 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1aa00611d0 con 0x7f1a8806cce0 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:11:47.353 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:11:47.354 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 msgr2=0x7f1a8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 0x7f1a8806f1a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0074880 tx=0x7f1a94005d50 comp rx=0 tx=0).stop 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 msgr2=0x7f1aa0073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000b5c0 tx=0x7f1a9000be20 comp rx=0 tx=0).stop 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 shutdown_connections 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1a8806cce0 0x7f1a8806f1a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1aa0100ea0 0x7f1aa0073090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 --2- 172.21.15.8:0/882504138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1aa0101850 0x7f1aa00735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.356+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 >> 172.21.15.8:0/882504138 conn(0x7f1aa00fc750 msgr2=0x7f1aa00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:47.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.357+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 shutdown_connections 2024-04-12T06:11:47.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.357+0000 7f1aa6490700 1 -- 172.21.15.8:0/882504138 wait complete. 2024-04-12T06:11:47.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:46 smithi008 ceph-mon[146084]: pgmap v502: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:47.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:46 smithi008 ceph-mon[146084]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:47.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:46 smithi008 ceph-mon[146084]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:47.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:46 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3954598688' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:11:47.462 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.463+0000 7fcc759f5700 1 -- 172.21.15.8:0/3893949615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 msgr2=0x7fcc701047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.463+0000 7fcc759f5700 1 --2- 172.21.15.8:0/3893949615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc701047a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009910 tx=0x7fcc64009c20 comp rx=0 tx=0).stop 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 -- 172.21.15.8:0/3893949615 shutdown_connections 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 --2- 172.21.15.8:0/3893949615 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc701047a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 --2- 172.21.15.8:0/3893949615 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 0x7fcc70100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 -- 172.21.15.8:0/3893949615 >> 172.21.15.8:0/3893949615 conn(0x7fcc700fb650 msgr2=0x7fcc700fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 -- 172.21.15.8:0/3893949615 shutdown_connections 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 -- 172.21.15.8:0/3893949615 wait complete. 2024-04-12T06:11:47.463 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 Processor -- start 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.464+0000 7fcc759f5700 1 -- start start 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc759f5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 0x7fcc701957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc759f5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc759f5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc701963d0 con 0x7fcc70100750 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc759f5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc70199f50 con 0x7fcc700ffda0 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:42802/0 (socket says 172.21.15.8:42802) 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 -- 172.21.15.8:0/4170606410 learned_addr learned my addr 172.21.15.8:0/4170606410 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6effd700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 0x7fcc701957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 -- 172.21.15.8:0/4170606410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 msgr2=0x7fcc701957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 0x7fcc701957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.465 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 -- 172.21.15.8:0/4170606410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640095f0 con 0x7fcc70100750 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc6e7fc700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400bd80 tx=0x7fcc6400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6401f070 con 0x7fcc70100750 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7019a1d0 con 0x7fcc70100750 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcc64003680 con 0x7fcc70100750 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.465+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6400f760 con 0x7fcc70100750 2024-04-12T06:11:47.466 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.466+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7019a6c0 con 0x7fcc70100750 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.467+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7004f2e0 con 0x7fcc70100750 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.467+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fcc64003c60 con 0x7fcc70100750 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.467+0000 7fcc749f3700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 0x7fcc5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.468+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc6408fb00 con 0x7fcc70100750 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.468+0000 7fcc6effd700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 0x7fcc5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:11:47.467 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.468+0000 7fcc6effd700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 0x7fcc5806f1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc60004630 tx=0x7fcc6000b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:11:47.470 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.472+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fcc6405d600 con 0x7fcc70100750 2024-04-12T06:11:47.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.758+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcc7019a8e0 con 0x7fcc70100750 2024-04-12T06:11:47.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.759+0000 7fcc749f3700 1 -- 172.21.15.8:0/4170606410 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fcc64005c00 con 0x7fcc70100750 2024-04-12T06:11:47.757 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:11:47.758 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:11:47.758 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:11:47.758 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:11:47.758 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 msgr2=0x7fcc5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 0x7fcc5806f1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc60004630 tx=0x7fcc6000b3c0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 msgr2=0x7fcc70195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400bd80 tx=0x7fcc6400bdb0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 shutdown_connections 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcc5806cce0 0x7fcc5806f1a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcc700ffda0 0x7fcc701957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 --2- 172.21.15.8:0/4170606410 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcc70100750 0x7fcc70195cf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:11:47.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 >> 172.21.15.8:0/4170606410 conn(0x7fcc700fb650 msgr2=0x7fcc700fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:11:47.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 shutdown_connections 2024-04-12T06:11:47.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:11:47.761+0000 7fcc759f5700 1 -- 172.21.15.8:0/4170606410 wait complete. 2024-04-12T06:11:48.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:47 smithi194 ceph-mon[106982]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:48.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:47 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2452693629' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:11:48.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:48.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:47 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4170606410' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:11:48.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:47 smithi008 ceph-mon[146084]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:48.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:47 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2452693629' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:11:48.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:11:48.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:47 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4170606410' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:11:49.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:48 smithi194 ceph-mon[106982]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:49.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:48 smithi194 ceph-mon[106982]: pgmap v503: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:49.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:48 smithi008 ceph-mon[146084]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:11:49.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:48 smithi008 ceph-mon[146084]: pgmap v503: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:51.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:50 smithi194 ceph-mon[106982]: pgmap v504: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:51.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:50 smithi008 ceph-mon[146084]: pgmap v504: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:53.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:52 smithi194 ceph-mon[106982]: pgmap v505: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:53.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:52 smithi008 ceph-mon[146084]: pgmap v505: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:55.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:55 smithi194 ceph-mon[106982]: pgmap v506: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:55.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:55 smithi008 ceph-mon[146084]: pgmap v506: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:57.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:57 smithi194 ceph-mon[106982]: pgmap v507: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:57.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:57 smithi008 ceph-mon[146084]: pgmap v507: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:59.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:11:59 smithi194 ceph-mon[106982]: pgmap v508: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:11:59.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:11:59 smithi008 ceph-mon[146084]: pgmap v508: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:01.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:01 smithi194 ceph-mon[106982]: pgmap v509: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:01.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:01 smithi008 ceph-mon[146084]: pgmap v509: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:03 smithi194 ceph-mon[106982]: pgmap v510: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:03.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:03 smithi008 ceph-mon[146084]: pgmap v510: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:03.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:05.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:05 smithi194 ceph-mon[106982]: pgmap v511: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:05.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:05 smithi008 ceph-mon[146084]: pgmap v511: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:07.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:07 smithi194 ceph-mon[106982]: pgmap v512: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:07.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:07 smithi008 ceph-mon[146084]: pgmap v512: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:09.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:09 smithi194 ceph-mon[106982]: pgmap v513: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:09.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:09 smithi008 ceph-mon[146084]: pgmap v513: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:11.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:11 smithi194 ceph-mon[106982]: pgmap v514: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:11.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:11 smithi008 ceph-mon[146084]: pgmap v514: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:13.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:13 smithi194 ceph-mon[106982]: pgmap v515: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:13.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:13 smithi008 ceph-mon[146084]: pgmap v515: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:15.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:15 smithi194 ceph-mon[106982]: pgmap v516: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:15.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:15 smithi008 ceph-mon[146084]: pgmap v516: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:17.300 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:17 smithi008 ceph-mon[146084]: pgmap v517: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:17.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:17 smithi194 ceph-mon[106982]: pgmap v517: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:17.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 -- 172.21.15.8:0/1497803665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50101e00 msgr2=0x7f8c501021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:17.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 --2- 172.21.15.8:0/1497803665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50101e00 0x7f8c501021e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009910 tx=0x7f8c44009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:17.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 -- 172.21.15.8:0/1497803665 shutdown_connections 2024-04-12T06:12:17.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 --2- 172.21.15.8:0/1497803665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50102720 0x7f8c5010ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:17.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 --2- 172.21.15.8:0/1497803665 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50101e00 0x7f8c501021e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:17.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 -- 172.21.15.8:0/1497803665 >> 172.21.15.8:0/1497803665 conn(0x7f8c500fb650 msgr2=0x7f8c500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:17.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 -- 172.21.15.8:0/1497803665 shutdown_connections 2024-04-12T06:12:17.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.882+0000 7f8c55497700 1 -- 172.21.15.8:0/1497803665 wait complete. 2024-04-12T06:12:17.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 Processor -- start 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 -- start start 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50101e00 0x7f8c5019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5019d200 con 0x7f8c50102720 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c55497700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501966b0 con 0x7f8c50101e00 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c4e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c4e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44470/0 (socket says 172.21.15.8:44470) 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c4e7fc700 1 -- 172.21.15.8:0/3458330538 learned_addr learned my addr 172.21.15.8:0/3458330538 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.883+0000 7f8c4effd700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50101e00 0x7f8c5019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c4e7fc700 1 -- 172.21.15.8:0/3458330538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50101e00 msgr2=0x7f8c5019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:17.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c4e7fc700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50101e00 0x7f8c5019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c4e7fc700 1 -- 172.21.15.8:0/3458330538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440095f0 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c4e7fc700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000d8d0 tx=0x7f8c4000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4000f860 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8c4000fe60 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c50196990 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c40019360 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.884+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c50196ee0 con 0x7f8c50102720 2024-04-12T06:12:17.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.885+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c50108310 con 0x7f8c50102720 2024-04-12T06:12:17.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.886+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8c40010460 con 0x7f8c50102720 2024-04-12T06:12:17.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.886+0000 7f8c37fff700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 0x7f8c3806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:17.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.886+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8c4008bfe0 con 0x7f8c50102720 2024-04-12T06:12:17.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.886+0000 7f8c4effd700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 0x7f8c3806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:17.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.887+0000 7f8c4effd700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 0x7f8c3806f1a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c4401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:17.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:17.891+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8c40059ae0 con 0x7f8c50102720 2024-04-12T06:12:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.112+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c50197160 con 0x7f8c3806cce0 2024-04-12T06:12:18.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.114+0000 7f8c37fff700 1 -- 172.21.15.8:0/3458330538 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8c50197160 con 0x7f8c3806cce0 2024-04-12T06:12:18.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 msgr2=0x7f8c3806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 0x7f8c3806f1a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8c44006010 tx=0x7f8c4401a040 comp rx=0 tx=0).stop 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 msgr2=0x7f8c5019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000d8d0 tx=0x7f8c4000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 shutdown_connections 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8c3806cce0 0x7f8c3806f1a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c50101e00 0x7f8c5019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 --2- 172.21.15.8:0/3458330538 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8c50102720 0x7f8c5019cb70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 >> 172.21.15.8:0/3458330538 conn(0x7f8c500fb650 msgr2=0x7f8c50105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 shutdown_connections 2024-04-12T06:12:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.116+0000 7f8c55497700 1 -- 172.21.15.8:0/3458330538 wait complete. 2024-04-12T06:12:18.125 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:12:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 -- 172.21.15.8:0/2413344480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 msgr2=0x7f1040105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 --2- 172.21.15.8:0/2413344480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040105fb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1030009910 tx=0x7f1030009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 -- 172.21.15.8:0/2413344480 shutdown_connections 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 --2- 172.21.15.8:0/2413344480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 0x7f1040100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 --2- 172.21.15.8:0/2413344480 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040105fb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 -- 172.21.15.8:0/2413344480 >> 172.21.15.8:0/2413344480 conn(0x7f1040075670 msgr2=0x7f1040075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 -- 172.21.15.8:0/2413344480 shutdown_connections 2024-04-12T06:12:18.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.223+0000 7f104604d700 1 -- 172.21.15.8:0/2413344480 wait complete. 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.224+0000 7f104604d700 1 Processor -- start 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.224+0000 7f104604d700 1 -- start start 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f104604d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 0x7f1040194590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f104604d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f104604d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10401951b0 con 0x7f1040105bd0 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f104604d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040198f40 con 0x7f10400ffc20 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44492/0 (socket says 172.21.15.8:44492) 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 -- 172.21.15.8:0/94328417 learned_addr learned my addr 172.21.15.8:0/94328417 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103f7fe700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 0x7f1040194590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 -- 172.21.15.8:0/94328417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 msgr2=0x7f1040194590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 0x7f1040194590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 -- 172.21.15.8:0/94328417 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300095f0 con 0x7f1040105bd0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103effd700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1034012fc0 tx=0x7f10340103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.225+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103400dba0 con 0x7f1040105bd0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.226+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10401991c0 con 0x7f1040105bd0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.226+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1034013040 con 0x7f1040105bd0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.226+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103400be50 con 0x7f1040105bd0 2024-04-12T06:12:18.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.226+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1040199710 con 0x7f1040105bd0 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.227+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10401080d0 con 0x7f1040105bd0 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.227+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1034010930 con 0x7f1040105bd0 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.227+0000 7f103cff9700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 0x7f102806f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.228+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f10340907a0 con 0x7f1040105bd0 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.228+0000 7f103f7fe700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 0x7f102806f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.228+0000 7f103f7fe700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 0x7f102806f140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030009f90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.232+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f103405a810 con 0x7f1040105bd0 2024-04-12T06:12:18.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:18.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:18.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.452+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1040199930 con 0x7f102806cc80 2024-04-12T06:12:18.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.453+0000 7f103cff9700 1 -- 172.21.15.8:0/94328417 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1040199930 con 0x7f102806cc80 2024-04-12T06:12:18.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 msgr2=0x7f102806f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 0x7f102806f140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1030000c00 tx=0x7f1030009f90 comp rx=0 tx=0).stop 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 msgr2=0x7f1040194ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1034012fc0 tx=0x7f10340103a0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 shutdown_connections 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102806cc80 0x7f102806f140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffc20 0x7f1040194590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 --2- 172.21.15.8:0/94328417 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bd0 0x7f1040194ad0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 >> 172.21.15.8:0/94328417 conn(0x7f1040075670 msgr2=0x7f10400feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 shutdown_connections 2024-04-12T06:12:18.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.455+0000 7f104604d700 1 -- 172.21.15.8:0/94328417 wait complete. 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 -- 172.21.15.8:0/2005014551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544100ea0 msgr2=0x7f1544101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 --2- 172.21.15.8:0/2005014551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544100ea0 0x7f1544101280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f153c00b0a0 tx=0x7f153c00b3b0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 -- 172.21.15.8:0/2005014551 shutdown_connections 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 --2- 172.21.15.8:0/2005014551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544101850 0x7f15441058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 --2- 172.21.15.8:0/2005014551 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544100ea0 0x7f1544101280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 -- 172.21.15.8:0/2005014551 >> 172.21.15.8:0/2005014551 conn(0x7f15440fc750 msgr2=0x7f15440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 -- 172.21.15.8:0/2005014551 shutdown_connections 2024-04-12T06:12:18.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.564+0000 7f154b98b700 1 -- 172.21.15.8:0/2005014551 wait complete. 2024-04-12T06:12:18.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 Processor -- start 2024-04-12T06:12:18.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 -- start start 2024-04-12T06:12:18.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544100ea0 0x7f1544073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1544073b10 con 0x7f1544101850 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f154b98b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1544073c50 con 0x7f1544100ea0 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44516/0 (socket says 172.21.15.8:44516) 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 -- 172.21.15.8:0/505969422 learned_addr learned my addr 172.21.15.8:0/505969422 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1549727700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544100ea0 0x7f1544073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 -- 172.21.15.8:0/505969422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544100ea0 msgr2=0x7f1544073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544100ea0 0x7f1544073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.565+0000 7f1548f26700 1 -- 172.21.15.8:0/505969422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c009d00 con 0x7f1544101850 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f1548f26700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f153400c8f0 tx=0x7f153400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1534007a50 con 0x7f1544101850 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1534007bb0 con 0x7f1544101850 2024-04-12T06:12:18.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15441a2960 con 0x7f1544101850 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1534018630 con 0x7f1544101850 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.566+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15441a2d70 con 0x7f1544101850 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.567+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1544109240 con 0x7f1544101850 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.567+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1534007d20 con 0x7f1544101850 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.568+0000 7f153a7fc700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 0x7f153006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.568+0000 7f1549727700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 0x7f153006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.568+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f153408c4a0 con 0x7f1544101850 2024-04-12T06:12:18.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.568+0000 7f1549727700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 0x7f153006f1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f153c009850 tx=0x7f153c0097c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.573+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1534056590 con 0x7f1544101850 2024-04-12T06:12:18.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.786+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f154402cce0 con 0x7f153006cce0 2024-04-12T06:12:18.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.796+0000 7f153a7fc700 1 -- 172.21.15.8:0/505969422 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f154402cce0 con 0x7f153006cce0 2024-04-12T06:12:18.795 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:12:18.795 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (14m) 7m ago 25m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:12:18.795 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (25m) 7m ago 25m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (24m) 7m ago 24m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (11m) 7m ago 25m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (11m) 7m ago 24m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (13m) 7m ago 25m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (7m) 7m ago 19m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (8m) 7m ago 19m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (7m) 7m ago 19m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (7m) 7m ago 19m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (16m) 7m ago 27m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (15m) 7m ago 24m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (12m) 7m ago 27m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:12:18.796 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (12m) 7m ago 24m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (15m) 7m ago 25m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (14m) 7m ago 24m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (11m) 7m ago 23m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (10m) 7m ago 23m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (10m) 7m ago 22m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (9m) 7m ago 22m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (9m) 7m ago 21m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (8m) 7m ago 21m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:12:18.797 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (14m) 7m ago 24m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.798+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 msgr2=0x7f153006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.798+0000 7f154b98b700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 0x7f153006f1a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f153c009850 tx=0x7f153c0097c0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.798+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 msgr2=0x7f15440735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.798+0000 7f154b98b700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f153400c8f0 tx=0x7f153400ccb0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 shutdown_connections 2024-04-12T06:12:18.798 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f153006cce0 0x7f153006f1a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1544100ea0 0x7f1544073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 --2- 172.21.15.8:0/505969422 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1544101850 0x7f15440735d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 >> 172.21.15.8:0/505969422 conn(0x7f15440fc750 msgr2=0x7f15440fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 shutdown_connections 2024-04-12T06:12:18.799 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.799+0000 7f154b98b700 1 -- 172.21.15.8:0/505969422 wait complete. 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 -- 172.21.15.8:0/3591325236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac105bd0 msgr2=0x7fc1ac105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3591325236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac105fb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009910 tx=0x7fc19c009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 -- 172.21.15.8:0/3591325236 shutdown_connections 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3591325236 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3591325236 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac105fb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 -- 172.21.15.8:0/3591325236 >> 172.21.15.8:0/3591325236 conn(0x7fc1ac075670 msgr2=0x7fc1ac075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 -- 172.21.15.8:0/3591325236 shutdown_connections 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.903+0000 7fc1b120a700 1 -- 172.21.15.8:0/3591325236 wait complete. 2024-04-12T06:12:18.903 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 Processor -- start 2024-04-12T06:12:18.904 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 -- start start 2024-04-12T06:12:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.906 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac196260 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.904+0000 7fc1b120a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac198f90 con 0x7fc1ac105bd0 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44532/0 (socket says 172.21.15.8:44532) 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 -- 172.21.15.8:0/3791804340 learned_addr learned my addr 172.21.15.8:0/3791804340 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aa59c700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 -- 172.21.15.8:0/3791804340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 msgr2=0x7fc1ac195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 -- 172.21.15.8:0/3791804340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0095f0 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aa59c700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1aad9d700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006010 tx=0x7fc19c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19c01d070 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.907 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc19c00f460 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac199210 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc19c021640 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.905+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac199700 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.906+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ac1080f0 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.907+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc19c00fa60 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.907+0000 7fc193fff700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 0x7fc19406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.908+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc19c08d7c0 con 0x7fc1ac0ffc20 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.908+0000 7fc1aa59c700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 0x7fc19406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:18.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.908+0000 7fc1aa59c700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 0x7fc19406f1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:18.910 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:18.912+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc19c0578b0 con 0x7fc1ac0ffc20 2024-04-12T06:12:19.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.201+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1ac1969f0 con 0x7fc1ac0ffc20 2024-04-12T06:12:19.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.201+0000 7fc193fff700 1 -- 172.21.15.8:0/3791804340 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc19c05aed0 con 0x7fc1ac0ffc20 2024-04-12T06:12:19.200 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:12:19.200 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:12:19.200 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:12:19.201 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:12:19.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.203+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 msgr2=0x7fc19406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.203+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 0x7fc19406f1a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0005dc0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.203+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 msgr2=0x7fc1ac195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.203+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc19c006010 tx=0x7fc19c00be20 comp rx=0 tx=0).stop 2024-04-12T06:12:19.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 shutdown_connections 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc19406cce0 0x7fc19406f1a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc1ac0ffc20 0x7fc1ac195690 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 --2- 172.21.15.8:0/3791804340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc1ac105bd0 0x7fc1ac195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 >> 172.21.15.8:0/3791804340 conn(0x7fc1ac075670 msgr2=0x7fc1ac0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 shutdown_connections 2024-04-12T06:12:19.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.204+0000 7fc1b120a700 1 -- 172.21.15.8:0/3791804340 wait complete. 2024-04-12T06:12:19.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 -- 172.21.15.8:0/481481445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba00ffc20 msgr2=0x7f4ba0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/481481445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba00ffc20 0x7f4ba0100080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009910 tx=0x7f4b94009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:19.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 -- 172.21.15.8:0/481481445 shutdown_connections 2024-04-12T06:12:19.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/481481445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba00ffc20 0x7f4ba0100080 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/481481445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba0105bd0 0x7f4ba0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 -- 172.21.15.8:0/481481445 >> 172.21.15.8:0/481481445 conn(0x7f4ba0075670 msgr2=0x7f4ba0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:19.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 -- 172.21.15.8:0/481481445 shutdown_connections 2024-04-12T06:12:19.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.309+0000 7f4ba5e56700 1 -- 172.21.15.8:0/481481445 wait complete. 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 Processor -- start 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 -- start start 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba00ffc20 0x7f4ba019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba019d150 con 0x7f4ba0105bd0 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.310+0000 7f4ba5e56700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba0196600 con 0x7f4ba00ffc20 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44546/0 (socket says 172.21.15.8:44546) 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 -- 172.21.15.8:0/1721596113 learned_addr learned my addr 172.21.15.8:0/1721596113 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:19.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9f7fe700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba00ffc20 0x7f4ba019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 -- 172.21.15.8:0/1721596113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba00ffc20 msgr2=0x7f4ba019c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba00ffc20 0x7f4ba019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 -- 172.21.15.8:0/1721596113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940095f0 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9effd700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b94006010 tx=0x7f4b9400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9401d070 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b9400f460 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b94021600 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba0196880 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.311+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba0196d20 con 0x7f4ba0105bd0 2024-04-12T06:12:19.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.312+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba004f2e0 con 0x7f4ba0105bd0 2024-04-12T06:12:19.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.313+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4b9400f5d0 con 0x7f4ba0105bd0 2024-04-12T06:12:19.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.313+0000 7f4b9cff9700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 0x7f4b8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.313+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b9408d800 con 0x7f4ba0105bd0 2024-04-12T06:12:19.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.313+0000 7f4b9f7fe700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 0x7f4b8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.314+0000 7f4b9f7fe700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 0x7f4b8806f1a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005950 tx=0x7f4b900058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:19.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.318+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4b9405b300 con 0x7f4ba0105bd0 2024-04-12T06:12:19.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:19 smithi194 ceph-mon[106982]: pgmap v518: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:19.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:19 smithi008 ceph-mon[146084]: pgmap v518: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:19.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.568+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ba0196fa0 con 0x7f4ba0105bd0 2024-04-12T06:12:19.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.569+0000 7f4b9cff9700 1 -- 172.21.15.8:0/1721596113 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4b9405ae90 con 0x7f4ba0105bd0 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:12:19.568 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:12:19.569 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:12:19.570 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:19.571 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:19.571 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:19.571 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:19.571 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 msgr2=0x7f4b8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 0x7f4b8806f1a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005950 tx=0x7f4b900058e0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 msgr2=0x7f4ba019cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4b94006010 tx=0x7f4b9400be20 comp rx=0 tx=0).stop 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 shutdown_connections 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b8806cce0 0x7f4b8806f1a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4ba00ffc20 0x7f4ba019c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 --2- 172.21.15.8:0/1721596113 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4ba0105bd0 0x7f4ba019cac0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 >> 172.21.15.8:0/1721596113 conn(0x7f4ba0075670 msgr2=0x7f4ba0068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 shutdown_connections 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.571+0000 7f4ba5e56700 1 -- 172.21.15.8:0/1721596113 wait complete. 2024-04-12T06:12:19.572 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 -- 172.21.15.8:0/1307025091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c106cd0 msgr2=0x7f219c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/1307025091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c106cd0 0x7f219c1070b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f218c009910 tx=0x7f218c009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 -- 172.21.15.8:0/1307025091 shutdown_connections 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/1307025091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c068580 0x7f219c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/1307025091 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c106cd0 0x7f219c1070b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 -- 172.21.15.8:0/1307025091 >> 172.21.15.8:0/1307025091 conn(0x7f219c075670 msgr2=0x7f219c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:19.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 -- 172.21.15.8:0/1307025091 shutdown_connections 2024-04-12T06:12:19.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.678+0000 7f21a1d7f700 1 -- 172.21.15.8:0/1307025091 wait complete. 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.679+0000 7f21a1d7f700 1 Processor -- start 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.679+0000 7f21a1d7f700 1 -- start start 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.679+0000 7f21a1d7f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.679+0000 7f21a1d7f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c106cd0 0x7f219c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f21a1d7f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c19d130 con 0x7f219c068580 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f21a1d7f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c196630 con 0x7f219c106cd0 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44560/0 (socket says 172.21.15.8:44560) 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 -- 172.21.15.8:0/3201548041 learned_addr learned my addr 172.21.15.8:0/3201548041 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219affd700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c106cd0 0x7f219c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 -- 172.21.15.8:0/3201548041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c106cd0 msgr2=0x7f219c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c106cd0 0x7f219c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 -- 172.21.15.8:0/3201548041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c0095f0 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f219b7fe700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f218c006010 tx=0x7f218c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c01d070 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f218c00f460 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c1968b0 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f218c021620 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.680+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c196da0 con 0x7f219c068580 2024-04-12T06:12:19.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.681+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c1091f0 con 0x7f219c068580 2024-04-12T06:12:19.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.682+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f218c003a60 con 0x7f219c068580 2024-04-12T06:12:19.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.682+0000 7f2198ff9700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 0x7f218806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:19.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.682+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f218c08d710 con 0x7f219c068580 2024-04-12T06:12:19.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.682+0000 7f219affd700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 0x7f218806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:19.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.683+0000 7f219affd700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 0x7f218806f1a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2184005950 tx=0x7f21840058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:19.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.687+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f218c05b290 con 0x7f219c068580 2024-04-12T06:12:19.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.909+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f219c197010 con 0x7f218806cce0 2024-04-12T06:12:19.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.911+0000 7f2198ff9700 1 -- 172.21.15.8:0/3201548041 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f219c197010 con 0x7f218806cce0 2024-04-12T06:12:19.909 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:12:19.909 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:12:19.910 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:12:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 msgr2=0x7f218806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 0x7f218806f1a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2184005950 tx=0x7f21840058e0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.911 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 msgr2=0x7f219c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f218c006010 tx=0x7f218c00be20 comp rx=0 tx=0).stop 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 shutdown_connections 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f218806cce0 0x7f218806f1a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f219c068580 0x7f219c19c560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 --2- 172.21.15.8:0/3201548041 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f219c106cd0 0x7f219c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 >> 172.21.15.8:0/3201548041 conn(0x7f219c075670 msgr2=0x7f219c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 shutdown_connections 2024-04-12T06:12:19.912 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:19.913+0000 7f21a1d7f700 1 -- 172.21.15.8:0/3201548041 wait complete. 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 -- 172.21.15.8:0/1192290352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 msgr2=0x7f5ca4105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 --2- 172.21.15.8:0/1192290352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4105fd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009910 tx=0x7f5c94009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 -- 172.21.15.8:0/1192290352 shutdown_connections 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 --2- 172.21.15.8:0/1192290352 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 0x7f5ca40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 --2- 172.21.15.8:0/1192290352 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4105fd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 -- 172.21.15.8:0/1192290352 >> 172.21.15.8:0/1192290352 conn(0x7f5ca40fb610 msgr2=0x7f5ca40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 -- 172.21.15.8:0/1192290352 shutdown_connections 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.019+0000 7f5caa721700 1 -- 172.21.15.8:0/1192290352 wait complete. 2024-04-12T06:12:20.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 Processor -- start 2024-04-12T06:12:20.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 -- start start 2024-04-12T06:12:20.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 0x7f5ca4196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:20.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:20.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41972d0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.020+0000 7f5caa721700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca419a000 con 0x7f5ca40ffb40 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44564/0 (socket says 172.21.15.8:44564) 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 -- 172.21.15.8:0/657502828 learned_addr learned my addr 172.21.15.8:0/657502828 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca3fff700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 0x7f5ca4196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 -- 172.21.15.8:0/657502828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 msgr2=0x7f5ca4196700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 0x7f5ca4196700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 -- 172.21.15.8:0/657502828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940095f0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca37fe700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d8d0 tx=0x7f5c8c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8c0098a0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c8c010460 con 0x7f5ca4105bf0 2024-04-12T06:12:20.022 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c8c009ea0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca419a2e0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.021+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca419a830 con 0x7f5ca4105bf0 2024-04-12T06:12:20.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.022+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca40679c0 con 0x7f5ca4105bf0 2024-04-12T06:12:20.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.023+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f5c8c009a00 con 0x7f5ca4105bf0 2024-04-12T06:12:20.023 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.023+0000 7f5ca17fa700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 0x7f5c9006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:20.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.023+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c8c08c370 con 0x7f5ca4105bf0 2024-04-12T06:12:20.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.023+0000 7f5ca3fff700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 0x7f5c9006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:20.024 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.024+0000 7f5ca3fff700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 0x7f5c9006f1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006010 tx=0x7f5c9401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:20.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.028+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5c8c056460 con 0x7f5ca4105bf0 2024-04-12T06:12:20.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.311+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5ca419ab40 con 0x7f5ca4105bf0 2024-04-12T06:12:20.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.312+0000 7f5ca17fa700 1 -- 172.21.15.8:0/657502828 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f5c8c059a80 con 0x7f5ca4105bf0 2024-04-12T06:12:20.310 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:12:20.311 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:12:20.311 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:12:20.311 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:12:20.311 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:12:20.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 msgr2=0x7f5c9006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 0x7f5c9006f1a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006010 tx=0x7f5c9401a040 comp rx=0 tx=0).stop 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 msgr2=0x7f5ca4196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00d8d0 tx=0x7f5c8c00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 shutdown_connections 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c9006cce0 0x7f5c9006f1a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ca40ffb40 0x7f5ca4196700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 --2- 172.21.15.8:0/657502828 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5ca4105bf0 0x7f5ca4196c40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 >> 172.21.15.8:0/657502828 conn(0x7f5ca40fb610 msgr2=0x7f5ca4068f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 shutdown_connections 2024-04-12T06:12:20.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:20.314+0000 7f5caa721700 1 -- 172.21.15.8:0/657502828 wait complete. 2024-04-12T06:12:20.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:20 smithi194 ceph-mon[106982]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:20 smithi194 ceph-mon[106982]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:20 smithi194 ceph-mon[106982]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:20 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3791804340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:12:20.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:20 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1721596113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:12:20.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:20 smithi008 ceph-mon[146084]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:20 smithi008 ceph-mon[146084]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:20 smithi008 ceph-mon[146084]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:20.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:20 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3791804340' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:12:20.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:20 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1721596113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:12:21.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:21 smithi194 ceph-mon[106982]: pgmap v519: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:21.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:21 smithi194 ceph-mon[106982]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:21.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:21 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/657502828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:12:21.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:21 smithi008 ceph-mon[146084]: pgmap v519: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:21.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:21 smithi008 ceph-mon[146084]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:21.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:21 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/657502828' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:12:23.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:23 smithi194 ceph-mon[106982]: pgmap v520: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:23.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:23 smithi008 ceph-mon[146084]: pgmap v520: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:25.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:25 smithi194 ceph-mon[106982]: pgmap v521: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:25.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:25 smithi008 ceph-mon[146084]: pgmap v521: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:27.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:27 smithi194 ceph-mon[106982]: pgmap v522: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:27.393 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:27 smithi008 ceph-mon[146084]: pgmap v522: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:29.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:29 smithi194 ceph-mon[106982]: pgmap v523: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:29.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:29 smithi008 ceph-mon[146084]: pgmap v523: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:31.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:31 smithi194 ceph-mon[106982]: pgmap v524: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:31.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:31 smithi008 ceph-mon[146084]: pgmap v524: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:33.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:33 smithi194 ceph-mon[106982]: pgmap v525: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:33.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:33.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:33 smithi008 ceph-mon[146084]: pgmap v525: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:33.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:34.406 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:34 smithi008 ceph-mon[146084]: pgmap v526: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:34.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:34 smithi194 ceph-mon[106982]: pgmap v526: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:36.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:36 smithi008 ceph-mon[146084]: pgmap v527: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:37.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:36 smithi194 ceph-mon[106982]: pgmap v527: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:38.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:38 smithi008 ceph-mon[146084]: pgmap v528: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:39.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:38 smithi194 ceph-mon[106982]: pgmap v528: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:40.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:40 smithi008 ceph-mon[146084]: pgmap v529: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:41.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:40 smithi194 ceph-mon[106982]: pgmap v529: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:42.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:42 smithi008 ceph-mon[146084]: pgmap v530: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:43.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:42 smithi194 ceph-mon[106982]: pgmap v530: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:44.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:44 smithi008 ceph-mon[146084]: pgmap v531: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:45.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:44 smithi194 ceph-mon[106982]: pgmap v531: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:46.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:46 smithi008 ceph-mon[146084]: pgmap v532: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:47.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:46 smithi194 ceph-mon[106982]: pgmap v532: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:47.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:48.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:12:48.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:48 smithi008 ceph-mon[146084]: pgmap v533: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:49.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:48 smithi194 ceph-mon[106982]: pgmap v533: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:50.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 -- 172.21.15.8:0/3355718267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c0ffc20 msgr2=0x7f7e9c100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 --2- 172.21.15.8:0/3355718267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c100080 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009910 tx=0x7f7e8c009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 -- 172.21.15.8:0/3355718267 shutdown_connections 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 --2- 172.21.15.8:0/3355718267 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c100080 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 --2- 172.21.15.8:0/3355718267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 -- 172.21.15.8:0/3355718267 >> 172.21.15.8:0/3355718267 conn(0x7f7e9c075670 msgr2=0x7f7e9c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 -- 172.21.15.8:0/3355718267 shutdown_connections 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.435+0000 7f7ea3559700 1 -- 172.21.15.8:0/3355718267 wait complete. 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 Processor -- start 2024-04-12T06:12:50.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 -- start start 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c196600 con 0x7f7e9c105bd0 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea3559700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c196770 con 0x7f7e9c0ffc20 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49570/0 (socket says 172.21.15.8:49570) 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 -- 172.21.15.8:0/4291131260 learned_addr learned my addr 172.21.15.8:0/4291131260 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea12f5700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 -- 172.21.15.8:0/4291131260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c0ffc20 msgr2=0x7f7e9c19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.436+0000 7f7ea0af4700 1 -- 172.21.15.8:0/4291131260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c0095f0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7ea0af4700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c01d070 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e8c00f460 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c1969f0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e8c021640 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.437+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c196ee0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.438+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c04f2e0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.439+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7e8c00f9e0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.439+0000 7f7e927fc700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 0x7f7e8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.439+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7e8c08dbd0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.439+0000 7f7ea12f5700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 0x7f7e8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.440+0000 7f7ea12f5700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 0x7f7e8806f1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e9800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:50.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.443+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7e8c057de0 con 0x7f7e9c105bd0 2024-04-12T06:12:50.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.666+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e9c1971a0 con 0x7f7e8806cce0 2024-04-12T06:12:50.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.667+0000 7f7e927fc700 1 -- 172.21.15.8:0/4291131260 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7e9c1971a0 con 0x7f7e8806cce0 2024-04-12T06:12:50.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.669+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 msgr2=0x7f7e8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.669+0000 7f7ea3559700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 0x7f7e8806f1a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e98005fd0 tx=0x7f7e9800a560 comp rx=0 tx=0).stop 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.669+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 msgr2=0x7f7e9c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.669+0000 7f7ea3559700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006010 tx=0x7f7e8c00be20 comp rx=0 tx=0).stop 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 shutdown_connections 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7e8806cce0 0x7f7e8806f1a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e9c0ffc20 0x7f7e9c19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 --2- 172.21.15.8:0/4291131260 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7e9c105bd0 0x7f7e9c19cb50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 >> 172.21.15.8:0/4291131260 conn(0x7f7e9c075670 msgr2=0x7f7e9c068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 shutdown_connections 2024-04-12T06:12:50.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.670+0000 7f7ea3559700 1 -- 172.21.15.8:0/4291131260 wait complete. 2024-04-12T06:12:50.679 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:12:50.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 -- 172.21.15.8:0/3471551860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 msgr2=0x7fc6d4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 --2- 172.21.15.8:0/3471551860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d4100080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009910 tx=0x7fc6c8009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:50.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 -- 172.21.15.8:0/3471551860 shutdown_connections 2024-04-12T06:12:50.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 --2- 172.21.15.8:0/3471551860 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d4100080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 --2- 172.21.15.8:0/3471551860 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 -- 172.21.15.8:0/3471551860 >> 172.21.15.8:0/3471551860 conn(0x7fc6d4075670 msgr2=0x7fc6d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:50.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 -- 172.21.15.8:0/3471551860 shutdown_connections 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.777+0000 7fc6d8982700 1 -- 172.21.15.8:0/3471551860 wait complete. 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 Processor -- start 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 -- start start 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4199570 con 0x7fc6d40ffc20 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d8982700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d419d380 con 0x7fc6d4105bd0 2024-04-12T06:12:50.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49576/0 (socket says 172.21.15.8:49576) 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 -- 172.21.15.8:0/816920688 learned_addr learned my addr 172.21.15.8:0/816920688 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d1d9b700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 -- 172.21.15.8:0/816920688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 msgr2=0x7fc6d4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d259c700 1 -- 172.21.15.8:0/816920688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80095f0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.778+0000 7fc6d1d9b700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:12:50.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6d259c700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400d8d0 tx=0x7fc6c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c4009940 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d419d660 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc6c4010460 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c400f5f0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.779+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d419dbb0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.780+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d404f2e0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.780+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc6c40105d0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.781+0000 7fc6c37fe700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 0x7fc6bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:50.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.781+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc6c408c5d0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.781+0000 7fc6d1d9b700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 0x7fc6bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:50.786 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.781+0000 7fc6d1d9b700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 0x7fc6bc06f1a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80059d0 tx=0x7fc6c8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:50.787 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:50.785+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc6c405a0d0 con 0x7fc6d40ffc20 2024-04-12T06:12:50.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:50 smithi008 ceph-mon[146084]: pgmap v534: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:51.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.004+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6d419de40 con 0x7fc6bc06cce0 2024-04-12T06:12:51.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.005+0000 7fc6c37fe700 1 -- 172.21.15.8:0/816920688 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc6d419de40 con 0x7fc6bc06cce0 2024-04-12T06:12:51.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 msgr2=0x7fc6bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 0x7fc6bc06f1a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80059d0 tx=0x7fc6c8005960 comp rx=0 tx=0).stop 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 msgr2=0x7fc6d41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400d8d0 tx=0x7fc6c400dc90 comp rx=0 tx=0).stop 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 shutdown_connections 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc6bc06cce0 0x7fc6bc06f1a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc6d40ffc20 0x7fc6d41989a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 --2- 172.21.15.8:0/816920688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc6d4105bd0 0x7fc6d4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.007+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 >> 172.21.15.8:0/816920688 conn(0x7fc6d4075670 msgr2=0x7fc6d40687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.008+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 shutdown_connections 2024-04-12T06:12:51.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.008+0000 7fc6d8982700 1 -- 172.21.15.8:0/816920688 wait complete. 2024-04-12T06:12:51.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:50 smithi194 ceph-mon[106982]: pgmap v534: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 -- 172.21.15.8:0/987018780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b40ffc20 msgr2=0x7f28b4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/987018780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b40ffc20 0x7f28b4100080 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009910 tx=0x7f28a4009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 -- 172.21.15.8:0/987018780 shutdown_connections 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/987018780 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b40ffc20 0x7f28b4100080 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/987018780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b4105bd0 0x7f28b4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 -- 172.21.15.8:0/987018780 >> 172.21.15.8:0/987018780 conn(0x7f28b4075670 msgr2=0x7f28b4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 -- 172.21.15.8:0/987018780 shutdown_connections 2024-04-12T06:12:51.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.115+0000 7f28bbc7b700 1 -- 172.21.15.8:0/987018780 wait complete. 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 Processor -- start 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 -- start start 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b40ffc20 0x7f28b419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b419d150 con 0x7f28b4105bd0 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28bbc7b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b4196600 con 0x7f28b40ffc20 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49594/0 (socket says 172.21.15.8:49594) 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 -- 172.21.15.8:0/3738375367 learned_addr learned my addr 172.21.15.8:0/3738375367 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9a17700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b40ffc20 0x7f28b419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 -- 172.21.15.8:0/3738375367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b40ffc20 msgr2=0x7f28b419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b40ffc20 0x7f28b419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.116+0000 7f28b9216700 1 -- 172.21.15.8:0/3738375367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40095f0 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28b9216700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f28a4000c00 tx=0x7f28a400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a401d070 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28a4005dc0 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28a4003940 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b4196880 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.117+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b4196d10 con 0x7f28b4105bd0 2024-04-12T06:12:51.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.118+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b404f2e0 con 0x7f28b4105bd0 2024-04-12T06:12:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.118+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f28a4004ae0 con 0x7f28b4105bd0 2024-04-12T06:12:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.119+0000 7f28aaffd700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 0x7f28a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.119+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f28a408db60 con 0x7f28b4105bd0 2024-04-12T06:12:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.119+0000 7f28b9a17700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 0x7f28a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.119+0000 7f28b9a17700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 0x7f28a006f1a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009c60 tx=0x7f28b0009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.123+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f28a405b660 con 0x7f28b4105bd0 2024-04-12T06:12:51.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.334+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f28b4196fc0 con 0x7f28a006cce0 2024-04-12T06:12:51.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.346+0000 7f28aaffd700 1 -- 172.21.15.8:0/3738375367 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f28b4196fc0 con 0x7f28a006cce0 2024-04-12T06:12:51.345 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:12:51.345 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (14m) 8m ago 26m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:12:51.345 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (26m) 8m ago 26m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:12:51.345 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (25m) 7m ago 25m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:12:51.346 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (12m) 8m ago 26m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:12:51.346 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (12m) 7m ago 25m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:12:51.346 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (14m) 8m ago 25m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:12:51.346 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (8m) 8m ago 19m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:12:51.346 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (8m) 8m ago 20m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (7m) 7m ago 19m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (8m) 7m ago 20m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (16m) 8m ago 27m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (15m) 7m ago 24m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (12m) 8m ago 27m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:12:51.347 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (12m) 7m ago 24m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:12:51.348 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (15m) 8m ago 26m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:12:51.348 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (15m) 7m ago 25m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:12:51.348 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (11m) 8m ago 24m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:12:51.348 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (11m) 8m ago 23m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:12:51.349 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (10m) 8m ago 23m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:12:51.349 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (10m) 7m ago 22m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:12:51.349 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (9m) 7m ago 22m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:12:51.349 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (9m) 7m ago 21m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:12:51.349 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (15m) 8m ago 25m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:12:51.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 msgr2=0x7f28a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 0x7f28a006f1a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009c60 tx=0x7f28b0009260 comp rx=0 tx=0).stop 2024-04-12T06:12:51.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 msgr2=0x7f28b419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f28a4000c00 tx=0x7f28a400bde0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 shutdown_connections 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28a006cce0 0x7f28a006f1a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28b40ffc20 0x7f28b419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 --2- 172.21.15.8:0/3738375367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28b4105bd0 0x7f28b419cac0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 >> 172.21.15.8:0/3738375367 conn(0x7f28b4075670 msgr2=0x7f28b4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 shutdown_connections 2024-04-12T06:12:51.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.348+0000 7f28bbc7b700 1 -- 172.21.15.8:0/3738375367 wait complete. 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.452+0000 7f8783840700 1 -- 172.21.15.8:0/2478357763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c106cd0 msgr2=0x7f877c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.452+0000 7f8783840700 1 --2- 172.21.15.8:0/2478357763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c106cd0 0x7f877c1070b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8778009910 tx=0x7f8778009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 -- 172.21.15.8:0/2478357763 shutdown_connections 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 --2- 172.21.15.8:0/2478357763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c068580 0x7f877c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 --2- 172.21.15.8:0/2478357763 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c106cd0 0x7f877c1070b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 -- 172.21.15.8:0/2478357763 >> 172.21.15.8:0/2478357763 conn(0x7f877c075670 msgr2=0x7f877c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 -- 172.21.15.8:0/2478357763 shutdown_connections 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 -- 172.21.15.8:0/2478357763 wait complete. 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 Processor -- start 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.453+0000 7f8783840700 1 -- start start 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8783840700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8783840700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 0x7f877c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8783840700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c100770 con 0x7f877c068580 2024-04-12T06:12:51.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8783840700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c1008b0 con 0x7f877c106cd0 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49616/0 (socket says 172.21.15.8:49616) 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 -- 172.21.15.8:0/245268215 learned_addr learned my addr 172.21.15.8:0/245268215 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 -- 172.21.15.8:0/245268215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 msgr2=0x7f877c100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8780ddb700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 0x7f877c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 0x7f877c100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 -- 172.21.15.8:0/245268215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87780095f0 con 0x7f877c068580 2024-04-12T06:12:51.458 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f8780ddb700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 0x7f877c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87815dc700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8778006010 tx=0x7f877800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.454+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f877801d070 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.455+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c100b30 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.455+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f877800f460 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.455+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8778021640 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.455+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c1a2c90 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.456+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c1091f0 con 0x7f877c068580 2024-04-12T06:12:51.459 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.456+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f877800fa60 con 0x7f877c068580 2024-04-12T06:12:51.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.457+0000 7f87727fc700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 0x7f876806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.457+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f877808d850 con 0x7f877c068580 2024-04-12T06:12:51.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.457+0000 7f8780ddb700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 0x7f876806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.460 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.457+0000 7f8780ddb700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 0x7f876806f1a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f876c005fd0 tx=0x7f876c005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.461 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.461+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f87780578c0 con 0x7f877c068580 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.747+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f877c108bd0 con 0x7f877c068580 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.747+0000 7f87727fc700 1 -- 172.21.15.8:0/245268215 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f877805aee0 con 0x7f877c068580 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:12:51.746 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:12:51.747 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 msgr2=0x7f876806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.748 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 0x7f876806f1a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f876c005fd0 tx=0x7f876c005dc0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 msgr2=0x7f877c0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8778006010 tx=0x7f877800be20 comp rx=0 tx=0).stop 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 shutdown_connections 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f876806cce0 0x7f876806f1a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f877c068580 0x7f877c0ffcf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 --2- 172.21.15.8:0/245268215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f877c106cd0 0x7f877c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 >> 172.21.15.8:0/245268215 conn(0x7f877c075670 msgr2=0x7f877c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 shutdown_connections 2024-04-12T06:12:51.749 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.749+0000 7f8783840700 1 -- 172.21.15.8:0/245268215 wait complete. 2024-04-12T06:12:51.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/1539434741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 msgr2=0x7fb7b4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/1539434741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4100080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009910 tx=0x7fb7a8009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/1539434741 shutdown_connections 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/1539434741 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4100080 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/1539434741 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/1539434741 >> 172.21.15.8:0/1539434741 conn(0x7fb7b4075670 msgr2=0x7fb7b4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/1539434741 shutdown_connections 2024-04-12T06:12:51.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.856+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/1539434741 wait complete. 2024-04-12T06:12:51.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 Processor -- start 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 -- start start 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41022c0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b8fe1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4102400 con 0x7fb7b4105bd0 2024-04-12T06:12:51.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.857+0000 7fb7b259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49628/0 (socket says 172.21.15.8:49628) 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 -- 172.21.15.8:0/316495440 learned_addr learned my addr 172.21.15.8:0/316495440 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b1d9b700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 -- 172.21.15.8:0/316495440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 msgr2=0x7fb7b4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:51.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 -- 172.21.15.8:0/316495440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80095f0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b1d9b700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b259c700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c8a0 tx=0x7fb7a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a400cea0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b41a2980 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7a4004830 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a4005500 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.858+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41a2d90 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.859+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b404f2e0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.860+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb7a40049a0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.860+0000 7fb7a37fe700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 0x7fb79c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:51.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.860+0000 7fb7b1d9b700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 0x7fb79c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:51.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.860+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb7a408c270 con 0x7fb7b40ffc20 2024-04-12T06:12:51.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.861+0000 7fb7b1d9b700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 0x7fb79c06f1a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80059d0 tx=0x7fb7a801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:51.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:51.865+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb7a40562e0 con 0x7fb7b40ffc20 2024-04-12T06:12:51.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:51 smithi008 ceph-mon[146084]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:51.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:51 smithi008 ceph-mon[146084]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:52.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:51 smithi194 ceph-mon[106982]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:52.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:51 smithi194 ceph-mon[106982]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.110+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb7b41a30a0 con 0x7fb7b40ffc20 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.110+0000 7fb7a37fe700 1 -- 172.21.15.8:0/316495440 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fb7a4059900 con 0x7fb7b40ffc20 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:12:52.109 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:12:52.110 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:12:52.111 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:12:52.112 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:52.113 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:52.113 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:52.113 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 msgr2=0x7fb79c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 0x7fb79c06f1a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb7a80059d0 tx=0x7fb7a801a040 comp rx=0 tx=0).stop 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 msgr2=0x7fb7b4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400c8a0 tx=0x7fb7a400cc60 comp rx=0 tx=0).stop 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 shutdown_connections 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb79c06cce0 0x7fb79c06f1a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb7b40ffc20 0x7fb7b4101840 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 --2- 172.21.15.8:0/316495440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7b4105bd0 0x7fb7b4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 >> 172.21.15.8:0/316495440 conn(0x7fb7b4075670 msgr2=0x7fb7b4068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 shutdown_connections 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.112+0000 7fb7b8fe1700 1 -- 172.21.15.8:0/316495440 wait complete. 2024-04-12T06:12:52.114 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 -- 172.21.15.8:0/384049723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 msgr2=0x7fa2c0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 --2- 172.21.15.8:0/384049723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c0101270 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009910 tx=0x7fa2b0009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 -- 172.21.15.8:0/384049723 shutdown_connections 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 --2- 172.21.15.8:0/384049723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 0x7fa2c0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 --2- 172.21.15.8:0/384049723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c0101270 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 -- 172.21.15.8:0/384049723 >> 172.21.15.8:0/384049723 conn(0x7fa2c00fc740 msgr2=0x7fa2c00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 -- 172.21.15.8:0/384049723 shutdown_connections 2024-04-12T06:12:52.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.219+0000 7fa2c6768700 1 -- 172.21.15.8:0/384049723 wait complete. 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 Processor -- start 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 -- start start 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 0x7fa2c019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c019d1b0 con 0x7fa2c0100e90 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2c6768700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c0196660 con 0x7fa2c0101840 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2bffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49632/0 (socket says 172.21.15.8:49632) 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2bffff700 1 -- 172.21.15.8:0/279763522 learned_addr learned my addr 172.21.15.8:0/279763522 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.220+0000 7fa2bf7fe700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 0x7fa2c019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bffff700 1 -- 172.21.15.8:0/279763522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 msgr2=0x7fa2c019cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bffff700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 0x7fa2c019cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bffff700 1 -- 172.21.15.8:0/279763522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b00095f0 con 0x7fa2c0100e90 2024-04-12T06:12:52.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bffff700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006010 tx=0x7fa2b000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b001d070 con 0x7fa2c0100e90 2024-04-12T06:12:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa2b0005710 con 0x7fa2c0100e90 2024-04-12T06:12:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c01968e0 con 0x7fa2c0100e90 2024-04-12T06:12:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2b000fe20 con 0x7fa2c0100e90 2024-04-12T06:12:52.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.221+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c0196dd0 con 0x7fa2c0100e90 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.222+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0109230 con 0x7fa2c0100e90 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.222+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa2b0003a60 con 0x7fa2c0100e90 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.223+0000 7fa2bd7fa700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.223+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa2b008d6e0 con 0x7fa2c0100e90 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.223+0000 7fa2bf7fe700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.223+0000 7fa2bf7fe700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 0x7fa2a806f1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00fd000 tx=0x7fa2b400b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:52.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.227+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa2b005b1e0 con 0x7fa2c0100e90 2024-04-12T06:12:52.448 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.449+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2c0196ff0 con 0x7fa2a806cce0 2024-04-12T06:12:52.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.450+0000 7fa2bd7fa700 1 -- 172.21.15.8:0/279763522 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa2c0196ff0 con 0x7fa2a806cce0 2024-04-12T06:12:52.449 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:12:52.449 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:12:52.449 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:12:52.449 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:12:52.450 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:12:52.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 msgr2=0x7fa2a806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 0x7fa2a806f1a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00fd000 tx=0x7fa2b400b3c0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 msgr2=0x7fa2c019c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0006010 tx=0x7fa2b000bd70 comp rx=0 tx=0).stop 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 shutdown_connections 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa2a806cce0 0x7fa2a806f1a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2c0100e90 0x7fa2c019c5e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 --2- 172.21.15.8:0/279763522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa2c0101840 0x7fa2c019cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.452+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 >> 172.21.15.8:0/279763522 conn(0x7fa2c00fc740 msgr2=0x7fa2c00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.453+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 shutdown_connections 2024-04-12T06:12:52.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.453+0000 7fa2c6768700 1 -- 172.21.15.8:0/279763522 wait complete. 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 -- 172.21.15.8:0/318476534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 msgr2=0x7f4158100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 --2- 172.21.15.8:0/318476534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158100080 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4144009910 tx=0x7f4144009c20 comp rx=0 tx=0).stop 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 -- 172.21.15.8:0/318476534 shutdown_connections 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 --2- 172.21.15.8:0/318476534 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158100080 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 --2- 172.21.15.8:0/318476534 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 -- 172.21.15.8:0/318476534 >> 172.21.15.8:0/318476534 conn(0x7f4158075670 msgr2=0x7f4158075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 -- 172.21.15.8:0/318476534 shutdown_connections 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.559+0000 7f415edf2700 1 -- 172.21.15.8:0/318476534 wait complete. 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 Processor -- start 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 -- start start 2024-04-12T06:12:52.559 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41581022c0 con 0x7f41580ffc20 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415edf2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158102400 con 0x7f4158105bd0 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49648/0 (socket says 172.21.15.8:49648) 2024-04-12T06:12:52.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 -- 172.21.15.8:0/1135104530 learned_addr learned my addr 172.21.15.8:0/1135104530 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:12:52.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f4157fff700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 -- 172.21.15.8:0/1135104530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 msgr2=0x7f4158101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f415cb8e700 1 -- 172.21.15.8:0/1135104530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440095f0 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.560+0000 7f4157fff700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f415cb8e700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f414c00d8d0 tx=0x7f414c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c009940 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f414c010460 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f414c00f5f0 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41581a2980 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.561+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41581a2d90 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.562+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415804f2e0 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.562+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f414c0105d0 con 0x7f41580ffc20 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.563+0000 7f4155ffb700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 0x7f414806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:12:52.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.563+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f414c08c4c0 con 0x7f41580ffc20 2024-04-12T06:12:52.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.563+0000 7f4157fff700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 0x7f414806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:12:52.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.563+0000 7f4157fff700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 0x7f414806f1a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f414400b5c0 tx=0x7f4144005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:12:52.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.567+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f414c0565b0 con 0x7f41580ffc20 2024-04-12T06:12:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.856+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f41581a2fd0 con 0x7f41580ffc20 2024-04-12T06:12:52.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.857+0000 7f4155ffb700 1 -- 172.21.15.8:0/1135104530 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f414c010880 con 0x7f41580ffc20 2024-04-12T06:12:52.856 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:12:52.856 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:12:52.856 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:12:52.856 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:12:52.856 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:12:52.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 msgr2=0x7f414806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 0x7f414806f1a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f414400b5c0 tx=0x7f4144005960 comp rx=0 tx=0).stop 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 msgr2=0x7f4158101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f414c00d8d0 tx=0x7f414c00dc90 comp rx=0 tx=0).stop 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 shutdown_connections 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f414806cce0 0x7f414806f1a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f41580ffc20 0x7f4158101840 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 --2- 172.21.15.8:0/1135104530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4158105bd0 0x7f4158101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 >> 172.21.15.8:0/1135104530 conn(0x7f4158075670 msgr2=0x7f4158068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 shutdown_connections 2024-04-12T06:12:52.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:12:52.859+0000 7f415edf2700 1 -- 172.21.15.8:0/1135104530 wait complete. 2024-04-12T06:12:52.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:52 smithi008 ceph-mon[146084]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:52.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:52 smithi008 ceph-mon[146084]: pgmap v535: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:52.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:52 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/245268215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:12:52.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:52 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/316495440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:12:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:52 smithi194 ceph-mon[106982]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:52 smithi194 ceph-mon[106982]: pgmap v535: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:52 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/245268215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:12:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:52 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/316495440' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:12:53.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:53 smithi008 ceph-mon[146084]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:53.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:53 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1135104530' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:12:54.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:53 smithi194 ceph-mon[106982]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:12:54.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:53 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1135104530' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:12:54.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:54 smithi008 ceph-mon[146084]: pgmap v536: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:55.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:54 smithi194 ceph-mon[106982]: pgmap v536: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:56.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:56 smithi008 ceph-mon[146084]: pgmap v537: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:57.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:56 smithi194 ceph-mon[106982]: pgmap v537: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:58.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:12:58 smithi008 ceph-mon[146084]: pgmap v538: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:12:59.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:12:58 smithi194 ceph-mon[106982]: pgmap v538: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:00.906 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:00 smithi008 ceph-mon[146084]: pgmap v539: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:00 smithi194 ceph-mon[106982]: pgmap v539: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:02.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:02 smithi008 ceph-mon[146084]: pgmap v540: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:02.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:02 smithi194 ceph-mon[106982]: pgmap v540: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:04.573 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:04 smithi194 ceph-mon[106982]: pgmap v541: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:04.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:04 smithi008 ceph-mon[146084]: pgmap v541: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:06.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:06 smithi008 ceph-mon[146084]: pgmap v542: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:07.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:06 smithi194 ceph-mon[106982]: pgmap v542: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:08.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:08 smithi194 ceph-mon[106982]: pgmap v543: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:08.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:08 smithi008 ceph-mon[146084]: pgmap v543: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:10.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:10 smithi008 ceph-mon[146084]: pgmap v544: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:10 smithi194 ceph-mon[106982]: pgmap v544: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:12.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:12 smithi008 ceph-mon[146084]: pgmap v545: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:13.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:12 smithi194 ceph-mon[106982]: pgmap v545: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:14.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:14 smithi008 ceph-mon[146084]: pgmap v546: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:15.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:14 smithi194 ceph-mon[106982]: pgmap v546: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:16.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:16 smithi008 ceph-mon[146084]: pgmap v547: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:17.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:16 smithi194 ceph-mon[106982]: pgmap v547: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:17.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:18.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:18.823 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:18 smithi194 ceph-mon[106982]: pgmap v548: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:18.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:18 smithi008 ceph-mon[146084]: pgmap v548: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:20.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:20 smithi008 ceph-mon[146084]: pgmap v549: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:21.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:20 smithi194 ceph-mon[106982]: pgmap v549: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:22.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:22 smithi008 ceph-mon[146084]: pgmap v550: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:22.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1724389904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38106cd0 msgr2=0x7fca381070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1724389904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38106cd0 0x7fca381070b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fca34009910 tx=0x7fca34009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1724389904 shutdown_connections 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1724389904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38068580 0x7fca380689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1724389904 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38106cd0 0x7fca381070b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.987+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1724389904 >> 172.21.15.8:0/1724389904 conn(0x7fca38075670 msgr2=0x7fca38075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1724389904 shutdown_connections 2024-04-12T06:13:22.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1724389904 wait complete. 2024-04-12T06:13:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 Processor -- start 2024-04-12T06:13:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 -- start start 2024-04-12T06:13:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38106cd0 0x7fca3819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:22.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3819d130 con 0x7fca38068580 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.988+0000 7fca3f4e6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38196630 con 0x7fca38106cd0 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40244/0 (socket says 172.21.15.8:40244) 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 -- 172.21.15.8:0/1546883574 learned_addr learned my addr 172.21.15.8:0/1546883574 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3ca81700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38106cd0 0x7fca3819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 -- 172.21.15.8:0/1546883574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38106cd0 msgr2=0x7fca3819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38106cd0 0x7fca3819caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 -- 172.21.15.8:0/1546883574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340095f0 con 0x7fca38068580 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3d282700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca34006010 tx=0x7fca3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca3401d070 con 0x7fca38068580 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca3400f460 con 0x7fca38068580 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca381968b0 con 0x7fca38068580 2024-04-12T06:13:22.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca34021640 con 0x7fca38068580 2024-04-12T06:13:22.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.989+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca38196da0 con 0x7fca38068580 2024-04-12T06:13:22.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.990+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca381091f0 con 0x7fca38068580 2024-04-12T06:13:22.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.991+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fca3400fa60 con 0x7fca38068580 2024-04-12T06:13:22.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.991+0000 7fca2e7fc700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 0x7fca2406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:22.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.991+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca3408d780 con 0x7fca38068580 2024-04-12T06:13:22.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.992+0000 7fca3ca81700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 0x7fca2406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:22.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.992+0000 7fca3ca81700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 0x7fca2406f1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca2800a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:22.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:22.996+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fca3405b300 con 0x7fca38068580 2024-04-12T06:13:23.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:22 smithi194 ceph-mon[106982]: pgmap v550: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:23.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.220+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca38197010 con 0x7fca2406cce0 2024-04-12T06:13:23.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.222+0000 7fca2e7fc700 1 -- 172.21.15.8:0/1546883574 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fca38197010 con 0x7fca2406cce0 2024-04-12T06:13:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 msgr2=0x7fca2406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 0x7fca2406f1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca2800a560 comp rx=0 tx=0).stop 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 msgr2=0x7fca3819c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fca34006010 tx=0x7fca3400be20 comp rx=0 tx=0).stop 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 shutdown_connections 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca2406cce0 0x7fca2406f1a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca38068580 0x7fca3819c560 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 --2- 172.21.15.8:0/1546883574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca38106cd0 0x7fca3819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 >> 172.21.15.8:0/1546883574 conn(0x7fca38075670 msgr2=0x7fca380fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 shutdown_connections 2024-04-12T06:13:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.224+0000 7fca3f4e6700 1 -- 172.21.15.8:0/1546883574 wait complete. 2024-04-12T06:13:23.233 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:13:23.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.331+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2745238284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 msgr2=0x7fc79c105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.331+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2745238284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c105890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc790009910 tx=0x7fc790009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:23.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2745238284 shutdown_connections 2024-04-12T06:13:23.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2745238284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c105890 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2745238284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 0x7fc79c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2745238284 >> 172.21.15.8:0/2745238284 conn(0x7fc79c0fc740 msgr2=0x7fc79c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2745238284 shutdown_connections 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2745238284 wait complete. 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 Processor -- start 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.332+0000 7fc7a29cb700 1 -- start start 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7a29cb700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 0x7fc79c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7a29cb700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7a29cb700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c19f2e0 con 0x7fc79c101840 2024-04-12T06:13:23.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7a29cb700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c198820 con 0x7fc79c100e90 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40252/0 (socket says 172.21.15.8:40252) 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2231487965 learned_addr learned my addr 172.21.15.8:0/2231487965 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79bfff700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 0x7fc79c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2231487965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 msgr2=0x7fc79c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 0x7fc79c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2231487965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900095f0 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc79b7fe700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc79000bda0 tx=0x7fc79000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79001d070 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c198aa0 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc79000f460 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.333+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc790021640 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.334+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c198f90 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.335+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79c105010 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.335+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc79000f9d0 con 0x7fc79c101840 2024-04-12T06:13:23.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.335+0000 7fc7997fa700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 0x7fc78406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.336+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc79008d820 con 0x7fc79c101840 2024-04-12T06:13:23.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.336+0000 7fc79bfff700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 0x7fc78406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.336+0000 7fc79bfff700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 0x7fc78406f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005ea0 tx=0x7fc78c005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:23.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.340+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc79005b320 con 0x7fc79c101840 2024-04-12T06:13:23.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.565+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc79c199e40 con 0x7fc78406cce0 2024-04-12T06:13:23.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.567+0000 7fc7997fa700 1 -- 172.21.15.8:0/2231487965 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc79c199e40 con 0x7fc78406cce0 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 msgr2=0x7fc78406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 0x7fc78406f1a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005ea0 tx=0x7fc78c005e30 comp rx=0 tx=0).stop 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 msgr2=0x7fc79c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc79000bda0 tx=0x7fc79000be80 comp rx=0 tx=0).stop 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 shutdown_connections 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc78406cce0 0x7fc78406f1a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc79c100e90 0x7fc79c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 --2- 172.21.15.8:0/2231487965 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc79c101840 0x7fc79c19ece0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 >> 172.21.15.8:0/2231487965 conn(0x7fc79c0fc740 msgr2=0x7fc79c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 shutdown_connections 2024-04-12T06:13:23.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.569+0000 7fc7a29cb700 1 -- 172.21.15.8:0/2231487965 wait complete. 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- 172.21.15.8:0/848455914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 msgr2=0x7f28480fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 --2- 172.21.15.8:0/848455914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f28480fffc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2834009910 tx=0x7f2834009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- 172.21.15.8:0/848455914 shutdown_connections 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 --2- 172.21.15.8:0/848455914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f28480fffc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 --2- 172.21.15.8:0/848455914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 0x7f2848105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- 172.21.15.8:0/848455914 >> 172.21.15.8:0/848455914 conn(0x7f28480fb610 msgr2=0x7f28480fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- 172.21.15.8:0/848455914 shutdown_connections 2024-04-12T06:13:23.675 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- 172.21.15.8:0/848455914 wait complete. 2024-04-12T06:13:23.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 Processor -- start 2024-04-12T06:13:23.676 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.676+0000 7f284ef87700 1 -- start start 2024-04-12T06:13:23.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284ef87700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284ef87700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 0x7f28481958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284ef87700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848195e00 con 0x7f28480ffb40 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284ef87700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2848195f70 con 0x7f2848105bf0 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33478/0 (socket says 172.21.15.8:33478) 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 -- 172.21.15.8:0/1000305412 learned_addr learned my addr 172.21.15.8:0/1000305412 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f2847fff700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 0x7f28481958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 -- 172.21.15.8:0/1000305412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 msgr2=0x7f28481958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 0x7f28481958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 -- 172.21.15.8:0/1000305412 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340095f0 con 0x7f28480ffb40 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284cd23700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d8d0 tx=0x7f283c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283c0098a0 con 0x7f28480ffb40 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f283c010460 con 0x7f28480ffb40 2024-04-12T06:13:23.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28481a29f0 con 0x7f28480ffb40 2024-04-12T06:13:23.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.677+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f283c009ea0 con 0x7f28480ffb40 2024-04-12T06:13:23.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.678+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28481a2d50 con 0x7f28480ffb40 2024-04-12T06:13:23.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.679+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28481080a0 con 0x7f28480ffb40 2024-04-12T06:13:23.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.679+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f283c0105d0 con 0x7f28480ffb40 2024-04-12T06:13:23.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.679+0000 7f2845ffb700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 0x7f283806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:23.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.680+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f283c08c300 con 0x7f28480ffb40 2024-04-12T06:13:23.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.680+0000 7f2847fff700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 0x7f283806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:23.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.680+0000 7f2847fff700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 0x7f283806f1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283400b5c0 tx=0x7f283401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:23.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.684+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f283c059e80 con 0x7f28480ffb40 2024-04-12T06:13:23.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.895+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f284802cce0 con 0x7f283806cce0 2024-04-12T06:13:23.905 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.906+0000 7f2845ffb700 1 -- 172.21.15.8:0/1000305412 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f284802cce0 con 0x7f283806cce0 2024-04-12T06:13:23.905 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:13:23.905 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (15m) 8m ago 26m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:13:23.905 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (26m) 8m ago 26m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:13:23.905 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (25m) 8m ago 25m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (12m) 8m ago 26m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (12m) 8m ago 25m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (14m) 8m ago 26m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (8m) 8m ago 20m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (9m) 8m ago 20m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (8m) 8m ago 20m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (8m) 8m ago 20m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (17m) 8m ago 28m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (16m) 8m ago 25m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (13m) 8m ago 28m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:13:23.906 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (13m) 8m ago 25m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (16m) 8m ago 26m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (15m) 8m ago 25m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (12m) 8m ago 24m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (11m) 8m ago 24m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (11m) 8m ago 23m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (10m) 8m ago 23m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (10m) 8m ago 22m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (9m) 8m ago 22m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:13:23.907 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (15m) 8m ago 26m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:13:23.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 msgr2=0x7f283806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 0x7f283806f1a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f283400b5c0 tx=0x7f283401a040 comp rx=0 tx=0).stop 2024-04-12T06:13:23.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 msgr2=0x7f2848195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:23.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f283c00d8d0 tx=0x7f283c00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.908 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 shutdown_connections 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f283806cce0 0x7f283806f1a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f28480ffb40 0x7f2848195380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 --2- 172.21.15.8:0/1000305412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2848105bf0 0x7f28481958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 >> 172.21.15.8:0/1000305412 conn(0x7f28480fb610 msgr2=0x7f28480fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 shutdown_connections 2024-04-12T06:13:23.909 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:23.908+0000 7f284ef87700 1 -- 172.21.15.8:0/1000305412 wait complete. 2024-04-12T06:13:24.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 -- 172.21.15.8:0/3730051996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b980ffc20 msgr2=0x7f8b98100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/3730051996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b980ffc20 0x7f8b98100080 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009910 tx=0x7f8b8c009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 -- 172.21.15.8:0/3730051996 shutdown_connections 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/3730051996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b980ffc20 0x7f8b98100080 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/3730051996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b98105bd0 0x7f8b98105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 -- 172.21.15.8:0/3730051996 >> 172.21.15.8:0/3730051996 conn(0x7f8b98075670 msgr2=0x7f8b98075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 -- 172.21.15.8:0/3730051996 shutdown_connections 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.013+0000 7f8b9f74d700 1 -- 172.21.15.8:0/3730051996 wait complete. 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 Processor -- start 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 -- start start 2024-04-12T06:13:24.014 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 0x7f8b9819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9819d150 con 0x7f8b98105bd0 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9f74d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b98196600 con 0x7f8b980ffc20 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33492/0 (socket says 172.21.15.8:33492) 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 -- 172.21.15.8:0/4144961802 learned_addr learned my addr 172.21.15.8:0/4144961802 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9d4e9700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 0x7f8b9819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 -- 172.21.15.8:0/4144961802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 msgr2=0x7f8b9819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 0x7f8b9819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 -- 172.21.15.8:0/4144961802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0095f0 con 0x7f8b98105bd0 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9d4e9700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 0x7f8b9819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.014+0000 7f8b9cce8700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0092e0 tx=0x7f8b8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.015+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8c01d070 con 0x7f8b98105bd0 2024-04-12T06:13:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.015+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8b8c005e00 con 0x7f8b98105bd0 2024-04-12T06:13:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.015+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b98196880 con 0x7f8b98105bd0 2024-04-12T06:13:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.015+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8c00fe60 con 0x7f8b98105bd0 2024-04-12T06:13:24.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.015+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b98196d70 con 0x7f8b98105bd0 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.016+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9804f2e0 con 0x7f8b98105bd0 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.016+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8b8c00f6d0 con 0x7f8b98105bd0 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.017+0000 7f8b8a7fc700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 0x7f8b8406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.017+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8b8c08d850 con 0x7f8b98105bd0 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.017+0000 7f8b9d4e9700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 0x7f8b8406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.017+0000 7f8b9d4e9700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 0x7f8b8406f1a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005950 tx=0x7f8b940058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.020 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.021+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8b8c05b350 con 0x7f8b98105bd0 2024-04-12T06:13:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.311+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b98196ff0 con 0x7f8b98105bd0 2024-04-12T06:13:24.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.311+0000 7f8b8a7fc700 1 -- 172.21.15.8:0/4144961802 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f8b8c05aee0 con 0x7f8b98105bd0 2024-04-12T06:13:24.310 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:13:24.311 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:13:24.312 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:24.312 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:13:24.312 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:13:24.312 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:13:24.312 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 msgr2=0x7f8b8406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 0x7f8b8406f1a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005950 tx=0x7f8b940058e0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 msgr2=0x7f8b9819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0092e0 tx=0x7f8b8c00be20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 shutdown_connections 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8b8406cce0 0x7f8b8406f1a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b980ffc20 0x7f8b9819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 --2- 172.21.15.8:0/4144961802 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8b98105bd0 0x7f8b9819cac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 >> 172.21.15.8:0/4144961802 conn(0x7f8b98075670 msgr2=0x7f8b98068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:24.313 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 shutdown_connections 2024-04-12T06:13:24.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.313+0000 7f8b9f74d700 1 -- 172.21.15.8:0/4144961802 wait complete. 2024-04-12T06:13:24.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 -- 172.21.15.8:0/885875929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 msgr2=0x7f7370101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 --2- 172.21.15.8:0/885875929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f7370101280 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7364009910 tx=0x7f7364009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 -- 172.21.15.8:0/885875929 shutdown_connections 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 --2- 172.21.15.8:0/885875929 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f73701058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 --2- 172.21.15.8:0/885875929 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f7370101280 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 -- 172.21.15.8:0/885875929 >> 172.21.15.8:0/885875929 conn(0x7f73700fc750 msgr2=0x7f73700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 -- 172.21.15.8:0/885875929 shutdown_connections 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.419+0000 7f73749b5700 1 -- 172.21.15.8:0/885875929 wait complete. 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 Processor -- start 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 -- start start 2024-04-12T06:13:24.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f737019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737019d0a0 con 0x7f7370100ea0 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f73749b5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73701965e0 con 0x7f7370101850 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33504/0 (socket says 172.21.15.8:33504) 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 -- 172.21.15.8:0/1052381703 learned_addr learned my addr 172.21.15.8:0/1052381703 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736dd9b700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f737019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 -- 172.21.15.8:0/1052381703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 msgr2=0x7f737019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f737019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736e59c700 1 -- 172.21.15.8:0/1052381703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73640095f0 con 0x7f7370100ea0 2024-04-12T06:13:24.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.420+0000 7f736dd9b700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f737019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f736e59c700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f736400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f736401d070 con 0x7f7370100ea0 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f736400f460 con 0x7f7370100ea0 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7370196860 con 0x7f7370100ea0 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7364021640 con 0x7f7370100ea0 2024-04-12T06:13:24.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.421+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7370196d50 con 0x7f7370100ea0 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.422+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7370109240 con 0x7f7370100ea0 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.423+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f736400fa20 con 0x7f7370100ea0 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.423+0000 7f735f7fe700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 0x7f735806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.423+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f736408d7a0 con 0x7f7370100ea0 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.423+0000 7f736dd9b700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 0x7f735806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.423+0000 7f736dd9b700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 0x7f735806f1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7360005fd0 tx=0x7f7360005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.428+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f736405b320 con 0x7f7370100ea0 2024-04-12T06:13:24.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.681+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7370108c20 con 0x7f7370100ea0 2024-04-12T06:13:24.680 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.681+0000 7f735f7fe700 1 -- 172.21.15.8:0/1052381703 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f736405aeb0 con 0x7f7370100ea0 2024-04-12T06:13:24.680 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:13:24.681 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:13:24.682 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:24.683 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 msgr2=0x7f735806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 0x7f735806f1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7360005fd0 tx=0x7f7360005e20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 msgr2=0x7f737019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7364006010 tx=0x7f736400be20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 shutdown_connections 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f735806cce0 0x7f735806f1a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7370100ea0 0x7f737019c560 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 --2- 172.21.15.8:0/1052381703 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7370101850 0x7f737019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.683+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 >> 172.21.15.8:0/1052381703 conn(0x7f73700fc750 msgr2=0x7f73700fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.684+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 shutdown_connections 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.684+0000 7f73749b5700 1 -- 172.21.15.8:0/1052381703 wait complete. 2024-04-12T06:13:24.685 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:13:24.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 -- 172.21.15.8:0/3431204985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 msgr2=0x7f7b34100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 --2- 172.21.15.8:0/3431204985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b34100080 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009910 tx=0x7f7b28009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 -- 172.21.15.8:0/3431204985 shutdown_connections 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 --2- 172.21.15.8:0/3431204985 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b34100080 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 --2- 172.21.15.8:0/3431204985 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b34105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 -- 172.21.15.8:0/3431204985 >> 172.21.15.8:0/3431204985 conn(0x7f7b34075670 msgr2=0x7f7b34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 -- 172.21.15.8:0/3431204985 shutdown_connections 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.790+0000 7f7b393e3700 1 -- 172.21.15.8:0/3431204985 wait complete. 2024-04-12T06:13:24.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 Processor -- start 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 -- start start 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b3419cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3419d1c0 con 0x7f7b340ffc20 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.791+0000 7f7b393e3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34196670 con 0x7f7b34105bd0 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33524/0 (socket says 172.21.15.8:33524) 2024-04-12T06:13:24.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 -- 172.21.15.8:0/2546288255 learned_addr learned my addr 172.21.15.8:0/2546288255 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b327fc700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b3419cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 -- 172.21.15.8:0/2546288255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 msgr2=0x7f7b3419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b3419cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 -- 172.21.15.8:0/2546288255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280095f0 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b327fc700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b3419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b32ffd700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d900 tx=0x7f7b2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b240041f0 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b34196950 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b24004cd0 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b240056d0 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.792+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b34196ea0 con 0x7f7b340ffc20 2024-04-12T06:13:24.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.793+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3404f2e0 con 0x7f7b340ffc20 2024-04-12T06:13:24.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.794+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7b24004e40 con 0x7f7b340ffc20 2024-04-12T06:13:24.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.794+0000 7f7b1bfff700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 0x7f7b1c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:24.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.794+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7b2408c6c0 con 0x7f7b340ffc20 2024-04-12T06:13:24.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.794+0000 7f7b327fc700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 0x7f7b1c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:24.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.795+0000 7f7b327fc700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 0x7f7b1c06f150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b5c0 tx=0x7f7b28005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:24.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:24.799+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7b2405a1c0 con 0x7f7b340ffc20 2024-04-12T06:13:24.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:24 smithi008 ceph-mon[146084]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:24.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:24 smithi008 ceph-mon[146084]: pgmap v551: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:24.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:24 smithi008 ceph-mon[146084]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:24.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:24 smithi008 ceph-mon[146084]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:24.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:24 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4144961802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:13:25.017 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.018+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b34197130 con 0x7f7b1c06cc90 2024-04-12T06:13:25.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.019+0000 7f7b1bfff700 1 -- 172.21.15.8:0/2546288255 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7b34197130 con 0x7f7b1c06cc90 2024-04-12T06:13:25.018 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:13:25.018 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:13:25.018 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:13:25.019 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 msgr2=0x7f7b1c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 0x7f7b1c06f150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b5c0 tx=0x7f7b28005960 comp rx=0 tx=0).stop 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 msgr2=0x7f7b3419c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7b2400d900 tx=0x7f7b2400dcc0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 shutdown_connections 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cc90 0x7f7b1c06f150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b340ffc20 0x7f7b3419c5f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 --2- 172.21.15.8:0/2546288255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34105bd0 0x7f7b3419cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 >> 172.21.15.8:0/2546288255 conn(0x7f7b34075670 msgr2=0x7f7b34103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 shutdown_connections 2024-04-12T06:13:25.021 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.022+0000 7f7b393e3700 1 -- 172.21.15.8:0/2546288255 wait complete. 2024-04-12T06:13:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:24 smithi194 ceph-mon[106982]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:24 smithi194 ceph-mon[106982]: pgmap v551: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:24 smithi194 ceph-mon[106982]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:24 smithi194 ceph-mon[106982]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:24 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4144961802' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:13:25.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 -- 172.21.15.8:0/1201890203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790106cd0 msgr2=0x7f47901070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.127 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 --2- 172.21.15.8:0/1201890203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790106cd0 0x7f47901070b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4780009910 tx=0x7f4780009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 -- 172.21.15.8:0/1201890203 shutdown_connections 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 --2- 172.21.15.8:0/1201890203 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790068580 0x7f47900689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 --2- 172.21.15.8:0/1201890203 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790106cd0 0x7f47901070b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 -- 172.21.15.8:0/1201890203 >> 172.21.15.8:0/1201890203 conn(0x7f4790075670 msgr2=0x7f4790075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 -- 172.21.15.8:0/1201890203 shutdown_connections 2024-04-12T06:13:25.128 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.128+0000 7f4795b38700 1 -- 172.21.15.8:0/1201890203 wait complete. 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 Processor -- start 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 -- start start 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790106cd0 0x7f479019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479019d0a0 con 0x7f4790068580 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.129+0000 7f4795b38700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790196630 con 0x7f4790106cd0 2024-04-12T06:13:25.130 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:33540/0 (socket says 172.21.15.8:33540) 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 -- 172.21.15.8:0/1603717301 learned_addr learned my addr 172.21.15.8:0/1603717301 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478effd700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790106cd0 0x7f479019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 -- 172.21.15.8:0/1603717301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790106cd0 msgr2=0x7f479019caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790106cd0 0x7f479019caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 -- 172.21.15.8:0/1603717301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800095f0 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478f7fe700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4780006010 tx=0x7f478000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478001d070 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4780005940 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47901968b0 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4780021410 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.130+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4790196da0 con 0x7f4790068580 2024-04-12T06:13:25.131 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.131+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47901091f0 con 0x7f4790068580 2024-04-12T06:13:25.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.132+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4780003a60 con 0x7f4790068580 2024-04-12T06:13:25.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.132+0000 7f478cff9700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 0x7f477806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:25.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.132+0000 7f478effd700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 0x7f477806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:25.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.132+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f478008d740 con 0x7f4790068580 2024-04-12T06:13:25.132 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.133+0000 7f478effd700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 0x7f477806f1a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47900fe390 tx=0x7f478400a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:25.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.137+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f478005b240 con 0x7f4790068580 2024-04-12T06:13:25.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.425+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f479004f2e0 con 0x7f4790068580 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.426+0000 7f478cff9700 1 -- 172.21.15.8:0/1603717301 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f478005add0 con 0x7f4790068580 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:13:25.425 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 msgr2=0x7f477806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 0x7f477806f1a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f47900fe390 tx=0x7f478400a380 comp rx=0 tx=0).stop 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 msgr2=0x7f479019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4780006010 tx=0x7f478000bd70 comp rx=0 tx=0).stop 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 shutdown_connections 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f477806cce0 0x7f477806f1a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4790068580 0x7f479019c560 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 --2- 172.21.15.8:0/1603717301 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4790106cd0 0x7f479019caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 >> 172.21.15.8:0/1603717301 conn(0x7f4790075670 msgr2=0x7f47900fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 shutdown_connections 2024-04-12T06:13:25.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:25.428+0000 7f4795b38700 1 -- 172.21.15.8:0/1603717301 wait complete. 2024-04-12T06:13:25.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:25 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1052381703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:13:25.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:25 smithi008 ceph-mon[146084]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:25.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:25 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1603717301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:13:26.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:25 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1052381703' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:13:26.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:25 smithi194 ceph-mon[106982]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:26.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:25 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1603717301' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:13:26.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:26 smithi008 ceph-mon[146084]: pgmap v552: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:27.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:26 smithi194 ceph-mon[106982]: pgmap v552: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:28.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:28 smithi008 ceph-mon[146084]: pgmap v553: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:29.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:28 smithi194 ceph-mon[106982]: pgmap v553: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:30.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:30 smithi008 ceph-mon[146084]: pgmap v554: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:31.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:30 smithi194 ceph-mon[106982]: pgmap v554: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:32.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:32 smithi008 ceph-mon[146084]: pgmap v555: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:32.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:33.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:32 smithi194 ceph-mon[106982]: pgmap v555: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:33.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:34.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:34 smithi008 ceph-mon[146084]: pgmap v556: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:35.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:34 smithi194 ceph-mon[106982]: pgmap v556: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:36.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:36 smithi008 ceph-mon[146084]: pgmap v557: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:37.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:36 smithi194 ceph-mon[106982]: pgmap v557: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:38.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:38 smithi008 ceph-mon[146084]: pgmap v558: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:39.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:38 smithi194 ceph-mon[106982]: pgmap v558: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:40.910 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:40 smithi008 ceph-mon[146084]: pgmap v559: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:41.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:40 smithi194 ceph-mon[106982]: pgmap v559: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:42.911 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:42 smithi008 ceph-mon[146084]: pgmap v560: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:43.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:42 smithi194 ceph-mon[106982]: pgmap v560: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:45.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:44 smithi194 ceph-mon[106982]: pgmap v561: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:45.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:44 smithi008 ceph-mon[146084]: pgmap v561: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:47.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:46 smithi194 ceph-mon[106982]: pgmap v562: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:47.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:46 smithi008 ceph-mon[146084]: pgmap v562: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:48.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:48.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:13:49.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:48 smithi194 ceph-mon[106982]: pgmap v563: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:49.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:48 smithi008 ceph-mon[146084]: pgmap v563: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:51.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:50 smithi194 ceph-mon[106982]: pgmap v564: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:51.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:50 smithi008 ceph-mon[146084]: pgmap v564: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:52 smithi194 ceph-mon[106982]: pgmap v565: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:53.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:52 smithi008 ceph-mon[146084]: pgmap v565: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:55.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:54 smithi194 ceph-mon[106982]: pgmap v566: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:55.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:54 smithi008 ceph-mon[146084]: pgmap v566: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:55.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3667896357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb40ffc20 msgr2=0x7fcbb4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3667896357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb40ffc20 0x7fcbb4100080 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009910 tx=0x7fcba4009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:55.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3667896357 shutdown_connections 2024-04-12T06:13:55.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3667896357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb40ffc20 0x7fcbb4100080 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3667896357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3667896357 >> 172.21.15.8:0/3667896357 conn(0x7fcbb4075670 msgr2=0x7fcbb4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3667896357 shutdown_connections 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.548+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3667896357 wait complete. 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 Processor -- start 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 -- start start 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb40ffc20 0x7fcbb41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb4199570 con 0x7fcbb4105bd0 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbbbbad700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb419d380 con 0x7fcbb40ffc20 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbb9148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbb9148700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57340/0 (socket says 172.21.15.8:57340) 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.549+0000 7fcbb9148700 1 -- 172.21.15.8:0/3310796485 learned_addr learned my addr 172.21.15.8:0/3310796485 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbb9949700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb40ffc20 0x7fcbb41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbb9148700 1 -- 172.21.15.8:0/3310796485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb40ffc20 msgr2=0x7fcbb41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbb9148700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb40ffc20 0x7fcbb41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbb9148700 1 -- 172.21.15.8:0/3310796485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40095f0 con 0x7fcbb4105bd0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbb9148700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006010 tx=0x7fcba400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba401d070 con 0x7fcbb4105bd0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcba4005710 con 0x7fcbb4105bd0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb419d600 con 0x7fcbb4105bd0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba400fe20 con 0x7fcbb4105bd0 2024-04-12T06:13:55.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.550+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb419daf0 con 0x7fcbb4105bd0 2024-04-12T06:13:55.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.551+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb404f2e0 con 0x7fcbb4105bd0 2024-04-12T06:13:55.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.552+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fcba4003a60 con 0x7fcbb4105bd0 2024-04-12T06:13:55.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.552+0000 7fcbaaffd700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 0x7fcba006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.552+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcba408d740 con 0x7fcbb4105bd0 2024-04-12T06:13:55.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.552+0000 7fcbb9949700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 0x7fcba006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.553+0000 7fcbb9949700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 0x7fcba006f1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005950 tx=0x7fcbb000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:55.556 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.557+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fcba405b1a0 con 0x7fcbb4105bd0 2024-04-12T06:13:55.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.777+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbb419dd60 con 0x7fcba006cce0 2024-04-12T06:13:55.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.778+0000 7fcbaaffd700 1 -- 172.21.15.8:0/3310796485 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcbb419dd60 con 0x7fcba006cce0 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 msgr2=0x7fcba006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 0x7fcba006f1a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0005950 tx=0x7fcbb000b410 comp rx=0 tx=0).stop 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 msgr2=0x7fcbb4198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006010 tx=0x7fcba400bd70 comp rx=0 tx=0).stop 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 shutdown_connections 2024-04-12T06:13:55.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcba006cce0 0x7fcba006f1a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcbb40ffc20 0x7fcbb41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 --2- 172.21.15.8:0/3310796485 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcbb4105bd0 0x7fcbb4198ee0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 >> 172.21.15.8:0/3310796485 conn(0x7fcbb4075670 msgr2=0x7fcbb40687c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:55.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 shutdown_connections 2024-04-12T06:13:55.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.780+0000 7fcbbbbad700 1 -- 172.21.15.8:0/3310796485 wait complete. 2024-04-12T06:13:55.789 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/2004210271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 msgr2=0x7f7b341070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/2004210271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b341070b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009910 tx=0x7f7b24009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/2004210271 shutdown_connections 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/2004210271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 0x7f7b340689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/2004210271 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b341070b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/2004210271 >> 172.21.15.8:0/2004210271 conn(0x7f7b34075670 msgr2=0x7f7b34075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:55.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/2004210271 shutdown_connections 2024-04-12T06:13:55.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.887+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/2004210271 wait complete. 2024-04-12T06:13:55.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 Processor -- start 2024-04-12T06:13:55.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 -- start start 2024-04-12T06:13:55.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 0x7f7b340ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34100770 con 0x7f7b34106cd0 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.888+0000 7f7b3a4e5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b341008b0 con 0x7f7b34068580 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57344/0 (socket says 172.21.15.8:57344) 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 -- 172.21.15.8:0/497078102 learned_addr learned my addr 172.21.15.8:0/497078102 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b33fff700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 0x7f7b340ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 -- 172.21.15.8:0/497078102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 msgr2=0x7f7b340ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 0x7f7b340ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 -- 172.21.15.8:0/497078102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240095f0 con 0x7f7b34106cd0 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b337fe700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d8d0 tx=0x7f7b2800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:55.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b280098a0 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b341a29b0 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7b28010460 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b28009ea0 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.889+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b341a2d10 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7b280105d0 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b341091d0 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b317fa700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 0x7f7b1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7b2808c440 con 0x7f7b34106cd0 2024-04-12T06:13:55.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b33fff700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 0x7f7b1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:55.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.891+0000 7f7b33fff700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 0x7f7b1c06f1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b2401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:55.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:55.896+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7b280564b0 con 0x7f7b34106cd0 2024-04-12T06:13:56.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.115+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b340611d0 con 0x7f7b1c06cce0 2024-04-12T06:13:56.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.116+0000 7f7b317fa700 1 -- 172.21.15.8:0/497078102 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7b340611d0 con 0x7f7b1c06cce0 2024-04-12T06:13:56.117 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 msgr2=0x7f7b1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 0x7f7b1c06f1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b24006010 tx=0x7f7b2401a040 comp rx=0 tx=0).stop 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 msgr2=0x7f7b34100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d8d0 tx=0x7f7b2800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 shutdown_connections 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7b1c06cce0 0x7f7b1c06f1a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7b34068580 0x7f7b340ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 --2- 172.21.15.8:0/497078102 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7b34106cd0 0x7f7b34100230 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.118+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 >> 172.21.15.8:0/497078102 conn(0x7f7b34075670 msgr2=0x7f7b340fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.119+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 shutdown_connections 2024-04-12T06:13:56.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.119+0000 7f7b3a4e5700 1 -- 172.21.15.8:0/497078102 wait complete. 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3351367719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0068580 msgr2=0x7f9ca00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3351367719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0068580 0x7f9ca00689e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009910 tx=0x7f9c90009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3351367719 shutdown_connections 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3351367719 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0068580 0x7f9ca00689e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3351367719 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0106cd0 0x7f9ca01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3351367719 >> 172.21.15.8:0/3351367719 conn(0x7f9ca0075670 msgr2=0x7f9ca0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3351367719 shutdown_connections 2024-04-12T06:13:56.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.225+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3351367719 wait complete. 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 Processor -- start 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 -- start start 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 0x7f9ca019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca019d110 con 0x7f9ca0106cd0 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca7fe4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca01965c0 con 0x7f9ca0068580 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57366/0 (socket says 172.21.15.8:57366) 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 -- 172.21.15.8:0/3431287893 learned_addr learned my addr 172.21.15.8:0/3431287893 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca5d80700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 0x7f9ca019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 -- 172.21.15.8:0/3431287893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 msgr2=0x7f9ca019c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 0x7f9ca019c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca557f700 1 -- 172.21.15.8:0/3431287893 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900095f0 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.226+0000 7f9ca5d80700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 0x7f9ca019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9ca557f700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c900092e0 tx=0x7f9c9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9001d070 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9c90005e00 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca0196840 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c9000fe60 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.227+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca0196d30 con 0x7f9ca0106cd0 2024-04-12T06:13:56.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.228+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca01091d0 con 0x7f9ca0106cd0 2024-04-12T06:13:56.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.228+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9c9000f6d0 con 0x7f9ca0106cd0 2024-04-12T06:13:56.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.229+0000 7f9c96ffd700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 0x7f9c8c06f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.229+0000 7f9ca5d80700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 0x7f9c8c06f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.229+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c9008d870 con 0x7f9ca0106cd0 2024-04-12T06:13:56.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.229+0000 7f9ca5d80700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 0x7f9c8c06f1f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c005950 tx=0x7f9c9c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.233+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9c9005b2d0 con 0x7f9ca0106cd0 2024-04-12T06:13:56.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.443+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ca0196fa0 con 0x7f9c8c06cd30 2024-04-12T06:13:56.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.454+0000 7f9c96ffd700 1 -- 172.21.15.8:0/3431287893 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9ca0196fa0 con 0x7f9c8c06cd30 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (15m) 9m ago 27m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (27m) 9m ago 27m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (26m) 8m ago 26m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (13m) 9m ago 27m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (13m) 8m ago 26m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (15m) 9m ago 26m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (9m) 9m ago 21m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (9m) 9m ago 21m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (8m) 8m ago 20m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (9m) 8m ago 21m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (17m) 9m ago 28m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:13:56.454 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (16m) 8m ago 26m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (14m) 9m ago 28m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (13m) 8m ago 25m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (16m) 9m ago 27m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (16m) 8m ago 26m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (13m) 9m ago 25m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (12m) 9m ago 24m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (11m) 9m ago 24m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (11m) 8m ago 23m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (10m) 8m ago 23m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (10m) 8m ago 22m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:13:56.455 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (16m) 9m ago 26m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 msgr2=0x7f9c8c06f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 0x7f9c8c06f1f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c005950 tx=0x7f9c9c0058e0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 msgr2=0x7f9ca019ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9c900092e0 tx=0x7f9c9000be20 comp rx=0 tx=0).stop 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 shutdown_connections 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9c8c06cd30 0x7f9c8c06f1f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.456+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ca0068580 0x7f9ca019c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.457+0000 7f9ca7fe4700 1 --2- 172.21.15.8:0/3431287893 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ca0106cd0 0x7f9ca019ca80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.456 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.457+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 >> 172.21.15.8:0/3431287893 conn(0x7f9ca0075670 msgr2=0x7f9ca00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.457+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 shutdown_connections 2024-04-12T06:13:56.457 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.457+0000 7f9ca7fe4700 1 -- 172.21.15.8:0/3431287893 wait complete. 2024-04-12T06:13:56.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 -- 172.21.15.8:0/1227437578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c4105bd0 msgr2=0x7ff8c4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 --2- 172.21.15.8:0/1227437578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4105fb0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009910 tx=0x7ff8c0009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:56.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 -- 172.21.15.8:0/1227437578 shutdown_connections 2024-04-12T06:13:56.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 --2- 172.21.15.8:0/1227437578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 --2- 172.21.15.8:0/1227437578 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4105fb0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 -- 172.21.15.8:0/1227437578 >> 172.21.15.8:0/1227437578 conn(0x7ff8c4075670 msgr2=0x7ff8c4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 -- 172.21.15.8:0/1227437578 shutdown_connections 2024-04-12T06:13:56.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.561+0000 7ff8cb790700 1 -- 172.21.15.8:0/1227437578 wait complete. 2024-04-12T06:13:56.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 Processor -- start 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 -- start start 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41022c0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8cb790700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4102400 con 0x7ff8c4105bd0 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57382/0 (socket says 172.21.15.8:57382) 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 -- 172.21.15.8:0/2764504415 learned_addr learned my addr 172.21.15.8:0/2764504415 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c8d2b700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 -- 172.21.15.8:0/2764504415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c4105bd0 msgr2=0x7ff8c4101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.562+0000 7ff8c952c700 1 -- 172.21.15.8:0/2764504415 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00095f0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8c952c700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c001d070 con 0x7ff8c40ffc20 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8c0005dc0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c0003940 con 0x7ff8c40ffc20 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c4102680 con 0x7ff8c40ffc20 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.563+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41a2c00 con 0x7ff8c40ffc20 2024-04-12T06:13:56.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.564+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c404f2e0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.565+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff8c0004ae0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.565+0000 7ff8ba7fc700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 0x7ff8b006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.565+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff8c008e230 con 0x7ff8c40ffc20 2024-04-12T06:13:56.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.565+0000 7ff8c8d2b700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 0x7ff8b006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.566+0000 7ff8c8d2b700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 0x7ff8b006f1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4103030 tx=0x7ff8b4009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.569+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff8c00582a0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.857+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8c4102d70 con 0x7ff8c40ffc20 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.858+0000 7ff8ba7fc700 1 -- 172.21.15.8:0/2764504415 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7ff8c005b8c0 con 0x7ff8c40ffc20 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:56.857 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:13:56.858 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 msgr2=0x7ff8b006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 0x7ff8b006f1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4103030 tx=0x7ff8b4009260 comp rx=0 tx=0).stop 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 msgr2=0x7ff8c4101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0000c00 tx=0x7ff8c000bde0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 shutdown_connections 2024-04-12T06:13:56.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff8b006cce0 0x7ff8b006f1a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff8c40ffc20 0x7ff8c4101840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 --2- 172.21.15.8:0/2764504415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff8c4105bd0 0x7ff8c4101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 >> 172.21.15.8:0/2764504415 conn(0x7ff8c4075670 msgr2=0x7ff8c4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 shutdown_connections 2024-04-12T06:13:56.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.860+0000 7ff8cb790700 1 -- 172.21.15.8:0/2764504415 wait complete. 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 -- 172.21.15.8:0/3045332344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500ff5a0 msgr2=0x7f22500ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 --2- 172.21.15.8:0/3045332344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500ff5a0 0x7f22500ff980 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f224c009910 tx=0x7f224c009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 -- 172.21.15.8:0/3045332344 shutdown_connections 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 --2- 172.21.15.8:0/3045332344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500fff50 0x7f2250105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 --2- 172.21.15.8:0/3045332344 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500ff5a0 0x7f22500ff980 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.966+0000 7f2257cc7700 1 -- 172.21.15.8:0/3045332344 >> 172.21.15.8:0/3045332344 conn(0x7f2250074e30 msgr2=0x7f2250075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:56.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 -- 172.21.15.8:0/3045332344 shutdown_connections 2024-04-12T06:13:56.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 -- 172.21.15.8:0/3045332344 wait complete. 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 Processor -- start 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 -- start start 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 0x7f225019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225019d170 con 0x7f22500fff50 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.967+0000 7f2257cc7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250196620 con 0x7f22500ff5a0 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57408/0 (socket says 172.21.15.8:57408) 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 -- 172.21.15.8:0/1542327843 learned_addr learned my addr 172.21.15.8:0/1542327843 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:56.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255a63700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 0x7f225019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 -- 172.21.15.8:0/1542327843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 msgr2=0x7f225019c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 0x7f225019c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 -- 172.21.15.8:0/1542327843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0095f0 con 0x7f22500fff50 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255a63700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 0x7f225019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2255262700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f224000d900 tx=0x7f224000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2240009900 con 0x7f22500fff50 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2240010460 con 0x7f22500fff50 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224000f5f0 con 0x7f22500fff50 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2250196900 con 0x7f22500fff50 2024-04-12T06:13:56.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.968+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2250196e50 con 0x7f22500fff50 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.969+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250109270 con 0x7f22500fff50 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.970+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f22400105d0 con 0x7f22500fff50 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.970+0000 7f2246ffd700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 0x7f223c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.970+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f224008c2f0 con 0x7f22500fff50 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.970+0000 7f2255a63700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 0x7f223c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:56.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.971+0000 7f2255a63700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 0x7f223c06f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f224c006010 tx=0x7f224c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:56.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:56.975+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2240059e70 con 0x7f22500fff50 2024-04-12T06:13:57.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:56 smithi194 ceph-mon[106982]: pgmap v567: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:57.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:56 smithi194 ceph-mon[106982]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:56 smithi008 ceph-mon[146084]: pgmap v567: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:56 smithi008 ceph-mon[146084]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.221+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f225004f2e0 con 0x7f22500fff50 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.221+0000 7f2246ffd700 1 -- 172.21.15.8:0/1542327843 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f2240017740 con 0x7f22500fff50 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:13:57.220 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:13:57.221 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:13:57.222 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:57.223 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 msgr2=0x7f223c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 0x7f223c06f1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f224c006010 tx=0x7f224c01a040 comp rx=0 tx=0).stop 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 msgr2=0x7f225019cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f224000d900 tx=0x7f224000dcc0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 shutdown_connections 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f223c06cce0 0x7f223c06f1a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22500ff5a0 0x7f225019c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 --2- 172.21.15.8:0/1542327843 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22500fff50 0x7f225019cae0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 >> 172.21.15.8:0/1542327843 conn(0x7f2250074e30 msgr2=0x7f22500fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:57.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 shutdown_connections 2024-04-12T06:13:57.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.223+0000 7f2257cc7700 1 -- 172.21.15.8:0/1542327843 wait complete. 2024-04-12T06:13:57.225 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 -- 172.21.15.8:0/3457593912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 msgr2=0x7f8594105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 --2- 172.21.15.8:0/3457593912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f8594105fb0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8590009910 tx=0x7f8590009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 -- 172.21.15.8:0/3457593912 shutdown_connections 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 --2- 172.21.15.8:0/3457593912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 0x7f8594100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 --2- 172.21.15.8:0/3457593912 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f8594105fb0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 -- 172.21.15.8:0/3457593912 >> 172.21.15.8:0/3457593912 conn(0x7f8594075670 msgr2=0x7f8594075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 -- 172.21.15.8:0/3457593912 shutdown_connections 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.330+0000 7f859b77c700 1 -- 172.21.15.8:0/3457593912 wait complete. 2024-04-12T06:13:57.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 Processor -- start 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 -- start start 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 0x7f859419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859419d110 con 0x7f8594105bd0 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f859b77c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594196630 con 0x7f85940ffc20 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f8598d17700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f8598d17700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57436/0 (socket says 172.21.15.8:57436) 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f8598d17700 1 -- 172.21.15.8:0/827070524 learned_addr learned my addr 172.21.15.8:0/827070524 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.331+0000 7f8599518700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 0x7f859419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f8598d17700 1 -- 172.21.15.8:0/827070524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 msgr2=0x7f859419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f8598d17700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 0x7f859419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f8598d17700 1 -- 172.21.15.8:0/827070524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900095f0 con 0x7f8594105bd0 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f8598d17700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f858800e920 tx=0x7f858800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:57.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858800c9d0 con 0x7f8594105bd0 2024-04-12T06:13:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8588010430 con 0x7f8594105bd0 2024-04-12T06:13:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8594196910 con 0x7f8594105bd0 2024-04-12T06:13:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588018650 con 0x7f8594105bd0 2024-04-12T06:13:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.332+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8594196e60 con 0x7f8594105bd0 2024-04-12T06:13:57.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.333+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859404f2e0 con 0x7f8594105bd0 2024-04-12T06:13:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.333+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f85880105a0 con 0x7f8594105bd0 2024-04-12T06:13:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.334+0000 7f85867fc700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 0x7f858006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.334+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8588014070 con 0x7f8594105bd0 2024-04-12T06:13:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.334+0000 7f8599518700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 0x7f858006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.334+0000 7f8599518700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 0x7f858006f1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8590009fd0 tx=0x7f8590005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:57.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.338+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8588052ab0 con 0x7f8594105bd0 2024-04-12T06:13:57.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.561+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85941971a0 con 0x7f858006cce0 2024-04-12T06:13:57.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.562+0000 7f85867fc700 1 -- 172.21.15.8:0/827070524 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f85941971a0 con 0x7f858006cce0 2024-04-12T06:13:57.562 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:13:57.562 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:13:57.562 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:13:57.562 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:13:57.562 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:13:57.563 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:13:57.564 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:13:57.564 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:13:57.564 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:13:57.564 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:13:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 msgr2=0x7f858006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 0x7f858006f1a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8590009fd0 tx=0x7f8590005940 comp rx=0 tx=0).stop 2024-04-12T06:13:57.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 msgr2=0x7f859419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f858800e920 tx=0x7f858800ec30 comp rx=0 tx=0).stop 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 shutdown_connections 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f858006cce0 0x7f858006f1a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85940ffc20 0x7f859419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 --2- 172.21.15.8:0/827070524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8594105bd0 0x7f859419ca80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.564+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 >> 172.21.15.8:0/827070524 conn(0x7f8594075670 msgr2=0x7f8594068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:57.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.565+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 shutdown_connections 2024-04-12T06:13:57.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.565+0000 7f859b77c700 1 -- 172.21.15.8:0/827070524 wait complete. 2024-04-12T06:13:57.677 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.671+0000 7f697cf91700 1 -- 172.21.15.8:0/3427476251 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 msgr2=0x7f69780689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.671+0000 7f697cf91700 1 --2- 172.21.15.8:0/3427476251 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f69780689e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f696c009910 tx=0x7f696c009c20 comp rx=0 tx=0).stop 2024-04-12T06:13:57.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.671+0000 7f697cf91700 1 -- 172.21.15.8:0/3427476251 shutdown_connections 2024-04-12T06:13:57.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 --2- 172.21.15.8:0/3427476251 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f69780689e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 --2- 172.21.15.8:0/3427476251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 0x7f69781070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 -- 172.21.15.8:0/3427476251 >> 172.21.15.8:0/3427476251 conn(0x7f6978075670 msgr2=0x7f6978075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 -- 172.21.15.8:0/3427476251 shutdown_connections 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 -- 172.21.15.8:0/3427476251 wait complete. 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 Processor -- start 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.672+0000 7f697cf91700 1 -- start start 2024-04-12T06:13:57.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697cf91700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697cf91700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 0x7f697819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697cf91700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697819d110 con 0x7f6978068580 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697cf91700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69781965c0 con 0x7f6978106cd0 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:57460/0 (socket says 172.21.15.8:57460) 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 -- 172.21.15.8:0/1424731544 learned_addr learned my addr 172.21.15.8:0/1424731544 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f6975d9b700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 0x7f697819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 -- 172.21.15.8:0/1424731544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 msgr2=0x7f697819ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 0x7f697819ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 -- 172.21.15.8:0/1424731544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696c0095f0 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697659c700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f696800e920 tx=0x7f696800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6968009790 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f696800cba0 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69781968a0 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.673+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6968018650 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.674+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6978196df0 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.674+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69781091d0 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.675+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f696800cd10 con 0x7f6978068580 2024-04-12T06:13:57.683 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.675+0000 7f69677fe700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 0x7f696006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:13:57.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.676+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6968014070 con 0x7f6978068580 2024-04-12T06:13:57.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.676+0000 7f6975d9b700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 0x7f696006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:13:57.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.676+0000 7f6975d9b700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 0x7f696006f1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f69780fe260 tx=0x7f696c009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:13:57.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.680+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6968052ab0 con 0x7f6978068580 2024-04-12T06:13:57.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.968+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f697804f2e0 con 0x7f6978068580 2024-04-12T06:13:57.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.968+0000 7f69677fe700 1 -- 172.21.15.8:0/1424731544 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f696805a1b0 con 0x7f6978068580 2024-04-12T06:13:57.967 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:13:57.967 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:13:57.967 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:13:57.968 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:13:57.968 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:13:57.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 msgr2=0x7f696006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 0x7f696006f1a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f69780fe260 tx=0x7f696c009f70 comp rx=0 tx=0).stop 2024-04-12T06:13:57.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 msgr2=0x7f697819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:13:57.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f696800e920 tx=0x7f696800ece0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 shutdown_connections 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f696006cce0 0x7f696006f1a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6978068580 0x7f697819c540 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 --2- 172.21.15.8:0/1424731544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6978106cd0 0x7f697819ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 >> 172.21.15.8:0/1424731544 conn(0x7f6978075670 msgr2=0x7f69780fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 shutdown_connections 2024-04-12T06:13:57.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:13:57.970+0000 7f697cf91700 1 -- 172.21.15.8:0/1424731544 wait complete. 2024-04-12T06:13:58.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:57 smithi008 ceph-mon[146084]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:58.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:57 smithi008 ceph-mon[146084]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:58.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:57 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2764504415' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:13:58.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:57 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1542327843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:13:58.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:57 smithi194 ceph-mon[106982]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:58.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:57 smithi194 ceph-mon[106982]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:58.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:57 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2764504415' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:13:58.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:57 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1542327843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:13:59.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:58 smithi194 ceph-mon[106982]: pgmap v568: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:59.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:58 smithi194 ceph-mon[106982]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:59.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:13:58 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1424731544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:13:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:58 smithi008 ceph-mon[146084]: pgmap v568: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:13:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:58 smithi008 ceph-mon[146084]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:13:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:13:58 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1424731544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:14:01.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:00 smithi194 ceph-mon[106982]: pgmap v569: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:01.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:00 smithi008 ceph-mon[146084]: pgmap v569: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:02 smithi194 ceph-mon[106982]: pgmap v570: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:02 smithi008 ceph-mon[146084]: pgmap v570: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:05.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:04 smithi194 ceph-mon[106982]: pgmap v571: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:05.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:04 smithi008 ceph-mon[146084]: pgmap v571: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:07.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:06 smithi194 ceph-mon[106982]: pgmap v572: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:06 smithi008 ceph-mon[146084]: pgmap v572: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:08.975 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:08 smithi194 ceph-mon[106982]: pgmap v573: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:09.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:08 smithi008 ceph-mon[146084]: pgmap v573: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:10 smithi194 ceph-mon[106982]: pgmap v574: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:11.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:10 smithi008 ceph-mon[146084]: pgmap v574: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:13.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:12 smithi194 ceph-mon[106982]: pgmap v575: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:13.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:12 smithi008 ceph-mon[146084]: pgmap v575: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:15.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:14 smithi194 ceph-mon[106982]: pgmap v576: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:15.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:14 smithi008 ceph-mon[146084]: pgmap v576: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:17.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:16 smithi194 ceph-mon[106982]: pgmap v577: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:17.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:16 smithi008 ceph-mon[146084]: pgmap v577: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:18.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:18.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:19.068 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:18 smithi194 ceph-mon[106982]: pgmap v578: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:19.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:18 smithi008 ceph-mon[146084]: pgmap v578: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:21.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:20 smithi194 ceph-mon[106982]: pgmap v579: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:21.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:20 smithi008 ceph-mon[146084]: pgmap v579: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:23.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:22 smithi194 ceph-mon[106982]: pgmap v580: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:23.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:22 smithi008 ceph-mon[146084]: pgmap v580: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:24 smithi194 ceph-mon[106982]: pgmap v581: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:25.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:24 smithi008 ceph-mon[146084]: pgmap v581: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:27.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:26 smithi194 ceph-mon[106982]: pgmap v582: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:27.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:26 smithi008 ceph-mon[146084]: pgmap v582: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 -- 172.21.15.8:0/4271260285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf780ffc20 msgr2=0x7faf78100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/4271260285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf780ffc20 0x7faf78100080 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009910 tx=0x7faf6c009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 -- 172.21.15.8:0/4271260285 shutdown_connections 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/4271260285 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf780ffc20 0x7faf78100080 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/4271260285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf78105bd0 0x7faf78105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 -- 172.21.15.8:0/4271260285 >> 172.21.15.8:0/4271260285 conn(0x7faf78075670 msgr2=0x7faf78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 -- 172.21.15.8:0/4271260285 shutdown_connections 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.092+0000 7faf7e8d8700 1 -- 172.21.15.8:0/4271260285 wait complete. 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 Processor -- start 2024-04-12T06:14:28.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 -- start start 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf780ffc20 0x7faf78195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78196260 con 0x7faf78105bd0 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf7e8d8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78198f90 con 0x7faf780ffc20 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54648/0 (socket says 172.21.15.8:54648) 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.093+0000 7faf777fe700 1 -- 172.21.15.8:0/3204463544 learned_addr learned my addr 172.21.15.8:0/3204463544 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:28.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf77fff700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf780ffc20 0x7faf78195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf777fe700 1 -- 172.21.15.8:0/3204463544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf780ffc20 msgr2=0x7faf78195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf777fe700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf780ffc20 0x7faf78195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf777fe700 1 -- 172.21.15.8:0/3204463544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c0095f0 con 0x7faf78105bd0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf777fe700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf6c006010 tx=0x7faf6c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf6c01f070 con 0x7faf78105bd0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faf6c005710 con 0x7faf78105bd0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf78199210 con 0x7faf78105bd0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf6c00f460 con 0x7faf78105bd0 2024-04-12T06:14:28.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.094+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf78199700 con 0x7faf78105bd0 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.095+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7804f2e0 con 0x7faf78105bd0 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.096+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7faf6c00f5c0 con 0x7faf78105bd0 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.096+0000 7faf757fa700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 0x7faf6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.096+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faf6c08f7d0 con 0x7faf78105bd0 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.096+0000 7faf77fff700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 0x7faf6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.097+0000 7faf77fff700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 0x7faf6006f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf68004630 tx=0x7faf6800b3c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.101+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7faf6c059840 con 0x7faf78105bd0 2024-04-12T06:14:28.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.323+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf78199970 con 0x7faf6006cce0 2024-04-12T06:14:28.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.324+0000 7faf757fa700 1 -- 172.21.15.8:0/3204463544 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faf78199970 con 0x7faf6006cce0 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 msgr2=0x7faf6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 0x7faf6006f1a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faf68004630 tx=0x7faf6800b3c0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 msgr2=0x7faf78195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faf6c006010 tx=0x7faf6c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 shutdown_connections 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faf6006cce0 0x7faf6006f1a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf780ffc20 0x7faf78195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 --2- 172.21.15.8:0/3204463544 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faf78105bd0 0x7faf78195bd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 >> 172.21.15.8:0/3204463544 conn(0x7faf78075670 msgr2=0x7faf780fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:28.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 shutdown_connections 2024-04-12T06:14:28.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.327+0000 7faf7e8d8700 1 -- 172.21.15.8:0/3204463544 wait complete. 2024-04-12T06:14:28.336 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 -- 172.21.15.8:0/3315202312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 msgr2=0x7f25780ff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 --2- 172.21.15.8:0/3315202312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f25780ff980 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2568009910 tx=0x7f2568009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 -- 172.21.15.8:0/3315202312 shutdown_connections 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 --2- 172.21.15.8:0/3315202312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 --2- 172.21.15.8:0/3315202312 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f25780ff980 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 -- 172.21.15.8:0/3315202312 >> 172.21.15.8:0/3315202312 conn(0x7f2578074e30 msgr2=0x7f2578075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 -- 172.21.15.8:0/3315202312 shutdown_connections 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.434+0000 7f257d38c700 1 -- 172.21.15.8:0/3315202312 wait complete. 2024-04-12T06:14:28.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 Processor -- start 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 -- start start 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578100a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2578100fa0 con 0x7f25780ff5a0 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.435+0000 7f257d38c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25781010e0 con 0x7f25780fff50 2024-04-12T06:14:28.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54676/0 (socket says 172.21.15.8:54676) 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 -- 172.21.15.8:0/2170663408 learned_addr learned my addr 172.21.15.8:0/2170663408 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f25767fc700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578100a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 -- 172.21.15.8:0/2170663408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 msgr2=0x7f2578100a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578100a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 -- 172.21.15.8:0/2170663408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680095f0 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f25767fc700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f2576ffd700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25680092e0 tx=0x7f256800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256801d070 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2568005e00 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2578101360 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256800fe60 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.436+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25781a2c90 con 0x7f25780ff5a0 2024-04-12T06:14:28.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.437+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578109270 con 0x7f25780ff5a0 2024-04-12T06:14:28.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.438+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f256800f6d0 con 0x7f25780ff5a0 2024-04-12T06:14:28.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.438+0000 7f255ffff700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 0x7f256006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.438+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f256808d7d0 con 0x7f25780ff5a0 2024-04-12T06:14:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.438+0000 7f25767fc700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 0x7f256006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.439+0000 7f25767fc700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 0x7f256006f1a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f256c005fd0 tx=0x7f256c005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.443+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2568057840 con 0x7f25780ff5a0 2024-04-12T06:14:28.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.663+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25780611d0 con 0x7f256006cce0 2024-04-12T06:14:28.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.664+0000 7f255ffff700 1 -- 172.21.15.8:0/2170663408 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f25780611d0 con 0x7f256006cce0 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 msgr2=0x7f256006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 0x7f256006f1a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f256c005fd0 tx=0x7f256c005ee0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 msgr2=0x7f2578100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25680092e0 tx=0x7f256800be20 comp rx=0 tx=0).stop 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 shutdown_connections 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f256006cce0 0x7f256006f1a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f25780ff5a0 0x7f2578100520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 --2- 172.21.15.8:0/2170663408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25780fff50 0x7f2578100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 >> 172.21.15.8:0/2170663408 conn(0x7f2578074e30 msgr2=0x7f25780fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:28.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 shutdown_connections 2024-04-12T06:14:28.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.666+0000 7f257d38c700 1 -- 172.21.15.8:0/2170663408 wait complete. 2024-04-12T06:14:28.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 -- 172.21.15.8:0/2639856630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 msgr2=0x7fab0c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 --2- 172.21.15.8:0/2639856630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c105fb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faafc009910 tx=0x7faafc009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:28.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 -- 172.21.15.8:0/2639856630 shutdown_connections 2024-04-12T06:14:28.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 --2- 172.21.15.8:0/2639856630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 0x7fab0c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 --2- 172.21.15.8:0/2639856630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c105fb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 -- 172.21.15.8:0/2639856630 >> 172.21.15.8:0/2639856630 conn(0x7fab0c075670 msgr2=0x7fab0c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:28.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 -- 172.21.15.8:0/2639856630 shutdown_connections 2024-04-12T06:14:28.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.772+0000 7fab111ef700 1 -- 172.21.15.8:0/2639856630 wait complete. 2024-04-12T06:14:28.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 Processor -- start 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 -- start start 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 0x7fab0c1953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c195e70 con 0x7fab0c105bd0 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab111ef700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c195fe0 con 0x7fab0c0ffc20 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab0a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab0a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54692/0 (socket says 172.21.15.8:54692) 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab0a59c700 1 -- 172.21.15.8:0/2037175951 learned_addr learned my addr 172.21.15.8:0/2037175951 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.773+0000 7fab0ad9d700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 0x7fab0c1953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab0a59c700 1 -- 172.21.15.8:0/2037175951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 msgr2=0x7fab0c1953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab0a59c700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 0x7fab0c1953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:28.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab0a59c700 1 -- 172.21.15.8:0/2037175951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0095f0 con 0x7fab0c105bd0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab0a59c700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d900 tx=0x7fab0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab000041f0 con 0x7fab0c105bd0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fab00004cd0 con 0x7fab0c105bd0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c1a2a60 con 0x7fab0c105bd0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab00005660 con 0x7fab0c105bd0 2024-04-12T06:14:28.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.774+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c1a2dc0 con 0x7fab0c105bd0 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.775+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c1080d0 con 0x7fab0c105bd0 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.775+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fab00004e40 con 0x7fab0c105bd0 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.776+0000 7faaf3fff700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 0x7faaf406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.776+0000 7fab0ad9d700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 0x7faaf406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.776+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab0008c270 con 0x7fab0c105bd0 2024-04-12T06:14:28.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.776+0000 7fab0ad9d700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 0x7faaf406f1a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faafc006010 tx=0x7faafc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:28.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.781+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fab00059df0 con 0x7fab0c105bd0 2024-04-12T06:14:28.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:28 smithi008 ceph-mon[146084]: pgmap v583: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:28.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:28.993+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab0c0611d0 con 0x7faaf406cce0 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.004+0000 7faaf3fff700 1 -- 172.21.15.8:0/2037175951 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fab0c0611d0 con 0x7faaf406cce0 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (16m) 9m ago 27m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (28m) 9m ago 28m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (26m) 9m ago 26m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:14:29.003 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (14m) 9m ago 27m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (13m) 9m ago 26m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (16m) 9m ago 27m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (10m) 9m ago 21m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (10m) 9m ago 21m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (9m) 9m ago 21m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (9m) 9m ago 21m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (18m) 9m ago 29m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (17m) 9m ago 26m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (14m) 9m ago 29m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:14:29.004 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (14m) 9m ago 26m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (17m) 9m ago 27m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (17m) 9m ago 26m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (13m) 9m ago 25m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (13m) 9m ago 25m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (12m) 9m ago 24m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (11m) 9m ago 24m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (11m) 9m ago 23m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (10m) 9m ago 23m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:14:29.005 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (16m) 9m ago 27m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 msgr2=0x7faaf406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 0x7faaf406f1a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7faafc006010 tx=0x7faafc01a040 comp rx=0 tx=0).stop 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 msgr2=0x7fab0c195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fab0000d900 tx=0x7fab0000dc10 comp rx=0 tx=0).stop 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 shutdown_connections 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faaf406cce0 0x7faaf406f1a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab0c0ffc20 0x7fab0c1953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 --2- 172.21.15.8:0/2037175951 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fab0c105bd0 0x7fab0c195930 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 >> 172.21.15.8:0/2037175951 conn(0x7fab0c075670 msgr2=0x7fab0c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 shutdown_connections 2024-04-12T06:14:29.007 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.006+0000 7fab111ef700 1 -- 172.21.15.8:0/2037175951 wait complete. 2024-04-12T06:14:29.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:28 smithi194 ceph-mon[106982]: pgmap v583: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:29.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- 172.21.15.8:0/2274585098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4068580 msgr2=0x7f1dc40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2274585098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4068580 0x7f1dc40689e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009910 tx=0x7f1dc0009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- 172.21.15.8:0/2274585098 shutdown_connections 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2274585098 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4068580 0x7f1dc40689e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2274585098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4106cd0 0x7f1dc41070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- 172.21.15.8:0/2274585098 >> 172.21.15.8:0/2274585098 conn(0x7f1dc4075670 msgr2=0x7f1dc4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- 172.21.15.8:0/2274585098 shutdown_connections 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- 172.21.15.8:0/2274585098 wait complete. 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 Processor -- start 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.112+0000 7f1dcc622700 1 -- start start 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4068580 0x7f1dc419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc419d110 con 0x7f1dc4106cd0 2024-04-12T06:14:29.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc41965c0 con 0x7f1dc4068580 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54708/0 (socket says 172.21.15.8:54708) 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 -- 172.21.15.8:0/2696888821 learned_addr learned my addr 172.21.15.8:0/2696888821 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dca3be700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4068580 0x7f1dc419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 -- 172.21.15.8:0/2696888821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4068580 msgr2=0x7f1dc419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4068580 0x7f1dc419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 -- 172.21.15.8:0/2696888821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dc4106cd0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dc9bbd700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc001d070 con 0x7f1dc4106cd0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1dc000f460 con 0x7f1dc4106cd0 2024-04-12T06:14:29.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4196840 con 0x7f1dc4106cd0 2024-04-12T06:14:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc0021620 con 0x7f1dc4106cd0 2024-04-12T06:14:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.113+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc4196d30 con 0x7f1dc4106cd0 2024-04-12T06:14:29.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.114+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc41091d0 con 0x7f1dc4106cd0 2024-04-12T06:14:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.115+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1dc000fa40 con 0x7f1dc4106cd0 2024-04-12T06:14:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.115+0000 7f1db77fe700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 0x7f1db006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.115+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1dc008d6b0 con 0x7f1dc4106cd0 2024-04-12T06:14:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.115+0000 7f1dca3be700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 0x7f1db006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.116+0000 7f1dca3be700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 0x7f1db006f1a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005950 tx=0x7f1db80058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.120+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1dc005b230 con 0x7f1dc4106cd0 2024-04-12T06:14:29.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.408+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1dc4108bb0 con 0x7f1dc4106cd0 2024-04-12T06:14:29.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.408+0000 7f1db77fe700 1 -- 172.21.15.8:0/2696888821 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1dc005adc0 con 0x7f1dc4106cd0 2024-04-12T06:14:29.407 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:14:29.408 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:14:29.409 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:14:29.409 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:14:29.409 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:14:29.409 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:14:29.409 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 msgr2=0x7f1db006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 0x7f1db006f1a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005950 tx=0x7f1db80058e0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 msgr2=0x7f1dc419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc000be20 comp rx=0 tx=0).stop 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 shutdown_connections 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1db006cce0 0x7f1db006f1a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dc4068580 0x7f1dc419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 --2- 172.21.15.8:0/2696888821 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dc4106cd0 0x7f1dc419ca80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 >> 172.21.15.8:0/2696888821 conn(0x7f1dc4075670 msgr2=0x7f1dc40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 shutdown_connections 2024-04-12T06:14:29.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.410+0000 7f1dcc622700 1 -- 172.21.15.8:0/2696888821 wait complete. 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- 172.21.15.8:0/2415995897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3350105bd0 msgr2=0x7f3350105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 --2- 172.21.15.8:0/2415995897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3350105bd0 0x7f3350105fb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3344009910 tx=0x7f3344009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- 172.21.15.8:0/2415995897 shutdown_connections 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 --2- 172.21.15.8:0/2415995897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f33500ffc20 0x7f3350100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 --2- 172.21.15.8:0/2415995897 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3350105bd0 0x7f3350105fb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- 172.21.15.8:0/2415995897 >> 172.21.15.8:0/2415995897 conn(0x7f3350075670 msgr2=0x7f3350075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- 172.21.15.8:0/2415995897 shutdown_connections 2024-04-12T06:14:29.517 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- 172.21.15.8:0/2415995897 wait complete. 2024-04-12T06:14:29.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 Processor -- start 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.517+0000 7f3356b1d700 1 -- start start 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f3356b1d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f3356b1d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 0x7f335019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f3356b1d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335019d150 con 0x7f33500ffc20 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f3356b1d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3350196600 con 0x7f3350105bd0 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54726/0 (socket says 172.21.15.8:54726) 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 -- 172.21.15.8:0/3572022653 learned_addr learned my addr 172.21.15.8:0/3572022653 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f334ffff700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 0x7f335019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 -- 172.21.15.8:0/3572022653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 msgr2=0x7f335019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 0x7f335019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 -- 172.21.15.8:0/3572022653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33440095f0 con 0x7f33500ffc20 2024-04-12T06:14:29.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f334ffff700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 0x7f335019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f33548b9700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3344006010 tx=0x7f334400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f334401d070 con 0x7f33500ffc20 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f334400f460 con 0x7f33500ffc20 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3350196880 con 0x7f33500ffc20 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.518+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3344021640 con 0x7f33500ffc20 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.519+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3350196d70 con 0x7f33500ffc20 2024-04-12T06:14:29.520 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.519+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335004f2e0 con 0x7f33500ffc20 2024-04-12T06:14:29.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.520+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f334400fa60 con 0x7f33500ffc20 2024-04-12T06:14:29.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.520+0000 7f334dffb700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 0x7f334006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.521+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f334408d820 con 0x7f33500ffc20 2024-04-12T06:14:29.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.521+0000 7f334ffff700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 0x7f334006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.521+0000 7f334ffff700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 0x7f334006f1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f333c005fd0 tx=0x7f333c005dc0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.525+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f334405b340 con 0x7f33500ffc20 2024-04-12T06:14:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.775+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f33501970a0 con 0x7f33500ffc20 2024-04-12T06:14:29.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.775+0000 7f334dffb700 1 -- 172.21.15.8:0/3572022653 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f334405aed0 con 0x7f33500ffc20 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:14:29.775 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:14:29.776 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:14:29.777 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:14:29.778 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:14:29.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 msgr2=0x7f334006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 0x7f334006f1a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f333c005fd0 tx=0x7f333c005dc0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 msgr2=0x7f335019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3344006010 tx=0x7f334400be20 comp rx=0 tx=0).stop 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 shutdown_connections 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f334006cce0 0x7f334006f1a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f33500ffc20 0x7f335019c580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 --2- 172.21.15.8:0/3572022653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3350105bd0 0x7f335019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 >> 172.21.15.8:0/3572022653 conn(0x7f3350075670 msgr2=0x7f3350068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 shutdown_connections 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.778+0000 7f3356b1d700 1 -- 172.21.15.8:0/3572022653 wait complete. 2024-04-12T06:14:29.779 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:14:29.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 -- 172.21.15.8:0/3628038601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 msgr2=0x7f2900101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 --2- 172.21.15.8:0/3628038601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f2900101270 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009910 tx=0x7f28f4009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 -- 172.21.15.8:0/3628038601 shutdown_connections 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 --2- 172.21.15.8:0/3628038601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 0x7f2900105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 --2- 172.21.15.8:0/3628038601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f2900101270 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 -- 172.21.15.8:0/3628038601 >> 172.21.15.8:0/3628038601 conn(0x7f29000fc740 msgr2=0x7f29000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 -- 172.21.15.8:0/3628038601 shutdown_connections 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.886+0000 7f2906d8a700 1 -- 172.21.15.8:0/3628038601 wait complete. 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 Processor -- start 2024-04-12T06:14:29.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 -- start start 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 0x7f290019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019f360 con 0x7f2900100e90 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2906d8a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900198810 con 0x7f2900101840 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54746/0 (socket says 172.21.15.8:54746) 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 -- 172.21.15.8:0/2098861910 learned_addr learned my addr 172.21.15.8:0/2098861910 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f28fffff700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 0x7f290019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 -- 172.21.15.8:0/2098861910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 msgr2=0x7f290019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:29.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 0x7f290019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 -- 172.21.15.8:0/2098861910 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40095f0 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.887+0000 7f2904b26700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006010 tx=0x7f28f400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.888+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f401d070 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.888+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28f4005710 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.888+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2900198a90 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.888+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f400fe20 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.888+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2900198f80 con 0x7f2900100e90 2024-04-12T06:14:29.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.889+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2900105010 con 0x7f2900100e90 2024-04-12T06:14:29.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.889+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f28f4003a60 con 0x7f2900100e90 2024-04-12T06:14:29.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.890+0000 7f28fdffb700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 0x7f28f006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:29.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.890+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f28f408d6a0 con 0x7f2900100e90 2024-04-12T06:14:29.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.890+0000 7f28fffff700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 0x7f28f006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:29.890 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.890+0000 7f28fffff700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 0x7f28f006f1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f29000fd000 tx=0x7f28ec00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:29.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:29.894+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f28f405b1a0 con 0x7f2900100e90 2024-04-12T06:14:30.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:29 smithi194 ceph-mon[106982]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:29 smithi194 ceph-mon[106982]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:29 smithi194 ceph-mon[106982]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:29 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2696888821' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:14:30.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.120+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f29001991a0 con 0x7f28f006cce0 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.121+0000 7f28fdffb700 1 -- 172.21.15.8:0/2098861910 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f29001991a0 con 0x7f28f006cce0 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:14:30.120 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:14:30.121 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:14:30.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 msgr2=0x7f28f006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 0x7f28f006f1a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f29000fd000 tx=0x7f28ec00b410 comp rx=0 tx=0).stop 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 msgr2=0x7f290019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006010 tx=0x7f28f400bd70 comp rx=0 tx=0).stop 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 shutdown_connections 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f28f006cce0 0x7f28f006f1a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2900100e90 0x7f290019e790 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 --2- 172.21.15.8:0/2098861910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2900101840 0x7f290019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 >> 172.21.15.8:0/2098861910 conn(0x7f29000fc740 msgr2=0x7f29000fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 shutdown_connections 2024-04-12T06:14:30.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.123+0000 7f2906d8a700 1 -- 172.21.15.8:0/2098861910 wait complete. 2024-04-12T06:14:30.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:29 smithi008 ceph-mon[146084]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:29 smithi008 ceph-mon[146084]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:29 smithi008 ceph-mon[146084]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:30.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:29 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2696888821' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 -- 172.21.15.8:0/3243220288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 msgr2=0x7fbc501070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 --2- 172.21.15.8:0/3243220288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc501070b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009910 tx=0x7fbc4c009c20 comp rx=0 tx=0).stop 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 -- 172.21.15.8:0/3243220288 shutdown_connections 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 --2- 172.21.15.8:0/3243220288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 0x7fbc500689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 --2- 172.21.15.8:0/3243220288 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc501070b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 -- 172.21.15.8:0/3243220288 >> 172.21.15.8:0/3243220288 conn(0x7fbc50075670 msgr2=0x7fbc50075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 -- 172.21.15.8:0/3243220288 shutdown_connections 2024-04-12T06:14:30.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.230+0000 7fbc57472700 1 -- 172.21.15.8:0/3243220288 wait complete. 2024-04-12T06:14:30.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 Processor -- start 2024-04-12T06:14:30.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 -- start start 2024-04-12T06:14:30.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 0x7fbc5019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:30.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5019d130 con 0x7fbc50106cd0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc57472700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc50196630 con 0x7fbc50068580 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:54768/0 (socket says 172.21.15.8:54768) 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 -- 172.21.15.8:0/1358463875 learned_addr learned my addr 172.21.15.8:0/1358463875 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc5520e700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 0x7fbc5019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 -- 172.21.15.8:0/1358463875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 msgr2=0x7fbc5019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 0x7fbc5019c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.231+0000 7fbc54a0d700 1 -- 172.21.15.8:0/1358463875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0095f0 con 0x7fbc50106cd0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc54a0d700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e920 tx=0x7fbc4000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc40009790 con 0x7fbc50106cd0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbc4000cba0 con 0x7fbc50106cd0 2024-04-12T06:14:30.232 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc50196910 con 0x7fbc50106cd0 2024-04-12T06:14:30.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc40018650 con 0x7fbc50106cd0 2024-04-12T06:14:30.233 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.232+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc50196e60 con 0x7fbc50106cd0 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.233+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc501091d0 con 0x7fbc50106cd0 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.233+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbc4000cd10 con 0x7fbc50106cd0 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.234+0000 7fbc467fc700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 0x7fbc3c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.234+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc40014070 con 0x7fbc50106cd0 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.234+0000 7fbc5520e700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 0x7fbc3c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:14:30.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.234+0000 7fbc5520e700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 0x7fbc3c06f1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009fd0 tx=0x7fbc4c005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:14:30.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.238+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbc40052ab0 con 0x7fbc50106cd0 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.530+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbc5004f2e0 con 0x7fbc50106cd0 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.531+0000 7fbc467fc700 1 -- 172.21.15.8:0/1358463875 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fbc4005a1b0 con 0x7fbc50106cd0 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:14:30.530 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 msgr2=0x7fbc3c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 0x7fbc3c06f1a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009fd0 tx=0x7fbc4c005940 comp rx=0 tx=0).stop 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 msgr2=0x7fbc5019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e920 tx=0x7fbc4000ece0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 shutdown_connections 2024-04-12T06:14:30.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbc3c06cce0 0x7fbc3c06f1a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbc50068580 0x7fbc5019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 --2- 172.21.15.8:0/1358463875 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbc50106cd0 0x7fbc5019caa0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:14:30.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 >> 172.21.15.8:0/1358463875 conn(0x7fbc50075670 msgr2=0x7fbc500fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:14:30.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 shutdown_connections 2024-04-12T06:14:30.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:14:30.533+0000 7fbc57472700 1 -- 172.21.15.8:0/1358463875 wait complete. 2024-04-12T06:14:31.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:30 smithi194 ceph-mon[106982]: pgmap v584: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:31.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:30 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3572022653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:14:31.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:30 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1358463875' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:14:31.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:30 smithi008 ceph-mon[146084]: pgmap v584: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:31.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:30 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3572022653' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:14:31.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:30 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1358463875' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:14:32.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:31 smithi194 ceph-mon[106982]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:32.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:31 smithi008 ceph-mon[146084]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:14:33.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:32 smithi194 ceph-mon[106982]: pgmap v585: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:32 smithi008 ceph-mon[146084]: pgmap v585: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:35.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:34 smithi194 ceph-mon[106982]: pgmap v586: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:35.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:34 smithi008 ceph-mon[146084]: pgmap v586: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:37.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:36 smithi194 ceph-mon[106982]: pgmap v587: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:37.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:36 smithi008 ceph-mon[146084]: pgmap v587: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:38.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:38 smithi008 ceph-mon[146084]: pgmap v588: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:39.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:38 smithi194 ceph-mon[106982]: pgmap v588: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:41.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:40 smithi194 ceph-mon[106982]: pgmap v589: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:41.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:40 smithi008 ceph-mon[146084]: pgmap v589: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:43.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:42 smithi194 ceph-mon[106982]: pgmap v590: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:43.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:42 smithi008 ceph-mon[146084]: pgmap v590: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:45.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:44 smithi194 ceph-mon[106982]: pgmap v591: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:45.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:44 smithi008 ceph-mon[146084]: pgmap v591: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:47.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:46 smithi194 ceph-mon[106982]: pgmap v592: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:47.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:46 smithi008 ceph-mon[146084]: pgmap v592: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:48.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:48.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:14:49.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:48 smithi194 ceph-mon[106982]: pgmap v593: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:49.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:48 smithi008 ceph-mon[146084]: pgmap v593: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:51.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:50 smithi194 ceph-mon[106982]: pgmap v594: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:51.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:50 smithi008 ceph-mon[146084]: pgmap v594: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:53.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:52 smithi194 ceph-mon[106982]: pgmap v595: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:53.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:52 smithi008 ceph-mon[146084]: pgmap v595: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:55.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:54 smithi194 ceph-mon[106982]: pgmap v596: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:55.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:54 smithi008 ceph-mon[146084]: pgmap v596: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:57.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:56 smithi194 ceph-mon[106982]: pgmap v597: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:56 smithi008 ceph-mon[146084]: pgmap v597: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:59.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:14:58 smithi194 ceph-mon[106982]: pgmap v598: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:14:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:14:58 smithi008 ceph-mon[146084]: pgmap v598: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:00.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 -- 172.21.15.8:0/62592567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 msgr2=0x7f9240105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:00.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 --2- 172.21.15.8:0/62592567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240105fb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9230009910 tx=0x7f9230009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:00.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 -- 172.21.15.8:0/62592567 shutdown_connections 2024-04-12T06:15:00.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 --2- 172.21.15.8:0/62592567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 0x7f9240100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 --2- 172.21.15.8:0/62592567 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240105fb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 -- 172.21.15.8:0/62592567 >> 172.21.15.8:0/62592567 conn(0x7f9240075670 msgr2=0x7f9240075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:00.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 -- 172.21.15.8:0/62592567 shutdown_connections 2024-04-12T06:15:00.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.654+0000 7f924650d700 1 -- 172.21.15.8:0/62592567 wait complete. 2024-04-12T06:15:00.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.655+0000 7f924650d700 1 Processor -- start 2024-04-12T06:15:00.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.655+0000 7f924650d700 1 -- start start 2024-04-12T06:15:00.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f924650d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 0x7f9240195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:00.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f924650d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f924650d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240196260 con 0x7f9240105bd0 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f924650d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240198f90 con 0x7f92400ffc20 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36828/0 (socket says 172.21.15.8:36828) 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 -- 172.21.15.8:0/2526674921 learned_addr learned my addr 172.21.15.8:0/2526674921 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:00.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923ffff700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 0x7f9240195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 -- 172.21.15.8:0/2526674921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 msgr2=0x7f9240195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 0x7f9240195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.656+0000 7f923f7fe700 1 -- 172.21.15.8:0/2526674921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92300095f0 con 0x7f9240105bd0 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f923f7fe700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9234007b50 tx=0x7f9234007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9234011070 con 0x7f9240105bd0 2024-04-12T06:15:00.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f923400c2e0 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923400e770 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9240199210 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.657+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240199760 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.658+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924004f2e0 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.658+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9234004a00 con 0x7f9240105bd0 2024-04-12T06:15:00.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.659+0000 7f923d7fa700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 0x7f922806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:00.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.659+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f923408dcf0 con 0x7f9240105bd0 2024-04-12T06:15:00.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.659+0000 7f923ffff700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 0x7f922806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:00.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.659+0000 7f923ffff700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 0x7f922806f1a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9230006010 tx=0x7f923001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:00.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.664+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9234057d60 con 0x7f9240105bd0 2024-04-12T06:15:00.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.886+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92401999f0 con 0x7f922806cce0 2024-04-12T06:15:00.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.887+0000 7f923d7fa700 1 -- 172.21.15.8:0/2526674921 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f92401999f0 con 0x7f922806cce0 2024-04-12T06:15:00.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 msgr2=0x7f922806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:00.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 0x7f922806f1a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9230006010 tx=0x7f923001a040 comp rx=0 tx=0).stop 2024-04-12T06:15:00.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 msgr2=0x7f9240195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9234007b50 tx=0x7f9234007f10 comp rx=0 tx=0).stop 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 shutdown_connections 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f922806cce0 0x7f922806f1a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f92400ffc20 0x7f9240195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 --2- 172.21.15.8:0/2526674921 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9240105bd0 0x7f9240195bd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 >> 172.21.15.8:0/2526674921 conn(0x7f9240075670 msgr2=0x7f92400fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 shutdown_connections 2024-04-12T06:15:00.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.889+0000 7f924650d700 1 -- 172.21.15.8:0/2526674921 wait complete. 2024-04-12T06:15:00.899 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:15:00.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/3018774972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4100ea0 msgr2=0x7fc5c4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:00.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/3018774972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4100ea0 0x7fc5c4101280 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009910 tx=0x7fc5b4009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:00.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/3018774972 shutdown_connections 2024-04-12T06:15:00.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/3018774972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4101850 0x7fc5c41058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/3018774972 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4100ea0 0x7fc5c4101280 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:00.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/3018774972 >> 172.21.15.8:0/3018774972 conn(0x7fc5c40fc750 msgr2=0x7fc5c40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:00.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/3018774972 shutdown_connections 2024-04-12T06:15:00.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/3018774972 wait complete. 2024-04-12T06:15:00.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.997+0000 7fc5c9c1e700 1 Processor -- start 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c9c1e700 1 -- start start 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c9c1e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4100ea0 0x7fc5c419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c9c1e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c9c1e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c419f360 con 0x7fc5c4101850 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c9c1e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c4198810 con 0x7fc5c4100ea0 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36856/0 (socket says 172.21.15.8:36856) 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 -- 172.21.15.8:0/39261543 learned_addr learned my addr 172.21.15.8:0/39261543 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c37fe700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4100ea0 0x7fc5c419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:00.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 -- 172.21.15.8:0/39261543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4100ea0 msgr2=0x7fc5c419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4100ea0 0x7fc5c419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 -- 172.21.15.8:0/39261543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40095f0 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.998+0000 7fc5c2ffd700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800c8f0 tx=0x7fc5b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.999+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5b8007a50 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.999+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc5b8007bb0 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.999+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c4198af0 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.999+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5b8018630 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:00.999+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5c4199040 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.000+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5c4105020 con 0x7fc5c4101850 2024-04-12T06:15:01.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.000+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc5b8004830 con 0x7fc5c4101850 2024-04-12T06:15:01.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.000+0000 7fc5c0ff9700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 0x7fc5ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.000+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc5b808c470 con 0x7fc5c4101850 2024-04-12T06:15:01.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.001+0000 7fc5c37fe700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 0x7fc5ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.001+0000 7fc5c37fe700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 0x7fc5ac06f1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006010 tx=0x7fc5b401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.005+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc5b805a060 con 0x7fc5c4101850 2024-04-12T06:15:01.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:00 smithi194 ceph-mon[106982]: pgmap v599: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:01.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:00 smithi008 ceph-mon[146084]: pgmap v599: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:01.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.225+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5c4199310 con 0x7fc5ac06cce0 2024-04-12T06:15:01.225 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.226+0000 7fc5c0ff9700 1 -- 172.21.15.8:0/39261543 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc5c4199310 con 0x7fc5ac06cce0 2024-04-12T06:15:01.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 msgr2=0x7fc5ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 0x7fc5ac06f1a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4006010 tx=0x7fc5b401a040 comp rx=0 tx=0).stop 2024-04-12T06:15:01.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 msgr2=0x7fc5c419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800c8f0 tx=0x7fc5b800ccb0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 shutdown_connections 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc5ac06cce0 0x7fc5ac06f1a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc5c4100ea0 0x7fc5c419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 --2- 172.21.15.8:0/39261543 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc5c4101850 0x7fc5c419ecd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 >> 172.21.15.8:0/39261543 conn(0x7fc5c40fc750 msgr2=0x7fc5c40fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 shutdown_connections 2024-04-12T06:15:01.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.228+0000 7fc5c9c1e700 1 -- 172.21.15.8:0/39261543 wait complete. 2024-04-12T06:15:01.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.334+0000 7f20fd3a2700 1 -- 172.21.15.8:0/2804471287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f80ffda0 msgr2=0x7f20f8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.334+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/2804471287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f80ffda0 0x7f20f8100180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009910 tx=0x7f20ec009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 -- 172.21.15.8:0/2804471287 shutdown_connections 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/2804471287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f8100750 0x7f20f81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/2804471287 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f80ffda0 0x7f20f8100180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 -- 172.21.15.8:0/2804471287 >> 172.21.15.8:0/2804471287 conn(0x7f20f80fb650 msgr2=0x7f20f80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 -- 172.21.15.8:0/2804471287 shutdown_connections 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 -- 172.21.15.8:0/2804471287 wait complete. 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 Processor -- start 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 -- start start 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.335+0000 7f20fd3a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80ffda0 0x7f20f819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20fd3a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20fd3a2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f819d150 con 0x7f20f8100750 2024-04-12T06:15:01.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20fd3a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8196600 con 0x7f20f80ffda0 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36870/0 (socket says 172.21.15.8:36870) 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 -- 172.21.15.8:0/3541347409 learned_addr learned my addr 172.21.15.8:0/3541347409 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f6ffd700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80ffda0 0x7f20f819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 -- 172.21.15.8:0/3541347409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80ffda0 msgr2=0x7f20f819c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80ffda0 0x7f20f819c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 -- 172.21.15.8:0/3541347409 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0095f0 con 0x7f20f8100750 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20f67fc700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d900 tx=0x7f20e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e8009900 con 0x7f20f8100750 2024-04-12T06:15:01.337 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20e8010460 con 0x7f20f8100750 2024-04-12T06:15:01.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81968e0 con 0x7f20f8100750 2024-04-12T06:15:01.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20e800f5f0 con 0x7f20f8100750 2024-04-12T06:15:01.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.336+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8196e30 con 0x7f20f8100750 2024-04-12T06:15:01.338 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.337+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f804f2e0 con 0x7f20f8100750 2024-04-12T06:15:01.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.338+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f20e80105d0 con 0x7f20f8100750 2024-04-12T06:15:01.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.339+0000 7f20dffff700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 0x7f20e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.339+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f20e808c2f0 con 0x7f20f8100750 2024-04-12T06:15:01.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.339+0000 7f20f6ffd700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 0x7f20e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.339 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.339+0000 7f20f6ffd700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 0x7f20e006f1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006010 tx=0x7f20ec01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.342 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.343+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f20e8059f10 con 0x7f20f8100750 2024-04-12T06:15:01.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.552+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f20f8197210 con 0x7f20e006cce0 2024-04-12T06:15:01.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.562+0000 7f20dffff700 1 -- 172.21.15.8:0/3541347409 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f20f8197210 con 0x7f20e006cce0 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (16m) 10m ago 28m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (28m) 10m ago 28m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (27m) 10m ago 27m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (14m) 10m ago 28m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (14m) 10m ago 27m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (16m) 10m ago 28m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (10m) 10m ago 22m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (10m) 10m ago 22m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:15:01.562 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (10m) 10m ago 22m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (10m) 10m ago 22m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (19m) 10m ago 29m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (18m) 10m ago 27m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (15m) 10m ago 30m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (14m) 10m ago 27m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (17m) 10m ago 28m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (17m) 10m ago 27m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (14m) 10m ago 26m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (13m) 10m ago 25m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (12m) 10m ago 25m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:15:01.563 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (12m) 10m ago 24m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:15:01.564 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (11m) 10m ago 24m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:15:01.564 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (11m) 10m ago 23m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:15:01.564 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (17m) 10m ago 27m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:15:01.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 msgr2=0x7f20e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 0x7f20e006f1a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006010 tx=0x7f20ec01a040 comp rx=0 tx=0).stop 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 msgr2=0x7f20f819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f20e800d900 tx=0x7f20e800dcc0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 shutdown_connections 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f20e006cce0 0x7f20e006f1a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f20f80ffda0 0x7f20f819c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 --2- 172.21.15.8:0/3541347409 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f20f8100750 0x7f20f819cac0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.564+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 >> 172.21.15.8:0/3541347409 conn(0x7f20f80fb650 msgr2=0x7f20f8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:01.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.565+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 shutdown_connections 2024-04-12T06:15:01.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.565+0000 7f20fd3a2700 1 -- 172.21.15.8:0/3541347409 wait complete. 2024-04-12T06:15:01.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 -- 172.21.15.8:0/2357116001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 msgr2=0x7f3a780689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 --2- 172.21.15.8:0/2357116001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780689e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009910 tx=0x7f3a6c009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:01.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 -- 172.21.15.8:0/2357116001 shutdown_connections 2024-04-12T06:15:01.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 --2- 172.21.15.8:0/2357116001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780689e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 --2- 172.21.15.8:0/2357116001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 0x7f3a781070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 -- 172.21.15.8:0/2357116001 >> 172.21.15.8:0/2357116001 conn(0x7f3a78075670 msgr2=0x7f3a78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:01.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 -- 172.21.15.8:0/2357116001 shutdown_connections 2024-04-12T06:15:01.670 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.670+0000 7f3a7d208700 1 -- 172.21.15.8:0/2357116001 wait complete. 2024-04-12T06:15:01.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 Processor -- start 2024-04-12T06:15:01.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 -- start start 2024-04-12T06:15:01.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 0x7f3a78100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a78100770 con 0x7f3a78068580 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7d208700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a781008b0 con 0x7f3a78106cd0 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a76d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a76d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36892/0 (socket says 172.21.15.8:36892) 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a76d9d700 1 -- 172.21.15.8:0/209242241 learned_addr learned my addr 172.21.15.8:0/209242241 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.671+0000 7f3a7659c700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 0x7f3a78100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a76d9d700 1 -- 172.21.15.8:0/209242241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 msgr2=0x7f3a78100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a76d9d700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 0x7f3a78100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a76d9d700 1 -- 172.21.15.8:0/209242241 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0095f0 con 0x7f3a78068580 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a76d9d700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800b4b0 tx=0x7f3a6800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68011800 con 0x7f3a78068580 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a781a2ac0 con 0x7f3a78068580 2024-04-12T06:15:01.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a68011e00 con 0x7f3a78068580 2024-04-12T06:15:01.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a68010e60 con 0x7f3a78068580 2024-04-12T06:15:01.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.672+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a781a2ed0 con 0x7f3a78068580 2024-04-12T06:15:01.673 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.673+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a781091d0 con 0x7f3a78068580 2024-04-12T06:15:01.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.674+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3a68011960 con 0x7f3a78068580 2024-04-12T06:15:01.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.674+0000 7f3a5ffff700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 0x7f3a6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:01.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.674+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3a6808c0a0 con 0x7f3a78068580 2024-04-12T06:15:01.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.674+0000 7f3a7659c700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 0x7f3a6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:01.674 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.675+0000 7f3a7659c700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 0x7f3a6006f1a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a781014e0 tx=0x7f3a6c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:01.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.679+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3a68056110 con 0x7f3a78068580 2024-04-12T06:15:01.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.971+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a78108bb0 con 0x7f3a78068580 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.972+0000 7f3a5ffff700 1 -- 172.21.15.8:0/209242241 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f3a68059730 con 0x7f3a78068580 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:15:01.971 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:15:01.972 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 msgr2=0x7f3a6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 0x7f3a6006f1a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3a781014e0 tx=0x7f3a6c01a040 comp rx=0 tx=0).stop 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 msgr2=0x7f3a780ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800b4b0 tx=0x7f3a6800b7c0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 shutdown_connections 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a6006cce0 0x7f3a6006f1a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a78068580 0x7f3a780ffcf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 --2- 172.21.15.8:0/209242241 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a78106cd0 0x7f3a78100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 >> 172.21.15.8:0/209242241 conn(0x7f3a78075670 msgr2=0x7f3a780fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:01.973 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 shutdown_connections 2024-04-12T06:15:01.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:01.974+0000 7f3a7d208700 1 -- 172.21.15.8:0/209242241 wait complete. 2024-04-12T06:15:02.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:01 smithi194 ceph-mon[106982]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3615752005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 msgr2=0x7f3d781070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3615752005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d781070b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009910 tx=0x7f3d74009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3615752005 shutdown_connections 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3615752005 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 0x7f3d780689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3615752005 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d781070b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3615752005 >> 172.21.15.8:0/3615752005 conn(0x7f3d78075670 msgr2=0x7f3d78075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:02.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3615752005 shutdown_connections 2024-04-12T06:15:02.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.081+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3615752005 wait complete. 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 Processor -- start 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 -- start start 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 0x7f3d7819c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7819d130 con 0x7f3d78106cd0 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7f3cd700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d78196630 con 0x7f3d78068580 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36918/0 (socket says 172.21.15.8:36918) 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 -- 172.21.15.8:0/3596516382 learned_addr learned my addr 172.21.15.8:0/3596516382 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7d169700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 0x7f3d7819c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 -- 172.21.15.8:0/3596516382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 msgr2=0x7f3d7819c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 0x7f3d7819c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.083 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.082+0000 7f3d7c968700 1 -- 172.21.15.8:0/3596516382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740095f0 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d7c968700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800d8d0 tx=0x7f3d6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d68009940 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d68010460 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d6800f5f0 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d78196910 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.083+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d78196e60 con 0x7f3d78106cd0 2024-04-12T06:15:02.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.084+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d781091d0 con 0x7f3d78106cd0 2024-04-12T06:15:02.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.084+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3d680105d0 con 0x7f3d78106cd0 2024-04-12T06:15:02.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.084+0000 7f3d6e7fc700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 0x7f3d6406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.085+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d6808c4c0 con 0x7f3d78106cd0 2024-04-12T06:15:02.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.085+0000 7f3d7d169700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 0x7f3d6406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.085+0000 7f3d7d169700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 0x7f3d6406f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d74006010 tx=0x7f3d7401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.089+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3d6805a040 con 0x7f3d78106cd0 2024-04-12T06:15:02.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:01 smithi008 ceph-mon[146084]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:02.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.337+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d7804f2e0 con 0x7f3d78106cd0 2024-04-12T06:15:02.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.337+0000 7f3d6e7fc700 1 -- 172.21.15.8:0/3596516382 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f3d68010880 con 0x7f3d78106cd0 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:15:02.337 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:15:02.338 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:02.339 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:02.340 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:02.340 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:02.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 msgr2=0x7f3d6406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 0x7f3d6406f1a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3d74006010 tx=0x7f3d7401a040 comp rx=0 tx=0).stop 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 msgr2=0x7f3d7819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800d8d0 tx=0x7f3d6800dc90 comp rx=0 tx=0).stop 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 shutdown_connections 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d6406cce0 0x7f3d6406f1a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d78068580 0x7f3d7819c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 --2- 172.21.15.8:0/3596516382 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d78106cd0 0x7f3d7819caa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 >> 172.21.15.8:0/3596516382 conn(0x7f3d78075670 msgr2=0x7f3d780fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 shutdown_connections 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.339+0000 7f3d7f3cd700 1 -- 172.21.15.8:0/3596516382 wait complete. 2024-04-12T06:15:02.341 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:15:02.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 -- 172.21.15.8:0/2482980962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 msgr2=0x7fa74c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 --2- 172.21.15.8:0/2482980962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c0689e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa740009910 tx=0x7fa740009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 -- 172.21.15.8:0/2482980962 shutdown_connections 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 --2- 172.21.15.8:0/2482980962 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c0689e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 --2- 172.21.15.8:0/2482980962 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 0x7fa74c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 -- 172.21.15.8:0/2482980962 >> 172.21.15.8:0/2482980962 conn(0x7fa74c075670 msgr2=0x7fa74c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 -- 172.21.15.8:0/2482980962 shutdown_connections 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.448+0000 7fa751e15700 1 -- 172.21.15.8:0/2482980962 wait complete. 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 Processor -- start 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 -- start start 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 0x7fa74c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c19d080 con 0x7fa74c068580 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.449+0000 7fa751e15700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c1965c0 con 0x7fa74c106cd0 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36932/0 (socket says 172.21.15.8:36932) 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 -- 172.21.15.8:0/611888771 learned_addr learned my addr 172.21.15.8:0/611888771 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74affd700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 0x7fa74c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 -- 172.21.15.8:0/611888771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 msgr2=0x7fa74c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 0x7fa74c19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 -- 172.21.15.8:0/611888771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400095f0 con 0x7fa74c068580 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa74b7fe700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c930 tx=0x7fa73c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c007ad0 con 0x7fa74c068580 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa73c007c30 con 0x7fa74c068580 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c1968a0 con 0x7fa74c068580 2024-04-12T06:15:02.451 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa73c018630 con 0x7fa74c068580 2024-04-12T06:15:02.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.450+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c196df0 con 0x7fa74c068580 2024-04-12T06:15:02.452 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.451+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c1091d0 con 0x7fa74c068580 2024-04-12T06:15:02.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.452+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa73c00f450 con 0x7fa74c068580 2024-04-12T06:15:02.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.452+0000 7fa748ff9700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 0x7fa73406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.453+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa73c08c930 con 0x7fa74c068580 2024-04-12T06:15:02.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.453+0000 7fa74affd700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 0x7fa73406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.453+0000 7fa74affd700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 0x7fa73406f1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0fe260 tx=0x7fa74001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.455 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.456+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa73c05a490 con 0x7fa74c068580 2024-04-12T06:15:02.678 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.678+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa74c197070 con 0x7fa73406cce0 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.680+0000 7fa748ff9700 1 -- 172.21.15.8:0/611888771 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa74c197070 con 0x7fa73406cce0 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:15:02.679 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:15:02.680 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:15:02.680 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:15:02.680 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:15:02.680 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:15:02.680 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 msgr2=0x7fa73406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 0x7fa73406f1a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0fe260 tx=0x7fa74001a040 comp rx=0 tx=0).stop 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 msgr2=0x7fa74c19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00c930 tx=0x7fa73c00ccf0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 shutdown_connections 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa73406cce0 0x7fa73406f1a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.681 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa74c068580 0x7fa74c19c540 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 --2- 172.21.15.8:0/611888771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa74c106cd0 0x7fa74c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 >> 172.21.15.8:0/611888771 conn(0x7fa74c075670 msgr2=0x7fa74c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 shutdown_connections 2024-04-12T06:15:02.682 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.682+0000 7fa751e15700 1 -- 172.21.15.8:0/611888771 wait complete. 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 -- 172.21.15.8:0/3895544150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 msgr2=0x7f66e8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 --2- 172.21.15.8:0/3895544150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e8100080 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f66e4009910 tx=0x7f66e4009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 -- 172.21.15.8:0/3895544150 shutdown_connections 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 --2- 172.21.15.8:0/3895544150 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e8100080 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 --2- 172.21.15.8:0/3895544150 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 0x7f66e8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 -- 172.21.15.8:0/3895544150 >> 172.21.15.8:0/3895544150 conn(0x7f66e8075670 msgr2=0x7f66e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 -- 172.21.15.8:0/3895544150 shutdown_connections 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.788+0000 7f66ef516700 1 -- 172.21.15.8:0/3895544150 wait complete. 2024-04-12T06:15:02.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 Processor -- start 2024-04-12T06:15:02.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 -- start start 2024-04-12T06:15:02.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 0x7f66e819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e819d150 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ef516700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e8196600 con 0x7f66e8105bd0 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36950/0 (socket says 172.21.15.8:36950) 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 -- 172.21.15.8:0/2687891510 learned_addr learned my addr 172.21.15.8:0/2687891510 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ecab1700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 0x7f66e819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 -- 172.21.15.8:0/2687891510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 msgr2=0x7f66e819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 0x7f66e819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.789+0000 7f66ed2b2700 1 -- 172.21.15.8:0/2687891510 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40095f0 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66ed2b2700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f66d800d8d0 tx=0x7f66d800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d80098a0 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66d8010460 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d8009ea0 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e81968e0 con 0x7f66e80ffc20 2024-04-12T06:15:02.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.790+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e8196e30 con 0x7f66e80ffc20 2024-04-12T06:15:02.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.791+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e804f2e0 con 0x7f66e80ffc20 2024-04-12T06:15:02.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.791+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f66d80105d0 con 0x7f66e80ffc20 2024-04-12T06:15:02.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.792+0000 7f66de7fc700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 0x7f66d406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:02.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.792+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f66d808c370 con 0x7f66e80ffc20 2024-04-12T06:15:02.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.792+0000 7f66ecab1700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 0x7f66d406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:02.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.792+0000 7f66ecab1700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 0x7f66d406f1a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f66e8075de0 tx=0x7f66e401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:02.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:02.796+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f66d8059ef0 con 0x7f66e80ffc20 2024-04-12T06:15:03.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: pgmap v600: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/209242241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:15:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3596516382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:15:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:03.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.079+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66e8197140 con 0x7f66e80ffc20 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.080+0000 7f66de7fc700 1 -- 172.21.15.8:0/2687891510 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f66d8010880 con 0x7f66e80ffc20 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:15:03.079 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 msgr2=0x7f66d406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 0x7f66d406f1a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f66e8075de0 tx=0x7f66e401a040 comp rx=0 tx=0).stop 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 msgr2=0x7f66e819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f66d800d8d0 tx=0x7f66d800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 shutdown_connections 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f66d406cce0 0x7f66d406f1a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f66e80ffc20 0x7f66e819c580 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 --2- 172.21.15.8:0/2687891510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f66e8105bd0 0x7f66e819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 >> 172.21.15.8:0/2687891510 conn(0x7f66e8075670 msgr2=0x7f66e8068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 shutdown_connections 2024-04-12T06:15:03.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:03.082+0000 7f66ef516700 1 -- 172.21.15.8:0/2687891510 wait complete. 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: pgmap v600: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/209242241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3596516382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:15:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:04.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:03 smithi194 ceph-mon[106982]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:04.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:03 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2687891510' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:15:04.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:03 smithi008 ceph-mon[146084]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:04.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:03 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2687891510' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:15:05.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:04 smithi194 ceph-mon[106982]: pgmap v601: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:05.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:04 smithi008 ceph-mon[146084]: pgmap v601: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:07.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:06 smithi194 ceph-mon[106982]: pgmap v602: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:06 smithi008 ceph-mon[146084]: pgmap v602: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:09.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:08 smithi194 ceph-mon[106982]: pgmap v603: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:09.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:08 smithi008 ceph-mon[146084]: pgmap v603: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:11.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:10 smithi194 ceph-mon[106982]: pgmap v604: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:11.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:10 smithi008 ceph-mon[146084]: pgmap v604: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:13.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:12 smithi194 ceph-mon[106982]: pgmap v605: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:13.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:12 smithi008 ceph-mon[146084]: pgmap v605: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:15.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:14 smithi194 ceph-mon[106982]: pgmap v606: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:15.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:14 smithi008 ceph-mon[146084]: pgmap v606: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:17.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:16 smithi194 ceph-mon[106982]: pgmap v607: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:17.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:16 smithi008 ceph-mon[146084]: pgmap v607: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:18.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:18.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:19.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:18 smithi008 ceph-mon[146084]: pgmap v608: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:19.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:18 smithi194 ceph-mon[106982]: pgmap v608: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:21.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:20 smithi008 ceph-mon[146084]: pgmap v609: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:21.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:20 smithi194 ceph-mon[106982]: pgmap v609: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:23.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:22 smithi008 ceph-mon[146084]: pgmap v610: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:23.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:22 smithi194 ceph-mon[106982]: pgmap v610: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:25.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:24 smithi194 ceph-mon[106982]: pgmap v611: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:25.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:24 smithi008 ceph-mon[146084]: pgmap v611: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:27.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:26 smithi008 ceph-mon[146084]: pgmap v612: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:27.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:26 smithi194 ceph-mon[106982]: pgmap v612: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:29.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:28 smithi008 ceph-mon[146084]: pgmap v613: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:29.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:28 smithi194 ceph-mon[106982]: pgmap v613: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:31.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:30 smithi008 ceph-mon[146084]: pgmap v614: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:31.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:30 smithi194 ceph-mon[106982]: pgmap v614: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:32 smithi008 ceph-mon[146084]: pgmap v615: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:33.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 -- 172.21.15.8:0/3748077905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 msgr2=0x7efd4410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 --2- 172.21.15.8:0/3748077905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4410abf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efd38009910 tx=0x7efd38009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 -- 172.21.15.8:0/3748077905 shutdown_connections 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 --2- 172.21.15.8:0/3748077905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4410abf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 --2- 172.21.15.8:0/3748077905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 -- 172.21.15.8:0/3748077905 >> 172.21.15.8:0/3748077905 conn(0x7efd440fb610 msgr2=0x7efd440fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 -- 172.21.15.8:0/3748077905 shutdown_connections 2024-04-12T06:15:33.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.203+0000 7efd4882d700 1 -- 172.21.15.8:0/3748077905 wait complete. 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 Processor -- start 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 -- start start 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd4419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4419d190 con 0x7efd44102700 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4882d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44196640 con 0x7efd44101de0 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd41d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd41d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37154/0 (socket says 172.21.15.8:37154) 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd41d9b700 1 -- 172.21.15.8:0/3715947479 learned_addr learned my addr 172.21.15.8:0/3715947479 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.204+0000 7efd4259c700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd4419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd41d9b700 1 -- 172.21.15.8:0/3715947479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 msgr2=0x7efd4419c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd41d9b700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd4419c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd41d9b700 1 -- 172.21.15.8:0/3715947479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380095f0 con 0x7efd44102700 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd4259c700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd4419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd41d9b700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efd380092e0 tx=0x7efd3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd3801d070 con 0x7efd44102700 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd441968c0 con 0x7efd44102700 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7efd38005dc0 con 0x7efd44102700 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd38021410 con 0x7efd44102700 2024-04-12T06:15:33.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.205+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd44196db0 con 0x7efd44102700 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.206+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd441082f0 con 0x7efd44102700 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.207+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7efd3800f6b0 con 0x7efd44102700 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.207+0000 7efd337fe700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 0x7efd2c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.207+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efd3808d7c0 con 0x7efd44102700 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.207+0000 7efd4259c700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 0x7efd2c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.208+0000 7efd4259c700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 0x7efd2c06f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd34005950 tx=0x7efd34005f50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.212+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7efd3805b2c0 con 0x7efd44102700 2024-04-12T06:15:33.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:32 smithi194 ceph-mon[106982]: pgmap v615: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:33.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:33.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.434+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd44196fd0 con 0x7efd2c06cce0 2024-04-12T06:15:33.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.436+0000 7efd337fe700 1 -- 172.21.15.8:0/3715947479 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efd44196fd0 con 0x7efd2c06cce0 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 msgr2=0x7efd2c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 0x7efd2c06f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd34005950 tx=0x7efd34005f50 comp rx=0 tx=0).stop 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 msgr2=0x7efd4419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7efd380092e0 tx=0x7efd3800be20 comp rx=0 tx=0).stop 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 shutdown_connections 2024-04-12T06:15:33.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7efd2c06cce0 0x7efd2c06f1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efd44101de0 0x7efd4419c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 --2- 172.21.15.8:0/3715947479 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7efd44102700 0x7efd4419cb00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 >> 172.21.15.8:0/3715947479 conn(0x7efd440fb610 msgr2=0x7efd44105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:33.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 shutdown_connections 2024-04-12T06:15:33.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.438+0000 7efd4882d700 1 -- 172.21.15.8:0/3715947479 wait complete. 2024-04-12T06:15:33.448 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:15:33.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 -- 172.21.15.8:0/1453545165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 msgr2=0x7feea810d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 --2- 172.21.15.8:0/1453545165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea810d290 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feea4009910 tx=0x7feea4009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:33.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 -- 172.21.15.8:0/1453545165 shutdown_connections 2024-04-12T06:15:33.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 --2- 172.21.15.8:0/1453545165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea810d290 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 --2- 172.21.15.8:0/1453545165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 0x7feea8068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 -- 172.21.15.8:0/1453545165 >> 172.21.15.8:0/1453545165 conn(0x7feea8075670 msgr2=0x7feea8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:33.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 -- 172.21.15.8:0/1453545165 shutdown_connections 2024-04-12T06:15:33.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.548+0000 7feeb04e8700 1 -- 172.21.15.8:0/1453545165 wait complete. 2024-04-12T06:15:33.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 Processor -- start 2024-04-12T06:15:33.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 -- start start 2024-04-12T06:15:33.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 0x7feea819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea819f450 con 0x7feea8068ea0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeb04e8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea8198900 con 0x7feea8068580 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41058/0 (socket says 172.21.15.8:41058) 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 -- 172.21.15.8:0/314355593 learned_addr learned my addr 172.21.15.8:0/314355593 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeae284700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 0x7feea819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 -- 172.21.15.8:0/314355593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 msgr2=0x7feea819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 0x7feea819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 -- 172.21.15.8:0/314355593 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40095f0 con 0x7feea8068ea0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.549+0000 7feeada83700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feea4006010 tx=0x7feea400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea401d070 con 0x7feea8068ea0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feea400f460 con 0x7feea8068ea0 2024-04-12T06:15:33.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea8198b80 con 0x7feea8068ea0 2024-04-12T06:15:33.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea4021640 con 0x7feea8068ea0 2024-04-12T06:15:33.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea8199070 con 0x7feea8068ea0 2024-04-12T06:15:33.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.550+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea810aa00 con 0x7feea8068ea0 2024-04-12T06:15:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.551+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7feea400f9e0 con 0x7feea8068ea0 2024-04-12T06:15:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.551+0000 7fee9b7fe700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 0x7fee9406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.552+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feea408d740 con 0x7feea8068ea0 2024-04-12T06:15:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.552+0000 7feeae284700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 0x7fee9406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.552+0000 7feeae284700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 0x7fee9406f1a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009a10 tx=0x7fee9c009190 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.555 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.556+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7feea405b1e0 con 0x7feea8068ea0 2024-04-12T06:15:33.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.779+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feea8199290 con 0x7fee9406cce0 2024-04-12T06:15:33.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.780+0000 7fee9b7fe700 1 -- 172.21.15.8:0/314355593 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feea8199290 con 0x7fee9406cce0 2024-04-12T06:15:33.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 msgr2=0x7fee9406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 0x7fee9406f1a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009a10 tx=0x7fee9c009190 comp rx=0 tx=0).stop 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 msgr2=0x7feea819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feea4006010 tx=0x7feea400bd70 comp rx=0 tx=0).stop 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 shutdown_connections 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fee9406cce0 0x7fee9406f1a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feea8068580 0x7feea819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 --2- 172.21.15.8:0/314355593 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7feea8068ea0 0x7feea819edc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 >> 172.21.15.8:0/314355593 conn(0x7feea8075670 msgr2=0x7feea80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 shutdown_connections 2024-04-12T06:15:33.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.782+0000 7feeb04e8700 1 -- 172.21.15.8:0/314355593 wait complete. 2024-04-12T06:15:33.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2495956177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d74105bd0 msgr2=0x7f4d74105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2495956177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d74105bd0 0x7f4d74105fb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009910 tx=0x7f4d64009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2495956177 shutdown_connections 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2495956177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d740ffc20 0x7f4d74100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2495956177 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d74105bd0 0x7f4d74105fb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2495956177 >> 172.21.15.8:0/2495956177 conn(0x7f4d74075670 msgr2=0x7f4d74075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2495956177 shutdown_connections 2024-04-12T06:15:33.889 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.889+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2495956177 wait complete. 2024-04-12T06:15:33.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 Processor -- start 2024-04-12T06:15:33.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 -- start start 2024-04-12T06:15:33.892 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 0x7f4d74101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d741022c0 con 0x7f4d740ffc20 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d78bbf700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d74102400 con 0x7f4d74105bd0 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41076/0 (socket says 172.21.15.8:41076) 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 -- 172.21.15.8:0/2088832764 learned_addr learned my addr 172.21.15.8:0/2088832764 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d71d9b700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 0x7f4d74101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 -- 172.21.15.8:0/2088832764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 msgr2=0x7f4d74101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:33.893 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 0x7f4d74101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d7259c700 1 -- 172.21.15.8:0/2088832764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640095f0 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.890+0000 7f4d71d9b700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 0x7f4d74101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d7259c700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006010 tx=0x7f4d6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d6401d070 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d6400f460 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d74102680 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d64021640 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.891+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d741a2c70 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.892+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7404f2e0 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.892+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4d6400fa20 con 0x7f4d740ffc20 2024-04-12T06:15:33.894 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.893+0000 7f4d6b7fe700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 0x7f4d6006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.893+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4d6408d880 con 0x7f4d740ffc20 2024-04-12T06:15:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.893+0000 7f4d71d9b700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 0x7f4d6006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:33.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.893+0000 7f4d71d9b700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 0x7f4d6006f1a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c005fd0 tx=0x7f4d5c005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:33.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:33.897+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4d640578f0 con 0x7f4d740ffc20 2024-04-12T06:15:34.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.108+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d74068e10 con 0x7f4d6006cce0 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.118+0000 7f4d6b7fe700 1 -- 172.21.15.8:0/2088832764 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4d74068e10 con 0x7f4d6006cce0 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (17m) 11m ago 28m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (29m) 11m ago 29m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (27m) 10m ago 27m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (15m) 11m ago 29m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (15m) 10m ago 27m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:15:34.118 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (17m) 11m ago 28m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (11m) 11m ago 22m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (11m) 11m ago 22m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (10m) 10m ago 22m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (10m) 10m ago 22m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (19m) 11m ago 30m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (18m) 10m ago 27m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (15m) 11m ago 30m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (15m) 10m ago 27m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (18m) 11m ago 28m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (18m) 10m ago 27m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (14m) 11m ago 26m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (14m) 11m ago 26m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (13m) 11m ago 25m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:15:34.119 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (12m) 10m ago 25m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:15:34.120 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (12m) 10m ago 24m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:15:34.120 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (11m) 10m ago 24m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:15:34.120 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (17m) 11m ago 28m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:15:34.120 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.120+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 msgr2=0x7f4d6006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.120+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 0x7f4d6006f1a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c005fd0 tx=0x7f4d5c005e30 comp rx=0 tx=0).stop 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.120+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 msgr2=0x7f4d74101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.120+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006010 tx=0x7f4d6400be20 comp rx=0 tx=0).stop 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 shutdown_connections 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4d6006cce0 0x7f4d6006f1a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4d740ffc20 0x7f4d74101840 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 --2- 172.21.15.8:0/2088832764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4d74105bd0 0x7f4d74101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 >> 172.21.15.8:0/2088832764 conn(0x7f4d74075670 msgr2=0x7f4d74068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 shutdown_connections 2024-04-12T06:15:34.121 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.121+0000 7f4d78bbf700 1 -- 172.21.15.8:0/2088832764 wait complete. 2024-04-12T06:15:34.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.226+0000 7f65db484700 1 -- 172.21.15.8:0/288057930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068ea0 msgr2=0x7f65d410d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.226+0000 7f65db484700 1 --2- 172.21.15.8:0/288057930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068ea0 0x7f65d410d290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009910 tx=0x7f65c4009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- 172.21.15.8:0/288057930 shutdown_connections 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 --2- 172.21.15.8:0/288057930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068ea0 0x7f65d410d290 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 --2- 172.21.15.8:0/288057930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068580 0x7f65d4068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- 172.21.15.8:0/288057930 >> 172.21.15.8:0/288057930 conn(0x7f65d4075670 msgr2=0x7f65d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- 172.21.15.8:0/288057930 shutdown_connections 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- 172.21.15.8:0/288057930 wait complete. 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 Processor -- start 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- start start 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 0x7f65d419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.228 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d419f3d0 con 0x7f65d4068580 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.227+0000 7f65db484700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d4198880 con 0x7f65d4068ea0 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41088/0 (socket says 172.21.15.8:41088) 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 -- 172.21.15.8:0/2546089138 learned_addr learned my addr 172.21.15.8:0/2546089138 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d8a1f700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 0x7f65d419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 -- 172.21.15.8:0/2546089138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 msgr2=0x7f65d419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 0x7f65d419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 -- 172.21.15.8:0/2546089138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40095f0 con 0x7f65d4068580 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d8a1f700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 0x7f65d419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65d9220700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f65d000d8d0 tx=0x7f65d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d0009940 con 0x7f65d4068580 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65d0010460 con 0x7f65d4068580 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d4198b60 con 0x7f65d4068580 2024-04-12T06:15:34.229 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65d000f5f0 con 0x7f65d4068580 2024-04-12T06:15:34.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.228+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d41990b0 con 0x7f65d4068580 2024-04-12T06:15:34.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.229+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d410aa00 con 0x7f65d4068580 2024-04-12T06:15:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.230+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f65d00105d0 con 0x7f65d4068580 2024-04-12T06:15:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.230+0000 7f65ca7fc700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 0x7f65c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.230+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f65d008c4c0 con 0x7f65d4068580 2024-04-12T06:15:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.230+0000 7f65d8a1f700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 0x7f65c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.231+0000 7f65d8a1f700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 0x7f65c006f1a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65d4101400 tx=0x7f65c401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:34.234 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.235+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f65d005a040 con 0x7f65d4068580 2024-04-12T06:15:34.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.525+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f65d404f2e0 con 0x7f65d4068580 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.526+0000 7f65ca7fc700 1 -- 172.21.15.8:0/2546089138 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f65d0010880 con 0x7f65d4068580 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:34.525 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:15:34.526 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 msgr2=0x7f65c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 0x7f65c006f1a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f65d4101400 tx=0x7f65c401a040 comp rx=0 tx=0).stop 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 msgr2=0x7f65d419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f65d000d8d0 tx=0x7f65d000dc90 comp rx=0 tx=0).stop 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 shutdown_connections 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65c006cce0 0x7f65c006f1a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65d4068580 0x7f65d419e800 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 --2- 172.21.15.8:0/2546089138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65d4068ea0 0x7f65d419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 >> 172.21.15.8:0/2546089138 conn(0x7f65d4075670 msgr2=0x7f65d40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:34.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 shutdown_connections 2024-04-12T06:15:34.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.528+0000 7f65db484700 1 -- 172.21.15.8:0/2546089138 wait complete. 2024-04-12T06:15:34.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 -- 172.21.15.8:0/933588878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c100ea0 msgr2=0x7f4b2c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 --2- 172.21.15.8:0/933588878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c101280 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009910 tx=0x7f4b20009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 -- 172.21.15.8:0/933588878 shutdown_connections 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 --2- 172.21.15.8:0/933588878 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c101850 0x7f4b2c1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 --2- 172.21.15.8:0/933588878 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c101280 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 -- 172.21.15.8:0/933588878 >> 172.21.15.8:0/933588878 conn(0x7f4b2c0fc750 msgr2=0x7f4b2c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 -- 172.21.15.8:0/933588878 shutdown_connections 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.633+0000 7f4b32aea700 1 -- 172.21.15.8:0/933588878 wait complete. 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 Processor -- start 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 -- start start 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c19d130 con 0x7f4b2c101850 2024-04-12T06:15:34.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b32aea700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c1965e0 con 0x7f4b2c100ea0 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b2bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41114/0 (socket says 172.21.15.8:41114) 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b2bfff700 1 -- 172.21.15.8:0/816935116 learned_addr learned my addr 172.21.15.8:0/816935116 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b30886700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b2bfff700 1 -- 172.21.15.8:0/816935116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c100ea0 msgr2=0x7f4b2c19c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.634+0000 7f4b2bfff700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c19c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b2bfff700 1 -- 172.21.15.8:0/816935116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200095f0 con 0x7f4b2c101850 2024-04-12T06:15:34.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b2bfff700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800d8d0 tx=0x7f4b1800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b180041f0 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c1968c0 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b18007620 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b18005610 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.635+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c196e10 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.636+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c109240 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.637+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4b18004750 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.637+0000 7f4b29ffb700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 0x7f4b1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.637+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b1808c480 con 0x7f4b2c101850 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.637+0000 7f4b30886700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 0x7f4b1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:34.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.637+0000 7f4b30886700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 0x7f4b1c06f1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b2001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:34.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.641+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4b18059f20 con 0x7f4b2c101850 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.891+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b2c04f2e0 con 0x7f4b2c101850 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.892+0000 7f4b29ffb700 1 -- 172.21.15.8:0/816935116 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4b18059ab0 con 0x7f4b2c101850 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:15:34.891 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:15:34.892 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:15:34.893 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:34.894 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:15:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 msgr2=0x7f4b1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 0x7f4b1c06f1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b2001a040 comp rx=0 tx=0).stop 2024-04-12T06:15:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 msgr2=0x7f4b2c19caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:34.895 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800d8d0 tx=0x7f4b1800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 shutdown_connections 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4b1c06cce0 0x7f4b1c06f1a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b2c100ea0 0x7f4b2c19c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 --2- 172.21.15.8:0/816935116 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4b2c101850 0x7f4b2c19caa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 >> 172.21.15.8:0/816935116 conn(0x7f4b2c0fc750 msgr2=0x7f4b2c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 shutdown_connections 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:34.894+0000 7f4b32aea700 1 -- 172.21.15.8:0/816935116 wait complete. 2024-04-12T06:15:34.896 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:15:35.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 -- 172.21.15.8:0/3345060816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 msgr2=0x7f809c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 --2- 172.21.15.8:0/3345060816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c105fb0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f808c009910 tx=0x7f808c009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 -- 172.21.15.8:0/3345060816 shutdown_connections 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 --2- 172.21.15.8:0/3345060816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 --2- 172.21.15.8:0/3345060816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c105fb0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 -- 172.21.15.8:0/3345060816 >> 172.21.15.8:0/3345060816 conn(0x7f809c075670 msgr2=0x7f809c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 -- 172.21.15.8:0/3345060816 shutdown_connections 2024-04-12T06:15:35.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.001+0000 7f80a148d700 1 -- 172.21.15.8:0/3345060816 wait complete. 2024-04-12T06:15:35.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 Processor -- start 2024-04-12T06:15:35.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 -- start start 2024-04-12T06:15:35.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c19d160 con 0x7f809c105bd0 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f80a148d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c196610 con 0x7f809c0ffc20 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41128/0 (socket says 172.21.15.8:41128) 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 -- 172.21.15.8:0/2891422464 learned_addr learned my addr 172.21.15.8:0/2891422464 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f809affd700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 -- 172.21.15.8:0/2891422464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 msgr2=0x7f809c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f8093fff700 1 -- 172.21.15.8:0/2891422464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0095f0 con 0x7f809c105bd0 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.002+0000 7f809affd700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:15:35.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f8093fff700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8d0 tx=0x7f808400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8084009940 con 0x7f809c105bd0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c1968f0 con 0x7f809c105bd0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8084010460 con 0x7f809c105bd0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808400f5f0 con 0x7f809c105bd0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.003+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809c196e40 con 0x7f809c105bd0 2024-04-12T06:15:35.005 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.004+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809c04f2e0 con 0x7f809c105bd0 2024-04-12T06:15:35.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.004+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f80840105d0 con 0x7f809c105bd0 2024-04-12T06:15:35.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.005+0000 7f8098ff9700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 0x7f808806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.005+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f808408c5d0 con 0x7f809c105bd0 2024-04-12T06:15:35.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.005+0000 7f809affd700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 0x7f808806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.005+0000 7f809affd700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 0x7f808806f1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b5c0 tx=0x7f808c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:35.008 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.009+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f808405a0d0 con 0x7f809c105bd0 2024-04-12T06:15:35.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:34 smithi194 ceph-mon[106982]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:35.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:34 smithi194 ceph-mon[106982]: pgmap v616: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:35.073 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:34 smithi194 ceph-mon[106982]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:35.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:34 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2546089138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:15:35.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:34 smithi008 ceph-mon[146084]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:35.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:34 smithi008 ceph-mon[146084]: pgmap v616: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:35.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:34 smithi008 ceph-mon[146084]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:35.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:34 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2546089138' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:15:35.230 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.231+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f809c1970f0 con 0x7f808806cce0 2024-04-12T06:15:35.231 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.232+0000 7f8098ff9700 1 -- 172.21.15.8:0/2891422464 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f809c1970f0 con 0x7f808806cce0 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:15:35.232 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:15:35.233 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:15:35.234 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:15:35.234 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:15:35.234 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:15:35.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 msgr2=0x7f808806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.235 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 0x7f808806f1a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b5c0 tx=0x7f808c005960 comp rx=0 tx=0).stop 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 msgr2=0x7f809c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f808400d8d0 tx=0x7f808400dc90 comp rx=0 tx=0).stop 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 shutdown_connections 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f808806cce0 0x7f808806f1a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f809c0ffc20 0x7f809c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 --2- 172.21.15.8:0/2891422464 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f809c105bd0 0x7f809c19cad0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 >> 172.21.15.8:0/2891422464 conn(0x7f809c075670 msgr2=0x7f809c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:35.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 shutdown_connections 2024-04-12T06:15:35.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.234+0000 7f80a148d700 1 -- 172.21.15.8:0/2891422464 wait complete. 2024-04-12T06:15:35.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 -- 172.21.15.8:0/327822029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c101850 msgr2=0x7fc96c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 --2- 172.21.15.8:0/327822029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c101850 0x7fc96c1058a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc960009910 tx=0x7fc960009c20 comp rx=0 tx=0).stop 2024-04-12T06:15:35.340 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 -- 172.21.15.8:0/327822029 shutdown_connections 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 --2- 172.21.15.8:0/327822029 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c101850 0x7fc96c1058a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 --2- 172.21.15.8:0/327822029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c100ea0 0x7fc96c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 -- 172.21.15.8:0/327822029 >> 172.21.15.8:0/327822029 conn(0x7fc96c0fc750 msgr2=0x7fc96c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 -- 172.21.15.8:0/327822029 shutdown_connections 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.340+0000 7fc97268d700 1 -- 172.21.15.8:0/327822029 wait complete. 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.341+0000 7fc97268d700 1 Processor -- start 2024-04-12T06:15:35.341 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.341+0000 7fc97268d700 1 -- start start 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc97268d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc97268d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 0x7fc96c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc97268d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c19d1f0 con 0x7fc96c100ea0 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc97268d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1966a0 con 0x7fc96c101850 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:41148/0 (socket says 172.21.15.8:41148) 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 -- 172.21.15.8:0/572733935 learned_addr learned my addr 172.21.15.8:0/572733935 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96b7fe700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 0x7fc96c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 -- 172.21.15.8:0/572733935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 msgr2=0x7fc96c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 0x7fc96c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 -- 172.21.15.8:0/572733935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600095f0 con 0x7fc96c100ea0 2024-04-12T06:15:35.343 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96b7fe700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 0x7fc96c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc96bfff700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00e920 tx=0x7fc95c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.342+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc95c009790 con 0x7fc96c100ea0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.343+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c196980 con 0x7fc96c100ea0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.343+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc95c00cba0 con 0x7fc96c100ea0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.343+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc95c018650 con 0x7fc96c100ea0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.343+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c196ed0 con 0x7fc96c100ea0 2024-04-12T06:15:35.344 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.344+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c1092a0 con 0x7fc96c100ea0 2024-04-12T06:15:35.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.344+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc95c00cd10 con 0x7fc96c100ea0 2024-04-12T06:15:35.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.345+0000 7fc9697fa700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 0x7fc95406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:15:35.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.345+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc95c014070 con 0x7fc96c100ea0 2024-04-12T06:15:35.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.345+0000 7fc96b7fe700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 0x7fc95406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:15:35.345 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.345+0000 7fc96b7fe700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 0x7fc95406f1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0fcfc0 tx=0x7fc960005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:15:35.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.349+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc95c052ab0 con 0x7fc96c100ea0 2024-04-12T06:15:35.633 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.634+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc96c04f2e0 con 0x7fc96c100ea0 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.634+0000 7fc9697fa700 1 -- 172.21.15.8:0/572733935 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc95c05a1b0 con 0x7fc96c100ea0 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:15:35.634 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.636+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 msgr2=0x7fc95406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.636+0000 7fc97268d700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 0x7fc95406f1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0fcfc0 tx=0x7fc960005fd0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.636+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 msgr2=0x7fc96c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.636+0000 7fc97268d700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00e920 tx=0x7fc95c00ece0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 shutdown_connections 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc95406cce0 0x7fc95406f1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc96c100ea0 0x7fc96c19c620 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 --2- 172.21.15.8:0/572733935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc96c101850 0x7fc96c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 >> 172.21.15.8:0/572733935 conn(0x7fc96c0fc750 msgr2=0x7fc96c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 shutdown_connections 2024-04-12T06:15:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:15:35.637+0000 7fc97268d700 1 -- 172.21.15.8:0/572733935 wait complete. 2024-04-12T06:15:36.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:35 smithi008 ceph-mon[146084]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:36.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:35 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/816935116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:15:36.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:35 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/572733935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:15:36.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:35 smithi194 ceph-mon[106982]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:36.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:35 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/816935116' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:15:36.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:35 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/572733935' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:15:37.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:36 smithi008 ceph-mon[146084]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:37.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:36 smithi008 ceph-mon[146084]: pgmap v617: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:37.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:36 smithi194 ceph-mon[106982]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:15:37.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:36 smithi194 ceph-mon[106982]: pgmap v617: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:39.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:38 smithi008 ceph-mon[146084]: pgmap v618: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:39.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:38 smithi194 ceph-mon[106982]: pgmap v618: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:41.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:40 smithi008 ceph-mon[146084]: pgmap v619: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:40 smithi194 ceph-mon[106982]: pgmap v619: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:43.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:42 smithi008 ceph-mon[146084]: pgmap v620: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:43.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:42 smithi194 ceph-mon[106982]: pgmap v620: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:45.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:44 smithi008 ceph-mon[146084]: pgmap v621: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:45.323 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:44 smithi194 ceph-mon[106982]: pgmap v621: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:47.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:46 smithi008 ceph-mon[146084]: pgmap v622: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:46 smithi194 ceph-mon[106982]: pgmap v622: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:48.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:48.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:15:49.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:48 smithi008 ceph-mon[146084]: pgmap v623: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:49.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:48 smithi194 ceph-mon[106982]: pgmap v623: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:51.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:50 smithi008 ceph-mon[146084]: pgmap v624: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:51.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:50 smithi194 ceph-mon[106982]: pgmap v624: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:53.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:52 smithi008 ceph-mon[146084]: pgmap v625: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:53.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:52 smithi194 ceph-mon[106982]: pgmap v625: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:55.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:54 smithi008 ceph-mon[146084]: pgmap v626: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:54 smithi194 ceph-mon[106982]: pgmap v626: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:56 smithi008 ceph-mon[146084]: pgmap v627: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:57.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:56 smithi194 ceph-mon[106982]: pgmap v627: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:15:58 smithi008 ceph-mon[146084]: pgmap v628: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:15:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:15:58 smithi194 ceph-mon[106982]: pgmap v628: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:01.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:00 smithi008 ceph-mon[146084]: pgmap v629: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:01.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:00 smithi194 ceph-mon[106982]: pgmap v629: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:02 smithi008 ceph-mon[146084]: pgmap v630: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:03.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:02 smithi194 ceph-mon[106982]: pgmap v630: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:05.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:04 smithi008 ceph-mon[146084]: pgmap v631: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:05.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:04 smithi194 ceph-mon[106982]: pgmap v631: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:05.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 -- 172.21.15.8:0/2304460740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf140ffda0 msgr2=0x7fcf14100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 --2- 172.21.15.8:0/2304460740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf140ffda0 0x7fcf14100180 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009910 tx=0x7fcf04009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 -- 172.21.15.8:0/2304460740 shutdown_connections 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 --2- 172.21.15.8:0/2304460740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf14100750 0x7fcf141047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 --2- 172.21.15.8:0/2304460740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf140ffda0 0x7fcf14100180 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 -- 172.21.15.8:0/2304460740 >> 172.21.15.8:0/2304460740 conn(0x7fcf140fb650 msgr2=0x7fcf140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 -- 172.21.15.8:0/2304460740 shutdown_connections 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 -- 172.21.15.8:0/2304460740 wait complete. 2024-04-12T06:16:05.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.758+0000 7fcf19531700 1 Processor -- start 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf19531700 1 -- start start 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf19531700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf140ffda0 0x7fcf141956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf19531700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf19531700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf141962a0 con 0x7fcf14100750 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf19531700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf14198fd0 con 0x7fcf140ffda0 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39826/0 (socket says 172.21.15.8:39826) 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 -- 172.21.15.8:0/1383553796 learned_addr learned my addr 172.21.15.8:0/1383553796 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf12ffd700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf140ffda0 0x7fcf141956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 -- 172.21.15.8:0/1383553796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf140ffda0 msgr2=0x7fcf141956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf140ffda0 0x7fcf141956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.759+0000 7fcf127fc700 1 -- 172.21.15.8:0/1383553796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf040095f0 con 0x7fcf14100750 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcf127fc700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d900 tx=0x7fcf0800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf08009900 con 0x7fcf14100750 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf141992b0 con 0x7fcf14100750 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcf08010460 con 0x7fcf14100750 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf0800f5f0 con 0x7fcf14100750 2024-04-12T06:16:05.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.760+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf14199800 con 0x7fcf14100750 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.761+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf14108140 con 0x7fcf14100750 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.762+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fcf080105d0 con 0x7fcf14100750 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.762+0000 7fcefbfff700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 0x7fcefc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.762+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf0808c080 con 0x7fcf14100750 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.762+0000 7fcf12ffd700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 0x7fcefc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:05.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.763+0000 7fcf12ffd700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 0x7fcefc06f1a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b5c0 tx=0x7fcf0401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:05.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.767+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fcf080560f0 con 0x7fcf14100750 2024-04-12T06:16:05.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.988+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf14196b10 con 0x7fcefc06cce0 2024-04-12T06:16:05.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.989+0000 7fcefbfff700 1 -- 172.21.15.8:0/1383553796 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcf14196b10 con 0x7fcefc06cce0 2024-04-12T06:16:05.990 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 msgr2=0x7fcefc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 0x7fcefc06f1a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b5c0 tx=0x7fcf0401a040 comp rx=0 tx=0).stop 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 msgr2=0x7fcf14195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d900 tx=0x7fcf0800dc10 comp rx=0 tx=0).stop 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 shutdown_connections 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcefc06cce0 0x7fcefc06f1a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf140ffda0 0x7fcf141956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 --2- 172.21.15.8:0/1383553796 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcf14100750 0x7fcf14195c10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 >> 172.21.15.8:0/1383553796 conn(0x7fcf140fb650 msgr2=0x7fcf140fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 shutdown_connections 2024-04-12T06:16:05.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:05.991+0000 7fcf19531700 1 -- 172.21.15.8:0/1383553796 wait complete. 2024-04-12T06:16:06.001 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:16:06.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 -- 172.21.15.8:0/1624311527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df40ffc20 msgr2=0x7f4df4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 --2- 172.21.15.8:0/1624311527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df40ffc20 0x7f4df4100080 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009910 tx=0x7f4de0009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:06.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 -- 172.21.15.8:0/1624311527 shutdown_connections 2024-04-12T06:16:06.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 --2- 172.21.15.8:0/1624311527 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df40ffc20 0x7f4df4100080 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 --2- 172.21.15.8:0/1624311527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df4105bd0 0x7f4df4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 -- 172.21.15.8:0/1624311527 >> 172.21.15.8:0/1624311527 conn(0x7f4df4075670 msgr2=0x7f4df4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 -- 172.21.15.8:0/1624311527 shutdown_connections 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.098+0000 7f4dfb015700 1 -- 172.21.15.8:0/1624311527 wait complete. 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 Processor -- start 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 -- start start 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df40ffc20 0x7f4df4101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df41022c0 con 0x7f4df4105bd0 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4dfb015700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4102400 con 0x7f4df40ffc20 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39846/0 (socket says 172.21.15.8:39846) 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4df3fff700 1 -- 172.21.15.8:0/413886183 learned_addr learned my addr 172.21.15.8:0/413886183 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.099+0000 7f4df8db1700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df40ffc20 0x7f4df4101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df3fff700 1 -- 172.21.15.8:0/413886183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df40ffc20 msgr2=0x7f4df4101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df3fff700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df40ffc20 0x7f4df4101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df3fff700 1 -- 172.21.15.8:0/413886183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00095f0 con 0x7f4df4105bd0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df3fff700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006010 tx=0x7f4de000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de001d070 con 0x7f4df4105bd0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4de000f460 con 0x7f4df4105bd0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de0021640 con 0x7f4df4105bd0 2024-04-12T06:16:06.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4102680 con 0x7f4df4105bd0 2024-04-12T06:16:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.100+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df41a2cb0 con 0x7f4df4105bd0 2024-04-12T06:16:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.101+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df404f2e0 con 0x7f4df4105bd0 2024-04-12T06:16:06.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.102+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4de000f9e0 con 0x7f4df4105bd0 2024-04-12T06:16:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.102+0000 7f4df1ffb700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 0x7f4de406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.102+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4de008d690 con 0x7f4df4105bd0 2024-04-12T06:16:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.102+0000 7f4df8db1700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 0x7f4de406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.103+0000 7f4df8db1700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 0x7f4de406f1f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005950 tx=0x7f4de800a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.107+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4de0057780 con 0x7f4df4105bd0 2024-04-12T06:16:06.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.327+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4df4068e30 con 0x7f4de406cd30 2024-04-12T06:16:06.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.328+0000 7f4df1ffb700 1 -- 172.21.15.8:0/413886183 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4df4068e30 con 0x7f4de406cd30 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 msgr2=0x7f4de406f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 0x7f4de406f1f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4de8005950 tx=0x7f4de800a300 comp rx=0 tx=0).stop 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 msgr2=0x7f4df4101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4de0006010 tx=0x7f4de000bd70 comp rx=0 tx=0).stop 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 shutdown_connections 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4de406cd30 0x7f4de406f1f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4df40ffc20 0x7f4df4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 --2- 172.21.15.8:0/413886183 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4df4105bd0 0x7f4df4101d80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.330+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 >> 172.21.15.8:0/413886183 conn(0x7f4df4075670 msgr2=0x7f4df4068720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:06.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.331+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 shutdown_connections 2024-04-12T06:16:06.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.331+0000 7f4dfb015700 1 -- 172.21.15.8:0/413886183 wait complete. 2024-04-12T06:16:06.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 -- 172.21.15.8:0/787852648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e8101060 msgr2=0x7f84e81047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/787852648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e8101060 0x7f84e81047b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009910 tx=0x7f84d8009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 -- 172.21.15.8:0/787852648 shutdown_connections 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/787852648 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e8101060 0x7f84e81047b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/787852648 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e81006b0 0x7f84e8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 -- 172.21.15.8:0/787852648 >> 172.21.15.8:0/787852648 conn(0x7f84e8074e30 msgr2=0x7f84e8075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 -- 172.21.15.8:0/787852648 shutdown_connections 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 -- 172.21.15.8:0/787852648 wait complete. 2024-04-12T06:16:06.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.437+0000 7f84ed4f9700 1 Processor -- start 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84ed4f9700 1 -- start start 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84ed4f9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84ed4f9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e8101060 0x7f84e819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84ed4f9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e819d120 con 0x7f84e81006b0 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84ed4f9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8196620 con 0x7f84e8101060 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39858/0 (socket says 172.21.15.8:39858) 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 -- 172.21.15.8:0/3644897623 learned_addr learned my addr 172.21.15.8:0/3644897623 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e67fc700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e8101060 0x7f84e819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 -- 172.21.15.8:0/3644897623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e8101060 msgr2=0x7f84e819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e8101060 0x7f84e819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 -- 172.21.15.8:0/3644897623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80095f0 con 0x7f84e81006b0 2024-04-12T06:16:06.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.438+0000 7f84e6ffd700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00d900 tx=0x7f84dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.439+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84dc0041f0 con 0x7f84e81006b0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.439+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84dc004cd0 con 0x7f84e81006b0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.439+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e8196900 con 0x7f84e81006b0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.439+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84dc0056d0 con 0x7f84e81006b0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.439+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e8196e50 con 0x7f84e81006b0 2024-04-12T06:16:06.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.440+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e804f2e0 con 0x7f84e81006b0 2024-04-12T06:16:06.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.440+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f84dc004e40 con 0x7f84e81006b0 2024-04-12T06:16:06.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.440+0000 7f84cffff700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 0x7f84d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.441+0000 7f84e67fc700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 0x7f84d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.441+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84dc08c5c0 con 0x7f84e81006b0 2024-04-12T06:16:06.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.441+0000 7f84e67fc700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 0x7f84d006f1a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84e8106160 tx=0x7f84d801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.445+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f84dc05a140 con 0x7f84e81006b0 2024-04-12T06:16:06.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.658+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84e8197150 con 0x7f84d006cce0 2024-04-12T06:16:06.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.669+0000 7f84cffff700 1 -- 172.21.15.8:0/3644897623 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f84e8197150 con 0x7f84d006cce0 2024-04-12T06:16:06.668 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:16:06.668 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (17m) 11m ago 29m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:16:06.668 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (29m) 11m ago 29m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (28m) 11m ago 28m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (15m) 11m ago 29m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (15m) 11m ago 28m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (17m) 11m ago 29m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (11m) 11m ago 23m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (12m) 11m ago 23m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (11m) 11m ago 23m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (11m) 11m ago 23m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (20m) 11m ago 30m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (19m) 11m ago 28m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (16m) 11m ago 31m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (15m) 11m ago 28m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (18m) 11m ago 29m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:16:06.669 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (18m) 11m ago 28m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (15m) 11m ago 27m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (14m) 11m ago 26m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (14m) 11m ago 26m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (13m) 11m ago 25m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (12m) 11m ago 25m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (12m) 11m ago 24m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:16:06.670 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (18m) 11m ago 28m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 msgr2=0x7f84d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 0x7f84d006f1a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f84e8106160 tx=0x7f84d801a040 comp rx=0 tx=0).stop 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 msgr2=0x7f84e819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00d900 tx=0x7f84dc00dcc0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 shutdown_connections 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84d006cce0 0x7f84d006f1a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84e81006b0 0x7f84e819c550 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 --2- 172.21.15.8:0/3644897623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84e8101060 0x7f84e819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 >> 172.21.15.8:0/3644897623 conn(0x7f84e8074e30 msgr2=0x7f84e8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 shutdown_connections 2024-04-12T06:16:06.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.671+0000 7f84ed4f9700 1 -- 172.21.15.8:0/3644897623 wait complete. 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 -- 172.21.15.8:0/1206163275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c106cd0 msgr2=0x7f586c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 --2- 172.21.15.8:0/1206163275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c106cd0 0x7f586c1070b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5860009910 tx=0x7f5860009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 -- 172.21.15.8:0/1206163275 shutdown_connections 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 --2- 172.21.15.8:0/1206163275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c068580 0x7f586c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 --2- 172.21.15.8:0/1206163275 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c106cd0 0x7f586c1070b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 -- 172.21.15.8:0/1206163275 >> 172.21.15.8:0/1206163275 conn(0x7f586c075670 msgr2=0x7f586c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 -- 172.21.15.8:0/1206163275 shutdown_connections 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.775+0000 7f5872b44700 1 -- 172.21.15.8:0/1206163275 wait complete. 2024-04-12T06:16:06.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 Processor -- start 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 -- start start 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 0x7f586c100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c100770 con 0x7f586c068580 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f5872b44700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1008b0 con 0x7f586c106cd0 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f58708e0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f58708e0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39868/0 (socket says 172.21.15.8:39868) 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.776+0000 7f58708e0700 1 -- 172.21.15.8:0/1057763548 learned_addr learned my addr 172.21.15.8:0/1057763548 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f586bfff700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 0x7f586c100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f58708e0700 1 -- 172.21.15.8:0/1057763548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 msgr2=0x7f586c100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f58708e0700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 0x7f586c100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f58708e0700 1 -- 172.21.15.8:0/1057763548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600095f0 con 0x7f586c068580 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f586bfff700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 0x7f586c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f58708e0700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5860000c00 tx=0x7f586000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f586001d070 con 0x7f586c068580 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5860005dc0 con 0x7f586c068580 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c100b30 con 0x7f586c068580 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58600039b0 con 0x7f586c068580 2024-04-12T06:16:06.779 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.777+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c1a2c30 con 0x7f586c068580 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.778+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c1091f0 con 0x7f586c068580 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.779+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f58600048e0 con 0x7f586c068580 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.779+0000 7f5869ffb700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 0x7f585c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.779+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f586008dc00 con 0x7f586c068580 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.779+0000 7f586bfff700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 0x7f585c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:06.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.780+0000 7f586bfff700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 0x7f585c06f1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f586c101490 tx=0x7f5858009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:06.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:06.784+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5860093020 con 0x7f586c068580 2024-04-12T06:16:07.072 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.073+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f586c1012d0 con 0x7f586c068580 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.073+0000 7f5869ffb700 1 -- 172.21.15.8:0/1057763548 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f5860057c70 con 0x7f586c068580 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:16:07.073 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:16:07.074 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 msgr2=0x7f585c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 0x7f585c06f1a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f586c101490 tx=0x7f5858009260 comp rx=0 tx=0).stop 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 msgr2=0x7f586c0ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5860000c00 tx=0x7f586000be90 comp rx=0 tx=0).stop 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 shutdown_connections 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f585c06cce0 0x7f585c06f1a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f586c068580 0x7f586c0ffcf0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 --2- 172.21.15.8:0/1057763548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f586c106cd0 0x7f586c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.075+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 >> 172.21.15.8:0/1057763548 conn(0x7f586c075670 msgr2=0x7f586c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.076+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 shutdown_connections 2024-04-12T06:16:07.076 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.076+0000 7f5872b44700 1 -- 172.21.15.8:0/1057763548 wait complete. 2024-04-12T06:16:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:06 smithi008 ceph-mon[146084]: pgmap v632: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:06 smithi008 ceph-mon[146084]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:07.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.181+0000 7f30e3293700 1 -- 172.21.15.8:0/3829987357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc100ea0 msgr2=0x7f30dc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.181+0000 7f30e3293700 1 --2- 172.21.15.8:0/3829987357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc100ea0 0x7f30dc101280 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009910 tx=0x7f30d8009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:07.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- 172.21.15.8:0/3829987357 shutdown_connections 2024-04-12T06:16:07.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 --2- 172.21.15.8:0/3829987357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc101850 0x7f30dc1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 --2- 172.21.15.8:0/3829987357 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc100ea0 0x7f30dc101280 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- 172.21.15.8:0/3829987357 >> 172.21.15.8:0/3829987357 conn(0x7f30dc0fc750 msgr2=0x7f30dc0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- 172.21.15.8:0/3829987357 shutdown_connections 2024-04-12T06:16:07.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- 172.21.15.8:0/3829987357 wait complete. 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 Processor -- start 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- start start 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc100ea0 0x7f30dc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc073b10 con 0x7f30dc101850 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.182+0000 7f30e3293700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30dc073c50 con 0x7f30dc100ea0 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39900/0 (socket says 172.21.15.8:39900) 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 -- 172.21.15.8:0/2958541214 learned_addr learned my addr 172.21.15.8:0/2958541214 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e102f700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc100ea0 0x7f30dc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 -- 172.21.15.8:0/2958541214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc100ea0 msgr2=0x7f30dc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc100ea0 0x7f30dc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 -- 172.21.15.8:0/2958541214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80095f0 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e082e700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d900 tx=0x7f30cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc009900 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30cc010460 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30dc1a2960 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30cc00f5f0 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.183+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30dc1a2d70 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.184+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30dc109240 con 0x7f30dc101850 2024-04-12T06:16:07.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.185+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f30cc0105d0 con 0x7f30dc101850 2024-04-12T06:16:07.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.185+0000 7f30d27fc700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 0x7f30c806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.185+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f30cc08c380 con 0x7f30dc101850 2024-04-12T06:16:07.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.185+0000 7f30e102f700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 0x7f30c806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.186+0000 7f30e102f700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 0x7f30c806f1a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006010 tx=0x7f30d801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.190+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f30cc0563f0 con 0x7f30dc101850 2024-04-12T06:16:07.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:06 smithi194 ceph-mon[106982]: pgmap v632: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:07.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:06 smithi194 ceph-mon[106982]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:07.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.436+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f30dc04f2e0 con 0x7f30dc101850 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.436+0000 7f30d27fc700 1 -- 172.21.15.8:0/2958541214 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f30cc059a10 con 0x7f30dc101850 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:16:07.436 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:16:07.437 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:07.438 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:07.439 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.438+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 msgr2=0x7f30c806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 0x7f30c806f1a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f30d8006010 tx=0x7f30d801a040 comp rx=0 tx=0).stop 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 msgr2=0x7f30dc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d900 tx=0x7f30cc00dcc0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 shutdown_connections 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f30c806cce0 0x7f30c806f1a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30dc100ea0 0x7f30dc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 --2- 172.21.15.8:0/2958541214 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f30dc101850 0x7f30dc0735d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 >> 172.21.15.8:0/2958541214 conn(0x7f30dc0fc750 msgr2=0x7f30dc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 shutdown_connections 2024-04-12T06:16:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.439+0000 7f30e3293700 1 -- 172.21.15.8:0/2958541214 wait complete. 2024-04-12T06:16:07.443 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 -- 172.21.15.8:0/1597069152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 msgr2=0x7f84f4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 --2- 172.21.15.8:0/1597069152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4105fb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009910 tx=0x7f84f0009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 -- 172.21.15.8:0/1597069152 shutdown_connections 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 --2- 172.21.15.8:0/1597069152 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 0x7f84f4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 --2- 172.21.15.8:0/1597069152 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4105fb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 -- 172.21.15.8:0/1597069152 >> 172.21.15.8:0/1597069152 conn(0x7f84f4075670 msgr2=0x7f84f4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 -- 172.21.15.8:0/1597069152 shutdown_connections 2024-04-12T06:16:07.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.546+0000 7f84fc33d700 1 -- 172.21.15.8:0/1597069152 wait complete. 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 Processor -- start 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 -- start start 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 0x7f84f4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f4199550 con 0x7f84f4105bd0 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fc33d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f419d360 con 0x7f84f40ffc20 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84f98d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84f98d8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39908/0 (socket says 172.21.15.8:39908) 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84f98d8700 1 -- 172.21.15.8:0/89264691 learned_addr learned my addr 172.21.15.8:0/89264691 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84fa0d9700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 0x7f84f4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.547+0000 7f84f98d8700 1 -- 172.21.15.8:0/89264691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 msgr2=0x7f84f4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84f98d8700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 0x7f84f4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84f98d8700 1 -- 172.21.15.8:0/89264691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f00095f0 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84f98d8700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d8d0 tx=0x7f84e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e4009940 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f419d640 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84e4010460 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e400f5f0 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.548+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f419db90 con 0x7f84f4105bd0 2024-04-12T06:16:07.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.549+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f404f2e0 con 0x7f84f4105bd0 2024-04-12T06:16:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.549+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f84e40105d0 con 0x7f84f4105bd0 2024-04-12T06:16:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.550+0000 7f84eb7fe700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 0x7f84e006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.550+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84e408c5d0 con 0x7f84f4105bd0 2024-04-12T06:16:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.550+0000 7f84fa0d9700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 0x7f84e006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.550+0000 7f84fa0d9700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 0x7f84e006f1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.554+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f84e405a0d0 con 0x7f84f4105bd0 2024-04-12T06:16:07.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.774+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84f419ddb0 con 0x7f84e006cce0 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.776+0000 7f84eb7fe700 1 -- 172.21.15.8:0/89264691 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f84f419ddb0 con 0x7f84e006cce0 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:16:07.775 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:16:07.776 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 msgr2=0x7f84e006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 0x7f84e006f1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f001a040 comp rx=0 tx=0).stop 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 msgr2=0x7f84f4198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84e400d8d0 tx=0x7f84e400dc90 comp rx=0 tx=0).stop 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 shutdown_connections 2024-04-12T06:16:07.777 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f84e006cce0 0x7f84e006f1a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f84f40ffc20 0x7f84f4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 --2- 172.21.15.8:0/89264691 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f84f4105bd0 0x7f84f4198ec0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 >> 172.21.15.8:0/89264691 conn(0x7f84f4075670 msgr2=0x7f84f4068760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 shutdown_connections 2024-04-12T06:16:07.778 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.778+0000 7f84fc33d700 1 -- 172.21.15.8:0/89264691 wait complete. 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297116813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54068580 msgr2=0x7f2a540689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297116813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54068580 0x7f2a540689e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009910 tx=0x7f2a44009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297116813 shutdown_connections 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297116813 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54068580 0x7f2a540689e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297116813 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54106cd0 0x7f2a541070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297116813 >> 172.21.15.8:0/1297116813 conn(0x7f2a54075670 msgr2=0x7f2a54075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:07.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297116813 shutdown_connections 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297116813 wait complete. 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 Processor -- start 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.884+0000 7f2a5c04c700 1 -- start start 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a5c04c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a5c04c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a5c04c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a54100770 con 0x7f2a54106cd0 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a5c04c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a541008b0 con 0x7f2a54068580 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:39928/0 (socket says 172.21.15.8:39928) 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 -- 172.21.15.8:0/1297393633 learned_addr learned my addr 172.21.15.8:0/1297393633 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a59de8700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 -- 172.21.15.8:0/1297393633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 msgr2=0x7f2a540ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:07.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 -- 172.21.15.8:0/1297393633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440095f0 con 0x7f2a54106cd0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a59de8700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a595e7700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a440092e0 tx=0x7f2a4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.885+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4401d070 con 0x7f2a54106cd0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.886+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2a44005e00 con 0x7f2a54106cd0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.886+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a54100b30 con 0x7f2a54106cd0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.886+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4400fe60 con 0x7f2a54106cd0 2024-04-12T06:16:07.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.886+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a541a2c70 con 0x7f2a54106cd0 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.886+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a541091d0 con 0x7f2a54106cd0 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.887+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2a4400f6d0 con 0x7f2a54106cd0 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.887+0000 7f2a4affd700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 0x7f2a4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.888+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2a4408d850 con 0x7f2a54106cd0 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.888+0000 7f2a59de8700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 0x7f2a4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:07.888 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.888+0000 7f2a59de8700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 0x7f2a4006f1a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005fd0 tx=0x7f2a50005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:07.891 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:07.892+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2a440578c0 con 0x7f2a54106cd0 2024-04-12T06:16:08.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.178+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2a5404f2e0 con 0x7f2a54106cd0 2024-04-12T06:16:08.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.178+0000 7f2a4affd700 1 -- 172.21.15.8:0/1297393633 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2a4405aee0 con 0x7f2a54106cd0 2024-04-12T06:16:08.178 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:16:08.178 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:16:08.178 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:16:08.178 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:16:08.178 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:16:08.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 msgr2=0x7f2a4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:08.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 0x7f2a4006f1a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005fd0 tx=0x7f2a50005ee0 comp rx=0 tx=0).stop 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 msgr2=0x7f2a54100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a440092e0 tx=0x7f2a4400be20 comp rx=0 tx=0).stop 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 shutdown_connections 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2a4006cce0 0x7f2a4006f1a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a54068580 0x7f2a540ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 --2- 172.21.15.8:0/1297393633 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2a54106cd0 0x7f2a54100230 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 >> 172.21.15.8:0/1297393633 conn(0x7f2a54075670 msgr2=0x7f2a540fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 shutdown_connections 2024-04-12T06:16:08.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:08.180+0000 7f2a5c04c700 1 -- 172.21.15.8:0/1297393633 wait complete. 2024-04-12T06:16:08.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:07 smithi194 ceph-mon[106982]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:08.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:07 smithi194 ceph-mon[106982]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:08.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:07 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1057763548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:16:08.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:07 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2958541214' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:16:08.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:07 smithi008 ceph-mon[146084]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:08.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:07 smithi008 ceph-mon[146084]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:08.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:07 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1057763548' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:16:08.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:07 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2958541214' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:16:09.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:08 smithi194 ceph-mon[106982]: pgmap v633: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:09.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:08 smithi194 ceph-mon[106982]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:09.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:08 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1297393633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:16:09.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:08 smithi008 ceph-mon[146084]: pgmap v633: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:09.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:08 smithi008 ceph-mon[146084]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:09.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:08 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1297393633' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:16:11.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:10 smithi194 ceph-mon[106982]: pgmap v634: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:11.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:10 smithi008 ceph-mon[146084]: pgmap v634: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:13.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:12 smithi194 ceph-mon[106982]: pgmap v635: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:13.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:12 smithi008 ceph-mon[146084]: pgmap v635: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:15.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:14 smithi194 ceph-mon[106982]: pgmap v636: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:15.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:14 smithi008 ceph-mon[146084]: pgmap v636: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:17.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:16 smithi194 ceph-mon[106982]: pgmap v637: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:17.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:16 smithi008 ceph-mon[146084]: pgmap v637: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:18.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:18.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:19.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:18 smithi194 ceph-mon[106982]: pgmap v638: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:19.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:18 smithi008 ceph-mon[146084]: pgmap v638: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:21.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:20 smithi194 ceph-mon[106982]: pgmap v639: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:21.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:20 smithi008 ceph-mon[146084]: pgmap v639: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:23.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:22 smithi194 ceph-mon[106982]: pgmap v640: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:23.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:22 smithi008 ceph-mon[146084]: pgmap v640: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:25.225 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:24 smithi194 ceph-mon[106982]: pgmap v641: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:25.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:24 smithi008 ceph-mon[146084]: pgmap v641: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:27.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:26 smithi194 ceph-mon[106982]: pgmap v642: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:27.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:26 smithi008 ceph-mon[146084]: pgmap v642: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:29.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:28 smithi194 ceph-mon[106982]: pgmap v643: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:28 smithi008 ceph-mon[146084]: pgmap v643: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:31.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:30 smithi194 ceph-mon[106982]: pgmap v644: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:31.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:30 smithi008 ceph-mon[146084]: pgmap v644: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:33.331 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:32 smithi194 ceph-mon[106982]: pgmap v645: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:33.332 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:33.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:32 smithi008 ceph-mon[146084]: pgmap v645: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:35.318 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:34 smithi194 ceph-mon[106982]: pgmap v646: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:35.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:34 smithi008 ceph-mon[146084]: pgmap v646: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:37.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:36 smithi194 ceph-mon[106982]: pgmap v647: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:37.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:36 smithi008 ceph-mon[146084]: pgmap v647: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 -- 172.21.15.8:0/807754596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc0ffc20 msgr2=0x7fbdbc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 --2- 172.21.15.8:0/807754596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc100080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009910 tx=0x7fbda4009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 -- 172.21.15.8:0/807754596 shutdown_connections 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 --2- 172.21.15.8:0/807754596 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc100080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 --2- 172.21.15.8:0/807754596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 -- 172.21.15.8:0/807754596 >> 172.21.15.8:0/807754596 conn(0x7fbdbc075670 msgr2=0x7fbdbc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 -- 172.21.15.8:0/807754596 shutdown_connections 2024-04-12T06:16:38.301 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.301+0000 7fbdc1116700 1 -- 172.21.15.8:0/807754596 wait complete. 2024-04-12T06:16:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.302+0000 7fbdc1116700 1 Processor -- start 2024-04-12T06:16:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.302+0000 7fbdc1116700 1 -- start start 2024-04-12T06:16:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdc1116700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc19b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdc1116700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.303 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdc1116700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc19c0f0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdc1116700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdbc1955a0 con 0x7fbdbc0ffc20 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59198/0 (socket says 172.21.15.8:59198) 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 -- 172.21.15.8:0/3073090055 learned_addr learned my addr 172.21.15.8:0/3073090055 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdbad9d700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc19b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 -- 172.21.15.8:0/3073090055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc0ffc20 msgr2=0x7fbdbc19b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc19b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.303+0000 7fbdb259c700 1 -- 172.21.15.8:0/3073090055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda40095f0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdb259c700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda401d070 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdbc195820 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbda4005dc0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbda4003940 con 0x7fbdbc105bd0 2024-04-12T06:16:38.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.304+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdbc195cb0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.305+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdbc1080d0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.306+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbda4004ae0 con 0x7fbdbc105bd0 2024-04-12T06:16:38.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.306+0000 7fbdb8d99700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 0x7fbda806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.306+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbda408e310 con 0x7fbdbc105bd0 2024-04-12T06:16:38.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.306+0000 7fbdbad9d700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 0x7fbda806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.307+0000 7fbdbad9d700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 0x7fbda806f1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009c60 tx=0x7fbdac009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.310+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbda405bd70 con 0x7fbdbc105bd0 2024-04-12T06:16:38.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.530+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdbc195f30 con 0x7fbda806cce0 2024-04-12T06:16:38.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.531+0000 7fbdb8d99700 1 -- 172.21.15.8:0/3073090055 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbdbc195f30 con 0x7fbda806cce0 2024-04-12T06:16:38.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 msgr2=0x7fbda806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 0x7fbda806f1a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009c60 tx=0x7fbdac009260 comp rx=0 tx=0).stop 2024-04-12T06:16:38.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 msgr2=0x7fbdbc19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbda4000c00 tx=0x7fbda400bde0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 shutdown_connections 2024-04-12T06:16:38.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbda806cce0 0x7fbda806f1a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbdbc0ffc20 0x7fbdbc19b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 --2- 172.21.15.8:0/3073090055 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbdbc105bd0 0x7fbdbc19ba60 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 >> 172.21.15.8:0/3073090055 conn(0x7fbdbc075670 msgr2=0x7fbdbc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:38.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 shutdown_connections 2024-04-12T06:16:38.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.533+0000 7fbdc1116700 1 -- 172.21.15.8:0/3073090055 wait complete. 2024-04-12T06:16:38.545 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:16:38.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 -- 172.21.15.8:0/1897653265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 msgr2=0x7f10e4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/1897653265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4105fb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b0a0 tx=0x7f10d400b3b0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 -- 172.21.15.8:0/1897653265 shutdown_connections 2024-04-12T06:16:38.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/1897653265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 0x7f10e4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/1897653265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4105fb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 -- 172.21.15.8:0/1897653265 >> 172.21.15.8:0/1897653265 conn(0x7f10e4075670 msgr2=0x7f10e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:38.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 -- 172.21.15.8:0/1897653265 shutdown_connections 2024-04-12T06:16:38.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.640+0000 7f10ea0e3700 1 -- 172.21.15.8:0/1897653265 wait complete. 2024-04-12T06:16:38.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 Processor -- start 2024-04-12T06:16:38.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 -- start start 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 0x7f10e4198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e4199560 con 0x7f10e4105bd0 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10ea0e3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e419d370 con 0x7f10e40ffc20 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59208/0 (socket says 172.21.15.8:59208) 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10e2ffd700 1 -- 172.21.15.8:0/2488350623 learned_addr learned my addr 172.21.15.8:0/2488350623 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.641+0000 7f10e37fe700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 0x7f10e4198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e2ffd700 1 -- 172.21.15.8:0/2488350623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 msgr2=0x7f10e4198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e2ffd700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 0x7f10e4198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e2ffd700 1 -- 172.21.15.8:0/2488350623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4009d00 con 0x7f10e4105bd0 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e2ffd700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d8d0 tx=0x7f10d800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d80098a0 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e419d650 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10d8010460 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10d8009ea0 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.642+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e419dba0 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.643+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e404f2e0 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.644+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f10d8009a00 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.644+0000 7f10e0ff9700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 0x7f10cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.644+0000 7f10e37fe700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 0x7f10cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.644+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f10d808c450 con 0x7f10e4105bd0 2024-04-12T06:16:38.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.645+0000 7f10e37fe700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 0x7f10cc06f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fd0 tx=0x7f10d40060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.649+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f10d8059f50 con 0x7f10e4105bd0 2024-04-12T06:16:38.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.871+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10e419ddc0 con 0x7f10cc06cce0 2024-04-12T06:16:38.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.872+0000 7f10e0ff9700 1 -- 172.21.15.8:0/2488350623 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f10e419ddc0 con 0x7f10cc06cce0 2024-04-12T06:16:38.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 msgr2=0x7f10cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 0x7f10cc06f1a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fd0 tx=0x7f10d40060c0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 msgr2=0x7f10e4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f10d800d8d0 tx=0x7f10d800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 shutdown_connections 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f10cc06cce0 0x7f10cc06f1a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10e40ffc20 0x7f10e4198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 --2- 172.21.15.8:0/2488350623 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10e4105bd0 0x7f10e4198ed0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 >> 172.21.15.8:0/2488350623 conn(0x7f10e4075670 msgr2=0x7f10e40686e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:38.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 shutdown_connections 2024-04-12T06:16:38.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.874+0000 7f10ea0e3700 1 -- 172.21.15.8:0/2488350623 wait complete. 2024-04-12T06:16:38.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- 172.21.15.8:0/684317940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724101840 msgr2=0x7ff724105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 --2- 172.21.15.8:0/684317940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724101840 0x7ff724105890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff720009910 tx=0x7ff720009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:38.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- 172.21.15.8:0/684317940 shutdown_connections 2024-04-12T06:16:38.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 --2- 172.21.15.8:0/684317940 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724101840 0x7ff724105890 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 --2- 172.21.15.8:0/684317940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724100e90 0x7ff724101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- 172.21.15.8:0/684317940 >> 172.21.15.8:0/684317940 conn(0x7ff7240fc740 msgr2=0x7ff7240feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:38.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- 172.21.15.8:0/684317940 shutdown_connections 2024-04-12T06:16:38.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- 172.21.15.8:0/684317940 wait complete. 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 Processor -- start 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.981+0000 7ff72ba39700 1 -- start start 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff72ba39700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff72ba39700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 0x7ff72419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff72ba39700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72419d200 con 0x7ff724100e90 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff72ba39700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7241966b0 con 0x7ff724101840 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59218/0 (socket says 172.21.15.8:59218) 2024-04-12T06:16:38.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 -- 172.21.15.8:0/4255213501 learned_addr learned my addr 172.21.15.8:0/4255213501 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff728fd4700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 0x7ff72419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 -- 172.21.15.8:0/4255213501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 msgr2=0x7ff72419cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 0x7ff72419cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 -- 172.21.15.8:0/4255213501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200095f0 con 0x7ff724100e90 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff728fd4700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 0x7ff72419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff7297d5700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c8a0 tx=0x7ff71400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.982+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71400cea0 con 0x7ff724100e90 2024-04-12T06:16:38.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.983+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff714004830 con 0x7ff724100e90 2024-04-12T06:16:38.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.983+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff724196990 con 0x7ff724100e90 2024-04-12T06:16:38.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.983+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff714005500 con 0x7ff724100e90 2024-04-12T06:16:38.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.983+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff724196ee0 con 0x7ff724100e90 2024-04-12T06:16:38.987 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.984+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff724109290 con 0x7ff724100e90 2024-04-12T06:16:38.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.984+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff7140049a0 con 0x7ff724100e90 2024-04-12T06:16:38.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.984+0000 7ff71a7fc700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 0x7ff71006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:38.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.985+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff71408c240 con 0x7ff724100e90 2024-04-12T06:16:38.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.985+0000 7ff728fd4700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 0x7ff71006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:38.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.985+0000 7ff728fd4700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 0x7ff71006f1a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b5c0 tx=0x7ff720005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:38.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:38.989+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff714059d40 con 0x7ff724100e90 2024-04-12T06:16:39.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.201+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff724197cd0 con 0x7ff71006cce0 2024-04-12T06:16:39.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.212+0000 7ff71a7fc700 1 -- 172.21.15.8:0/4255213501 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff724197cd0 con 0x7ff71006cce0 2024-04-12T06:16:39.215 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:16:39.215 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (18m) 12m ago 29m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:16:39.215 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (30m) 12m ago 30m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:16:39.216 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (29m) 11m ago 28m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:16:39.216 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (16m) 12m ago 30m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:16:39.216 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (16m) 11m ago 28m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:16:39.216 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (18m) 12m ago 29m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (12m) 12m ago 23m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (12m) 12m ago 23m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (11m) 11m ago 23m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (11m) 11m ago 23m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (20m) 12m ago 31m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:16:39.217 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (19m) 11m ago 28m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (16m) 12m ago 31m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (16m) 11m ago 28m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (19m) 12m ago 30m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (19m) 11m ago 28m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (15m) 12m ago 27m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (15m) 12m ago 27m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (14m) 12m ago 26m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (13m) 11m ago 26m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (13m) 11m ago 25m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:16:39.218 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (12m) 11m ago 25m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:16:39.219 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (18m) 12m ago 29m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:16:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 msgr2=0x7ff71006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 0x7ff71006f1a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b5c0 tx=0x7ff720005fd0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 msgr2=0x7ff72419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c8a0 tx=0x7ff71400cc60 comp rx=0 tx=0).stop 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 shutdown_connections 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff71006cce0 0x7ff71006f1a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff724100e90 0x7ff72419c630 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 --2- 172.21.15.8:0/4255213501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff724101840 0x7ff72419cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 >> 172.21.15.8:0/4255213501 conn(0x7ff7240fc740 msgr2=0x7ff7240fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:39.220 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 shutdown_connections 2024-04-12T06:16:39.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.214+0000 7ff72ba39700 1 -- 172.21.15.8:0/4255213501 wait complete. 2024-04-12T06:16:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 -- 172.21.15.8:0/1287328844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca680ffc20 msgr2=0x7fca68100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 --2- 172.21.15.8:0/1287328844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca680ffc20 0x7fca68100080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fca54009910 tx=0x7fca54009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 -- 172.21.15.8:0/1287328844 shutdown_connections 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 --2- 172.21.15.8:0/1287328844 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca680ffc20 0x7fca68100080 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 --2- 172.21.15.8:0/1287328844 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca68105bd0 0x7fca68105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 -- 172.21.15.8:0/1287328844 >> 172.21.15.8:0/1287328844 conn(0x7fca68075670 msgr2=0x7fca68075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 -- 172.21.15.8:0/1287328844 shutdown_connections 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.319+0000 7fca6ebde700 1 -- 172.21.15.8:0/1287328844 wait complete. 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 Processor -- start 2024-04-12T06:16:39.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 -- start start 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca680ffc20 0x7fca6819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6819c0f0 con 0x7fca68105bd0 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6ebde700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca681955a0 con 0x7fca680ffc20 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59238/0 (socket says 172.21.15.8:59238) 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 -- 172.21.15.8:0/2927246039 learned_addr learned my addr 172.21.15.8:0/2927246039 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca6c97a700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca680ffc20 0x7fca6819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 -- 172.21.15.8:0/2927246039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca680ffc20 msgr2=0x7fca6819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca680ffc20 0x7fca6819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 -- 172.21.15.8:0/2927246039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca540095f0 con 0x7fca68105bd0 2024-04-12T06:16:39.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:38 smithi194 ceph-mon[106982]: pgmap v648: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:39.325 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.320+0000 7fca67fff700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fca54006010 tx=0x7fca5400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.321+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5401f070 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.321+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fca54005710 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.321+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca68195820 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.321+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5400f460 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.321+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca68195d10 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.322+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6804f2e0 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.323+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fca5400f5c0 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.323+0000 7fca65ffb700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 0x7fca5806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.323+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca5408f7d0 con 0x7fca68105bd0 2024-04-12T06:16:39.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.323+0000 7fca6c97a700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 0x7fca5806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.324+0000 7fca6c97a700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 0x7fca5806f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005950 tx=0x7fca5c00b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:39.336 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.328+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fca5405d2d0 con 0x7fca68105bd0 2024-04-12T06:16:39.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:38 smithi008 ceph-mon[146084]: pgmap v648: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.612+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fca68196ab0 con 0x7fca68105bd0 2024-04-12T06:16:39.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.613+0000 7fca65ffb700 1 -- 172.21.15.8:0/2927246039 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fca5405ce60 con 0x7fca68105bd0 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:16:39.613 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:16:39.614 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:16:39.615 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 msgr2=0x7fca5806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 0x7fca5806f1a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005950 tx=0x7fca5c00b410 comp rx=0 tx=0).stop 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 msgr2=0x7fca6819ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fca54006010 tx=0x7fca5400bd70 comp rx=0 tx=0).stop 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 shutdown_connections 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fca5806cce0 0x7fca5806f1a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fca680ffc20 0x7fca6819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 --2- 172.21.15.8:0/2927246039 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fca68105bd0 0x7fca6819ba60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 >> 172.21.15.8:0/2927246039 conn(0x7fca68075670 msgr2=0x7fca680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:39.616 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 shutdown_connections 2024-04-12T06:16:39.617 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.615+0000 7fca6ebde700 1 -- 172.21.15.8:0/2927246039 wait complete. 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 -- 172.21.15.8:0/2366851671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 msgr2=0x7f642c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 --2- 172.21.15.8:0/2366851671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c105fb0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6428009910 tx=0x7f6428009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 -- 172.21.15.8:0/2366851671 shutdown_connections 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 --2- 172.21.15.8:0/2366851671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 --2- 172.21.15.8:0/2366851671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c105fb0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.721+0000 7f6433666700 1 -- 172.21.15.8:0/2366851671 >> 172.21.15.8:0/2366851671 conn(0x7f642c075670 msgr2=0x7f642c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:39.724 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.722+0000 7f6433666700 1 -- 172.21.15.8:0/2366851671 shutdown_connections 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.722+0000 7f6433666700 1 -- 172.21.15.8:0/2366851671 wait complete. 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.722+0000 7f6433666700 1 Processor -- start 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.722+0000 7f6433666700 1 -- start start 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6433666700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c1953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6433666700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6433666700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c195e70 con 0x7f642c105bd0 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6433666700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c195fe0 con 0x7f642c0ffc20 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59254/0 (socket says 172.21.15.8:59254) 2024-04-12T06:16:39.725 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 -- 172.21.15.8:0/1879763896 learned_addr learned my addr 172.21.15.8:0/1879763896 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6431402700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c1953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 -- 172.21.15.8:0/1879763896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 msgr2=0x7f642c1953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c1953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 -- 172.21.15.8:0/1879763896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280095f0 con 0x7f642c105bd0 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6431402700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c1953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6430c01700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f641c00d8d0 tx=0x7f641c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f641c009940 con 0x7f642c105bd0 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f641c010460 con 0x7f642c105bd0 2024-04-12T06:16:39.726 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.723+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c1a2a60 con 0x7f642c105bd0 2024-04-12T06:16:39.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.724+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f641c00f5f0 con 0x7f642c105bd0 2024-04-12T06:16:39.727 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.724+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c1a2dc0 con 0x7f642c105bd0 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.725+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c1080d0 con 0x7f642c105bd0 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.725+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f641c0105d0 con 0x7f642c105bd0 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.726+0000 7f64227fc700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 0x7f641806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.726+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f641c08c530 con 0x7f642c105bd0 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.726+0000 7f6431402700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 0x7f641806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:39.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.726+0000 7f6431402700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 0x7f641806f1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6428006010 tx=0x7f642801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:39.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.732+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f641c05a030 con 0x7f642c105bd0 2024-04-12T06:16:39.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.989+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f642c107ab0 con 0x7f642c105bd0 2024-04-12T06:16:39.989 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.990+0000 7f64227fc700 1 -- 172.21.15.8:0/1879763896 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f641c010880 con 0x7f642c105bd0 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:16:39.990 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:16:39.991 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:16:39.992 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:16:39.993 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:39.994 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:16:39.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 msgr2=0x7f641806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 0x7f641806f1a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6428006010 tx=0x7f642801a040 comp rx=0 tx=0).stop 2024-04-12T06:16:39.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 msgr2=0x7f642c195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f641c00d8d0 tx=0x7f641c00dc90 comp rx=0 tx=0).stop 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 shutdown_connections 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f641806cce0 0x7f641806f1a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f642c0ffc20 0x7f642c1953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 --2- 172.21.15.8:0/1879763896 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f642c105bd0 0x7f642c195930 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 >> 172.21.15.8:0/1879763896 conn(0x7f642c075670 msgr2=0x7f642c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 shutdown_connections 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:39.992+0000 7f6433666700 1 -- 172.21.15.8:0/1879763896 wait complete. 2024-04-12T06:16:39.996 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 -- 172.21.15.8:0/2037803367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 msgr2=0x7f0980100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 --2- 172.21.15.8:0/2037803367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f0980100180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0970009910 tx=0x7f0970009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 -- 172.21.15.8:0/2037803367 shutdown_connections 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 --2- 172.21.15.8:0/2037803367 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 0x7f09801047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 --2- 172.21.15.8:0/2037803367 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f0980100180 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 -- 172.21.15.8:0/2037803367 >> 172.21.15.8:0/2037803367 conn(0x7f09800fb650 msgr2=0x7f09800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 -- 172.21.15.8:0/2037803367 shutdown_connections 2024-04-12T06:16:40.099 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.099+0000 7f0986487700 1 -- 172.21.15.8:0/2037803367 wait complete. 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 Processor -- start 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 -- start start 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 0x7f098019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098019d240 con 0x7f09800ffda0 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f0986487700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801966f0 con 0x7f0980100750 2024-04-12T06:16:40.101 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f097ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f097ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59276/0 (socket says 172.21.15.8:59276) 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.100+0000 7f097ffff700 1 -- 172.21.15.8:0/3213572157 learned_addr learned my addr 172.21.15.8:0/3213572157 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097f7fe700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 0x7f098019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097ffff700 1 -- 172.21.15.8:0/3213572157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 msgr2=0x7f098019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097ffff700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 0x7f098019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097ffff700 1 -- 172.21.15.8:0/3213572157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700095f0 con 0x7f09800ffda0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097ffff700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f097000b5c0 tx=0x7f097000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097001d070 con 0x7f09800ffda0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f097000f460 con 0x7f09800ffda0 2024-04-12T06:16:40.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0970021640 con 0x7f09800ffda0 2024-04-12T06:16:40.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0980196970 con 0x7f09800ffda0 2024-04-12T06:16:40.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.101+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0980196e60 con 0x7f09800ffda0 2024-04-12T06:16:40.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.102+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098004f2e0 con 0x7f09800ffda0 2024-04-12T06:16:40.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.103+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f097000f9b0 con 0x7f09800ffda0 2024-04-12T06:16:40.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.103+0000 7f097d7fa700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 0x7f096806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.103+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f097008d8c0 con 0x7f09800ffda0 2024-04-12T06:16:40.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.103+0000 7f097f7fe700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 0x7f096806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.104 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.104+0000 7f097f7fe700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 0x7f096806f1a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0974006fd0 tx=0x7f0974008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:40.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.108+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f097005b3c0 con 0x7f09800ffda0 2024-04-12T06:16:40.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:39 smithi008 ceph-mon[146084]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:40.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:39 smithi008 ceph-mon[146084]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:40.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2927246039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:16:40.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:39 smithi194 ceph-mon[106982]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:40.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:39 smithi194 ceph-mon[106982]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:40.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2927246039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:16:40.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.329+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09801970f0 con 0x7f096806cce0 2024-04-12T06:16:40.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.331+0000 7f097d7fa700 1 -- 172.21.15.8:0/3213572157 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f09801970f0 con 0x7f096806cce0 2024-04-12T06:16:40.330 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:16:40.330 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:16:40.331 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:16:40.332 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:16:40.332 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:16:40.332 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:16:40.332 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:16:40.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 msgr2=0x7f096806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 0x7f096806f1a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0974006fd0 tx=0x7f0974008040 comp rx=0 tx=0).stop 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 msgr2=0x7f098019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f097000b5c0 tx=0x7f097000be20 comp rx=0 tx=0).stop 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 shutdown_connections 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f096806cce0 0x7f096806f1a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f09800ffda0 0x7f098019c670 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 --2- 172.21.15.8:0/3213572157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0980100750 0x7f098019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 >> 172.21.15.8:0/3213572157 conn(0x7f09800fb650 msgr2=0x7f09801040c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:40.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 shutdown_connections 2024-04-12T06:16:40.335 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.333+0000 7f0986487700 1 -- 172.21.15.8:0/3213572157 wait complete. 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 -- 172.21.15.8:0/3305920097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 msgr2=0x7f10b00689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 --2- 172.21.15.8:0/3305920097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00689e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009910 tx=0x7f10a4009c20 comp rx=0 tx=0).stop 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 -- 172.21.15.8:0/3305920097 shutdown_connections 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 --2- 172.21.15.8:0/3305920097 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00689e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 --2- 172.21.15.8:0/3305920097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 0x7f10b01070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 -- 172.21.15.8:0/3305920097 >> 172.21.15.8:0/3305920097 conn(0x7f10b0075670 msgr2=0x7f10b0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 -- 172.21.15.8:0/3305920097 shutdown_connections 2024-04-12T06:16:40.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.441+0000 7f10b5b88700 1 -- 172.21.15.8:0/3305920097 wait complete. 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 Processor -- start 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 -- start start 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 0x7f10b0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0100770 con 0x7f10b0068580 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10b5b88700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01008b0 con 0x7f10b0106cd0 2024-04-12T06:16:40.443 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.442+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:59288/0 (socket says 172.21.15.8:59288) 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 -- 172.21.15.8:0/4127297360 learned_addr learned my addr 172.21.15.8:0/4127297360 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10aeffd700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 0x7f10b0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 -- 172.21.15.8:0/4127297360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 msgr2=0x7f10b0100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 0x7f10b0100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 -- 172.21.15.8:0/4127297360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40095f0 con 0x7f10b0068580 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10af7fe700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f10a000cfd0 tx=0x7f10a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a000ad70 con 0x7f10b0068580 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f10a00128c0 con 0x7f10b0068580 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b01a2940 con 0x7f10b0068580 2024-04-12T06:16:40.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b01a2ca0 con 0x7f10b0068580 2024-04-12T06:16:40.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.443+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10a00117b0 con 0x7f10b0068580 2024-04-12T06:16:40.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.445+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b01091d0 con 0x7f10b0068580 2024-04-12T06:16:40.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.445+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f10a0012a30 con 0x7f10b0068580 2024-04-12T06:16:40.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.445+0000 7f10acff9700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 0x7f109806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:16:40.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.445+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f10a008caa0 con 0x7f10b0068580 2024-04-12T06:16:40.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.445+0000 7f10aeffd700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 0x7f109806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:16:40.446 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.446+0000 7f10aeffd700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 0x7f109806f1a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10b01014e0 tx=0x7f10a401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:16:40.449 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.450+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f10a0056b10 con 0x7f10b0068580 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.735+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f10b0101320 con 0x7f10b0068580 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.735+0000 7f10acff9700 1 -- 172.21.15.8:0/4127297360 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f10a0012ce0 con 0x7f10b0068580 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:16:40.735 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 msgr2=0x7f109806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 0x7f109806f1a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f10b01014e0 tx=0x7f10a401a040 comp rx=0 tx=0).stop 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 msgr2=0x7f10b00ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f10a000cfd0 tx=0x7f10a000a340 comp rx=0 tx=0).stop 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 shutdown_connections 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f109806cce0 0x7f109806f1a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10b0068580 0x7f10b00ffcf0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 --2- 172.21.15.8:0/4127297360 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10b0106cd0 0x7f10b0100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.737+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 >> 172.21.15.8:0/4127297360 conn(0x7f10b0075670 msgr2=0x7f10b00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:16:40.737 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.738+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 shutdown_connections 2024-04-12T06:16:40.738 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:16:40.738+0000 7f10b5b88700 1 -- 172.21.15.8:0/4127297360 wait complete. 2024-04-12T06:16:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:40 smithi194 ceph-mon[106982]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:40 smithi194 ceph-mon[106982]: pgmap v649: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:40 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1879763896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:16:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:40 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4127297360' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:16:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:40 smithi008 ceph-mon[146084]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:40 smithi008 ceph-mon[146084]: pgmap v649: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:40 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1879763896' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:16:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:40 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4127297360' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:16:42.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:41 smithi194 ceph-mon[106982]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:42.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:41 smithi008 ceph-mon[146084]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:16:43.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:42 smithi194 ceph-mon[106982]: pgmap v650: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:43.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:42 smithi008 ceph-mon[146084]: pgmap v650: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:45.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:44 smithi008 ceph-mon[146084]: pgmap v651: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:45.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:44 smithi194 ceph-mon[106982]: pgmap v651: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:46 smithi194 ceph-mon[106982]: pgmap v652: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:47.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:46 smithi008 ceph-mon[146084]: pgmap v652: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:48.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:48.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:16:49.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:48 smithi194 ceph-mon[106982]: pgmap v653: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:49.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:48 smithi008 ceph-mon[146084]: pgmap v653: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:51.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:50 smithi194 ceph-mon[106982]: pgmap v654: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:51.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:50 smithi008 ceph-mon[146084]: pgmap v654: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:53.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:52 smithi194 ceph-mon[106982]: pgmap v655: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:53.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:52 smithi008 ceph-mon[146084]: pgmap v655: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:55.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:54 smithi008 ceph-mon[146084]: pgmap v656: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:54 smithi194 ceph-mon[106982]: pgmap v656: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:57.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:57 smithi194 ceph-mon[106982]: pgmap v657: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:57.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:57 smithi008 ceph-mon[146084]: pgmap v657: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:16:59 smithi194 ceph-mon[106982]: pgmap v658: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:16:59.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:16:59 smithi008 ceph-mon[146084]: pgmap v658: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:01.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:01 smithi194 ceph-mon[106982]: pgmap v659: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:01.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:01 smithi008 ceph-mon[146084]: pgmap v659: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:03 smithi194 ceph-mon[106982]: pgmap v660: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:03.407 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:03 smithi008 ceph-mon[146084]: pgmap v660: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:04.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:04 smithi194 ceph-mon[106982]: pgmap v661: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:04.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:04 smithi008 ceph-mon[146084]: pgmap v661: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:07.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:06 smithi194 ceph-mon[106982]: pgmap v662: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:06 smithi008 ceph-mon[146084]: pgmap v662: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:09.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:08 smithi194 ceph-mon[106982]: pgmap v663: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:09.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:08 smithi008 ceph-mon[146084]: pgmap v663: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:10.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 -- 172.21.15.8:0/3831139905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83540ffc20 msgr2=0x7f8354100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:10.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 --2- 172.21.15.8:0/3831139905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83540ffc20 0x7f8354100080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8348009910 tx=0x7f8348009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 -- 172.21.15.8:0/3831139905 shutdown_connections 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 --2- 172.21.15.8:0/3831139905 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f83540ffc20 0x7f8354100080 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 --2- 172.21.15.8:0/3831139905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8354105bd0 0x7f8354105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.859+0000 7f83595de700 1 -- 172.21.15.8:0/3831139905 >> 172.21.15.8:0/3831139905 conn(0x7f8354075670 msgr2=0x7f8354075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.860+0000 7f83595de700 1 -- 172.21.15.8:0/3831139905 shutdown_connections 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.860+0000 7f83595de700 1 -- 172.21.15.8:0/3831139905 wait complete. 2024-04-12T06:17:10.860 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.860+0000 7f83595de700 1 Processor -- start 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.860+0000 7f83595de700 1 -- start start 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83595de700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f83540ffc20 0x7f835419c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83595de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83595de700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835419d070 con 0x7f8354105bd0 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83595de700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8354196600 con 0x7f83540ffc20 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37804/0 (socket says 172.21.15.8:37804) 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 -- 172.21.15.8:0/5447601 learned_addr learned my addr 172.21.15.8:0/5447601 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f8352ffd700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f83540ffc20 0x7f835419c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 -- 172.21.15.8:0/5447601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f83540ffc20 msgr2=0x7f835419c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f83540ffc20 0x7f835419c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 -- 172.21.15.8:0/5447601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83480095f0 con 0x7f8354105bd0 2024-04-12T06:17:10.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f83527fc700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:10.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834801d070 con 0x7f8354105bd0 2024-04-12T06:17:10.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.861+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f83480044c0 con 0x7f8354105bd0 2024-04-12T06:17:10.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.862+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8354196880 con 0x7f8354105bd0 2024-04-12T06:17:10.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.862+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83480039d0 con 0x7f8354105bd0 2024-04-12T06:17:10.863 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.862+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8354196d10 con 0x7f8354105bd0 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.863+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835404f2e0 con 0x7f8354105bd0 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.863+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f83480048e0 con 0x7f8354105bd0 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.863+0000 7f833bfff700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 0x7f833c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.864+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f834808dc00 con 0x7f8354105bd0 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.864+0000 7f8352ffd700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 0x7f833c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:10.864 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.864+0000 7f8352ffd700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 0x7f833c06f1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8344009bf0 tx=0x7f8344009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:10.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:10.868+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f834805b6e0 con 0x7f8354105bd0 2024-04-12T06:17:11.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:10 smithi194 ceph-mon[106982]: pgmap v664: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:11.087 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.088+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8354197010 con 0x7f833c06cce0 2024-04-12T06:17:11.088 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.089+0000 7f833bfff700 1 -- 172.21.15.8:0/5447601 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8354197010 con 0x7f833c06cce0 2024-04-12T06:17:11.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 msgr2=0x7f833c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 0x7f833c06f1a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8344009bf0 tx=0x7f8344009260 comp rx=0 tx=0).stop 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 msgr2=0x7f835419ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8348000c00 tx=0x7f834800be90 comp rx=0 tx=0).stop 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 shutdown_connections 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f833c06cce0 0x7f833c06f1a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f83540ffc20 0x7f835419c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 --2- 172.21.15.8:0/5447601 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8354105bd0 0x7f835419ca50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 >> 172.21.15.8:0/5447601 conn(0x7f8354075670 msgr2=0x7f8354068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 shutdown_connections 2024-04-12T06:17:11.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.091+0000 7f83595de700 1 -- 172.21.15.8:0/5447601 wait complete. 2024-04-12T06:17:11.101 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:17:11.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:10 smithi008 ceph-mon[146084]: pgmap v664: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:11.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.198+0000 7f431c251700 1 -- 172.21.15.8:0/1628898067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 msgr2=0x7f4314105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.198+0000 7f431c251700 1 --2- 172.21.15.8:0/1628898067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f4314105fb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4310009910 tx=0x7f4310009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:11.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 -- 172.21.15.8:0/1628898067 shutdown_connections 2024-04-12T06:17:11.198 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 --2- 172.21.15.8:0/1628898067 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 0x7f4314100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 --2- 172.21.15.8:0/1628898067 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f4314105fb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 -- 172.21.15.8:0/1628898067 >> 172.21.15.8:0/1628898067 conn(0x7f4314075670 msgr2=0x7f4314075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 -- 172.21.15.8:0/1628898067 shutdown_connections 2024-04-12T06:17:11.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 -- 172.21.15.8:0/1628898067 wait complete. 2024-04-12T06:17:11.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 Processor -- start 2024-04-12T06:17:11.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.199+0000 7f431c251700 1 -- start start 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 0x7f431419c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431419d160 con 0x7f4314105bd0 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4314196610 con 0x7f43140ffc20 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37812/0 (socket says 172.21.15.8:37812) 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 -- 172.21.15.8:0/3160206568 learned_addr learned my addr 172.21.15.8:0/3160206568 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f4319fed700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 0x7f431419c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 -- 172.21.15.8:0/3160206568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 msgr2=0x7f431419c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 0x7f431419c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 -- 172.21.15.8:0/3160206568 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100095f0 con 0x7f4314105bd0 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f43197ec700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f430400e920 tx=0x7f430400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4304009790 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f430400cba0 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43141968f0 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4304018650 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.200+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4314196e40 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.201+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431404f2e0 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.202+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f430400cd10 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.202+0000 7f430affd700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 0x7f430006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.202+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4304014070 con 0x7f4314105bd0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.202+0000 7f4319fed700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 0x7f430006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.203+0000 7f4319fed700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 0x7f430006f1a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4310009fd0 tx=0x7f4310005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.207+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4304052ab0 con 0x7f4314105bd0 2024-04-12T06:17:11.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.426+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43141970c0 con 0x7f430006cce0 2024-04-12T06:17:11.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.427+0000 7f430affd700 1 -- 172.21.15.8:0/3160206568 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f43141970c0 con 0x7f430006cce0 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 msgr2=0x7f430006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 0x7f430006f1a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4310009fd0 tx=0x7f4310005940 comp rx=0 tx=0).stop 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 msgr2=0x7f431419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f430400e920 tx=0x7f430400ece0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 shutdown_connections 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f430006cce0 0x7f430006f1a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43140ffc20 0x7f431419c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 --2- 172.21.15.8:0/3160206568 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4314105bd0 0x7f431419cad0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 >> 172.21.15.8:0/3160206568 conn(0x7f4314075670 msgr2=0x7f4314068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 shutdown_connections 2024-04-12T06:17:11.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.429+0000 7f431c251700 1 -- 172.21.15.8:0/3160206568 wait complete. 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 -- 172.21.15.8:0/1090790834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c105bd0 msgr2=0x7f5c2c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 --2- 172.21.15.8:0/1090790834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c105fb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009910 tx=0x7f5c20009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 -- 172.21.15.8:0/1090790834 shutdown_connections 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 --2- 172.21.15.8:0/1090790834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 --2- 172.21.15.8:0/1090790834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c105fb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 -- 172.21.15.8:0/1090790834 >> 172.21.15.8:0/1090790834 conn(0x7f5c2c075670 msgr2=0x7f5c2c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 -- 172.21.15.8:0/1090790834 shutdown_connections 2024-04-12T06:17:11.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.535+0000 7f5c30c84700 1 -- 172.21.15.8:0/1090790834 wait complete. 2024-04-12T06:17:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 Processor -- start 2024-04-12T06:17:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 -- start start 2024-04-12T06:17:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.537 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c1022c0 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c30c84700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2c102400 con 0x7f5c2c105bd0 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c2a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c2a59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37830/0 (socket says 172.21.15.8:37830) 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.536+0000 7f5c2a59c700 1 -- 172.21.15.8:0/558990996 learned_addr learned my addr 172.21.15.8:0/558990996 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c29d9b700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c2a59c700 1 -- 172.21.15.8:0/558990996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c105bd0 msgr2=0x7f5c2c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c2a59c700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c2a59c700 1 -- 172.21.15.8:0/558990996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200095f0 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c2a59c700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006010 tx=0x7f5c2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c2001d070 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c2000f460 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c2c102680 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c20021640 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.537+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c1a2d80 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.538+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2c04f2e0 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.539+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f5c2000f9e0 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.539+0000 7f5c1b7fe700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 0x7f5c1406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.539+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c2008d8c0 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.539+0000 7f5c29d9b700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 0x7f5c1406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.540+0000 7f5c29d9b700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 0x7f5c1406f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006fd0 tx=0x7f5c1c008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.544+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5c20057930 con 0x7f5c2c0ffc20 2024-04-12T06:17:11.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.757+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c2c104990 con 0x7f5c1406cce0 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.768+0000 7f5c1b7fe700 1 -- 172.21.15.8:0/558990996 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5c2c104990 con 0x7f5c1406cce0 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (19m) 12m ago 30m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (30m) 12m ago 30m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (29m) 12m ago 29m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (16m) 12m ago 30m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (16m) 12m ago 29m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (18m) 12m ago 30m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (12m) 12m ago 24m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:17:11.768 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (13m) 12m ago 24m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (12m) 12m ago 24m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (12m) 12m ago 24m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (21m) 12m ago 32m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (20m) 12m ago 29m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (17m) 12m ago 32m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (17m) 12m ago 29m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (19m) 12m ago 30m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (19m) 12m ago 29m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (16m) 12m ago 28m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (15m) 12m ago 27m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (15m) 12m ago 27m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (14m) 12m ago 26m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (13m) 12m ago 26m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:17:11.769 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (13m) 12m ago 25m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:17:11.770 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (19m) 12m ago 29m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:17:11.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.770+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 msgr2=0x7f5c1406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.770+0000 7f5c30c84700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 0x7f5c1406f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006fd0 tx=0x7f5c1c008040 comp rx=0 tx=0).stop 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.770+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 msgr2=0x7f5c2c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.770+0000 7f5c30c84700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5c20006010 tx=0x7f5c2000be20 comp rx=0 tx=0).stop 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 shutdown_connections 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f5c1406cce0 0x7f5c1406f1a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5c2c0ffc20 0x7f5c2c101840 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 --2- 172.21.15.8:0/558990996 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5c2c105bd0 0x7f5c2c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 >> 172.21.15.8:0/558990996 conn(0x7f5c2c075670 msgr2=0x7f5c2c068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 shutdown_connections 2024-04-12T06:17:11.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.771+0000 7f5c30c84700 1 -- 172.21.15.8:0/558990996 wait complete. 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 -- 172.21.15.8:0/2556562619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068ea0 msgr2=0x7f1dfc10d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 --2- 172.21.15.8:0/2556562619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc10d290 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009910 tx=0x7f1dec009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 -- 172.21.15.8:0/2556562619 shutdown_connections 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 --2- 172.21.15.8:0/2556562619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc10d290 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 --2- 172.21.15.8:0/2556562619 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068580 0x7f1dfc068960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.875+0000 7f1e040b0700 1 -- 172.21.15.8:0/2556562619 >> 172.21.15.8:0/2556562619 conn(0x7f1dfc075670 msgr2=0x7f1dfc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 -- 172.21.15.8:0/2556562619 shutdown_connections 2024-04-12T06:17:11.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 -- 172.21.15.8:0/2556562619 wait complete. 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 Processor -- start 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 -- start start 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc19f450 con 0x7f1dfc068580 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e040b0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc198900 con 0x7f1dfc068ea0 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.876+0000 7f1e01e4c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37840/0 (socket says 172.21.15.8:37840) 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 -- 172.21.15.8:0/230195524 learned_addr learned my addr 172.21.15.8:0/230195524 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e0164b700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 -- 172.21.15.8:0/230195524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 msgr2=0x7f1dfc19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:11.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 -- 172.21.15.8:0/230195524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec0095f0 con 0x7f1dfc068580 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e0164b700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e01e4c700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b4b0 tx=0x7f1df800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df8011800 con 0x7f1dfc068580 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1df8011e00 con 0x7f1dfc068580 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc198be0 con 0x7f1dfc068580 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df8010e50 con 0x7f1dfc068580 2024-04-12T06:17:11.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.877+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc199130 con 0x7f1dfc068580 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.878+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc10aa00 con 0x7f1dfc068580 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.879+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1df8010380 con 0x7f1dfc068580 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.879+0000 7f1df2ffd700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 0x7f1de806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.879+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1df808c120 con 0x7f1dfc068580 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.879+0000 7f1e0164b700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 0x7f1de806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:11.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.879+0000 7f1e0164b700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 0x7f1de806f1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b5c0 tx=0x7f1dec005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:11.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:11.884+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1df8059c20 con 0x7f1dfc068580 2024-04-12T06:17:12.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:11 smithi194 ceph-mon[106982]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:12.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:11 smithi008 ceph-mon[146084]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:12.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.173+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1dfc04f2e0 con 0x7f1dfc068580 2024-04-12T06:17:12.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.174+0000 7f1df2ffd700 1 -- 172.21.15.8:0/230195524 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1df80597b0 con 0x7f1dfc068580 2024-04-12T06:17:12.173 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:17:12.173 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:17:12.173 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:17:12.174 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 msgr2=0x7f1de806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 0x7f1de806f1a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00b5c0 tx=0x7f1dec005960 comp rx=0 tx=0).stop 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 msgr2=0x7f1dfc19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b4b0 tx=0x7f1df800b870 comp rx=0 tx=0).stop 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 shutdown_connections 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1de806cce0 0x7f1de806f1a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1dfc068580 0x7f1dfc19e880 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 --2- 172.21.15.8:0/230195524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc068ea0 0x7f1dfc19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 >> 172.21.15.8:0/230195524 conn(0x7f1dfc075670 msgr2=0x7f1dfc0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 shutdown_connections 2024-04-12T06:17:12.177 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.176+0000 7f1e040b0700 1 -- 172.21.15.8:0/230195524 wait complete. 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3482888492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec4105bf0 msgr2=0x7f3ec4105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3482888492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4105fd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009910 tx=0x7f3eb8009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3482888492 shutdown_connections 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3482888492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec40ffb40 0x7f3ec40fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3482888492 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4105fd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3482888492 >> 172.21.15.8:0/3482888492 conn(0x7f3ec40fb610 msgr2=0x7f3ec40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3482888492 shutdown_connections 2024-04-12T06:17:12.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.282+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3482888492 wait complete. 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 Processor -- start 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 -- start start 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4196260 con 0x7f3ec40ffb40 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ecb0d0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec4199ff0 con 0x7f3ec4105bf0 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37868/0 (socket says 172.21.15.8:37868) 2024-04-12T06:17:12.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 -- 172.21.15.8:0/3411404265 learned_addr learned my addr 172.21.15.8:0/3411404265 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec3fff700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 -- 172.21.15.8:0/3411404265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 msgr2=0x7f3ec4195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.283+0000 7f3ec8e6c700 1 -- 172.21.15.8:0/3411404265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb80095f0 con 0x7f3ec40ffb40 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ec3fff700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ec8e6c700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb801f070 con 0x7f3ec40ffb40 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec419a270 con 0x7f3ec40ffb40 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3eb8019610 con 0x7f3ec40ffb40 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb800f670 con 0x7f3ec40ffb40 2024-04-12T06:17:12.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.284+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec419a760 con 0x7f3ec40ffb40 2024-04-12T06:17:12.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.285+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec404f2e0 con 0x7f3ec40ffb40 2024-04-12T06:17:12.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.286+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3eb8019bd0 con 0x7f3ec40ffb40 2024-04-12T06:17:12.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.286+0000 7f3ec1ffb700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 0x7f3eb406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.286+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3eb808f9c0 con 0x7f3ec40ffb40 2024-04-12T06:17:12.286 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.286+0000 7f3ec3fff700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 0x7f3eb406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.286+0000 7f3ec3fff700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 0x7f3eb406f1a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005fd0 tx=0x7f3eb0005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:12.290 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.291+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3eb8059a30 con 0x7f3ec40ffb40 2024-04-12T06:17:12.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.542+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3ec419aae0 con 0x7f3ec40ffb40 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.542+0000 7f3ec1ffb700 1 -- 172.21.15.8:0/3411404265 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f3eb805d050 con 0x7f3ec40ffb40 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:17:12.542 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:17:12.543 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:17:12.544 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:12.545 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 msgr2=0x7f3eb406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 0x7f3eb406f1a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005fd0 tx=0x7f3eb0005ee0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 msgr2=0x7f3ec4195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8006010 tx=0x7f3eb800be20 comp rx=0 tx=0).stop 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 shutdown_connections 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3eb406cce0 0x7f3eb406f1a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3ec40ffb40 0x7f3ec4195640 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 --2- 172.21.15.8:0/3411404265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3ec4105bf0 0x7f3ec4195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 >> 172.21.15.8:0/3411404265 conn(0x7f3ec40fb610 msgr2=0x7f3ec40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 shutdown_connections 2024-04-12T06:17:12.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.544+0000 7f3ecb0d0700 1 -- 172.21.15.8:0/3411404265 wait complete. 2024-04-12T06:17:12.547 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 -- 172.21.15.8:0/771409336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 msgr2=0x7fa4e4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 --2- 172.21.15.8:0/771409336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e4100080 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009910 tx=0x7fa4d8009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 -- 172.21.15.8:0/771409336 shutdown_connections 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 --2- 172.21.15.8:0/771409336 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e4100080 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 --2- 172.21.15.8:0/771409336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 -- 172.21.15.8:0/771409336 >> 172.21.15.8:0/771409336 conn(0x7fa4e4075670 msgr2=0x7fa4e4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 -- 172.21.15.8:0/771409336 shutdown_connections 2024-04-12T06:17:12.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.651+0000 7fa4e9133700 1 -- 172.21.15.8:0/771409336 wait complete. 2024-04-12T06:17:12.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.652+0000 7fa4e9133700 1 Processor -- start 2024-04-12T06:17:12.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.652+0000 7fa4e9133700 1 -- start start 2024-04-12T06:17:12.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.653 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e419d150 con 0x7fa4e40ffc20 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4196600 con 0x7fa4e4105bd0 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37888/0 (socket says 172.21.15.8:37888) 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 -- 172.21.15.8:0/202958757 learned_addr learned my addr 172.21.15.8:0/202958757 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e259c700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 -- 172.21.15.8:0/202958757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 msgr2=0x7fa4e419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 -- 172.21.15.8:0/202958757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d80095f0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e259c700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e2d9d700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c8a0 tx=0x7fa4d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d400cea0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4d4004830 con 0x7fa4e40ffc20 2024-04-12T06:17:12.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d4005500 con 0x7fa4e40ffc20 2024-04-12T06:17:12.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e41968e0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.653+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e4196e30 con 0x7fa4e40ffc20 2024-04-12T06:17:12.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.654+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e404f2e0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.655+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa4d40049a0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.655+0000 7fa4cbfff700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 0x7fa4cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.655+0000 7fa4e259c700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 0x7fa4cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.655+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa4d408c1c0 con 0x7fa4e40ffc20 2024-04-12T06:17:12.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.656+0000 7fa4e259c700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 0x7fa4cc06f1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8006010 tx=0x7fa4d800b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:12.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.660+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa4d4059d40 con 0x7fa4e40ffc20 2024-04-12T06:17:12.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.883+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e41971f0 con 0x7fa4cc06cce0 2024-04-12T06:17:12.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.884+0000 7fa4cbfff700 1 -- 172.21.15.8:0/202958757 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa4e41971f0 con 0x7fa4cc06cce0 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:17:12.884 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:17:12.885 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 msgr2=0x7fa4cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 0x7fa4cc06f1a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8006010 tx=0x7fa4d800b540 comp rx=0 tx=0).stop 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 msgr2=0x7fa4e419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c8a0 tx=0x7fa4d400cc60 comp rx=0 tx=0).stop 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 shutdown_connections 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa4cc06cce0 0x7fa4cc06f1a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa4e40ffc20 0x7fa4e419c580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 --2- 172.21.15.8:0/202958757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa4e4105bd0 0x7fa4e419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 >> 172.21.15.8:0/202958757 conn(0x7fa4e4075670 msgr2=0x7fa4e4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 shutdown_connections 2024-04-12T06:17:12.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.886+0000 7fa4e9133700 1 -- 172.21.15.8:0/202958757 wait complete. 2024-04-12T06:17:12.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 -- 172.21.15.8:0/1702940834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134068580 msgr2=0x7f31340689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 --2- 172.21.15.8:0/1702940834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134068580 0x7f31340689e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3128009910 tx=0x7f3128009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 -- 172.21.15.8:0/1702940834 shutdown_connections 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 --2- 172.21.15.8:0/1702940834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134068580 0x7f31340689e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 --2- 172.21.15.8:0/1702940834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134106cd0 0x7f31341070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 -- 172.21.15.8:0/1702940834 >> 172.21.15.8:0/1702940834 conn(0x7f3134075670 msgr2=0x7f3134075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 -- 172.21.15.8:0/1702940834 shutdown_connections 2024-04-12T06:17:12.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.993+0000 7f313a8a5700 1 -- 172.21.15.8:0/1702940834 wait complete. 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 Processor -- start 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 -- start start 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134068580 0x7f313419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313419d110 con 0x7f3134106cd0 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f313a8a5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31341965c0 con 0x7f3134068580 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:37892/0 (socket says 172.21.15.8:37892) 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 -- 172.21.15.8:0/2667153338 learned_addr learned my addr 172.21.15.8:0/2667153338 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f3133fff700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134068580 0x7f313419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 -- 172.21.15.8:0/2667153338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134068580 msgr2=0x7f313419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:12.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134068580 0x7f313419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.994+0000 7f31337fe700 1 -- 172.21.15.8:0/2667153338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31280095f0 con 0x7f3134106cd0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f31337fe700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f312800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f312801d070 con 0x7f3134106cd0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f31280044c0 con 0x7f3134106cd0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134196840 con 0x7f3134106cd0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31280039d0 con 0x7f3134106cd0 2024-04-12T06:17:12.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.995+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134196cd0 con 0x7f3134106cd0 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.996+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31341091d0 con 0x7f3134106cd0 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.996+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f31280048e0 con 0x7f3134106cd0 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.997+0000 7f31317fa700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 0x7f311c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.997+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f312808db50 con 0x7f3134106cd0 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.997+0000 7f3133fff700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 0x7f311c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:12.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:12.997+0000 7f3133fff700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 0x7f311c06f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3124009b30 tx=0x7f3124009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:13.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.002+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f312805b6d0 con 0x7f3134106cd0 2024-04-12T06:17:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:12 smithi194 ceph-mon[106982]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:12 smithi194 ceph-mon[106982]: pgmap v665: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:12 smithi194 ceph-mon[106982]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:13.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:12 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/230195524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:17:13.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:12 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3411404265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:17:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:12 smithi008 ceph-mon[146084]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:12 smithi008 ceph-mon[146084]: pgmap v665: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:13.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:12 smithi008 ceph-mon[146084]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:13.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:12 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/230195524' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:17:13.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:12 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3411404265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.289+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3134197b70 con 0x7f3134106cd0 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.289+0000 7f31317fa700 1 -- 172.21.15.8:0/2667153338 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3128026090 con 0x7f3134106cd0 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:17:13.289 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 msgr2=0x7f311c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 0x7f311c06f1a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3124009b30 tx=0x7f3124009260 comp rx=0 tx=0).stop 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 msgr2=0x7f313419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3128000c00 tx=0x7f312800be90 comp rx=0 tx=0).stop 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 shutdown_connections 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f311c06cce0 0x7f311c06f1a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3134068580 0x7f313419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:13.291 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 --2- 172.21.15.8:0/2667153338 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3134106cd0 0x7f313419ca80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:13.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 >> 172.21.15.8:0/2667153338 conn(0x7f3134075670 msgr2=0x7f31340fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:13.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 shutdown_connections 2024-04-12T06:17:13.292 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:13.291+0000 7f313a8a5700 1 -- 172.21.15.8:0/2667153338 wait complete. 2024-04-12T06:17:14.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:13 smithi194 ceph-mon[106982]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:14.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:13 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2667153338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:17:14.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:13 smithi008 ceph-mon[146084]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:14.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:13 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2667153338' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:17:15.052 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:14 smithi008 ceph-mon[146084]: pgmap v666: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:15.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:14 smithi194 ceph-mon[106982]: pgmap v666: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:17.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:16 smithi194 ceph-mon[106982]: pgmap v667: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:17.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:16 smithi008 ceph-mon[146084]: pgmap v667: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:18.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:18 smithi194 ceph-mon[106982]: pgmap v668: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:19.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:18 smithi008 ceph-mon[146084]: pgmap v668: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:21.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:20 smithi194 ceph-mon[106982]: pgmap v669: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:21.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:20 smithi008 ceph-mon[146084]: pgmap v669: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:22 smithi194 ceph-mon[106982]: pgmap v670: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:23.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:22 smithi008 ceph-mon[146084]: pgmap v670: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:25.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:24 smithi194 ceph-mon[106982]: pgmap v671: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:25.146 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:24 smithi008 ceph-mon[146084]: pgmap v671: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:27.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:26 smithi194 ceph-mon[106982]: pgmap v672: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:27.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:26 smithi008 ceph-mon[146084]: pgmap v672: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:29.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:28 smithi194 ceph-mon[106982]: pgmap v673: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:29.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:28 smithi008 ceph-mon[146084]: pgmap v673: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:31.084 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:30 smithi194 ceph-mon[106982]: pgmap v674: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:31.167 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:30 smithi008 ceph-mon[146084]: pgmap v674: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:32 smithi194 ceph-mon[106982]: pgmap v675: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:33.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:32 smithi008 ceph-mon[146084]: pgmap v675: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:35.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:34 smithi194 ceph-mon[106982]: pgmap v676: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:35.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:34 smithi008 ceph-mon[146084]: pgmap v676: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:37.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:36 smithi194 ceph-mon[106982]: pgmap v677: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:37.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:36 smithi008 ceph-mon[146084]: pgmap v677: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:39.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:38 smithi194 ceph-mon[106982]: pgmap v678: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:39.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:38 smithi008 ceph-mon[146084]: pgmap v678: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:41.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:40 smithi194 ceph-mon[106982]: pgmap v679: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:41.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:40 smithi008 ceph-mon[146084]: pgmap v679: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:43.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:42 smithi194 ceph-mon[106982]: pgmap v680: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:43.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:42 smithi008 ceph-mon[146084]: pgmap v680: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 -- 172.21.15.8:0/418490321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 msgr2=0x7fa0701058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 --2- 172.21.15.8:0/418490321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0701058a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa05c009910 tx=0x7fa05c009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 -- 172.21.15.8:0/418490321 shutdown_connections 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 --2- 172.21.15.8:0/418490321 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0701058a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 --2- 172.21.15.8:0/418490321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 0x7fa070101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 -- 172.21.15.8:0/418490321 >> 172.21.15.8:0/418490321 conn(0x7fa0700fc750 msgr2=0x7fa0700feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 -- 172.21.15.8:0/418490321 shutdown_connections 2024-04-12T06:17:43.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.412+0000 7fa076d16700 1 -- 172.21.15.8:0/418490321 wait complete. 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 Processor -- start 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 -- start start 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 0x7fa070073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa070073b10 con 0x7fa070101850 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa076d16700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa070073c50 con 0x7fa070100ea0 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa06ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa06ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52146/0 (socket says 172.21.15.8:52146) 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa06ffff700 1 -- 172.21.15.8:0/2161009549 learned_addr learned my addr 172.21.15.8:0/2161009549 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa074ab2700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 0x7fa070073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.413+0000 7fa06ffff700 1 -- 172.21.15.8:0/2161009549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 msgr2=0x7fa070073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06ffff700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 0x7fa070073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06ffff700 1 -- 172.21.15.8:0/2161009549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0095f0 con 0x7fa070101850 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06ffff700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000c00 tx=0x7fa05c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa05c01d070 con 0x7fa070101850 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa070073ed0 con 0x7fa070101850 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa05c003cf0 con 0x7fa070101850 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa05c021410 con 0x7fa070101850 2024-04-12T06:17:43.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.414+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0701a2cf0 con 0x7fa070101850 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.416+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa05c003850 con 0x7fa070101850 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.416+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0701092a0 con 0x7fa070101850 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.416+0000 7fa06dffb700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 0x7fa06006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.416+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa05c08d7d0 con 0x7fa070101850 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.416+0000 7fa074ab2700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 0x7fa06006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.417+0000 7fa074ab2700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 0x7fa06006f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa064009bf0 tx=0x7fa064009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:43.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.421+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa05c057840 con 0x7fa070101850 2024-04-12T06:17:43.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.643+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0700611d0 con 0x7fa06006cce0 2024-04-12T06:17:43.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.644+0000 7fa06dffb700 1 -- 172.21.15.8:0/2161009549 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa0700611d0 con 0x7fa06006cce0 2024-04-12T06:17:43.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 msgr2=0x7fa06006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 0x7fa06006f1a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa064009bf0 tx=0x7fa064009260 comp rx=0 tx=0).stop 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 msgr2=0x7fa0700735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa05c000c00 tx=0x7fa05c00bb20 comp rx=0 tx=0).stop 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 shutdown_connections 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa06006cce0 0x7fa06006f1a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa070100ea0 0x7fa070073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 --2- 172.21.15.8:0/2161009549 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa070101850 0x7fa0700735d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 >> 172.21.15.8:0/2161009549 conn(0x7fa0700fc750 msgr2=0x7fa0700fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 shutdown_connections 2024-04-12T06:17:43.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.646+0000 7fa076d16700 1 -- 172.21.15.8:0/2161009549 wait complete. 2024-04-12T06:17:43.656 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 -- 172.21.15.8:0/233015638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 msgr2=0x7f6768101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 --2- 172.21.15.8:0/233015638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768101280 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f675800b0a0 tx=0x7f675800b3b0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 -- 172.21.15.8:0/233015638 shutdown_connections 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 --2- 172.21.15.8:0/233015638 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 0x7f67681058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 --2- 172.21.15.8:0/233015638 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768101280 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.753 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 -- 172.21.15.8:0/233015638 >> 172.21.15.8:0/233015638 conn(0x7f67680fc750 msgr2=0x7f67680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 -- 172.21.15.8:0/233015638 shutdown_connections 2024-04-12T06:17:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.753+0000 7f676cea7700 1 -- 172.21.15.8:0/233015638 wait complete. 2024-04-12T06:17:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 Processor -- start 2024-04-12T06:17:43.754 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 -- start start 2024-04-12T06:17:43.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 0x7f6768198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6768199550 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676cea7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676819d360 con 0x7f6768101850 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52158/0 (socket says 172.21.15.8:52158) 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 -- 172.21.15.8:0/1490008553 learned_addr learned my addr 172.21.15.8:0/1490008553 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f6765d9b700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 0x7f6768198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 -- 172.21.15.8:0/1490008553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 msgr2=0x7f6768198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 0x7f6768198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.754+0000 7f676659c700 1 -- 172.21.15.8:0/1490008553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758009d00 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f676659c700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f675800d010 tx=0x7f6758007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675800e040 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6758003de0 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676819d5e0 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f675801d6b0 con 0x7f6768100ea0 2024-04-12T06:17:43.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.755+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676819dad0 con 0x7f6768100ea0 2024-04-12T06:17:43.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.756+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6768105020 con 0x7f6768100ea0 2024-04-12T06:17:43.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.756+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f6758004670 con 0x7f6768100ea0 2024-04-12T06:17:43.757 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.757+0000 7f675f7fe700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 0x7f675406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:43.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.757+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f675808de70 con 0x7f6768100ea0 2024-04-12T06:17:43.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.757+0000 7f6765d9b700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 0x7f675406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:43.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.757+0000 7f6765d9b700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 0x7f675406f1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6768199fa0 tx=0x7f675000b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:43.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.761+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f675805b970 con 0x7f6768100ea0 2024-04-12T06:17:43.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.983+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6768199df0 con 0x7f675406cce0 2024-04-12T06:17:43.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.984+0000 7f675f7fe700 1 -- 172.21.15.8:0/1490008553 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6768199df0 con 0x7f675406cce0 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 msgr2=0x7f675406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 0x7f675406f1a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6768199fa0 tx=0x7f675000b410 comp rx=0 tx=0).stop 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 msgr2=0x7f6768198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f675800d010 tx=0x7f6758007b30 comp rx=0 tx=0).stop 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 shutdown_connections 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f675406cce0 0x7f675406f1a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6768100ea0 0x7f6768198980 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 --2- 172.21.15.8:0/1490008553 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6768101850 0x7f6768198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 >> 172.21.15.8:0/1490008553 conn(0x7f67680fc750 msgr2=0x7f67680fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 shutdown_connections 2024-04-12T06:17:43.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:43.986+0000 7f676cea7700 1 -- 172.21.15.8:0/1490008553 wait complete. 2024-04-12T06:17:44.092 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.092+0000 7fc7b0a61700 1 -- 172.21.15.8:0/1713577723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0fefc0 msgr2=0x7fc7ac0ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.092+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/1713577723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac0ff3a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009940 tx=0x7fc79c009c50 comp rx=0 tx=0).stop 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 -- 172.21.15.8:0/1713577723 shutdown_connections 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/1713577723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac10cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/1713577723 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac0ff3a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 -- 172.21.15.8:0/1713577723 >> 172.21.15.8:0/1713577723 conn(0x7fc7ac075e10 msgr2=0x7fc7ac076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 -- 172.21.15.8:0/1713577723 shutdown_connections 2024-04-12T06:17:44.093 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 -- 172.21.15.8:0/1713577723 wait complete. 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 Processor -- start 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.093+0000 7fc7b0a61700 1 -- start start 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7b0a61700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7b0a61700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7b0a61700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac199560 con 0x7fc7ac0ff970 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7b0a61700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ac19d370 con 0x7fc7ac0fefc0 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52184/0 (socket says 172.21.15.8:52184) 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 -- 172.21.15.8:0/2061615373 learned_addr learned my addr 172.21.15.8:0/2061615373 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7aa59c700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 -- 172.21.15.8:0/2061615373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0fefc0 msgr2=0x7fc7ac198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 -- 172.21.15.8:0/2061615373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0095f0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7a9d9b700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d8d0 tx=0x7fc7a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a0009940 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7a0010460 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7ac19d650 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.094+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a000f5f0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.095+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7ac19dba0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.095+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7ac10a6a0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.096+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc7a00105d0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.096+0000 7fc79b7fe700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 0x7fc79406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.096+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc7a008c5d0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.096+0000 7fc7aa59c700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 0x7fc79406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.097 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.097+0000 7fc7aa59c700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 0x7fc79406f1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006010 tx=0x7fc79c005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.100 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.101+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc7a005a0d0 con 0x7fc7ac0ff970 2024-04-12T06:17:44.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.316+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7ac199db0 con 0x7fc79406cce0 2024-04-12T06:17:44.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.327+0000 7fc79b7fe700 1 -- 172.21.15.8:0/2061615373 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc7ac199db0 con 0x7fc79406cce0 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (19m) 13m ago 30m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (31m) 13m ago 31m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (30m) 12m ago 30m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (17m) 13m ago 31m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (17m) 12m ago 30m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (19m) 13m ago 30m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:17:44.327 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (13m) 13m ago 24m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (13m) 13m ago 24m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (12m) 12m ago 24m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (13m) 12m ago 24m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (21m) 13m ago 32m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (20m) 12m ago 29m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (17m) 13m ago 32m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (17m) 12m ago 29m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:17:44.328 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (20m) 13m ago 31m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (20m) 12m ago 29m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (16m) 13m ago 28m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (16m) 13m ago 28m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (15m) 13m ago 27m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (15m) 12m ago 27m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (14m) 12m ago 27m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (13m) 12m ago 26m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:17:44.329 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (19m) 13m ago 30m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:17:44.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 msgr2=0x7fc79406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 0x7fc79406f1a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc79c006010 tx=0x7fc79c005940 comp rx=0 tx=0).stop 2024-04-12T06:17:44.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 msgr2=0x7fc7ac198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d8d0 tx=0x7fc7a000dc90 comp rx=0 tx=0).stop 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 shutdown_connections 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc79406cce0 0x7fc79406f1a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc7ac0fefc0 0x7fc7ac198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 --2- 172.21.15.8:0/2061615373 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc7ac0ff970 0x7fc7ac198ed0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 >> 172.21.15.8:0/2061615373 conn(0x7fc7ac075e10 msgr2=0x7fc7ac0fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:44.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 shutdown_connections 2024-04-12T06:17:44.332 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.329+0000 7fc7b0a61700 1 -- 172.21.15.8:0/2061615373 wait complete. 2024-04-12T06:17:44.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/4179711108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 msgr2=0x7fc3e8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/4179711108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e8100080 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009910 tx=0x7fc3d4009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:44.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/4179711108 shutdown_connections 2024-04-12T06:17:44.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/4179711108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e8100080 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.434 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/4179711108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/4179711108 >> 172.21.15.8:0/4179711108 conn(0x7fc3e8075670 msgr2=0x7fc3e8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:44.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/4179711108 shutdown_connections 2024-04-12T06:17:44.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.434+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/4179711108 wait complete. 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 Processor -- start 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 -- start start 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8198200 con 0x7fc3e80ffc20 2024-04-12T06:17:44.437 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.435+0000 7fc3eeb1a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e8198370 con 0x7fc3e8105bd0 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52204/0 (socket says 172.21.15.8:52204) 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 -- 172.21.15.8:0/329127788 learned_addr learned my addr 172.21.15.8:0/329127788 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3e7fff700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 -- 172.21.15.8:0/329127788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 msgr2=0x7fc3e8197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 -- 172.21.15.8:0/329127788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40095f0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3e7fff700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3ec8b6700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00d8d0 tx=0x7fc3dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc009940 con 0x7fc3e80ffc20 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3dc010460 con 0x7fc3e80ffc20 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e81a0710 con 0x7fc3e80ffc20 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3dc00f5f0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.436+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e81a0c00 con 0x7fc3e80ffc20 2024-04-12T06:17:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.437+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e81080d0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.438+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc3dc0105d0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.438+0000 7fc3e5ffb700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 0x7fc3d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.438+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc3dc08c4c0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.438+0000 7fc3e7fff700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 0x7fc3d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.440 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.439+0000 7fc3e7fff700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 0x7fc3d806f1a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d400b540 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.442 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.443+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc3dc0565b0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:44 smithi194 ceph-mon[106982]: pgmap v681: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:44.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:44 smithi194 ceph-mon[106982]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:44.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:44 smithi194 ceph-mon[106982]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:44.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:44 smithi008 ceph-mon[146084]: pgmap v681: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:44.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:44 smithi008 ceph-mon[146084]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:44.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:44 smithi008 ceph-mon[146084]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:44.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.728+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc3e8107ab0 con 0x7fc3e80ffc20 2024-04-12T06:17:44.728 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.729+0000 7fc3e5ffb700 1 -- 172.21.15.8:0/329127788 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc3dc010880 con 0x7fc3e80ffc20 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:17:44.729 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:17:44.730 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:17:44.730 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:17:44.730 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:17:44.730 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 msgr2=0x7fc3d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 0x7fc3d806f1a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4006010 tx=0x7fc3d400b540 comp rx=0 tx=0).stop 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 msgr2=0x7fc3e81976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00d8d0 tx=0x7fc3dc00dc90 comp rx=0 tx=0).stop 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 shutdown_connections 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc3d806cce0 0x7fc3d806f1a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc3e80ffc20 0x7fc3e81976a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 --2- 172.21.15.8:0/329127788 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc3e8105bd0 0x7fc3e8197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 >> 172.21.15.8:0/329127788 conn(0x7fc3e8075670 msgr2=0x7fc3e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 shutdown_connections 2024-04-12T06:17:44.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.731+0000 7fc3eeb1a700 1 -- 172.21.15.8:0/329127788 wait complete. 2024-04-12T06:17:44.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- 172.21.15.8:0/2122379380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 msgr2=0x7f13f0100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 --2- 172.21.15.8:0/2122379380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f0100080 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009910 tx=0x7f13e0009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- 172.21.15.8:0/2122379380 shutdown_connections 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 --2- 172.21.15.8:0/2122379380 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f0100080 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 --2- 172.21.15.8:0/2122379380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 0x7f13f0105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- 172.21.15.8:0/2122379380 >> 172.21.15.8:0/2122379380 conn(0x7f13f0075670 msgr2=0x7f13f0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- 172.21.15.8:0/2122379380 shutdown_connections 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- 172.21.15.8:0/2122379380 wait complete. 2024-04-12T06:17:44.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 Processor -- start 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.837+0000 7f13f807a700 1 -- start start 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f807a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f807a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 0x7f13f019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f807a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f019c0f0 con 0x7f13f00ffc20 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f807a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f01955a0 con 0x7f13f0105bd0 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52234/0 (socket says 172.21.15.8:52234) 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 -- 172.21.15.8:0/3578825200 learned_addr learned my addr 172.21.15.8:0/3578825200 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5615700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 0x7f13f019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 -- 172.21.15.8:0/3578825200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 msgr2=0x7f13f019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:44.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 0x7f13f019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 -- 172.21.15.8:0/3578825200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00095f0 con 0x7f13f00ffc20 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f5e16700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00d8d0 tx=0x7f13ec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ec0098a0 con 0x7f13f00ffc20 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f13ec010460 con 0x7f13f00ffc20 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f0195880 con 0x7f13f00ffc20 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.838+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ec009ea0 con 0x7f13f00ffc20 2024-04-12T06:17:44.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.839+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f0195dd0 con 0x7f13f00ffc20 2024-04-12T06:17:44.842 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.839+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f01080d0 con 0x7f13f00ffc20 2024-04-12T06:17:44.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.840+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f13ec0105d0 con 0x7f13f00ffc20 2024-04-12T06:17:44.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.840+0000 7f13e6ffd700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 0x7f13dc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:44.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.840+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f13ec08c440 con 0x7f13f00ffc20 2024-04-12T06:17:44.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.840+0000 7f13f5615700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 0x7f13dc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:44.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.841+0000 7f13f5615700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 0x7f13dc06f1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13f00fe2d0 tx=0x7f13e001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:44.844 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:44.845+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f13ec059f40 con 0x7f13f00ffc20 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.090+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13f004f2e0 con 0x7f13f00ffc20 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.090+0000 7f13e6ffd700 1 -- 172.21.15.8:0/3578825200 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f13ec010880 con 0x7f13f00ffc20 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:17:45.090 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:17:45.091 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:17:45.092 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:45.093 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:45.094 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:17:45.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 msgr2=0x7f13dc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.094 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 0x7f13dc06f1a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f13f00fe2d0 tx=0x7f13e001a040 comp rx=0 tx=0).stop 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 msgr2=0x7f13f019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00d8d0 tx=0x7f13ec00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 shutdown_connections 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f13dc06cce0 0x7f13dc06f1a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f13f00ffc20 0x7f13f019b520 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 --2- 172.21.15.8:0/3578825200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13f0105bd0 0x7f13f019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.092+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 >> 172.21.15.8:0/3578825200 conn(0x7f13f0075670 msgr2=0x7f13f00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.093+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 shutdown_connections 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.093+0000 7f13f807a700 1 -- 172.21.15.8:0/3578825200 wait complete. 2024-04-12T06:17:45.095 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:17:45.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.199+0000 7febbc9de700 1 -- 172.21.15.8:0/2911778501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8101de0 msgr2=0x7febb81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.199 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.199+0000 7febbc9de700 1 --2- 172.21.15.8:0/2911778501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8101de0 0x7febb81021c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feba8009910 tx=0x7feba8009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 -- 172.21.15.8:0/2911778501 shutdown_connections 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 --2- 172.21.15.8:0/2911778501 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8102700 0x7febb810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 --2- 172.21.15.8:0/2911778501 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8101de0 0x7febb81021c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 -- 172.21.15.8:0/2911778501 >> 172.21.15.8:0/2911778501 conn(0x7febb80fb610 msgr2=0x7febb80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 -- 172.21.15.8:0/2911778501 shutdown_connections 2024-04-12T06:17:45.200 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.200+0000 7febbc9de700 1 -- 172.21.15.8:0/2911778501 wait complete. 2024-04-12T06:17:45.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 Processor -- start 2024-04-12T06:17:45.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 -- start start 2024-04-12T06:17:45.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8101de0 0x7febb819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb819d190 con 0x7febb8102700 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febbc9de700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb8196640 con 0x7febb8101de0 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52250/0 (socket says 172.21.15.8:52250) 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 -- 172.21.15.8:0/733176730 learned_addr learned my addr 172.21.15.8:0/733176730 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb659c700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8101de0 0x7febb819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 -- 172.21.15.8:0/733176730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8101de0 msgr2=0x7febb819c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8101de0 0x7febb819c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.201+0000 7febb5d9b700 1 -- 172.21.15.8:0/733176730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba80095f0 con 0x7febb8102700 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7febb5d9b700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febac00d6d0 tx=0x7febac00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac00f860 con 0x7febb8102700 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb8196920 con 0x7febb8102700 2024-04-12T06:17:45.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febac00fe60 con 0x7febb8102700 2024-04-12T06:17:45.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febac00e3b0 con 0x7febb8102700 2024-04-12T06:17:45.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.202+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb8196e70 con 0x7febb8102700 2024-04-12T06:17:45.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.203+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb81082f0 con 0x7febb8102700 2024-04-12T06:17:45.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.203+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7febac010460 con 0x7febb8102700 2024-04-12T06:17:45.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.204+0000 7feba77fe700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 0x7feba006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.204+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7febac08c150 con 0x7febb8102700 2024-04-12T06:17:45.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.204+0000 7febb659c700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 0x7feba006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.204+0000 7febb659c700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 0x7feba006f1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:45.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.208+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7febac059cd0 con 0x7febb8102700 2024-04-12T06:17:45.435 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.435+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febb8197140 con 0x7feba006cce0 2024-04-12T06:17:45.436 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.437+0000 7feba77fe700 1 -- 172.21.15.8:0/733176730 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7febb8197140 con 0x7feba006cce0 2024-04-12T06:17:45.436 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:17:45.436 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:17:45.436 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:17:45.436 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:17:45.437 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:17:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 msgr2=0x7feba006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 0x7feba006f1a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7feba8006010 tx=0x7feba801a040 comp rx=0 tx=0).stop 2024-04-12T06:17:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 msgr2=0x7febb819cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.438 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7febac00d6d0 tx=0x7febac00d9e0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 shutdown_connections 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7feba006cce0 0x7feba006f1a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febb8101de0 0x7febb819c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 --2- 172.21.15.8:0/733176730 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febb8102700 0x7febb819cb00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 >> 172.21.15.8:0/733176730 conn(0x7febb80fb610 msgr2=0x7febb8105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 shutdown_connections 2024-04-12T06:17:45.439 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.439+0000 7febbc9de700 1 -- 172.21.15.8:0/733176730 wait complete. 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 -- 172.21.15.8:0/3163293048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa874100750 msgr2=0x7fa8741047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 --2- 172.21.15.8:0/3163293048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa874100750 0x7fa8741047a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009910 tx=0x7fa85c009c20 comp rx=0 tx=0).stop 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 -- 172.21.15.8:0/3163293048 shutdown_connections 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 --2- 172.21.15.8:0/3163293048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa874100750 0x7fa8741047a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 --2- 172.21.15.8:0/3163293048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa8740ffda0 0x7fa874100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 -- 172.21.15.8:0/3163293048 >> 172.21.15.8:0/3163293048 conn(0x7fa8740fb650 msgr2=0x7fa8740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 -- 172.21.15.8:0/3163293048 shutdown_connections 2024-04-12T06:17:45.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.545+0000 7fa8798e7700 1 -- 172.21.15.8:0/3163293048 wait complete. 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 Processor -- start 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 -- start start 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa874100750 0x7fa87419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87419d220 con 0x7fa8740ffda0 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8798e7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8741966d0 con 0x7fa874100750 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa872ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.547 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa872ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:52272/0 (socket says 172.21.15.8:52272) 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa872ffd700 1 -- 172.21.15.8:0/2523927839 learned_addr learned my addr 172.21.15.8:0/2523927839 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.546+0000 7fa8727fc700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa874100750 0x7fa87419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa872ffd700 1 -- 172.21.15.8:0/2523927839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa874100750 msgr2=0x7fa87419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa872ffd700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa874100750 0x7fa87419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa872ffd700 1 -- 172.21.15.8:0/2523927839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0095f0 con 0x7fa8740ffda0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa872ffd700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c8a0 tx=0x7fa86400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86400cea0 con 0x7fa8740ffda0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8741969b0 con 0x7fa8740ffda0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa864004830 con 0x7fa8740ffda0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa864005500 con 0x7fa8740ffda0 2024-04-12T06:17:45.548 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.547+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874196f00 con 0x7fa8740ffda0 2024-04-12T06:17:45.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.548+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8741091b0 con 0x7fa8740ffda0 2024-04-12T06:17:45.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.548+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa8640049a0 con 0x7fa8740ffda0 2024-04-12T06:17:45.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.549+0000 7fa8788e5700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 0x7fa86006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:17:45.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.549+0000 7fa8727fc700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 0x7fa86006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:17:45.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.549+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa86408c270 con 0x7fa8740ffda0 2024-04-12T06:17:45.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.549+0000 7fa8727fc700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 0x7fa86006f1a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa874197b40 tx=0x7fa85c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:17:45.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.554+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa864059d70 con 0x7fa8740ffda0 2024-04-12T06:17:45.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:45 smithi194 ceph-mon[106982]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:45.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:45 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/329127788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:17:45.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:45 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3578825200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:17:45.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:45 smithi008 ceph-mon[146084]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:45.657 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:45 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/329127788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:17:45.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:45 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3578825200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.839+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa874197be0 con 0x7fa8740ffda0 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.839+0000 7fa8788e5700 1 -- 172.21.15.8:0/2523927839 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa864059900 con 0x7fa8740ffda0 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:17:45.839 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 msgr2=0x7fa86006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 0x7fa86006f1a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa874197b40 tx=0x7fa85c01a040 comp rx=0 tx=0).stop 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 msgr2=0x7fa87419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa86400c8a0 tx=0x7fa86400cc60 comp rx=0 tx=0).stop 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 shutdown_connections 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa86006cce0 0x7fa86006f1a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa8740ffda0 0x7fa87419c650 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 --2- 172.21.15.8:0/2523927839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa874100750 0x7fa87419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 >> 172.21.15.8:0/2523927839 conn(0x7fa8740fb650 msgr2=0x7fa874069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 shutdown_connections 2024-04-12T06:17:45.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:17:45.841+0000 7fa8798e7700 1 -- 172.21.15.8:0/2523927839 wait complete. 2024-04-12T06:17:46.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:46 smithi194 ceph-mon[106982]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:46.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:46 smithi194 ceph-mon[106982]: pgmap v682: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:46.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:46 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2523927839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:17:46.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:46 smithi008 ceph-mon[146084]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:17:46.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:46 smithi008 ceph-mon[146084]: pgmap v682: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:46.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:46 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2523927839' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:17:47.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:47.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:17:48.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:48 smithi194 ceph-mon[106982]: pgmap v683: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:48.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:48 smithi008 ceph-mon[146084]: pgmap v683: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:51.014 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:50 smithi194 ceph-mon[106982]: pgmap v684: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:51.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:50 smithi008 ceph-mon[146084]: pgmap v684: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:53.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:52 smithi194 ceph-mon[106982]: pgmap v685: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:53.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:52 smithi008 ceph-mon[146084]: pgmap v685: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:55.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:54 smithi194 ceph-mon[106982]: pgmap v686: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:55.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:54 smithi008 ceph-mon[146084]: pgmap v686: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:56 smithi194 ceph-mon[106982]: pgmap v687: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:56 smithi008 ceph-mon[146084]: pgmap v687: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:59.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:17:58 smithi194 ceph-mon[106982]: pgmap v688: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:17:59.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:17:58 smithi008 ceph-mon[146084]: pgmap v688: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:00.907 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:00 smithi008 ceph-mon[146084]: pgmap v689: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:01.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:00 smithi194 ceph-mon[106982]: pgmap v689: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:02 smithi194 ceph-mon[106982]: pgmap v690: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:02 smithi008 ceph-mon[146084]: pgmap v690: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:04.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:04 smithi194 ceph-mon[106982]: pgmap v691: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:04.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:04 smithi008 ceph-mon[146084]: pgmap v691: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:07.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:06 smithi194 ceph-mon[106982]: pgmap v692: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:07.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:06 smithi008 ceph-mon[146084]: pgmap v692: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:09.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:08 smithi194 ceph-mon[106982]: pgmap v693: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:09.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:08 smithi008 ceph-mon[146084]: pgmap v693: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:10.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:10 smithi008 ceph-mon[146084]: pgmap v694: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:11.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:10 smithi194 ceph-mon[106982]: pgmap v694: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:12 smithi194 ceph-mon[106982]: pgmap v695: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:12 smithi008 ceph-mon[146084]: pgmap v695: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:15.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:14 smithi194 ceph-mon[106982]: pgmap v696: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:15.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:14 smithi008 ceph-mon[146084]: pgmap v696: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:15.962 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 -- 172.21.15.8:0/3851002671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 msgr2=0x7f32641047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 --2- 172.21.15.8:0/3851002671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f32641047a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3254009910 tx=0x7f3254009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 -- 172.21.15.8:0/3851002671 shutdown_connections 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 --2- 172.21.15.8:0/3851002671 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f32641047a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 --2- 172.21.15.8:0/3851002671 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 0x7f3264100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 -- 172.21.15.8:0/3851002671 >> 172.21.15.8:0/3851002671 conn(0x7f32640fb650 msgr2=0x7f32640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 -- 172.21.15.8:0/3851002671 shutdown_connections 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.962+0000 7f326c558700 1 -- 172.21.15.8:0/3851002671 wait complete. 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 Processor -- start 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 -- start start 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 0x7f326419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:15.963 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326419d1a0 con 0x7f3264100750 2024-04-12T06:18:15.964 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326c558700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264196650 con 0x7f32640ffda0 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f3269af3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f3269af3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56872/0 (socket says 172.21.15.8:56872) 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f3269af3700 1 -- 172.21.15.8:0/3566559547 learned_addr learned my addr 172.21.15.8:0/3566559547 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.963+0000 7f326a2f4700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 0x7f326419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f3269af3700 1 -- 172.21.15.8:0/3566559547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 msgr2=0x7f326419c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f3269af3700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 0x7f326419c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f3269af3700 1 -- 172.21.15.8:0/3566559547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32540095f0 con 0x7f3264100750 2024-04-12T06:18:15.965 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f3269af3700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f325400bd70 tx=0x7f325400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325401d070 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f325400f460 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32641968d0 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3254021640 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.964+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3264196dc0 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.965+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326404f2e0 con 0x7f3264100750 2024-04-12T06:18:15.966 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.965+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f325400f9e0 con 0x7f3264100750 2024-04-12T06:18:15.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.966+0000 7f325b7fe700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 0x7f325006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:15.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.966+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f325408d650 con 0x7f3264100750 2024-04-12T06:18:15.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.966+0000 7f326a2f4700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 0x7f325006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:15.967 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.967+0000 7f326a2f4700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 0x7f325006f1a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3260007be0 tx=0x7f32600073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:15.970 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:15.971+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f325405b170 con 0x7f3264100750 2024-04-12T06:18:16.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.192+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32641970a0 con 0x7f325006cce0 2024-04-12T06:18:16.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.194+0000 7f325b7fe700 1 -- 172.21.15.8:0/3566559547 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32641970a0 con 0x7f325006cce0 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 msgr2=0x7f325006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 0x7f325006f1a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3260007be0 tx=0x7f32600073d0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 msgr2=0x7f326419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f325400bd70 tx=0x7f325400bda0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 shutdown_connections 2024-04-12T06:18:16.196 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f325006cce0 0x7f325006f1a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32640ffda0 0x7f326419c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 --2- 172.21.15.8:0/3566559547 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3264100750 0x7f326419cb10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 >> 172.21.15.8:0/3566559547 conn(0x7f32640fb650 msgr2=0x7f32640fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 shutdown_connections 2024-04-12T06:18:16.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.196+0000 7f326c558700 1 -- 172.21.15.8:0/3566559547 wait complete. 2024-04-12T06:18:16.206 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:18:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 -- 172.21.15.8:0/1290904724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec100ea0 msgr2=0x7f63ec101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/1290904724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec100ea0 0x7f63ec101280 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f63dc009910 tx=0x7f63dc009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 -- 172.21.15.8:0/1290904724 shutdown_connections 2024-04-12T06:18:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/1290904724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec101850 0x7f63ec1058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.304 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/1290904724 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec100ea0 0x7f63ec101280 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 -- 172.21.15.8:0/1290904724 >> 172.21.15.8:0/1290904724 conn(0x7f63ec0fc750 msgr2=0x7f63ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 -- 172.21.15.8:0/1290904724 shutdown_connections 2024-04-12T06:18:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 -- 172.21.15.8:0/1290904724 wait complete. 2024-04-12T06:18:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.304+0000 7f63f1c5b700 1 Processor -- start 2024-04-12T06:18:16.305 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63f1c5b700 1 -- start start 2024-04-12T06:18:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63f1c5b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec100ea0 0x7f63ec073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63f1c5b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63f1c5b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec074e40 con 0x7f63ec101850 2024-04-12T06:18:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63f1c5b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec074fb0 con 0x7f63ec100ea0 2024-04-12T06:18:16.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56890/0 (socket says 172.21.15.8:56890) 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 -- 172.21.15.8:0/216171170 learned_addr learned my addr 172.21.15.8:0/216171170 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eb7fe700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec100ea0 0x7f63ec073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 -- 172.21.15.8:0/216171170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec100ea0 msgr2=0x7f63ec073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec100ea0 0x7f63ec073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 -- 172.21.15.8:0/216171170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc0095f0 con 0x7f63ec101850 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.305+0000 7f63eaffd700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e8f0 tx=0x7f63d400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.306+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d4009790 con 0x7f63ec101850 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.306+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f63d400c810 con 0x7f63ec101850 2024-04-12T06:18:16.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.306+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec073bd0 con 0x7f63ec101850 2024-04-12T06:18:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.306+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63d4005520 con 0x7f63ec101850 2024-04-12T06:18:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.306+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec1a7300 con 0x7f63ec101850 2024-04-12T06:18:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.307+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec109240 con 0x7f63ec101850 2024-04-12T06:18:16.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.308+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f63d400c980 con 0x7f63ec101850 2024-04-12T06:18:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.308+0000 7f63e8ff9700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 0x7f63d806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.308+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f63d4014070 con 0x7f63ec101850 2024-04-12T06:18:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.308+0000 7f63eb7fe700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 0x7f63d806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.309+0000 7f63eb7fe700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 0x7f63d806f1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b6e0 tx=0x7f63dc005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.312+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f63d4056ae0 con 0x7f63ec101850 2024-04-12T06:18:16.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.532+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63ec0611d0 con 0x7f63d806cce0 2024-04-12T06:18:16.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.533+0000 7f63e8ff9700 1 -- 172.21.15.8:0/216171170 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f63ec0611d0 con 0x7f63d806cce0 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 msgr2=0x7f63d806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 0x7f63d806f1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b6e0 tx=0x7f63dc005a90 comp rx=0 tx=0).stop 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 msgr2=0x7f63ec0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f63d400e8f0 tx=0x7f63d400ec00 comp rx=0 tx=0).stop 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 shutdown_connections 2024-04-12T06:18:16.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f63d806cce0 0x7f63d806f1a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f63ec100ea0 0x7f63ec073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 --2- 172.21.15.8:0/216171170 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f63ec101850 0x7f63ec0735d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 >> 172.21.15.8:0/216171170 conn(0x7f63ec0fc750 msgr2=0x7f63ec0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.535+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 shutdown_connections 2024-04-12T06:18:16.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.536+0000 7f63f1c5b700 1 -- 172.21.15.8:0/216171170 wait complete. 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- 172.21.15.8:0/2857812443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c106cd0 msgr2=0x7f975c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 --2- 172.21.15.8:0/2857812443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c106cd0 0x7f975c1070b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f974c009910 tx=0x7f974c009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- 172.21.15.8:0/2857812443 shutdown_connections 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 --2- 172.21.15.8:0/2857812443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c068580 0x7f975c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 --2- 172.21.15.8:0/2857812443 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c106cd0 0x7f975c1070b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- 172.21.15.8:0/2857812443 >> 172.21.15.8:0/2857812443 conn(0x7f975c075670 msgr2=0x7f975c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- 172.21.15.8:0/2857812443 shutdown_connections 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- 172.21.15.8:0/2857812443 wait complete. 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 Processor -- start 2024-04-12T06:18:16.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.642+0000 7f9761ff4700 1 -- start start 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f9761ff4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f9761ff4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c106cd0 0x7f975c19caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f9761ff4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c19d130 con 0x7f975c068580 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f9761ff4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c196630 con 0x7f975c106cd0 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56898/0 (socket says 172.21.15.8:56898) 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 -- 172.21.15.8:0/3075003746 learned_addr learned my addr 172.21.15.8:0/3075003746 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975affd700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c106cd0 0x7f975c19caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 -- 172.21.15.8:0/3075003746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c106cd0 msgr2=0x7f975c19caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c106cd0 0x7f975c19caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 -- 172.21.15.8:0/3075003746 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c0095f0 con 0x7f975c068580 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f975b7fe700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.643+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974c01d070 con 0x7f975c068580 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.644+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f974c005dc0 con 0x7f975c068580 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.644+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c1968b0 con 0x7f975c068580 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.644+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974c003940 con 0x7f975c068580 2024-04-12T06:18:16.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.644+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c196d40 con 0x7f975c068580 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.645+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c1091f0 con 0x7f975c068580 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.645+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f974c004ae0 con 0x7f975c068580 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.645+0000 7f9758ff9700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 0x7f974406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.646+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f974c08db60 con 0x7f975c068580 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.646+0000 7f975affd700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 0x7f974406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.646+0000 7f975affd700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 0x7f974406f1a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f975c0fe390 tx=0x7f9750009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.649 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.650+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f974c05b660 con 0x7f975c068580 2024-04-12T06:18:16.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.863+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f975c196fc0 con 0x7f974406cce0 2024-04-12T06:18:16.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.873+0000 7f9758ff9700 1 -- 172.21.15.8:0/3075003746 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f975c196fc0 con 0x7f974406cce0 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (20m) 13m ago 31m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (31m) 13m ago 31m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (30m) 13m ago 30m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (17m) 13m ago 31m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (17m) 13m ago 30m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (19m) 13m ago 31m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (13m) 13m ago 25m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:18:16.874 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (14m) 13m ago 25m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (13m) 13m ago 25m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (13m) 13m ago 25m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (22m) 13m ago 33m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (21m) 13m ago 30m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (18m) 13m ago 33m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (18m) 13m ago 30m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (21m) 13m ago 31m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (20m) 13m ago 30m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (17m) 13m ago 29m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (16m) 13m ago 29m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:18:16.875 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (16m) 13m ago 28m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:18:16.876 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (15m) 13m ago 28m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:18:16.876 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (15m) 13m ago 27m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:18:16.876 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (14m) 13m ago 27m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:18:16.876 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (20m) 13m ago 30m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:18:16.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 msgr2=0x7f974406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 0x7f974406f1a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f975c0fe390 tx=0x7f9750009260 comp rx=0 tx=0).stop 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 msgr2=0x7f975c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f974c000c00 tx=0x7f974c00bde0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 shutdown_connections 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f974406cce0 0x7f974406f1a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f975c068580 0x7f975c19c560 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 --2- 172.21.15.8:0/3075003746 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f975c106cd0 0x7f975c19caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 >> 172.21.15.8:0/3075003746 conn(0x7f975c075670 msgr2=0x7f975c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 shutdown_connections 2024-04-12T06:18:16.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.876+0000 7f9761ff4700 1 -- 172.21.15.8:0/3075003746 wait complete. 2024-04-12T06:18:16.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 -- 172.21.15.8:0/1917169565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4101850 msgr2=0x7f22d41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 --2- 172.21.15.8:0/1917169565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4101850 0x7f22d41058a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009910 tx=0x7f22c8009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:16.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 -- 172.21.15.8:0/1917169565 shutdown_connections 2024-04-12T06:18:16.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 --2- 172.21.15.8:0/1917169565 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4101850 0x7f22d41058a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 --2- 172.21.15.8:0/1917169565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4100ea0 0x7f22d4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.980+0000 7f22d9764700 1 -- 172.21.15.8:0/1917169565 >> 172.21.15.8:0/1917169565 conn(0x7f22d40fc750 msgr2=0x7f22d40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.981+0000 7f22d9764700 1 -- 172.21.15.8:0/1917169565 shutdown_connections 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.981+0000 7f22d9764700 1 -- 172.21.15.8:0/1917169565 wait complete. 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.981+0000 7f22d9764700 1 Processor -- start 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.981+0000 7f22d9764700 1 -- start start 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.981+0000 7f22d9764700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d9764700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4101850 0x7f22d40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d9764700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4073b10 con 0x7f22d4100ea0 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d9764700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4073c50 con 0x7f22d4101850 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56914/0 (socket says 172.21.15.8:56914) 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 -- 172.21.15.8:0/710183075 learned_addr learned my addr 172.21.15.8:0/710183075 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:16.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d27fc700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4101850 0x7f22d40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 -- 172.21.15.8:0/710183075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4101850 msgr2=0x7f22d40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4101850 0x7f22d40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 -- 172.21.15.8:0/710183075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80095f0 con 0x7f22d4100ea0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d2ffd700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d8d0 tx=0x7f22c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c40098a0 con 0x7f22d4100ea0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41a2a40 con 0x7f22d4100ea0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f22c4010460 con 0x7f22d4100ea0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22c4009ea0 con 0x7f22d4100ea0 2024-04-12T06:18:16.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.982+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d41a2e50 con 0x7f22d4100ea0 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.984+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d41092a0 con 0x7f22d4100ea0 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.984+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f22c40105d0 con 0x7f22d4100ea0 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.984+0000 7f22bbfff700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 0x7f22bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.985+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f22c408c440 con 0x7f22d4100ea0 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.985+0000 7f22d27fc700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 0x7f22bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:16.985 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.985+0000 7f22d27fc700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 0x7f22bc06f1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22d4074880 tx=0x7f22c801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:16.988 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:16.989+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f22c40564b0 con 0x7f22d4100ea0 2024-04-12T06:18:17.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:16 smithi194 ceph-mon[106982]: pgmap v697: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:17.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:16 smithi008 ceph-mon[146084]: pgmap v697: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:17.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.275+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22d4108c80 con 0x7f22d4100ea0 2024-04-12T06:18:17.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.276+0000 7f22bbfff700 1 -- 172.21.15.8:0/710183075 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f22c4010880 con 0x7f22d4100ea0 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:18:17.276 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:17.277 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:18:17.277 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:18:17.277 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:18:17.277 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:18:17.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 msgr2=0x7f22bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 0x7f22bc06f1a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f22d4074880 tx=0x7f22c801a040 comp rx=0 tx=0).stop 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 msgr2=0x7f22d4073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d8d0 tx=0x7f22c400dbe0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 shutdown_connections 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f22bc06cce0 0x7f22bc06f1a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f22d4100ea0 0x7f22d4073090 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 --2- 172.21.15.8:0/710183075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f22d4101850 0x7f22d40735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 >> 172.21.15.8:0/710183075 conn(0x7f22d40fc750 msgr2=0x7f22d40fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 shutdown_connections 2024-04-12T06:18:17.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.278+0000 7f22d9764700 1 -- 172.21.15.8:0/710183075 wait complete. 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 -- 172.21.15.8:0/3403480740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b0105bd0 msgr2=0x7ff0b0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3403480740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0105fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b0a0 tx=0x7ff0a000b3b0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 -- 172.21.15.8:0/3403480740 shutdown_connections 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3403480740 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3403480740 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0105fb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 -- 172.21.15.8:0/3403480740 >> 172.21.15.8:0/3403480740 conn(0x7ff0b0075670 msgr2=0x7ff0b0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 -- 172.21.15.8:0/3403480740 shutdown_connections 2024-04-12T06:18:17.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.384+0000 7ff0b509b700 1 -- 172.21.15.8:0/3403480740 wait complete. 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 Processor -- start 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 -- start start 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0196260 con 0x7ff0b00ffc20 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0b509b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b0198f90 con 0x7ff0b0105bd0 2024-04-12T06:18:17.386 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0aed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0aed9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56928/0 (socket says 172.21.15.8:56928) 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.385+0000 7ff0aed9d700 1 -- 172.21.15.8:0/3909776531 learned_addr learned my addr 172.21.15.8:0/3909776531 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0ae59c700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0aed9d700 1 -- 172.21.15.8:0/3909776531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b0105bd0 msgr2=0x7ff0b0195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0aed9d700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0aed9d700 1 -- 172.21.15.8:0/3909776531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0009d00 con 0x7ff0b00ffc20 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0aed9d700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d010 tx=0x7ff0a0007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a000e040 con 0x7ff0b00ffc20 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff0a00044e0 con 0x7ff0b00ffc20 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b0199210 con 0x7ff0b00ffc20 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a001d910 con 0x7ff0b00ffc20 2024-04-12T06:18:17.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.386+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b0199700 con 0x7ff0b00ffc20 2024-04-12T06:18:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.387+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b01080f0 con 0x7ff0b00ffc20 2024-04-12T06:18:17.388 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.388+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff0a0004650 con 0x7ff0b00ffc20 2024-04-12T06:18:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.388+0000 7ff0a7fff700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 0x7ff09c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.388+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff0a008e010 con 0x7ff0b00ffc20 2024-04-12T06:18:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.388+0000 7ff0ae59c700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 0x7ff09c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.389 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.389+0000 7ff0ae59c700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 0x7ff09c06f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0196cb0 tx=0x7ff09800a300 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:17.392 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.393+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff0a0058080 con 0x7ff0b00ffc20 2024-04-12T06:18:17.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.657+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff0b0196ac0 con 0x7ff0b00ffc20 2024-04-12T06:18:17.657 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.658+0000 7ff0a7fff700 1 -- 172.21.15.8:0/3909776531 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7ff0a0004900 con 0x7ff0b00ffc20 2024-04-12T06:18:17.657 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:18:17.658 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:18:17.659 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:17.660 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 msgr2=0x7ff09c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 0x7ff09c06f1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0196cb0 tx=0x7ff09800a300 comp rx=0 tx=0).stop 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 msgr2=0x7ff0b0195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d010 tx=0x7ff0a0007b30 comp rx=0 tx=0).stop 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 shutdown_connections 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff09c06cce0 0x7ff09c06f1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff0b00ffc20 0x7ff0b0195690 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 --2- 172.21.15.8:0/3909776531 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0b0105bd0 0x7ff0b0195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 >> 172.21.15.8:0/3909776531 conn(0x7ff0b0075670 msgr2=0x7ff0b00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 shutdown_connections 2024-04-12T06:18:17.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.660+0000 7ff0b509b700 1 -- 172.21.15.8:0/3909776531 wait complete. 2024-04-12T06:18:17.662 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 -- 172.21.15.8:0/4149731000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 msgr2=0x7fb744105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 --2- 172.21.15.8:0/4149731000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744105fb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb740009910 tx=0x7fb740009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 -- 172.21.15.8:0/4149731000 shutdown_connections 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 --2- 172.21.15.8:0/4149731000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 0x7fb744100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 --2- 172.21.15.8:0/4149731000 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744105fb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 -- 172.21.15.8:0/4149731000 >> 172.21.15.8:0/4149731000 conn(0x7fb744075670 msgr2=0x7fb744075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:17.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 -- 172.21.15.8:0/4149731000 shutdown_connections 2024-04-12T06:18:17.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.768+0000 7fb74c51e700 1 -- 172.21.15.8:0/4149731000 wait complete. 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 Processor -- start 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 -- start start 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 0x7fb744195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441961d0 con 0x7fb744105bd0 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74c51e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb744196340 con 0x7fb7440ffc20 2024-04-12T06:18:17.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb749ab9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb749ab9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56952/0 (socket says 172.21.15.8:56952) 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb749ab9700 1 -- 172.21.15.8:0/2551279887 learned_addr learned my addr 172.21.15.8:0/2551279887 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.769+0000 7fb74a2ba700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 0x7fb744195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb749ab9700 1 -- 172.21.15.8:0/2551279887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 msgr2=0x7fb744195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb749ab9700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 0x7fb744195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb749ab9700 1 -- 172.21.15.8:0/2551279887 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400095f0 con 0x7fb744105bd0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb749ab9700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c930 tx=0x7fb73800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb738007ad0 con 0x7fb744105bd0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb73800ce80 con 0x7fb744105bd0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7441990e0 con 0x7fb744105bd0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb738018630 con 0x7fb744105bd0 2024-04-12T06:18:17.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.770+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb744199630 con 0x7fb744105bd0 2024-04-12T06:18:17.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.771+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7441080d0 con 0x7fb744105bd0 2024-04-12T06:18:17.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.771+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb738007c30 con 0x7fb744105bd0 2024-04-12T06:18:17.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.772+0000 7fb7377fe700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 0x7fb73006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:17.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.772+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb73808c570 con 0x7fb744105bd0 2024-04-12T06:18:17.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.772+0000 7fb74a2ba700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 0x7fb73006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:17.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.772+0000 7fb74a2ba700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 0x7fb73006f1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb740006010 tx=0x7fb74001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:17.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.776+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb7380565e0 con 0x7fb744105bd0 2024-04-12T06:18:17.998 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.998+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb74402ccc0 con 0x7fb73006cce0 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:17.999+0000 7fb7377fe700 1 -- 172.21.15.8:0/2551279887 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb74402ccc0 con 0x7fb73006cce0 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:18:17.999 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:18:18.000 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:18:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 msgr2=0x7fb73006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 0x7fb73006f1a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb740006010 tx=0x7fb74001a040 comp rx=0 tx=0).stop 2024-04-12T06:18:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 msgr2=0x7fb744195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb73800c930 tx=0x7fb73800cc40 comp rx=0 tx=0).stop 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 shutdown_connections 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb73006cce0 0x7fb73006f1a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7440ffc20 0x7fb744195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 --2- 172.21.15.8:0/2551279887 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb744105bd0 0x7fb744195bd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.001+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 >> 172.21.15.8:0/2551279887 conn(0x7fb744075670 msgr2=0x7fb7440fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.002+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 shutdown_connections 2024-04-12T06:18:18.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.002+0000 7fb74c51e700 1 -- 172.21.15.8:0/2551279887 wait complete. 2024-04-12T06:18:18.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.028 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.028 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.028 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/710183075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:18:18.028 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:18.028 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:17 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3909776531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:18:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/710183075' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:18:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:18.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:17 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3909776531' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 -- 172.21.15.8:0/3185854089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 msgr2=0x7f555c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 --2- 172.21.15.8:0/3185854089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c10abf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5544009910 tx=0x7f5544009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 -- 172.21.15.8:0/3185854089 shutdown_connections 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 --2- 172.21.15.8:0/3185854089 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c10abf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 --2- 172.21.15.8:0/3185854089 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 0x7f555c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 -- 172.21.15.8:0/3185854089 >> 172.21.15.8:0/3185854089 conn(0x7f555c0fb610 msgr2=0x7f555c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 -- 172.21.15.8:0/3185854089 shutdown_connections 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 -- 172.21.15.8:0/3185854089 wait complete. 2024-04-12T06:18:18.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.108+0000 7f5561b47700 1 Processor -- start 2024-04-12T06:18:18.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f5561b47700 1 -- start start 2024-04-12T06:18:18.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f5561b47700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 0x7f555c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f5561b47700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f5561b47700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c19d190 con 0x7f555c102700 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f5561b47700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c196640 con 0x7f555c101de0 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56966/0 (socket says 172.21.15.8:56966) 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 -- 172.21.15.8:0/1828747974 learned_addr learned my addr 172.21.15.8:0/1828747974 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555b7fe700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 0x7f555c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 -- 172.21.15.8:0/1828747974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 msgr2=0x7f555c19c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 0x7f555c19c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 -- 172.21.15.8:0/1828747974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440095f0 con 0x7f555c102700 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.109+0000 7f555affd700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55440092e0 tx=0x7f554400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.110+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554401d070 con 0x7f555c102700 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.110+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5544003d30 con 0x7f555c102700 2024-04-12T06:18:18.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.110+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c1968c0 con 0x7f555c102700 2024-04-12T06:18:18.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.110+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554400fdb0 con 0x7f555c102700 2024-04-12T06:18:18.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.110+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c196db0 con 0x7f555c102700 2024-04-12T06:18:18.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.111+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c1082f0 con 0x7f555c102700 2024-04-12T06:18:18.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.111+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f554400f5f0 con 0x7f555c102700 2024-04-12T06:18:18.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.111+0000 7f5558ff9700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 0x7f554806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:18.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.112+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f554408d670 con 0x7f555c102700 2024-04-12T06:18:18.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.112+0000 7f555b7fe700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 0x7f554806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:18.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.112+0000 7f555b7fe700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 0x7f554806f1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f554c007be0 tx=0x7f554c0073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:18.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.116+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f554405b170 con 0x7f555c102700 2024-04-12T06:18:18.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.404+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f555c04f2e0 con 0x7f555c102700 2024-04-12T06:18:18.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.405+0000 7f5558ff9700 1 -- 172.21.15.8:0/1828747974 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f554405ad00 con 0x7f555c102700 2024-04-12T06:18:18.405 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:18:18.405 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:18:18.405 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:18:18.405 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:18:18.405 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:18:18.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 msgr2=0x7f554806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 0x7f554806f1a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f554c007be0 tx=0x7f554c0073d0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 msgr2=0x7f555c19cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f55440092e0 tx=0x7f554400bd70 comp rx=0 tx=0).stop 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 shutdown_connections 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f554806cce0 0x7f554806f1a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f555c101de0 0x7f555c19c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 --2- 172.21.15.8:0/1828747974 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f555c102700 0x7f555c19cb00 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 >> 172.21.15.8:0/1828747974 conn(0x7f555c0fb610 msgr2=0x7f555c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 shutdown_connections 2024-04-12T06:18:18.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:18.407+0000 7f5561b47700 1 -- 172.21.15.8:0/1828747974 wait complete. 2024-04-12T06:18:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:18 smithi194 ceph-mon[106982]: pgmap v698: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:18 smithi194 ceph-mon[106982]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:18 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1828747974' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:18:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:18 smithi008 ceph-mon[146084]: pgmap v698: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:18 smithi008 ceph-mon[146084]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:18 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1828747974' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:18:21.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:20 smithi194 ceph-mon[106982]: pgmap v699: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:21.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:20 smithi008 ceph-mon[146084]: pgmap v699: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:22 smithi194 ceph-mon[106982]: pgmap v700: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:23.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:22 smithi008 ceph-mon[146084]: pgmap v700: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:25.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:24 smithi194 ceph-mon[106982]: pgmap v701: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:25.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:24 smithi008 ceph-mon[146084]: pgmap v701: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:27.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:26 smithi194 ceph-mon[106982]: pgmap v702: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:27.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:26 smithi008 ceph-mon[146084]: pgmap v702: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:29.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:28 smithi194 ceph-mon[106982]: pgmap v703: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:29.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:28 smithi008 ceph-mon[146084]: pgmap v703: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:31.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:30 smithi194 ceph-mon[106982]: pgmap v704: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:31.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:30 smithi008 ceph-mon[146084]: pgmap v704: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:32 smithi194 ceph-mon[106982]: pgmap v705: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:33.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:32 smithi008 ceph-mon[146084]: pgmap v705: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:35.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:34 smithi194 ceph-mon[106982]: pgmap v706: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:35.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:34 smithi008 ceph-mon[146084]: pgmap v706: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:37.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:36 smithi194 ceph-mon[106982]: pgmap v707: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:37.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:36 smithi008 ceph-mon[146084]: pgmap v707: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:39.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:38 smithi194 ceph-mon[106982]: pgmap v708: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:39.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:38 smithi008 ceph-mon[146084]: pgmap v708: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:41.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:40 smithi194 ceph-mon[106982]: pgmap v709: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:41.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:40 smithi008 ceph-mon[146084]: pgmap v709: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:43.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:42 smithi194 ceph-mon[106982]: pgmap v710: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:43.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:42 smithi008 ceph-mon[146084]: pgmap v710: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:45.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:44 smithi194 ceph-mon[106982]: pgmap v711: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:45.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:44 smithi008 ceph-mon[146084]: pgmap v711: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:46.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:46 smithi194 ceph-mon[106982]: pgmap v712: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:47.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:46 smithi008 ceph-mon[146084]: pgmap v712: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:48.027 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:48.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:18:48.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 -- 172.21.15.8:0/2923013165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 msgr2=0x7fe11c1070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 --2- 172.21.15.8:0/2923013165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c1070b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe104009910 tx=0x7fe104009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 -- 172.21.15.8:0/2923013165 shutdown_connections 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 --2- 172.21.15.8:0/2923013165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 0x7fe11c0689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 --2- 172.21.15.8:0/2923013165 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c1070b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 -- 172.21.15.8:0/2923013165 >> 172.21.15.8:0/2923013165 conn(0x7fe11c075670 msgr2=0x7fe11c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 -- 172.21.15.8:0/2923013165 shutdown_connections 2024-04-12T06:18:48.533 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.532+0000 7fe120c48700 1 -- 172.21.15.8:0/2923013165 wait complete. 2024-04-12T06:18:48.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 Processor -- start 2024-04-12T06:18:48.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 -- start start 2024-04-12T06:18:48.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 0x7fe11c19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c19d140 con 0x7fe11c106cd0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.533+0000 7fe120c48700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c196640 con 0x7fe11c068580 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58034/0 (socket says 172.21.15.8:58034) 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 -- 172.21.15.8:0/3788366458 learned_addr learned my addr 172.21.15.8:0/3788366458 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe11a59c700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 0x7fe11c19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 -- 172.21.15.8:0/3788366458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 msgr2=0x7fe11c19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 0x7fe11c19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 -- 172.21.15.8:0/3788366458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040095f0 con 0x7fe11c106cd0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe119d9b700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c8f0 tx=0x7fe10c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c007a70 con 0x7fe11c106cd0 2024-04-12T06:18:48.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe10c004cd0 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c196920 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c005580 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.534+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c196e70 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.535+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c1091d0 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.536+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe10c004750 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.536+0000 7fe1137fe700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 0x7fe10806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.537+0000 7fe11a59c700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 0x7fe10806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.537+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe10c013070 con 0x7fe11c106cd0 2024-04-12T06:18:48.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.537+0000 7fe11a59c700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 0x7fe10806f1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe104006010 tx=0x7fe10401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:48.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.541+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe10c059eb0 con 0x7fe11c106cd0 2024-04-12T06:18:48.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.764+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11c197140 con 0x7fe10806cce0 2024-04-12T06:18:48.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.765+0000 7fe1137fe700 1 -- 172.21.15.8:0/3788366458 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe11c197140 con 0x7fe10806cce0 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 msgr2=0x7fe10806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 0x7fe10806f1a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe104006010 tx=0x7fe10401a040 comp rx=0 tx=0).stop 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 msgr2=0x7fe11c19cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00c8f0 tx=0x7fe10c00ccb0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 shutdown_connections 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe10806cce0 0x7fe10806f1a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe11c068580 0x7fe11c19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 --2- 172.21.15.8:0/3788366458 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe11c106cd0 0x7fe11c19cab0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 >> 172.21.15.8:0/3788366458 conn(0x7fe11c075670 msgr2=0x7fe11c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:48.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 shutdown_connections 2024-04-12T06:18:48.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.767+0000 7fe120c48700 1 -- 172.21.15.8:0/3788366458 wait complete. 2024-04-12T06:18:48.777 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:18:48.875 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 -- 172.21.15.8:0/2755716586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 msgr2=0x7f47b4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 --2- 172.21.15.8:0/2755716586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b4105fb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009910 tx=0x7f47b0009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 -- 172.21.15.8:0/2755716586 shutdown_connections 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 --2- 172.21.15.8:0/2755716586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 --2- 172.21.15.8:0/2755716586 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b4105fb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 -- 172.21.15.8:0/2755716586 >> 172.21.15.8:0/2755716586 conn(0x7f47b4075670 msgr2=0x7f47b4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 -- 172.21.15.8:0/2755716586 shutdown_connections 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.874+0000 7f47bc337700 1 -- 172.21.15.8:0/2755716586 wait complete. 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 Processor -- start 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 -- start start 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.876 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b419d150 con 0x7f47b4105bd0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47bc337700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b4196600 con 0x7f47b40ffc20 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58058/0 (socket says 172.21.15.8:58058) 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 -- 172.21.15.8:0/3812778036 learned_addr learned my addr 172.21.15.8:0/3812778036 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47ba0d3700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 -- 172.21.15.8:0/3812778036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 msgr2=0x7f47b419c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b419c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47b98d2700 1 -- 172.21.15.8:0/3812778036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b00095f0 con 0x7f47b4105bd0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.875+0000 7f47ba0d3700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47b98d2700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e920 tx=0x7f47a800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a8009790 con 0x7f47b4105bd0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47a800cba0 con 0x7f47b4105bd0 2024-04-12T06:18:48.877 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b41968e0 con 0x7f47b4105bd0 2024-04-12T06:18:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47a8018650 con 0x7f47b4105bd0 2024-04-12T06:18:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.876+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b4196e30 con 0x7f47b4105bd0 2024-04-12T06:18:48.878 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.877+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b404f2e0 con 0x7f47b4105bd0 2024-04-12T06:18:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.877+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f47a800cd10 con 0x7f47b4105bd0 2024-04-12T06:18:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.878+0000 7f47a77fe700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.878+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f47a8014070 con 0x7f47b4105bd0 2024-04-12T06:18:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.878+0000 7f47ba0d3700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:48.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.878+0000 7f47ba0d3700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47b00059d0 tx=0x7f47b0005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:48.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:48.882+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f47a8052ab0 con 0x7f47b4105bd0 2024-04-12T06:18:49.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:48 smithi194 ceph-mon[106982]: pgmap v713: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:49.102 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.102+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b41970b0 con 0x7f47a006cce0 2024-04-12T06:18:49.103 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.104+0000 7f47a77fe700 1 -- 172.21.15.8:0/3812778036 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f47b41970b0 con 0x7f47a006cce0 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 msgr2=0x7f47a006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f47b00059d0 tx=0x7f47b0005940 comp rx=0 tx=0).stop 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 msgr2=0x7f47b419cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e920 tx=0x7f47a800ece0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 shutdown_connections 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f47a006cce0 0x7f47a006f1a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f47b40ffc20 0x7f47b419c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 --2- 172.21.15.8:0/3812778036 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f47b4105bd0 0x7f47b419cac0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 >> 172.21.15.8:0/3812778036 conn(0x7f47b4075670 msgr2=0x7f47b4068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 shutdown_connections 2024-04-12T06:18:49.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.106+0000 7f47bc337700 1 -- 172.21.15.8:0/3812778036 wait complete. 2024-04-12T06:18:49.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:48 smithi008 ceph-mon[146084]: pgmap v713: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:49.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 -- 172.21.15.8:0/4054001919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d4105bd0 msgr2=0x7f77d4105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 --2- 172.21.15.8:0/4054001919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d4105bd0 0x7f77d4105fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009910 tx=0x7f77c8009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:49.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 -- 172.21.15.8:0/4054001919 shutdown_connections 2024-04-12T06:18:49.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 --2- 172.21.15.8:0/4054001919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d40ffc20 0x7f77d4100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.211 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 --2- 172.21.15.8:0/4054001919 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d4105bd0 0x7f77d4105fb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 -- 172.21.15.8:0/4054001919 >> 172.21.15.8:0/4054001919 conn(0x7f77d4075670 msgr2=0x7f77d4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 -- 172.21.15.8:0/4054001919 shutdown_connections 2024-04-12T06:18:49.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.211+0000 7f77d8b56700 1 -- 172.21.15.8:0/4054001919 wait complete. 2024-04-12T06:18:49.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 Processor -- start 2024-04-12T06:18:49.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 -- start start 2024-04-12T06:18:49.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 0x7f77d419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d419cff0 con 0x7f77d40ffc20 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.212+0000 7f77d8b56700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4196610 con 0x7f77d4105bd0 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58062/0 (socket says 172.21.15.8:58062) 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 -- 172.21.15.8:0/3989496460 learned_addr learned my addr 172.21.15.8:0/3989496460 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d1d9b700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 0x7f77d419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 -- 172.21.15.8:0/3989496460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 msgr2=0x7f77d419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 0x7f77d419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 -- 172.21.15.8:0/3989496460 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80095f0 con 0x7f77d40ffc20 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d1d9b700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 0x7f77d419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d259c700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b5c0 tx=0x7f77c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77c801d070 con 0x7f77d40ffc20 2024-04-12T06:18:49.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d4196890 con 0x7f77d40ffc20 2024-04-12T06:18:49.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f77c800f460 con 0x7f77d40ffc20 2024-04-12T06:18:49.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77c8021640 con 0x7f77d40ffc20 2024-04-12T06:18:49.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.213+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d4196d80 con 0x7f77d40ffc20 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.214+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d404f2e0 con 0x7f77d40ffc20 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.215+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f77c800f9b0 con 0x7f77d40ffc20 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.215+0000 7f77c37fe700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 0x7f77bc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.215+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f77c808d8c0 con 0x7f77d40ffc20 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.215+0000 7f77d1d9b700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 0x7f77bc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.216+0000 7f77d1d9b700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 0x7f77bc06f1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005fd0 tx=0x7f77c4005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.220+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f77c805b3c0 con 0x7f77d40ffc20 2024-04-12T06:18:49.431 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.431+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f77d4196fa0 con 0x7f77bc06cce0 2024-04-12T06:18:49.441 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.442+0000 7f77c37fe700 1 -- 172.21.15.8:0/3989496460 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f77d4196fa0 con 0x7f77bc06cce0 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (20m) 14m ago 32m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (32m) 14m ago 32m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (31m) 13m ago 31m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (18m) 14m ago 32m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (18m) 13m ago 31m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (20m) 14m ago 31m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (14m) 14m ago 25m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (14m) 14m ago 26m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (13m) 13m ago 25m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (14m) 13m ago 25m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (22m) 14m ago 33m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:18:49.442 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (21m) 13m ago 30m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (18m) 14m ago 33m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (18m) 13m ago 30m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (21m) 14m ago 32m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (21m) 13m ago 31m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (17m) 14m ago 30m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (17m) 14m ago 29m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (16m) 14m ago 29m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (16m) 13m ago 28m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (15m) 13m ago 28m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (15m) 13m ago 27m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:18:49.443 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (20m) 14m ago 31m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 msgr2=0x7f77bc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 0x7f77bc06f1a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005fd0 tx=0x7f77c4005e20 comp rx=0 tx=0).stop 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 msgr2=0x7f77d419c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b5c0 tx=0x7f77c800be20 comp rx=0 tx=0).stop 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 shutdown_connections 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f77bc06cce0 0x7f77bc06f1a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f77d40ffc20 0x7f77d419c520 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 --2- 172.21.15.8:0/3989496460 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77d4105bd0 0x7f77d419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.444 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 >> 172.21.15.8:0/3989496460 conn(0x7f77d4075670 msgr2=0x7f77d40686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 shutdown_connections 2024-04-12T06:18:49.445 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.444+0000 7f77d8b56700 1 -- 172.21.15.8:0/3989496460 wait complete. 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 -- 172.21.15.8:0/15908044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 msgr2=0x7fb010100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 --2- 172.21.15.8:0/15908044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb010100080 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faff8009910 tx=0x7faff8009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 -- 172.21.15.8:0/15908044 shutdown_connections 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 --2- 172.21.15.8:0/15908044 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb010100080 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 --2- 172.21.15.8:0/15908044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 0x7fb010105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 -- 172.21.15.8:0/15908044 >> 172.21.15.8:0/15908044 conn(0x7fb010075670 msgr2=0x7fb010075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 -- 172.21.15.8:0/15908044 shutdown_connections 2024-04-12T06:18:49.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.549+0000 7fb014b35700 1 -- 172.21.15.8:0/15908044 wait complete. 2024-04-12T06:18:49.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 Processor -- start 2024-04-12T06:18:49.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 -- start start 2024-04-12T06:18:49.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 0x7fb01019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01019d150 con 0x7fb0100ffc20 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb014b35700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010196600 con 0x7fb010105bd0 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb00e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb00e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58066/0 (socket says 172.21.15.8:58066) 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb00e59c700 1 -- 172.21.15.8:0/2005934744 learned_addr learned my addr 172.21.15.8:0/2005934744 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.550+0000 7fb00dd9b700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 0x7fb01019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb00e59c700 1 -- 172.21.15.8:0/2005934744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 msgr2=0x7fb01019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb00e59c700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 0x7fb01019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb00e59c700 1 -- 172.21.15.8:0/2005934744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80095f0 con 0x7fb0100ffc20 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb00e59c700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb00000d8d0 tx=0x7fb00000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0000098a0 con 0x7fb0100ffc20 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb000010460 con 0x7fb0100ffc20 2024-04-12T06:18:49.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0101968e0 con 0x7fb0100ffc20 2024-04-12T06:18:49.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb000009ea0 con 0x7fb0100ffc20 2024-04-12T06:18:49.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.551+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb010196e30 con 0x7fb0100ffc20 2024-04-12T06:18:49.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.552+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01004f2e0 con 0x7fb0100ffc20 2024-04-12T06:18:49.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.552+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb0000105d0 con 0x7fb0100ffc20 2024-04-12T06:18:49.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.553+0000 7fb0077fe700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 0x7faffc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.553+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb00008c370 con 0x7fb0100ffc20 2024-04-12T06:18:49.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.553+0000 7fb00dd9b700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 0x7faffc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.554 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.553+0000 7fb00dd9b700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 0x7faffc06f1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb010075d70 tx=0x7faff801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.557 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.557+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb000059ef0 con 0x7fb0100ffc20 2024-04-12T06:18:49.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.843+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb010197140 con 0x7fb0100ffc20 2024-04-12T06:18:49.843 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.844+0000 7fb0077fe700 1 -- 172.21.15.8:0/2005934744 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb000010880 con 0x7fb0100ffc20 2024-04-12T06:18:49.843 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:18:49.843 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:18:49.844 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:18:49.845 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:18:49.845 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:18:49.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.845+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 msgr2=0x7faffc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 0x7faffc06f1a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb010075d70 tx=0x7faff801a040 comp rx=0 tx=0).stop 2024-04-12T06:18:49.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 msgr2=0x7fb01019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb00000d8d0 tx=0x7fb00000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 shutdown_connections 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faffc06cce0 0x7faffc06f1a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb0100ffc20 0x7fb01019c580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 --2- 172.21.15.8:0/2005934744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb010105bd0 0x7fb01019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 >> 172.21.15.8:0/2005934744 conn(0x7fb010075670 msgr2=0x7fb010068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 shutdown_connections 2024-04-12T06:18:49.846 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.846+0000 7fb014b35700 1 -- 172.21.15.8:0/2005934744 wait complete. 2024-04-12T06:18:49.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 -- 172.21.15.8:0/3028356627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 msgr2=0x7fad00107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 --2- 172.21.15.8:0/3028356627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad00107190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7facf0009910 tx=0x7facf0009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:49.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 -- 172.21.15.8:0/3028356627 shutdown_connections 2024-04-12T06:18:49.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 --2- 172.21.15.8:0/3028356627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad000ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 --2- 172.21.15.8:0/3028356627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad00107190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.951+0000 7fad0514a700 1 -- 172.21.15.8:0/3028356627 >> 172.21.15.8:0/3028356627 conn(0x7fad00074e30 msgr2=0x7fad00075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:49.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.952+0000 7fad0514a700 1 -- 172.21.15.8:0/3028356627 shutdown_connections 2024-04-12T06:18:49.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.952+0000 7fad0514a700 1 -- 172.21.15.8:0/3028356627 wait complete. 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.952+0000 7fad0514a700 1 Processor -- start 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.952+0000 7fad0514a700 1 -- start start 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7fad0514a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad0019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7fad0514a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7fad0514a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0019d1a0 con 0x7fad00106db0 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7fad0514a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001966b0 con 0x7fad000ff410 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.954 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58088/0 (socket says 172.21.15.8:58088) 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 -- 172.21.15.8:0/761472834 learned_addr learned my addr 172.21.15.8:0/761472834 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfed9d700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad0019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 -- 172.21.15.8:0/761472834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 msgr2=0x7fad0019c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad0019c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 -- 172.21.15.8:0/761472834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf00095f0 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfed9d700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad0019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7facfe59c700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7facf400c930 tx=0x7facf400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.953+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facf4007ad0 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.954+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7facf4007c30 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.954+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad00196990 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.954+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facf4018630 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.954+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad00196ee0 con 0x7fad00106db0 2024-04-12T06:18:49.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.954+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad00109320 con 0x7fad00106db0 2024-04-12T06:18:49.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.955+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7facf400f450 con 0x7fad00106db0 2024-04-12T06:18:49.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.956+0000 7face7fff700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 0x7face806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:49.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.956+0000 7facfed9d700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 0x7face806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:49.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.956+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7facf408c7a0 con 0x7fad00106db0 2024-04-12T06:18:49.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.956+0000 7facfed9d700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 0x7face806f1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7facf000b5c0 tx=0x7facf0005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:49.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:49.960+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7facf405a2a0 con 0x7fad00106db0 2024-04-12T06:18:50.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:49 smithi194 ceph-mon[106982]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:50.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:49 smithi194 ceph-mon[106982]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:50.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:49 smithi008 ceph-mon[146084]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:50.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:49 smithi008 ceph-mon[146084]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:50.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.207+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad00197c80 con 0x7fad00106db0 2024-04-12T06:18:50.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.208+0000 7face7fff700 1 -- 172.21.15.8:0/761472834 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7facf4059e30 con 0x7fad00106db0 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:18:50.208 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:18:50.209 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:18:50.210 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:50.211 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:50.212 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 msgr2=0x7face806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 0x7face806f1a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7facf000b5c0 tx=0x7facf0005fd0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 msgr2=0x7fad0019cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7facf400c930 tx=0x7facf400ccf0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 shutdown_connections 2024-04-12T06:18:50.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7face806cce0 0x7face806f1a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad000ff410 0x7fad0019c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 --2- 172.21.15.8:0/761472834 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad00106db0 0x7fad0019cb50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 >> 172.21.15.8:0/761472834 conn(0x7fad00074e30 msgr2=0x7fad000fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 shutdown_connections 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.210+0000 7fad0514a700 1 -- 172.21.15.8:0/761472834 wait complete. 2024-04-12T06:18:50.214 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 -- 172.21.15.8:0/2137792636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c105bd0 msgr2=0x7ff08c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 --2- 172.21.15.8:0/2137792636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c105bd0 0x7ff08c105fb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff088009910 tx=0x7ff088009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 -- 172.21.15.8:0/2137792636 shutdown_connections 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 --2- 172.21.15.8:0/2137792636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c0ffc20 0x7ff08c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 --2- 172.21.15.8:0/2137792636 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c105bd0 0x7ff08c105fb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.317+0000 7ff093b0d700 1 -- 172.21.15.8:0/2137792636 >> 172.21.15.8:0/2137792636 conn(0x7ff08c075670 msgr2=0x7ff08c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.318+0000 7ff093b0d700 1 -- 172.21.15.8:0/2137792636 shutdown_connections 2024-04-12T06:18:50.318 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.318+0000 7ff093b0d700 1 -- 172.21.15.8:0/2137792636 wait complete. 2024-04-12T06:18:50.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.318+0000 7ff093b0d700 1 Processor -- start 2024-04-12T06:18:50.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.318+0000 7ff093b0d700 1 -- start start 2024-04-12T06:18:50.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff093b0d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff093b0d700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 0x7ff08c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff093b0d700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c198140 con 0x7ff08c0ffc20 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff093b0d700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c1982b0 con 0x7ff08c105bd0 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58108/0 (socket says 172.21.15.8:58108) 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 -- 172.21.15.8:0/4198107562 learned_addr learned my addr 172.21.15.8:0/4198107562 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0910a8700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 0x7ff08c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 -- 172.21.15.8:0/4198107562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 msgr2=0x7ff08c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 0x7ff08c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 -- 172.21.15.8:0/4198107562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880095f0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0910a8700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 0x7ff08c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.319+0000 7ff0918a9700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b5c0 tx=0x7ff08800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:50.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.320+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08801d070 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.320+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c1a0650 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.320+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff088005e00 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.320+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08800fe60 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.320+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c1a0ac0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.321+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c1080f0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.321+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff08800f6d0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.321+0000 7ff082ffd700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 0x7ff07806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.322+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff08808d8f0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.322+0000 7ff0910a8700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 0x7ff07806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.322+0000 7ff0910a8700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 0x7ff07806f1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff08c068e50 tx=0x7ff07c00b2c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:50.326 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.326+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff0880578c0 con 0x7ff08c0ffc20 2024-04-12T06:18:50.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.549+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff08c02cce0 con 0x7ff07806cce0 2024-04-12T06:18:50.550 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.550+0000 7ff082ffd700 1 -- 172.21.15.8:0/4198107562 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff08c02cce0 con 0x7ff07806cce0 2024-04-12T06:18:50.550 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:18:50.550 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:18:50.550 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:18:50.551 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.552+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 msgr2=0x7ff07806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.552+0000 7ff093b0d700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 0x7ff07806f1a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff08c068e50 tx=0x7ff07c00b2c0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.552+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 msgr2=0x7ff08c1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.552+0000 7ff093b0d700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b5c0 tx=0x7ff08800be20 comp rx=0 tx=0).stop 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.552+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 shutdown_connections 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff07806cce0 0x7ff07806f1a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ff08c0ffc20 0x7ff08c1975e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 --2- 172.21.15.8:0/4198107562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff08c105bd0 0x7ff08c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 >> 172.21.15.8:0/4198107562 conn(0x7ff08c075670 msgr2=0x7ff08c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 shutdown_connections 2024-04-12T06:18:50.553 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.553+0000 7ff093b0d700 1 -- 172.21.15.8:0/4198107562 wait complete. 2024-04-12T06:18:50.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.658+0000 7ffa0788e700 1 -- 172.21.15.8:0/2227843056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00101850 msgr2=0x7ffa001058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.658+0000 7ffa0788e700 1 --2- 172.21.15.8:0/2227843056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00101850 0x7ffa001058a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009910 tx=0x7ff9f0009c20 comp rx=0 tx=0).stop 2024-04-12T06:18:50.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 -- 172.21.15.8:0/2227843056 shutdown_connections 2024-04-12T06:18:50.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 --2- 172.21.15.8:0/2227843056 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00101850 0x7ffa001058a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.660 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 --2- 172.21.15.8:0/2227843056 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00100ea0 0x7ffa00101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 -- 172.21.15.8:0/2227843056 >> 172.21.15.8:0/2227843056 conn(0x7ffa000fc750 msgr2=0x7ffa000feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 -- 172.21.15.8:0/2227843056 shutdown_connections 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 -- 172.21.15.8:0/2227843056 wait complete. 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 Processor -- start 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 -- start start 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.659+0000 7ffa0788e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0788e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 0x7ffa000735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0788e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00073b10 con 0x7ffa00100ea0 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0788e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00073c50 con 0x7ffa00101850 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58116/0 (socket says 172.21.15.8:58116) 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 -- 172.21.15.8:0/1326364949 learned_addr learned my addr 172.21.15.8:0/1326364949 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa04e29700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 0x7ffa000735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 -- 172.21.15.8:0/1326364949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 msgr2=0x7ffa000735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 0x7ffa000735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 -- 172.21.15.8:0/1326364949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00095f0 con 0x7ffa00100ea0 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa04e29700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 0x7ffa000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0562a700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c930 tx=0x7ff9fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:50.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9fc007ad0 con 0x7ffa00100ea0 2024-04-12T06:18:50.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa001a2a20 con 0x7ffa00100ea0 2024-04-12T06:18:50.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001a2d80 con 0x7ffa00100ea0 2024-04-12T06:18:50.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff9fc007c30 con 0x7ffa00100ea0 2024-04-12T06:18:50.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.660+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9fc018630 con 0x7ffa00100ea0 2024-04-12T06:18:50.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.661+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e4005320 con 0x7ffa00100ea0 2024-04-12T06:18:50.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.662+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7ff9fc018790 con 0x7ffa00100ea0 2024-04-12T06:18:50.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.662+0000 7ff9f67fc700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 0x7ff9ec06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:18:50.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.662+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff9fc08c520 con 0x7ffa00100ea0 2024-04-12T06:18:50.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.662+0000 7ffa04e29700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 0x7ff9ec06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:18:50.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.663+0000 7ffa04e29700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 0x7ff9ec06f1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa00074880 tx=0x7ff9f001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:18:50.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.667+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7ff9fc056610 con 0x7ffa00100ea0 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.958+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff9e4005190 con 0x7ffa00100ea0 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.958+0000 7ff9f67fc700 1 -- 172.21.15.8:0/1326364949 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff9fc059c30 con 0x7ffa00100ea0 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:18:50.958 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 msgr2=0x7ff9ec06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 0x7ff9ec06f1a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa00074880 tx=0x7ff9f001a040 comp rx=0 tx=0).stop 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 msgr2=0x7ffa00073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c930 tx=0x7ff9fc00ccf0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 shutdown_connections 2024-04-12T06:18:50.960 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7ff9ec06cce0 0x7ff9ec06f1a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7ffa00100ea0 0x7ffa00073090 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 --2- 172.21.15.8:0/1326364949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffa00101850 0x7ffa000735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:18:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 >> 172.21.15.8:0/1326364949 conn(0x7ffa000fc750 msgr2=0x7ffa000fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:18:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 shutdown_connections 2024-04-12T06:18:50.961 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:18:50.960+0000 7ffa0788e700 1 -- 172.21.15.8:0/1326364949 wait complete. 2024-04-12T06:18:51.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:50 smithi194 ceph-mon[106982]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:51.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:50 smithi194 ceph-mon[106982]: pgmap v714: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:51.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:50 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2005934744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:18:51.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:50 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/761472834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:18:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:50 smithi008 ceph-mon[146084]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:50 smithi008 ceph-mon[146084]: pgmap v714: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:50 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2005934744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:18:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:50 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/761472834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:18:51.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:51 smithi194 ceph-mon[106982]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:51.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:51 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1326364949' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:18:52.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:51 smithi008 ceph-mon[146084]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:18:52.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:51 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1326364949' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:18:53.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:52 smithi194 ceph-mon[106982]: pgmap v715: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:53.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:52 smithi008 ceph-mon[146084]: pgmap v715: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:55.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:54 smithi194 ceph-mon[106982]: pgmap v716: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:55.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:54 smithi008 ceph-mon[146084]: pgmap v716: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:57.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:56 smithi194 ceph-mon[106982]: pgmap v717: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:57.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:56 smithi008 ceph-mon[146084]: pgmap v717: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:59.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:18:58 smithi194 ceph-mon[106982]: pgmap v718: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:18:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:18:58 smithi008 ceph-mon[146084]: pgmap v718: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:01.038 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:00 smithi008 ceph-mon[146084]: pgmap v719: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:01.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:00 smithi194 ceph-mon[106982]: pgmap v719: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:02 smithi194 ceph-mon[106982]: pgmap v720: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:03.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:02 smithi008 ceph-mon[146084]: pgmap v720: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:05.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:04 smithi194 ceph-mon[106982]: pgmap v721: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:05.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:04 smithi008 ceph-mon[146084]: pgmap v721: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:07.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:06 smithi194 ceph-mon[106982]: pgmap v722: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:07.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:06 smithi008 ceph-mon[146084]: pgmap v722: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:09.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:08 smithi194 ceph-mon[106982]: pgmap v723: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:09.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:08 smithi008 ceph-mon[146084]: pgmap v723: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:11.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:10 smithi194 ceph-mon[106982]: pgmap v724: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:11.131 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:10 smithi008 ceph-mon[146084]: pgmap v724: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:12 smithi194 ceph-mon[106982]: pgmap v725: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:12 smithi008 ceph-mon[146084]: pgmap v725: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:15.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:14 smithi194 ceph-mon[106982]: pgmap v726: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:14 smithi008 ceph-mon[146084]: pgmap v726: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:17.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:16 smithi194 ceph-mon[106982]: pgmap v727: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:17.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:16 smithi008 ceph-mon[146084]: pgmap v727: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:18.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:18.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:19.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:18 smithi194 ceph-mon[106982]: pgmap v728: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:19.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:18 smithi008 ceph-mon[146084]: pgmap v728: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:21.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:20 smithi194 ceph-mon[106982]: pgmap v729: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:21.081 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.081+0000 7faea659b700 1 -- 172.21.15.8:0/1672544573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0102720 msgr2=0x7faea010ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.081+0000 7faea659b700 1 --2- 172.21.15.8:0/1672544573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0102720 0x7faea010ac10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae94009910 tx=0x7fae94009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 -- 172.21.15.8:0/1672544573 shutdown_connections 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 --2- 172.21.15.8:0/1672544573 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0102720 0x7faea010ac10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 --2- 172.21.15.8:0/1672544573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0101e00 0x7faea01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 -- 172.21.15.8:0/1672544573 >> 172.21.15.8:0/1672544573 conn(0x7faea00fb650 msgr2=0x7faea00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 -- 172.21.15.8:0/1672544573 shutdown_connections 2024-04-12T06:19:21.082 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 -- 172.21.15.8:0/1672544573 wait complete. 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 Processor -- start 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.082+0000 7faea659b700 1 -- start start 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7faea659b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7faea659b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 0x7faea019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7faea659b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019f2f0 con 0x7faea0101e00 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7faea659b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0198830 con 0x7faea0102720 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40810/0 (socket says 172.21.15.8:40810) 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 -- 172.21.15.8:0/3555289597 learned_addr learned my addr 172.21.15.8:0/3555289597 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9f7fe700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 0x7faea019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.084 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 -- 172.21.15.8:0/3555289597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 msgr2=0x7faea019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 0x7faea019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9ffff700 1 -- 172.21.15.8:0/3555289597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940095f0 con 0x7faea0101e00 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.083+0000 7fae9f7fe700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 0x7faea019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7fae9ffff700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c930 tx=0x7fae9000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae90007ad0 con 0x7faea0101e00 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae90007c30 con 0x7faea0101e00 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea0198b10 con 0x7faea0101e00 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae90018630 con 0x7faea0101e00 2024-04-12T06:19:21.085 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.084+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea0199060 con 0x7faea0101e00 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.085+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea010a770 con 0x7faea0101e00 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.085+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fae9000f450 con 0x7faea0101e00 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.086+0000 7fae9d7fa700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 0x7fae8806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.086+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae9008c750 con 0x7faea0101e00 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.086+0000 7fae9f7fe700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 0x7fae8806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.086 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.087+0000 7fae9f7fe700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 0x7fae8806f1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae94006010 tx=0x7fae94005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.091 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.091+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fae9005a380 con 0x7faea0101e00 2024-04-12T06:19:21.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:20 smithi008 ceph-mon[146084]: pgmap v729: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:21.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.312+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faea0199330 con 0x7fae8806cce0 2024-04-12T06:19:21.312 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.313+0000 7fae9d7fa700 1 -- 172.21.15.8:0/3555289597 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faea0199330 con 0x7fae8806cce0 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 msgr2=0x7fae8806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 0x7fae8806f1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae94006010 tx=0x7fae94005960 comp rx=0 tx=0).stop 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 msgr2=0x7faea019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c930 tx=0x7fae9000ccf0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 shutdown_connections 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae8806cce0 0x7fae8806f1a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faea0101e00 0x7faea019e7b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 --2- 172.21.15.8:0/3555289597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faea0102720 0x7faea019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 >> 172.21.15.8:0/3555289597 conn(0x7faea00fb650 msgr2=0x7faea0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:21.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 shutdown_connections 2024-04-12T06:19:21.316 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.315+0000 7faea659b700 1 -- 172.21.15.8:0/3555289597 wait complete. 2024-04-12T06:19:21.326 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:19:21.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2840353401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb00687e0 msgr2=0x7f0fb0068bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2840353401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb00687e0 0x7f0fb0068bc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009910 tx=0x7f0fa0009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:21.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2840353401 shutdown_connections 2024-04-12T06:19:21.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2840353401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb0069190 0x7f0fb0105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2840353401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb00687e0 0x7f0fb0068bc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2840353401 >> 172.21.15.8:0/2840353401 conn(0x7f0fb00754a0 msgr2=0x7f0fb00758b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:21.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2840353401 shutdown_connections 2024-04-12T06:19:21.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.422+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2840353401 wait complete. 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 Processor -- start 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 -- start start 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb00687e0 0x7f0fb0196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0197250 con 0x7f0fb0069190 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.423+0000 7f0fb5bdc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb0199f80 con 0x7f0fb00687e0 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40838/0 (socket says 172.21.15.8:40838) 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 -- 172.21.15.8:0/2578598486 learned_addr learned my addr 172.21.15.8:0/2578598486 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faf7fe700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb00687e0 0x7f0fb0196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 -- 172.21.15.8:0/2578598486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb00687e0 msgr2=0x7f0fb0196680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb00687e0 0x7f0fb0196680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 -- 172.21.15.8:0/2578598486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00095f0 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0faeffd700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400d900 tx=0x7f0fa400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa4009900 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0fa4010460 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb019a260 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fa400f5f0 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.424+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb019a7b0 con 0x7f0fb0069190 2024-04-12T06:19:21.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.425+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb0109110 con 0x7f0fb0069190 2024-04-12T06:19:21.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.426+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f0fa40105d0 con 0x7f0fb0069190 2024-04-12T06:19:21.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.426+0000 7f0facff9700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 0x7f0f9806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.426+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0fa408bfe0 con 0x7f0fb0069190 2024-04-12T06:19:21.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.426+0000 7f0faf7fe700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 0x7f0f9806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.427+0000 7f0faf7fe700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 0x7f0f9806f1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b5c0 tx=0x7f0fa001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.431+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0fa40560d0 con 0x7f0fb0069190 2024-04-12T06:19:21.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.655+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fb019aa20 con 0x7f0f9806cce0 2024-04-12T06:19:21.656 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.656+0000 7f0facff9700 1 -- 172.21.15.8:0/2578598486 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0fb019aa20 con 0x7f0f9806cce0 2024-04-12T06:19:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 msgr2=0x7f0f9806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 0x7f0f9806f1a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000b5c0 tx=0x7f0fa001a040 comp rx=0 tx=0).stop 2024-04-12T06:19:21.658 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 msgr2=0x7f0fb0196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400d900 tx=0x7f0fa400dc10 comp rx=0 tx=0).stop 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 shutdown_connections 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0f9806cce0 0x7f0f9806f1a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fb00687e0 0x7f0fb0196680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.658+0000 7f0fb5bdc700 1 --2- 172.21.15.8:0/2578598486 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0fb0069190 0x7f0fb0196bc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.659+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 >> 172.21.15.8:0/2578598486 conn(0x7f0fb00754a0 msgr2=0x7f0fb00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.659+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 shutdown_connections 2024-04-12T06:19:21.659 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.659+0000 7f0fb5bdc700 1 -- 172.21.15.8:0/2578598486 wait complete. 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 -- 172.21.15.8:0/3220252931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 msgr2=0x7f6388105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 --2- 172.21.15.8:0/3220252931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f6388105890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f637c009910 tx=0x7f637c009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 -- 172.21.15.8:0/3220252931 shutdown_connections 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 --2- 172.21.15.8:0/3220252931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f6388105890 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 --2- 172.21.15.8:0/3220252931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 0x7f6388101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 -- 172.21.15.8:0/3220252931 >> 172.21.15.8:0/3220252931 conn(0x7f63880fc740 msgr2=0x7f63880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 -- 172.21.15.8:0/3220252931 shutdown_connections 2024-04-12T06:19:21.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.765+0000 7f638e398700 1 -- 172.21.15.8:0/3220252931 wait complete. 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 Processor -- start 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 -- start start 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 0x7f6388073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388073b10 con 0x7f6388101840 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f638e398700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388073c50 con 0x7f6388100e90 2024-04-12T06:19:21.767 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f63877fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f63877fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40856/0 (socket says 172.21.15.8:40856) 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f63877fe700 1 -- 172.21.15.8:0/929672576 learned_addr learned my addr 172.21.15.8:0/929672576 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.766+0000 7f6387fff700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 0x7f6388073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63877fe700 1 -- 172.21.15.8:0/929672576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 msgr2=0x7f6388073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63877fe700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 0x7f6388073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63877fe700 1 -- 172.21.15.8:0/929672576 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0095f0 con 0x7f6388101840 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63877fe700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f637c0092e0 tx=0x7f637c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c01d070 con 0x7f6388101840 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388073ed0 con 0x7f6388101840 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f637c005dc0 con 0x7f6388101840 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c00fea0 con 0x7f6388101840 2024-04-12T06:19:21.768 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.767+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63881a2d60 con 0x7f6388101840 2024-04-12T06:19:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.768+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6388109290 con 0x7f6388101840 2024-04-12T06:19:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.768+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f637c003a60 con 0x7f6388101840 2024-04-12T06:19:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.769+0000 7f63857fa700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 0x7f637006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.769+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f637c08d780 con 0x7f6388101840 2024-04-12T06:19:21.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.769+0000 7f6387fff700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 0x7f637006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:21.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.769+0000 7f6387fff700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 0x7f637006f1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6378007be0 tx=0x7f63780073d0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:21.773 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.773+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f637c057750 con 0x7f6388101840 2024-04-12T06:19:21.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.986+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f63880746d0 con 0x7f637006cce0 2024-04-12T06:19:21.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.997+0000 7f63857fa700 1 -- 172.21.15.8:0/929672576 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f63880746d0 con 0x7f637006cce0 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (21m) 14m ago 32m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (32m) 14m ago 32m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (31m) 14m ago 31m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (18m) 14m ago 32m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (18m) 14m ago 31m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (20m) 14m ago 32m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (14m) 14m ago 26m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (15m) 14m ago 26m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (14m) 14m ago 26m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (14m) 14m ago 26m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:19:21.997 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (23m) 14m ago 34m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (22m) 14m ago 31m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (19m) 14m ago 34m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (19m) 14m ago 31m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (22m) 14m ago 32m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (21m) 14m ago 31m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (18m) 14m ago 30m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (17m) 14m ago 30m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (17m) 14m ago 29m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (16m) 14m ago 29m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (16m) 14m ago 28m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (15m) 14m ago 28m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:19:21.998 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (21m) 14m ago 32m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:19:21.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.999+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 msgr2=0x7f637006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.999+0000 7f638e398700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 0x7f637006f1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6378007be0 tx=0x7f63780073d0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.999+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 msgr2=0x7f63880735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:21.999+0000 7f638e398700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f637c0092e0 tx=0x7f637c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 shutdown_connections 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f637006cce0 0x7f637006f1a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6388100e90 0x7f6388073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 --2- 172.21.15.8:0/929672576 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6388101840 0x7f63880735d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 >> 172.21.15.8:0/929672576 conn(0x7f63880fc740 msgr2=0x7f63880fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 shutdown_connections 2024-04-12T06:19:22.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.000+0000 7f638e398700 1 -- 172.21.15.8:0/929672576 wait complete. 2024-04-12T06:19:22.106 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3421178270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 msgr2=0x7fb868105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3421178270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868105fd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb858009910 tx=0x7fb858009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3421178270 shutdown_connections 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3421178270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 0x7fb8680fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3421178270 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868105fd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3421178270 >> 172.21.15.8:0/3421178270 conn(0x7fb8680fb610 msgr2=0x7fb8680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3421178270 shutdown_connections 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3421178270 wait complete. 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.105+0000 7fb86fdc9700 1 Processor -- start 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86fdc9700 1 -- start start 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86fdc9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 0x7fb8681017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86fdc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.107 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86fdc9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb868102230 con 0x7fb868105bf0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86fdc9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb868102370 con 0x7fb8680ffb40 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40868/0 (socket says 172.21.15.8:40868) 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 -- 172.21.15.8:0/3700501772 learned_addr learned my addr 172.21.15.8:0/3700501772 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86db65700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 0x7fb8681017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 -- 172.21.15.8:0/3700501772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 msgr2=0x7fb8681017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 0x7fb8681017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 -- 172.21.15.8:0/3700501772 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580095f0 con 0x7fb868105bf0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb86d364700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c8a0 tx=0x7fb86400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.106+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb86400cea0 con 0x7fb868105bf0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.107+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb864004830 con 0x7fb868105bf0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.107+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8681a2940 con 0x7fb868105bf0 2024-04-12T06:19:22.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.107+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb864005500 con 0x7fb868105bf0 2024-04-12T06:19:22.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.107+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8681a2cf0 con 0x7fb868105bf0 2024-04-12T06:19:22.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.108+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86804f2e0 con 0x7fb868105bf0 2024-04-12T06:19:22.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.108+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb8640049a0 con 0x7fb868105bf0 2024-04-12T06:19:22.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.109+0000 7fb85effd700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 0x7fb85406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.109+0000 7fb86db65700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 0x7fb85406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.109+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb86408c120 con 0x7fb868105bf0 2024-04-12T06:19:22.110 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.109+0000 7fb86db65700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 0x7fb85406f1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb858006010 tx=0x7fb85801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.113+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb864056210 con 0x7fb868105bf0 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.403+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb868102c90 con 0x7fb868105bf0 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.403+0000 7fb85effd700 1 -- 172.21.15.8:0/3700501772 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb864059830 con 0x7fb868105bf0 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:19:22.403 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:19:22.404 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:19:22.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 msgr2=0x7fb85406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 0x7fb85406f1a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb858006010 tx=0x7fb85801a040 comp rx=0 tx=0).stop 2024-04-12T06:19:22.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 msgr2=0x7fb868101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c8a0 tx=0x7fb86400cc60 comp rx=0 tx=0).stop 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 shutdown_connections 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb85406cce0 0x7fb85406f1a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb8680ffb40 0x7fb8681017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.405+0000 7fb86fdc9700 1 --2- 172.21.15.8:0/3700501772 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb868105bf0 0x7fb868101cf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.406+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 >> 172.21.15.8:0/3700501772 conn(0x7fb8680fb610 msgr2=0x7fb868068fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.406+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 shutdown_connections 2024-04-12T06:19:22.406 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.406+0000 7fb86fdc9700 1 -- 172.21.15.8:0/3700501772 wait complete. 2024-04-12T06:19:22.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 -- 172.21.15.8:0/2619036015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 msgr2=0x7fc41c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 --2- 172.21.15.8:0/2619036015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c105fb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc41400b0a0 tx=0x7fc41400b3b0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 -- 172.21.15.8:0/2619036015 shutdown_connections 2024-04-12T06:19:22.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 --2- 172.21.15.8:0/2619036015 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 0x7fc41c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 --2- 172.21.15.8:0/2619036015 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c105fb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 -- 172.21.15.8:0/2619036015 >> 172.21.15.8:0/2619036015 conn(0x7fc41c075670 msgr2=0x7fc41c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 -- 172.21.15.8:0/2619036015 shutdown_connections 2024-04-12T06:19:22.512 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.511+0000 7fc4240f6700 1 -- 172.21.15.8:0/2619036015 wait complete. 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 Processor -- start 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 -- start start 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 0x7fc41c195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c196260 con 0x7fc41c105bd0 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc4240f6700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c198f90 con 0x7fc41c0ffc20 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc421691700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc421691700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40896/0 (socket says 172.21.15.8:40896) 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc421691700 1 -- 172.21.15.8:0/3527912625 learned_addr learned my addr 172.21.15.8:0/3527912625 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.512+0000 7fc421e92700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 0x7fc41c195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc421691700 1 -- 172.21.15.8:0/3527912625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 msgr2=0x7fc41c195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc421691700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 0x7fc41c195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc421691700 1 -- 172.21.15.8:0/3527912625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc414009d00 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc421691700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc410009970 tx=0x7fc410009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc410012070 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc410007ad0 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c199210 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc410017bb0 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.513+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c199760 con 0x7fc41c105bd0 2024-04-12T06:19:22.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.514+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c04f2e0 con 0x7fc41c105bd0 2024-04-12T06:19:22.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.515+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc410007c40 con 0x7fc41c105bd0 2024-04-12T06:19:22.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.515+0000 7fc41affd700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 0x7fc40406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.515+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc41008d9a0 con 0x7fc41c105bd0 2024-04-12T06:19:22.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.515+0000 7fc421e92700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 0x7fc40406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.515+0000 7fc421e92700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 0x7fc40406f1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc414009fd0 tx=0x7fc4140060c0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.519+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc410057b40 con 0x7fc41c105bd0 2024-04-12T06:19:22.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.770+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc41c1969f0 con 0x7fc41c105bd0 2024-04-12T06:19:22.770 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.770+0000 7fc41affd700 1 -- 172.21.15.8:0/3527912625 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fc41001c740 con 0x7fc41c105bd0 2024-04-12T06:19:22.770 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:19:22.770 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:19:22.770 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:19:22.771 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:19:22.772 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:22.773 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:22.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 msgr2=0x7fc40406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.774 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 0x7fc40406f1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc414009fd0 tx=0x7fc4140060c0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 msgr2=0x7fc41c195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc410009970 tx=0x7fc410009c80 comp rx=0 tx=0).stop 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 shutdown_connections 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40406cce0 0x7fc40406f1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c0ffc20 0x7fc41c195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 --2- 172.21.15.8:0/3527912625 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c105bd0 0x7fc41c195bd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 >> 172.21.15.8:0/3527912625 conn(0x7fc41c075670 msgr2=0x7fc41c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 shutdown_connections 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.772+0000 7fc4240f6700 1 -- 172.21.15.8:0/3527912625 wait complete. 2024-04-12T06:19:22.775 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:19:22.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/4040246473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 msgr2=0x7fbb181070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/4040246473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb181070b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009910 tx=0x7fbb08009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:22.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/4040246473 shutdown_connections 2024-04-12T06:19:22.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/4040246473 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 0x7fbb180689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.879 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/4040246473 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb181070b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/4040246473 >> 172.21.15.8:0/4040246473 conn(0x7fbb18075670 msgr2=0x7fbb18075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:22.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/4040246473 shutdown_connections 2024-04-12T06:19:22.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.879+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/4040246473 wait complete. 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 Processor -- start 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 -- start start 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 0x7fbb1819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1819f360 con 0x7fbb18106cd0 2024-04-12T06:19:22.881 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb1dd6c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18198810 con 0x7fbb18068580 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb16ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40916/0 (socket says 172.21.15.8:40916) 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.880+0000 7fbb16ffd700 1 -- 172.21.15.8:0/1383906717 learned_addr learned my addr 172.21.15.8:0/1383906717 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb177fe700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 0x7fbb1819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb16ffd700 1 -- 172.21.15.8:0/1383906717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 msgr2=0x7fbb1819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb16ffd700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 0x7fbb1819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb16ffd700 1 -- 172.21.15.8:0/1383906717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080095f0 con 0x7fbb18106cd0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb16ffd700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d8d0 tx=0x7fbb0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb0c0098a0 con 0x7fbb18106cd0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb18198af0 con 0x7fbb18106cd0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbb0c010460 con 0x7fbb18106cd0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb0c009ea0 con 0x7fbb18106cd0 2024-04-12T06:19:22.882 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.881+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18199040 con 0x7fbb18106cd0 2024-04-12T06:19:22.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.882+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1804f2e0 con 0x7fbb18106cd0 2024-04-12T06:19:22.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.883+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbb0c0105d0 con 0x7fbb18106cd0 2024-04-12T06:19:22.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.883+0000 7fbb14ff9700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 0x7fbb0006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:22.883 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.883+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbb0c08c370 con 0x7fbb18106cd0 2024-04-12T06:19:22.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.883+0000 7fbb177fe700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 0x7fbb0006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:22.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.884+0000 7fbb177fe700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 0x7fbb0006f1a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb0801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:22.887 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:22.888+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbb0c059ef0 con 0x7fbb18106cd0 2024-04-12T06:19:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: pgmap v730: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:23.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3700501772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:23.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:22 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3527912625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:19:23.108 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.109+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb18199320 con 0x7fbb0006cce0 2024-04-12T06:19:23.109 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.110+0000 7fbb14ff9700 1 -- 172.21.15.8:0/1383906717 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbb18199320 con 0x7fbb0006cce0 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:19:23.110 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:19:23.111 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:19:23.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 msgr2=0x7fbb0006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 0x7fbb0006f1a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbb08006010 tx=0x7fbb0801a040 comp rx=0 tx=0).stop 2024-04-12T06:19:23.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 msgr2=0x7fbb1819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.112 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00d8d0 tx=0x7fbb0c00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 shutdown_connections 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbb0006cce0 0x7fbb0006f1a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb18068580 0x7fbb1819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 --2- 172.21.15.8:0/1383906717 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbb18106cd0 0x7fbb1819ecd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 >> 172.21.15.8:0/1383906717 conn(0x7fbb18075670 msgr2=0x7fbb180feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 shutdown_connections 2024-04-12T06:19:23.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.112+0000 7fbb1dd6c700 1 -- 172.21.15.8:0/1383906717 wait complete. 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: pgmap v730: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3700501772' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:22 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3527912625' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:19:23.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 -- 172.21.15.8:0/1138823998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c068580 msgr2=0x7fc41c0689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 --2- 172.21.15.8:0/1138823998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c068580 0x7fc41c0689e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009910 tx=0x7fc40c009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:23.221 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 -- 172.21.15.8:0/1138823998 shutdown_connections 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 --2- 172.21.15.8:0/1138823998 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c068580 0x7fc41c0689e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 --2- 172.21.15.8:0/1138823998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c106cd0 0x7fc41c1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 -- 172.21.15.8:0/1138823998 >> 172.21.15.8:0/1138823998 conn(0x7fc41c075670 msgr2=0x7fc41c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 -- 172.21.15.8:0/1138823998 shutdown_connections 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.218+0000 7fc42449a700 1 -- 172.21.15.8:0/1138823998 wait complete. 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 Processor -- start 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 -- start start 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 0x7fc41c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c199530 con 0x7fc41c106cd0 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc42449a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c19d340 con 0x7fc41c068580 2024-04-12T06:19:23.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc421a35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc421a35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:40934/0 (socket says 172.21.15.8:40934) 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc421a35700 1 -- 172.21.15.8:0/2478624048 learned_addr learned my addr 172.21.15.8:0/2478624048 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.219+0000 7fc422236700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 0x7fc41c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc421a35700 1 -- 172.21.15.8:0/2478624048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 msgr2=0x7fc41c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc421a35700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 0x7fc41c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc421a35700 1 -- 172.21.15.8:0/2478624048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0095f0 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc422236700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 0x7fc41c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc421a35700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006010 tx=0x7fc40c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c01d070 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc40c00f460 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c19d5c0 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c021620 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.220+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c19dab0 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.221+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c04f2e0 con 0x7fc41c106cd0 2024-04-12T06:19:23.223 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.221+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc40c00fa40 con 0x7fc41c106cd0 2024-04-12T06:19:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.222+0000 7fc4137fe700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 0x7fc40806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.222+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc40c08d690 con 0x7fc41c106cd0 2024-04-12T06:19:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.222+0000 7fc422236700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 0x7fc40806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:23.224 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.222+0000 7fc422236700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 0x7fc40806f1a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc418005950 tx=0x7fc4180058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:23.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.227+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc40c05b210 con 0x7fc41c106cd0 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.516+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc41c19dde0 con 0x7fc41c106cd0 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.516+0000 7fc4137fe700 1 -- 172.21.15.8:0/2478624048 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc40c05ada0 con 0x7fc41c106cd0 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:19:23.516 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 msgr2=0x7fc40806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 0x7fc40806f1a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc418005950 tx=0x7fc4180058e0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 msgr2=0x7fc41c198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc40c006010 tx=0x7fc40c00be20 comp rx=0 tx=0).stop 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 shutdown_connections 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc40806cce0 0x7fc40806f1a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc41c068580 0x7fc41c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 --2- 172.21.15.8:0/2478624048 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc41c106cd0 0x7fc41c198ea0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:23.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 >> 172.21.15.8:0/2478624048 conn(0x7fc41c075670 msgr2=0x7fc41c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:23.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 shutdown_connections 2024-04-12T06:19:23.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:23.518+0000 7fc42449a700 1 -- 172.21.15.8:0/2478624048 wait complete. 2024-04-12T06:19:24.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:23 smithi194 ceph-mon[106982]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:24.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:23 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2478624048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:19:24.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:23 smithi008 ceph-mon[146084]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:24.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:23 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2478624048' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:19:25.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:24 smithi194 ceph-mon[106982]: pgmap v731: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:25.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:24 smithi008 ceph-mon[146084]: pgmap v731: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:27.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:26 smithi194 ceph-mon[106982]: pgmap v732: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:27.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:26 smithi008 ceph-mon[146084]: pgmap v732: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:29.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:28 smithi194 ceph-mon[106982]: pgmap v733: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:29.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:28 smithi008 ceph-mon[146084]: pgmap v733: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:31.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:30 smithi194 ceph-mon[106982]: pgmap v734: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:31.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:30 smithi008 ceph-mon[146084]: pgmap v734: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:32 smithi194 ceph-mon[106982]: pgmap v735: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:33.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:32 smithi008 ceph-mon[146084]: pgmap v735: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:35.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:34 smithi008 ceph-mon[146084]: pgmap v736: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:34 smithi194 ceph-mon[106982]: pgmap v736: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:37.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:36 smithi008 ceph-mon[146084]: pgmap v737: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:37.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:36 smithi194 ceph-mon[106982]: pgmap v737: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:39.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:38 smithi008 ceph-mon[146084]: pgmap v738: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:39.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:38 smithi194 ceph-mon[106982]: pgmap v738: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:41.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:40 smithi008 ceph-mon[146084]: pgmap v739: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:40 smithi194 ceph-mon[106982]: pgmap v739: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:43.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:42 smithi008 ceph-mon[146084]: pgmap v740: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:43.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:42 smithi194 ceph-mon[106982]: pgmap v740: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:45.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:44 smithi008 ceph-mon[146084]: pgmap v741: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:45.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:44 smithi194 ceph-mon[106982]: pgmap v741: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:47.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:46 smithi008 ceph-mon[146084]: pgmap v742: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:46 smithi194 ceph-mon[106982]: pgmap v742: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:48.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:19:49.157 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:48 smithi008 ceph-mon[146084]: pgmap v743: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:49.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:48 smithi194 ceph-mon[106982]: pgmap v743: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:50 smithi008 ceph-mon[146084]: pgmap v744: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:51.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:50 smithi194 ceph-mon[106982]: pgmap v744: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:53.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:52 smithi008 ceph-mon[146084]: pgmap v745: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:53.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:52 smithi194 ceph-mon[106982]: pgmap v745: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- 172.21.15.8:0/874356378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa14105bd0 msgr2=0x7faa14105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/874356378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa14105bd0 0x7faa14105fb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faa10009910 tx=0x7faa10009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- 172.21.15.8:0/874356378 shutdown_connections 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/874356378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa140ffc20 0x7faa14100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/874356378 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa14105bd0 0x7faa14105fb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- 172.21.15.8:0/874356378 >> 172.21.15.8:0/874356378 conn(0x7faa14075670 msgr2=0x7faa14075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- 172.21.15.8:0/874356378 shutdown_connections 2024-04-12T06:19:53.639 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- 172.21.15.8:0/874356378 wait complete. 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 Processor -- start 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- start start 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa14105bd0 0x7faa1419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1419d150 con 0x7faa140ffc20 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1bbb2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14196600 con 0x7faa14105bd0 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49660/0 (socket says 172.21.15.8:49660) 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 -- 172.21.15.8:0/334833590 learned_addr learned my addr 172.21.15.8:0/334833590 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1914d700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa14105bd0 0x7faa1419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 -- 172.21.15.8:0/334833590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa14105bd0 msgr2=0x7faa1419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa14105bd0 0x7faa1419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 -- 172.21.15.8:0/334833590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa100095f0 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa1994e700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.639+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa1001d070 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.640+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faa1000f460 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.640+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa14196880 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.640+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa10021620 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.640+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa14196d70 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.641+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1404f2e0 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.641+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7faa1000fa40 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.641+0000 7faa06ffd700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 0x7faa0006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.642+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faa1008d6b0 con 0x7faa140ffc20 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.642+0000 7faa1914d700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 0x7faa0006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.642+0000 7faa1914d700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 0x7faa0006f1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa08005950 tx=0x7faa080058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:53.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.647+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7faa1005b230 con 0x7faa140ffc20 2024-04-12T06:19:53.868 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.867+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa14197070 con 0x7faa0006cce0 2024-04-12T06:19:53.869 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.868+0000 7faa06ffd700 1 -- 172.21.15.8:0/334833590 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faa14197070 con 0x7faa0006cce0 2024-04-12T06:19:53.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 msgr2=0x7faa0006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 0x7faa0006f1a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7faa08005950 tx=0x7faa080058e0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 msgr2=0x7faa1419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.871 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faa10006010 tx=0x7faa1000be20 comp rx=0 tx=0).stop 2024-04-12T06:19:53.872 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 shutdown_connections 2024-04-12T06:19:53.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7faa0006cce0 0x7faa0006f1a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7faa140ffc20 0x7faa1419c580 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.873 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 --2- 172.21.15.8:0/334833590 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faa14105bd0 0x7faa1419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 >> 172.21.15.8:0/334833590 conn(0x7faa14075670 msgr2=0x7faa14068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:53.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 shutdown_connections 2024-04-12T06:19:53.874 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.871+0000 7faa1bbb2700 1 -- 172.21.15.8:0/334833590 wait complete. 2024-04-12T06:19:53.882 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:19:53.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 -- 172.21.15.8:0/4185673512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 msgr2=0x7f16241058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 --2- 172.21.15.8:0/4185673512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f16241058a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1614009910 tx=0x7f1614009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:53.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 -- 172.21.15.8:0/4185673512 shutdown_connections 2024-04-12T06:19:53.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 --2- 172.21.15.8:0/4185673512 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f16241058a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 --2- 172.21.15.8:0/4185673512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f1624101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 -- 172.21.15.8:0/4185673512 >> 172.21.15.8:0/4185673512 conn(0x7f16240fc750 msgr2=0x7f16240feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:53.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 -- 172.21.15.8:0/4185673512 shutdown_connections 2024-04-12T06:19:53.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.977+0000 7f1629d40700 1 -- 172.21.15.8:0/4185673512 wait complete. 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 Processor -- start 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 -- start start 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f162419c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162419d160 con 0x7f1624101850 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.978+0000 7f1629d40700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16241966a0 con 0x7f1624100ea0 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49672/0 (socket says 172.21.15.8:49672) 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 -- 172.21.15.8:0/2079110930 learned_addr learned my addr 172.21.15.8:0/2079110930 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f16237fe700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f162419c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 -- 172.21.15.8:0/2079110930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 msgr2=0x7f162419c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:53.981 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f162419c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 -- 172.21.15.8:0/2079110930 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140095f0 con 0x7f1624101850 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f16237fe700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f162419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1622ffd700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f161400bda0 tx=0x7f161400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161401d070 con 0x7f1624101850 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f161400f460 con 0x7f1624101850 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1624196920 con 0x7f1624101850 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1614021640 con 0x7f1624101850 2024-04-12T06:19:53.982 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.979+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624196e10 con 0x7f1624101850 2024-04-12T06:19:53.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.980+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16241092a0 con 0x7f1624101850 2024-04-12T06:19:53.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.981+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f161400f9b0 con 0x7f1624101850 2024-04-12T06:19:53.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.981+0000 7f1620ff9700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 0x7f160c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:53.983 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.982+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f161408d850 con 0x7f1624101850 2024-04-12T06:19:53.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.982+0000 7f16237fe700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 0x7f160c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:53.984 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.982+0000 7f16237fe700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 0x7f160c06f1a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:53.986 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:53.987+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f161405b350 con 0x7f1624101850 2024-04-12T06:19:54.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.209+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1624197c70 con 0x7f160c06cce0 2024-04-12T06:19:54.210 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.210+0000 7f1620ff9700 1 -- 172.21.15.8:0/2079110930 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1624197c70 con 0x7f160c06cce0 2024-04-12T06:19:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 msgr2=0x7f160c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 0x7f160c06f1a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1618005fd0 tx=0x7f1618005e20 comp rx=0 tx=0).stop 2024-04-12T06:19:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 msgr2=0x7f162419cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.212 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f161400bda0 tx=0x7f161400be80 comp rx=0 tx=0).stop 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 shutdown_connections 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f160c06cce0 0x7f160c06f1a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.212+0000 7f1629d40700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1624100ea0 0x7f162419c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.213+0000 7f1629d40700 1 --2- 172.21.15.8:0/2079110930 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1624101850 0x7f162419cb60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.213+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 >> 172.21.15.8:0/2079110930 conn(0x7f16240fc750 msgr2=0x7f16240fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.213+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 shutdown_connections 2024-04-12T06:19:54.213 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.213+0000 7f1629d40700 1 -- 172.21.15.8:0/2079110930 wait complete. 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 -- 172.21.15.8:0/3255393276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 msgr2=0x7f81c41021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 --2- 172.21.15.8:0/3255393276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c41021e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009910 tx=0x7f81b4009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 -- 172.21.15.8:0/3255393276 shutdown_connections 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 --2- 172.21.15.8:0/3255393276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 0x7f81c410ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 --2- 172.21.15.8:0/3255393276 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c41021e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.319 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 -- 172.21.15.8:0/3255393276 >> 172.21.15.8:0/3255393276 conn(0x7f81c40fb650 msgr2=0x7f81c40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 -- 172.21.15.8:0/3255393276 shutdown_connections 2024-04-12T06:19:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 -- 172.21.15.8:0/3255393276 wait complete. 2024-04-12T06:19:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.319+0000 7f81c999b700 1 Processor -- start 2024-04-12T06:19:54.320 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c999b700 1 -- start start 2024-04-12T06:19:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c999b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c999b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 0x7f81c40fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.321 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c999b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c41004e0 con 0x7f81c4101e00 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c999b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c4100620 con 0x7f81c4102720 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49682/0 (socket says 172.21.15.8:49682) 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 -- 172.21.15.8:0/2349002603 learned_addr learned my addr 172.21.15.8:0/2349002603 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c27fc700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 0x7f81c40fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 -- 172.21.15.8:0/2349002603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 msgr2=0x7f81c40fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 0x7f81c40fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 -- 172.21.15.8:0/2349002603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b40095f0 con 0x7f81c4101e00 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.320+0000 7f81c2ffd700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b5c0 tx=0x7f81b400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.321+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b401d070 con 0x7f81c4101e00 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.321+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f81b400f460 con 0x7f81c4101e00 2024-04-12T06:19:54.322 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.321+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c41008a0 con 0x7f81c4101e00 2024-04-12T06:19:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.321+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b4021640 con 0x7f81c4101e00 2024-04-12T06:19:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.321+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c41a2d60 con 0x7f81c4101e00 2024-04-12T06:19:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.322+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c4108310 con 0x7f81c4101e00 2024-04-12T06:19:54.323 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.322+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f81b402b440 con 0x7f81c4101e00 2024-04-12T06:19:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.322+0000 7f81c8999700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 0x7f81ac06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.322+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f81b408d700 con 0x7f81c4101e00 2024-04-12T06:19:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.323+0000 7f81c27fc700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 0x7f81ac06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.324 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.323+0000 7f81c27fc700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 0x7f81ac06f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81c4101250 tx=0x7f81b800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:54.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.327+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f81b4057770 con 0x7f81c4101e00 2024-04-12T06:19:54.538 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.538+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f81c40611d0 con 0x7f81ac06cce0 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.549+0000 7f81c8999700 1 -- 172.21.15.8:0/2349002603 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f81c40611d0 con 0x7f81ac06cce0 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (21m) 15m ago 33m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (33m) 15m ago 33m 8456k - 18.2.1 5be31c24972a c03c2c2b2b93 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (32m) 14m ago 32m 8442k - 18.2.1 5be31c24972a 4f783589ded7 2024-04-12T06:19:54.549 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (19m) 15m ago 33m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (19m) 14m ago 32m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (21m) 15m ago 32m 86.6M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (15m) 15m ago 27m 14.1M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (15m) 15m ago 27m 15.9M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (14m) 14m ago 26m 12.5M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (15m) 14m ago 27m 13.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (23m) 15m ago 34m 513M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (22m) 14m ago 32m 452M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (20m) 15m ago 34m 55.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (19m) 14m ago 31m 42.6M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (22m) 15m ago 33m 17.6M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (22m) 14m ago 32m 18.4M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (19m) 15m ago 31m 101M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (18m) 15m ago 30m 108M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:19:54.550 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (17m) 15m ago 30m 91.3M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (17m) 14m ago 29m 106M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (16m) 14m ago 29m 109M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (16m) 14m ago 28m 85.7M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (22m) 15m ago 32m 77.2M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 msgr2=0x7f81ac06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.551 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 0x7f81ac06f1a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f81c4101250 tx=0x7f81b800b410 comp rx=0 tx=0).stop 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 msgr2=0x7f81c40ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81b400b5c0 tx=0x7f81b400bd70 comp rx=0 tx=0).stop 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 shutdown_connections 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f81ac06cce0 0x7f81ac06f1a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f81c4101e00 0x7f81c40ffa60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 --2- 172.21.15.8:0/2349002603 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f81c4102720 0x7f81c40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 >> 172.21.15.8:0/2349002603 conn(0x7f81c40fb650 msgr2=0x7f81c4105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 shutdown_connections 2024-04-12T06:19:54.552 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.551+0000 7f81c999b700 1 -- 172.21.15.8:0/2349002603 wait complete. 2024-04-12T06:19:54.661 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 -- 172.21.15.8:0/3842966086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 msgr2=0x7f45081058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3842966086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45081058a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009910 tx=0x7f44fc009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:54.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 -- 172.21.15.8:0/3842966086 shutdown_connections 2024-04-12T06:19:54.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3842966086 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45081058a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3842966086 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 0x7f4508101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.662 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 -- 172.21.15.8:0/3842966086 >> 172.21.15.8:0/3842966086 conn(0x7f45080fc750 msgr2=0x7f45080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 -- 172.21.15.8:0/3842966086 shutdown_connections 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.656+0000 7f450f5b1700 1 -- 172.21.15.8:0/3842966086 wait complete. 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 Processor -- start 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 -- start start 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 0x7f4508073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.663 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4508073b10 con 0x7f4508101850 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450f5b1700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4508073c50 con 0x7f4508100ea0 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450cb4c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450cb4c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49712/0 (socket says 172.21.15.8:49712) 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.657+0000 7f450cb4c700 1 -- 172.21.15.8:0/3380556920 learned_addr learned my addr 172.21.15.8:0/3380556920 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:54.664 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450d34d700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 0x7f4508073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450cb4c700 1 -- 172.21.15.8:0/3380556920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 msgr2=0x7f4508073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450cb4c700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 0x7f4508073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450cb4c700 1 -- 172.21.15.8:0/3380556920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc0095f0 con 0x7f4508101850 2024-04-12T06:19:54.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450cb4c700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000c00 tx=0x7f44fc00ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:54.665 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44fc01d070 con 0x7f4508101850 2024-04-12T06:19:54.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44fc00bed0 con 0x7f4508101850 2024-04-12T06:19:54.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508073ed0 con 0x7f4508101850 2024-04-12T06:19:54.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44fc00fd90 con 0x7f4508101850 2024-04-12T06:19:54.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.658+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45081a2cf0 con 0x7f4508101850 2024-04-12T06:19:54.666 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.659+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45081092a0 con 0x7f4508101850 2024-04-12T06:19:54.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.660+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f44fc021410 con 0x7f4508101850 2024-04-12T06:19:54.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.660+0000 7f44fa7fc700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 0x7f44f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:54.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.660+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f44fc08d720 con 0x7f4508101850 2024-04-12T06:19:54.667 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.660+0000 7f450d34d700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 0x7f44f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:54.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.661+0000 7f450d34d700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 0x7f44f406f1a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4504009c60 tx=0x7f4504009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:54.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.665+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f44fc057790 con 0x7f4508101850 2024-04-12T06:19:54.949 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.950+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f450804f2e0 con 0x7f4508101850 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.950+0000 7f44fa7fc700 1 -- 172.21.15.8:0/3380556920 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f44fc05adb0 con 0x7f4508101850 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:19:54.950 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:19:54.951 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 msgr2=0x7f44f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 0x7f44f406f1a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4504009c60 tx=0x7f4504009260 comp rx=0 tx=0).stop 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 msgr2=0x7f45080735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000c00 tx=0x7f44fc00ba70 comp rx=0 tx=0).stop 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 shutdown_connections 2024-04-12T06:19:54.952 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f44f406cce0 0x7f44f406f1a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4508100ea0 0x7f4508073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 --2- 172.21.15.8:0/3380556920 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4508101850 0x7f45080735d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:54.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 >> 172.21.15.8:0/3380556920 conn(0x7f45080fc750 msgr2=0x7f45080fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:54.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 shutdown_connections 2024-04-12T06:19:54.953 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:54.952+0000 7f450f5b1700 1 -- 172.21.15.8:0/3380556920 wait complete. 2024-04-12T06:19:55.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 -- 172.21.15.8:0/4210882014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778102760 msgr2=0x7f577810ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 --2- 172.21.15.8:0/4210882014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778102760 0x7f577810ac50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f576c009940 tx=0x7f576c009c50 comp rx=0 tx=0).stop 2024-04-12T06:19:55.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 -- 172.21.15.8:0/4210882014 shutdown_connections 2024-04-12T06:19:55.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 --2- 172.21.15.8:0/4210882014 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778102760 0x7f577810ac50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 --2- 172.21.15.8:0/4210882014 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778101e40 0x7f5778102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 -- 172.21.15.8:0/4210882014 >> 172.21.15.8:0/4210882014 conn(0x7f57780fb610 msgr2=0x7f57780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 -- 172.21.15.8:0/4210882014 shutdown_connections 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.058+0000 7f577de69700 1 -- 172.21.15.8:0/4210882014 wait complete. 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 Processor -- start 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 -- start start 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778102760 0x7f577819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819d170 con 0x7f5778101e40 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.059+0000 7f577de69700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5778196620 con 0x7f5778102760 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49724/0 (socket says 172.21.15.8:49724) 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 -- 172.21.15.8:0/1914093257 learned_addr learned my addr 172.21.15.8:0/1914093257 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f5776ffd700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778102760 0x7f577819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 -- 172.21.15.8:0/1914093257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778102760 msgr2=0x7f577819cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778102760 0x7f577819cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 -- 172.21.15.8:0/1914093257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0095f0 con 0x7f5778101e40 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f57777fe700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f576800d8d0 tx=0x7f576800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57680098a0 con 0x7f5778101e40 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5768010460 con 0x7f5778101e40 2024-04-12T06:19:55.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778196900 con 0x7f5778101e40 2024-04-12T06:19:55.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5768009ea0 con 0x7f5778101e40 2024-04-12T06:19:55.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.060+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778196e20 con 0x7f5778101e40 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.061+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778108350 con 0x7f5778101e40 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.062+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f57680105d0 con 0x7f5778101e40 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.062+0000 7f5774ff9700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 0x7f576006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.062+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f576808c440 con 0x7f5778101e40 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.062+0000 7f5776ffd700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 0x7f576006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.063+0000 7f5776ffd700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 0x7f576006f1a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f576c006010 tx=0x7f576c005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.067+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5768059f40 con 0x7f5778101e40 2024-04-12T06:19:55.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:54 smithi008 ceph-mon[146084]: pgmap v746: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:55.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:54 smithi008 ceph-mon[146084]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.311+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5778197040 con 0x7f5778101e40 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.311+0000 7f5774ff9700 1 -- 172.21.15.8:0/1914093257 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f5768010880 con 0x7f5778101e40 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout:e36 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:55.311 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 27 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:03:56.166804+0000 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:19:55.312 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:up {} 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:failed 0,1 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout:Standby daemons: 2024-04-12T06:19:55.313 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:19:55.314 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{-1:34330} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:55.314 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{-1:34362} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:55.314 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{-1:44331} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:55.314 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{-1:44361} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:19:55.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 msgr2=0x7f576006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 0x7f576006f1a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f576c006010 tx=0x7f576c005960 comp rx=0 tx=0).stop 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 msgr2=0x7f577819c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f576800d8d0 tx=0x7f576800dbe0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 shutdown_connections 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f576006cce0 0x7f576006f1a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5778101e40 0x7f577819c5a0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 --2- 172.21.15.8:0/1914093257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5778102760 0x7f577819cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 >> 172.21.15.8:0/1914093257 conn(0x7f57780fb610 msgr2=0x7f5778105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 shutdown_connections 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.313+0000 7f577de69700 1 -- 172.21.15.8:0/1914093257 wait complete. 2024-04-12T06:19:55.315 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:19:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:54 smithi194 ceph-mon[106982]: pgmap v746: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:54 smithi194 ceph-mon[106982]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 -- 172.21.15.8:0/3554779676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 msgr2=0x7f57680fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 --2- 172.21.15.8:0/3554779676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57680fffc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f575c009910 tx=0x7f575c009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 -- 172.21.15.8:0/3554779676 shutdown_connections 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 --2- 172.21.15.8:0/3554779676 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57680fffc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 --2- 172.21.15.8:0/3554779676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 0x7f5768105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 -- 172.21.15.8:0/3554779676 >> 172.21.15.8:0/3554779676 conn(0x7f57680fb610 msgr2=0x7f57680fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 -- 172.21.15.8:0/3554779676 shutdown_connections 2024-04-12T06:19:55.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.422+0000 7f576e651700 1 -- 172.21.15.8:0/3554779676 wait complete. 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 Processor -- start 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 -- start start 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 0x7f5768101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768102230 con 0x7f57680ffb40 2024-04-12T06:19:55.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f576e651700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768102370 con 0x7f5768105bf0 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49746/0 (socket says 172.21.15.8:49746) 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 -- 172.21.15.8:0/2324127054 learned_addr learned my addr 172.21.15.8:0/2324127054 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f57677fe700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 0x7f5768101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 -- 172.21.15.8:0/2324127054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 msgr2=0x7f5768101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 0x7f5768101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.423+0000 7f5767fff700 1 -- 172.21.15.8:0/2324127054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0095f0 con 0x7f57680ffb40 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f5767fff700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f575800e920 tx=0x7f575800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5758009790 con 0x7f57680ffb40 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f575800cba0 con 0x7f57680ffb40 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57681a28f0 con 0x7f57680ffb40 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5758018650 con 0x7f57680ffb40 2024-04-12T06:19:55.425 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.424+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57681a2d00 con 0x7f57680ffb40 2024-04-12T06:19:55.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.425+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576804f2e0 con 0x7f57680ffb40 2024-04-12T06:19:55.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.425+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f575800cd10 con 0x7f57680ffb40 2024-04-12T06:19:55.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.426+0000 7f57657fa700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.426+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5758014070 con 0x7f57680ffb40 2024-04-12T06:19:55.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.426+0000 7f57677fe700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.426+0000 7f57677fe700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 0x7f575006f1a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5768102fa0 tx=0x7f575c009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.430+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f5758052ab0 con 0x7f57680ffb40 2024-04-12T06:19:55.651 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.651+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5768069660 con 0x7f575006cce0 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.652+0000 7f57657fa700 1 -- 172.21.15.8:0/2324127054 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5768069660 con 0x7f575006cce0 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:19:55.652 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "16/23 daemons upgraded", 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading mds daemons", 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:19:55.653 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:19:55.654 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.654+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 msgr2=0x7f575006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.654+0000 7f576e651700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 0x7f575006f1a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5768102fa0 tx=0x7f575c009f70 comp rx=0 tx=0).stop 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.654+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 msgr2=0x7f57681017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.654+0000 7f576e651700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f575800e920 tx=0x7f575800ece0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 shutdown_connections 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f575006cce0 0x7f575006f1a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f57680ffb40 0x7f57681017b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 --2- 172.21.15.8:0/2324127054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5768105bf0 0x7f5768101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 >> 172.21.15.8:0/2324127054 conn(0x7f57680fb610 msgr2=0x7f5768068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 shutdown_connections 2024-04-12T06:19:55.655 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.655+0000 7f576e651700 1 -- 172.21.15.8:0/2324127054 wait complete. 2024-04-12T06:19:55.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 -- 172.21.15.8:0/154232949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 msgr2=0x7f49cc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/154232949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc105fb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009910 tx=0x7f49bc009c20 comp rx=0 tx=0).stop 2024-04-12T06:19:55.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 -- 172.21.15.8:0/154232949 shutdown_connections 2024-04-12T06:19:55.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/154232949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 0x7f49cc100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.761 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/154232949 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc105fb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 -- 172.21.15.8:0/154232949 >> 172.21.15.8:0/154232949 conn(0x7f49cc075670 msgr2=0x7f49cc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:55.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 -- 172.21.15.8:0/154232949 shutdown_connections 2024-04-12T06:19:55.762 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.761+0000 7f49d0ab2700 1 -- 172.21.15.8:0/154232949 wait complete. 2024-04-12T06:19:55.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 Processor -- start 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 -- start start 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 0x7f49cc19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc19d150 con 0x7f49cc105bd0 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49d0ab2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc196600 con 0x7f49cc0ffc20 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:49762/0 (socket says 172.21.15.8:49762) 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 -- 172.21.15.8:0/2886665816 learned_addr learned my addr 172.21.15.8:0/2886665816 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49ca59c700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 0x7f49cc19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 -- 172.21.15.8:0/2886665816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 msgr2=0x7f49cc19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 0x7f49cc19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:55.764 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.762+0000 7f49c9d9b700 1 -- 172.21.15.8:0/2886665816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0095f0 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49c9d9b700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c8a0 tx=0x7f49c000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49c00078e0 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49c000f450 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc1968e0 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49c000e5e0 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.763+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc196e30 con 0x7f49cc105bd0 2024-04-12T06:19:55.765 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.764+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc04f2e0 con 0x7f49cc105bd0 2024-04-12T06:19:55.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.764+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f49c000f5c0 con 0x7f49cc105bd0 2024-04-12T06:19:55.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.765+0000 7f49bb7fe700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 0x7f49b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:19:55.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.765+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f49c008c460 con 0x7f49cc105bd0 2024-04-12T06:19:55.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.765+0000 7f49ca59c700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 0x7f49b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:19:55.766 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.765+0000 7f49ca59c700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 0x7f49b406f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005f50 tx=0x7f49bc01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:19:55.769 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:55.769+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f49c0059fe0 con 0x7f49cc105bd0 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.057+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f49cc197160 con 0x7f49cc105bd0 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.057+0000 7f49bb7fe700 1 -- 172.21.15.8:0/2886665816 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f49c0059b70 con 0x7f49cc105bd0 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is degraded 2024-04-12T06:19:56.057 INFO:teuthology.orchestra.run.smithi008.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-12T06:19:56.058 INFO:teuthology.orchestra.run.smithi008.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 msgr2=0x7f49b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 0x7f49b406f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005f50 tx=0x7f49bc01a040 comp rx=0 tx=0).stop 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 msgr2=0x7f49cc19cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c8a0 tx=0x7f49c000cbb0 comp rx=0 tx=0).stop 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 shutdown_connections 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f49b406cce0 0x7f49b406f1a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49cc0ffc20 0x7f49cc19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:56.059 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 --2- 172.21.15.8:0/2886665816 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f49cc105bd0 0x7f49cc19cac0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:19:56.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 >> 172.21.15.8:0/2886665816 conn(0x7f49cc075670 msgr2=0x7f49cc068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:19:56.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 shutdown_connections 2024-04-12T06:19:56.060 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:19:56.059+0000 7f49d0ab2700 1 -- 172.21.15.8:0/2886665816 wait complete. 2024-04-12T06:19:56.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:55 smithi008 ceph-mon[146084]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:56.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:55 smithi008 ceph-mon[146084]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:56.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:55 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3380556920' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:56.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:55 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1914093257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:19:56.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:55 smithi194 ceph-mon[106982]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:56.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:55 smithi194 ceph-mon[106982]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:56.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:55 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3380556920' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:56.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:55 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1914093257' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:19:57.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:56 smithi008 ceph-mon[146084]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:57.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:56 smithi008 ceph-mon[146084]: pgmap v747: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:57.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:56 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2886665816' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:19:57.172 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:56 smithi194 ceph-mon[106982]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:19:57.172 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:56 smithi194 ceph-mon[106982]: pgmap v747: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:57.172 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:56 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2886665816' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.lmvepz"}]': finished 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.rhdhsr"}]': finished 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:19:58.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:57 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.gbfdvc"}]': finished 2024-04-12T06:19:58.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:19:58.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.lmvepz"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.lmvepz"}]': finished 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.rhdhsr"}]: dispatch 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi008.rhdhsr"}]': finished 2024-04-12T06:19:58.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.gbfdvc"}]: dispatch 2024-04-12T06:19:58.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:57 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.gbfdvc"}]': finished 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: pgmap v748: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: pgmap v749: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all mds 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-12T06:19:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.uzfpyu"}]': finished 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Standby daemon mds.cephfs.smithi008.rhdhsr assigned to filesystem cephfs as rank 0 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Standby daemon mds.cephfs.smithi008.lmvepz assigned to filesystem cephfs as rank 1 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:replay} 2 up:standby 2024-04-12T06:19:59.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:58 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-12T06:19:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: pgmap v748: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: pgmap v749: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all mds 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.uzfpyu"}]: dispatch 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi194.uzfpyu"}]': finished 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Standby daemon mds.cephfs.smithi008.rhdhsr assigned to filesystem cephfs as rank 0 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Standby daemon mds.cephfs.smithi008.lmvepz assigned to filesystem cephfs as rank 1 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:replay} 2 up:standby 2024-04-12T06:19:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:58 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:replay} 2 up:standby 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:19:59 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:00.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-12T06:20:00.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:replay} 2 up:standby 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:00.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:19:59 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:01.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all rgw 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: pgmap v750: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: Health detail: HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: host smithi194 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: /usr/bin/podman: timeout after 895 seconds 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 -e NODE_NAME=smithi194 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp0xbl8sy6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 inventory --format=json-pretty --filter-for-batch 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: Traceback (most recent call last): 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: "__main__", mod_spec) 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: exec(code, run_globals) 2024-04-12T06:20:01.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 10889, in 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 10877, in main 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 -e NODE_NAME=smithi194 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp0xbl8sy6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 inventory --format=json-pretty --filter-for-batch 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:20:01.160 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:00 smithi008 ceph-mon[146084]: fs cephfs is degraded 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all rgw 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all rbd-mirror 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: pgmap v750: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 0 op/s 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: Health detail: HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: host smithi194 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi194/config 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: /usr/bin/podman: timeout after 895 seconds 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 -e NODE_NAME=smithi194 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp0xbl8sy6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 inventory --format=json-pretty --filter-for-batch 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: Traceback (most recent call last): 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main 2024-04-12T06:20:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: "__main__", mod_spec) 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: exec(code, run_globals) 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 10889, in 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 10877, in main 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2576, in _infer_config 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2492, in _infer_fsid 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2604, in _infer_image 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2479, in _validate_fsid 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 7145, in command_ceph_volume 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: File "/tmp/tmpxnq8s67_.cephadm.build/__main__.py", line 2267, in call_throws 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 -e NODE_NAME=smithi194 -e CEPH_USE_RANDOM_NONCE=1 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp0xbl8sy6:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3 inventory --format=json-pretty --filter-for-batch 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-12T06:20:01.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:00 smithi194 ceph-mon[106982]: fs cephfs is degraded 2024-04-12T06:20:02.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: Upgrade: Updating ceph-exporter.smithi008 (1/2) 2024-04-12T06:20:02.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:02.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T06:20:02.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:02.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: Deploying daemon ceph-exporter.smithi008 on smithi008 2024-04-12T06:20:02.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:02 smithi194 ceph-mon[106982]: pgmap v751: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 3 op/s 2024-04-12T06:20:02.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: Upgrade: Updating ceph-exporter.smithi008 (1/2) 2024-04-12T06:20:02.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:02.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi008", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T06:20:02.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:02.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: Deploying daemon ceph-exporter.smithi008 on smithi008 2024-04-12T06:20:02.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:02 smithi008 ceph-mon[146084]: pgmap v751: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.6 MiB/s rd, 3 op/s 2024-04-12T06:20:03.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:03.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:03.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:03.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:04.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:04 smithi194 ceph-mon[106982]: pgmap v752: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-04-12T06:20:04.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:04 smithi008 ceph-mon[146084]: pgmap v752: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-04-12T06:20:06.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:06 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:resolve 2024-04-12T06:20:06.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:06 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:resolve} 2 up:standby 2024-04-12T06:20:06.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:06 smithi194 ceph-mon[106982]: pgmap v753: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-12T06:20:06.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:06 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:resolve 2024-04-12T06:20:06.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:06 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:replay,1=cephfs.smithi008.lmvepz=up:resolve} 2 up:standby 2024-04-12T06:20:06.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:06 smithi008 ceph-mon[146084]: pgmap v753: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-12T06:20:07.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:07 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:resolve 2024-04-12T06:20:07.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:07 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:resolve,1=cephfs.smithi008.lmvepz=up:resolve} 2 up:standby 2024-04-12T06:20:07.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:07 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:resolve 2024-04-12T06:20:07.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:07 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:resolve,1=cephfs.smithi008.lmvepz=up:resolve} 2 up:standby 2024-04-12T06:20:08.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: reconnect by client.14532 192.168.0.1:0/530473219 after 0.000999989 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: reconnect by client.14532 192.168.0.1:0/530473219 after 0.000999989 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: reconnect by client.24331 192.168.0.1:0/4244588227 after 0.000999989 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: reconnect by client.24331 192.168.0.1:0/4244588227 after 0.000999989 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:reconnect 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:reconnect 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:reconnect,1=cephfs.smithi008.lmvepz=up:reconnect} 2 up:standby 2024-04-12T06:20:08.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:08 smithi194 ceph-mon[106982]: pgmap v754: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: reconnect by client.14532 192.168.0.1:0/530473219 after 0.000999989 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: reconnect by client.14532 192.168.0.1:0/530473219 after 0.000999989 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: reconnect by client.24331 192.168.0.1:0/4244588227 after 0.000999989 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: reconnect by client.24331 192.168.0.1:0/4244588227 after 0.000999989 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:reconnect 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:reconnect 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:reconnect,1=cephfs.smithi008.lmvepz=up:reconnect} 2 up:standby 2024-04-12T06:20:08.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:08 smithi008 ceph-mon[146084]: pgmap v754: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-04-12T06:20:09.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:09 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:rejoin 2024-04-12T06:20:09.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:09 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:rejoin 2024-04-12T06:20:09.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:09 smithi194 ceph-mon[106982]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:rejoin,1=cephfs.smithi008.lmvepz=up:rejoin} 2 up:standby 2024-04-12T06:20:09.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:09 smithi194 ceph-mon[106982]: daemon mds.cephfs.smithi008.lmvepz is now active in filesystem cephfs as rank 1 2024-04-12T06:20:09.825 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:09 smithi194 ceph-mon[106982]: daemon mds.cephfs.smithi008.rhdhsr is now active in filesystem cephfs as rank 0 2024-04-12T06:20:09.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:09 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:rejoin 2024-04-12T06:20:09.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:09 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:rejoin 2024-04-12T06:20:09.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:09 smithi008 ceph-mon[146084]: fsmap cephfs:2/2 {0=cephfs.smithi008.rhdhsr=up:rejoin,1=cephfs.smithi008.lmvepz=up:rejoin} 2 up:standby 2024-04-12T06:20:09.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:09 smithi008 ceph-mon[146084]: daemon mds.cephfs.smithi008.lmvepz is now active in filesystem cephfs as rank 1 2024-04-12T06:20:09.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:09 smithi008 ceph-mon[146084]: daemon mds.cephfs.smithi008.rhdhsr is now active in filesystem cephfs as rank 0 2024-04-12T06:20:10.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T06:20:10.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:active 2024-04-12T06:20:10.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:active 2024-04-12T06:20:10.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby 2024-04-12T06:20:10.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby-replay 2024-04-12T06:20:10.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:10 smithi194 ceph-mon[106982]: pgmap v755: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] up:active 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: mds.? [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] up:active 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby-replay 2024-04-12T06:20:10.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:10 smithi008 ceph-mon[146084]: pgmap v755: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-12T06:20:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:12 smithi194 ceph-mon[106982]: pgmap v756: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-12T06:20:13.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:13.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:12 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:12 smithi008 ceph-mon[146084]: pgmap v756: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-04-12T06:20:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:13.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:12 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:15.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:14 smithi194 ceph-mon[106982]: pgmap v757: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-12T06:20:15.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:15.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T06:20:15.071 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:14 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:14 smithi008 ceph-mon[146084]: pgmap v757: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 11 KiB/s wr, 19 op/s 2024-04-12T06:20:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi194", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-12T06:20:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:14 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:15.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:15 smithi194 ceph-mon[106982]: Upgrade: Updating ceph-exporter.smithi194 (2/2) 2024-04-12T06:20:15.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:15 smithi194 ceph-mon[106982]: Deploying daemon ceph-exporter.smithi194 on smithi194 2024-04-12T06:20:16.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:15 smithi008 ceph-mon[146084]: Upgrade: Updating ceph-exporter.smithi194 (2/2) 2024-04-12T06:20:16.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:15 smithi008 ceph-mon[146084]: Deploying daemon ceph-exporter.smithi194 on smithi194 2024-04-12T06:20:16.963 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:16 smithi194 ceph-mon[106982]: pgmap v758: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:17.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:16 smithi008 ceph-mon[146084]: pgmap v758: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:18.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:18.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: pgmap v759: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:18.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.826 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.827 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:18.827 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.827 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.827 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.827 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: pgmap v759: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:18.909 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.909 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.909 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:18.909 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:21.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:20 smithi194 ceph-mon[106982]: pgmap v760: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:21.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:20 smithi008 ceph-mon[146084]: pgmap v760: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:22 smithi194 ceph-mon[106982]: pgmap v761: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:23.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:22 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:22 smithi008 ceph-mon[146084]: pgmap v761: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 11 KiB/s wr, 18 op/s 2024-04-12T06:20:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:22 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.304 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.305 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.305 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.305 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.305 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.305 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:24 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:24.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:24 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:25.332 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:25 smithi008 ceph-mon[146084]: pgmap v762: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-12T06:20:25.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:25 smithi194 ceph-mon[106982]: pgmap v762: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 6.0 KiB/s wr, 13 op/s 2024-04-12T06:20:26.172 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 -- 172.21.15.8:0/1228664401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 msgr2=0x7febdc100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 --2- 172.21.15.8:0/1228664401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc100080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7febc4009910 tx=0x7febc4009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 -- 172.21.15.8:0/1228664401 shutdown_connections 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 --2- 172.21.15.8:0/1228664401 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc100080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 --2- 172.21.15.8:0/1228664401 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 -- 172.21.15.8:0/1228664401 >> 172.21.15.8:0/1228664401 conn(0x7febdc075670 msgr2=0x7febdc075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 -- 172.21.15.8:0/1228664401 shutdown_connections 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.170+0000 7febe0d13700 1 -- 172.21.15.8:0/1228664401 wait complete. 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 Processor -- start 2024-04-12T06:20:26.173 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 -- start start 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1022c0 con 0x7febdc0ffc20 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febe0d13700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc102400 con 0x7febdc105bd0 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febda59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febda59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38606/0 (socket says 172.21.15.8:38606) 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.171+0000 7febda59c700 1 -- 172.21.15.8:0/925231019 learned_addr learned my addr 172.21.15.8:0/925231019 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febd9d9b700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febda59c700 1 -- 172.21.15.8:0/925231019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 msgr2=0x7febdc101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febda59c700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febda59c700 1 -- 172.21.15.8:0/925231019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40095f0 con 0x7febdc0ffc20 2024-04-12T06:20:26.174 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febd9d9b700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febda59c700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7febd40060b0 tx=0x7febd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd400dda0 con 0x7febdc0ffc20 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7febd400f040 con 0x7febdc0ffc20 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc1a2980 con 0x7febdc0ffc20 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd40146f0 con 0x7febdc0ffc20 2024-04-12T06:20:26.175 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.172+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1a2d90 con 0x7febdc0ffc20 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.173+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc04f2e0 con 0x7febdc0ffc20 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.174+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7febd4015400 con 0x7febdc0ffc20 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.174+0000 7febd37fe700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 0x7febc806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.174+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7febd408c050 con 0x7febdc0ffc20 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.174+0000 7febd9d9b700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 0x7febc806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.176 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.175+0000 7febd9d9b700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 0x7febc806f1a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7febc400b5c0 tx=0x7febc4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.179 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.180+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7febd4056170 con 0x7febdc0ffc20 2024-04-12T06:20:26.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.411+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febdc068e10 con 0x7febc806cce0 2024-04-12T06:20:26.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.412+0000 7febd37fe700 1 -- 172.21.15.8:0/925231019 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7febdc068e10 con 0x7febc806cce0 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.414+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 msgr2=0x7febc806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.414+0000 7febe0d13700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 0x7febc806f1a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7febc400b5c0 tx=0x7febc4005960 comp rx=0 tx=0).stop 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.414+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 msgr2=0x7febdc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.414+0000 7febe0d13700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7febd40060b0 tx=0x7febd400d3b0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 shutdown_connections 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7febc806cce0 0x7febc806f1a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7febdc0ffc20 0x7febdc101840 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 --2- 172.21.15.8:0/925231019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc105bd0 0x7febdc101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 >> 172.21.15.8:0/925231019 conn(0x7febdc075670 msgr2=0x7febdc068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 shutdown_connections 2024-04-12T06:20:26.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.415+0000 7febe0d13700 1 -- 172.21.15.8:0/925231019 wait complete. 2024-04-12T06:20:26.426 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 -- 172.21.15.8:0/946680619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec100750 msgr2=0x7f9aec1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 --2- 172.21.15.8:0/946680619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec100750 0x7f9aec1047a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009910 tx=0x7f9ae0009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 -- 172.21.15.8:0/946680619 shutdown_connections 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 --2- 172.21.15.8:0/946680619 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec100750 0x7f9aec1047a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 --2- 172.21.15.8:0/946680619 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec0ffda0 0x7f9aec100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 -- 172.21.15.8:0/946680619 >> 172.21.15.8:0/946680619 conn(0x7f9aec0fb650 msgr2=0x7f9aec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:26.528 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 -- 172.21.15.8:0/946680619 shutdown_connections 2024-04-12T06:20:26.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.528+0000 7f9af141f700 1 -- 172.21.15.8:0/946680619 wait complete. 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 Processor -- start 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 -- start start 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 0x7f9aec197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec198260 con 0x7f9aec0ffda0 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9af141f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1983a0 con 0x7f9aec100750 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9aeaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9aea7fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 0x7f9aec197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9aeaffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38622/0 (socket says 172.21.15.8:38622) 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9aea7fc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 0x7f9aec197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.8:54212/0 (socket says 172.21.15.8:54212) 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.529+0000 7f9aeaffd700 1 -- 172.21.15.8:0/66009021 learned_addr learned my addr 172.21.15.8:0/66009021 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.530+0000 7f9aeaffd700 1 -- 172.21.15.8:0/66009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 msgr2=0x7f9aec197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.530+0000 7f9aeaffd700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 0x7f9aec197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.530+0000 7f9aeaffd700 1 -- 172.21.15.8:0/66009021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00095f0 con 0x7f9aec0ffda0 2024-04-12T06:20:26.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.530+0000 7f9aeaffd700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d900 tx=0x7f9adc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.531+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc0041f0 con 0x7f9aec0ffda0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.531+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9adc004cd0 con 0x7f9aec0ffda0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.531+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc005660 con 0x7f9aec0ffda0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.532+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec1a0770 con 0x7f9aec0ffda0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.532+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec1a0be0 con 0x7f9aec0ffda0 2024-04-12T06:20:26.532 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.532+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec1081a0 con 0x7f9aec0ffda0 2024-04-12T06:20:26.534 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.535+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9adc004e40 con 0x7f9aec0ffda0 2024-04-12T06:20:26.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.535+0000 7f9ad3fff700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 0x7f9ad406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.535+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9adc08c890 con 0x7f9aec0ffda0 2024-04-12T06:20:26.535 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.536+0000 7f9aea7fc700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 0x7f9ad406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.536 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.536+0000 7f9aea7fc700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 0x7f9ad406f1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006010 tx=0x7f9ae001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.541+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9adc056980 con 0x7f9aec0ffda0 2024-04-12T06:20:26.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.806+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9aec069580 con 0x7f9ad406cce0 2024-04-12T06:20:26.807 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.807+0000 7f9ad3fff700 1 -- 172.21.15.8:0/66009021 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f9aec069580 con 0x7f9ad406cce0 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.809+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 msgr2=0x7f9ad406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.809+0000 7f9af141f700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 0x7f9ad406f1a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006010 tx=0x7f9ae001a040 comp rx=0 tx=0).stop 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.809+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 msgr2=0x7f9aec197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.809+0000 7f9af141f700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00d900 tx=0x7f9adc00dc10 comp rx=0 tx=0).stop 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 shutdown_connections 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9ad406cce0 0x7f9ad406f1a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9aec0ffda0 0x7f9aec197700 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 --2- 172.21.15.8:0/66009021 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9aec100750 0x7f9aec197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 >> 172.21.15.8:0/66009021 conn(0x7f9aec0fb650 msgr2=0x7f9aec0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 shutdown_connections 2024-04-12T06:20:26.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.810+0000 7f9af141f700 1 -- 172.21.15.8:0/66009021 wait complete. 2024-04-12T06:20:26.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 -- 172.21.15.8:0/892463084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318068580 msgr2=0x7fc3180689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/892463084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318068580 0x7fc3180689e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009910 tx=0x7fc30c009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 -- 172.21.15.8:0/892463084 shutdown_connections 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/892463084 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318068580 0x7fc3180689e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/892463084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318106cd0 0x7fc3181070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 -- 172.21.15.8:0/892463084 >> 172.21.15.8:0/892463084 conn(0x7fc318075670 msgr2=0x7fc318075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 -- 172.21.15.8:0/892463084 shutdown_connections 2024-04-12T06:20:26.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.925+0000 7fc31d4b0700 1 -- 172.21.15.8:0/892463084 wait complete. 2024-04-12T06:20:26.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 Processor -- start 2024-04-12T06:20:26.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 -- start start 2024-04-12T06:20:26.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318068580 0x7fc3180ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318100770 con 0x7fc318106cd0 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.926+0000 7fc31d4b0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3181008b0 con 0x7fc318068580 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38640/0 (socket says 172.21.15.8:38640) 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 -- 172.21.15.8:0/3863136626 learned_addr learned my addr 172.21.15.8:0/3863136626 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc316ffd700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318068580 0x7fc3180ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 -- 172.21.15.8:0/3863136626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318068580 msgr2=0x7fc3180ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318068580 0x7fc3180ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:26.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 -- 172.21.15.8:0/3863136626 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0095f0 con 0x7fc318106cd0 2024-04-12T06:20:26.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc3167fc700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc30c006010 tx=0x7fc30c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c01d070 con 0x7fc318106cd0 2024-04-12T06:20:26.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318100b30 con 0x7fc318106cd0 2024-04-12T06:20:26.929 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc30c005710 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c00fe20 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.927+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3181a2c70 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.929+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3181091d0 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.929+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc30c003a60 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.929+0000 7fc2fffff700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 0x7fc30006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.930+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc30c08d730 con 0x7fc318106cd0 2024-04-12T06:20:26.930 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.931+0000 7fc316ffd700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 0x7fc30006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:26.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.931+0000 7fc316ffd700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 0x7fc30006f1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc308005950 tx=0x7fc30800b410 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:26.934 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:26.934+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc30c0577a0 con 0x7fc318106cd0 2024-04-12T06:20:27.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.149+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3180611d0 con 0x7fc30006cce0 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.159+0000 7fc2fffff700 1 -- 172.21.15.8:0/3863136626 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc3180611d0 con 0x7fc30006cce0 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (22m) 4s ago 33m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (15s) 4s ago 33m 7075k - 18.2.2-1187-gd540ebac bb2c5eac26a4 dd25dcb3a496 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (9s) 4s ago 32m 6874k - 18.2.2-1187-gd540ebac bb2c5eac26a4 e155129f1ace 2024-04-12T06:20:27.160 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (20m) 4s ago 33m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (19m) 4s ago 32m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (21m) 4s ago 33m 93.5M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (16m) 4s ago 27m 60.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (16m) 4s ago 27m 74.7M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (15m) 4s ago 27m 58.6M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (15m) 4s ago 27m 70.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (24m) 4s ago 35m 517M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:20:27.161 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (23m) 4s ago 32m 458M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (20m) 4s ago 35m 73.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (20m) 4s ago 32m 50.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (23m) 4s ago 33m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (23m) 4s ago 32m 18.7M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (19m) 4s ago 31m 111M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (18m) 4s ago 31m 121M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (18m) 4s ago 30m 104M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (17m) 4s ago 30m 122M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (17m) 4s ago 29m 119M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (16m) 4s ago 29m 97.0M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:20:27.162 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (22m) 4s ago 33m 82.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:20:27.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 msgr2=0x7fc30006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 0x7fc30006f1a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc308005950 tx=0x7fc30800b410 comp rx=0 tx=0).stop 2024-04-12T06:20:27.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 msgr2=0x7fc318100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc30c006010 tx=0x7fc30c00bd70 comp rx=0 tx=0).stop 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 shutdown_connections 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc30006cce0 0x7fc30006f1a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc318068580 0x7fc3180ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 --2- 172.21.15.8:0/3863136626 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc318106cd0 0x7fc318100230 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.161+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 >> 172.21.15.8:0/3863136626 conn(0x7fc318075670 msgr2=0x7fc3180fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.162+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 shutdown_connections 2024-04-12T06:20:27.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.162+0000 7fc31d4b0700 1 -- 172.21.15.8:0/3863136626 wait complete. 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 -- 172.21.15.8:0/1182974076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 msgr2=0x7f1288105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1182974076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f1288105890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1278009910 tx=0x7f1278009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 -- 172.21.15.8:0/1182974076 shutdown_connections 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1182974076 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f1288105890 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1182974076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 0x7f1288101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.275 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 -- 172.21.15.8:0/1182974076 >> 172.21.15.8:0/1182974076 conn(0x7f12880fc740 msgr2=0x7f12880feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 -- 172.21.15.8:0/1182974076 shutdown_connections 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.273+0000 7f128d2e7700 1 -- 172.21.15.8:0/1182974076 wait complete. 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 Processor -- start 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 -- start start 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 0x7f128819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128819f350 con 0x7f1288101840 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f128d2e7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1288198890 con 0x7f1288100e90 2024-04-12T06:20:27.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38658/0 (socket says 172.21.15.8:38658) 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 -- 172.21.15.8:0/1143949903 learned_addr learned my addr 172.21.15.8:0/1143949903 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f1286ffd700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 0x7f128819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 -- 172.21.15.8:0/1143949903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 msgr2=0x7f128819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 0x7f128819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.274+0000 7f12867fc700 1 -- 172.21.15.8:0/1143949903 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12780095f0 con 0x7f1288101840 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f12867fc700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f127800bda0 tx=0x7f127800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f127801d070 con 0x7f1288101840 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f127800f460 con 0x7f1288101840 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1288198b10 con 0x7f1288101840 2024-04-12T06:20:27.277 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1278021640 con 0x7f1288101840 2024-04-12T06:20:27.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.275+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1288199000 con 0x7f1288101840 2024-04-12T06:20:27.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.276+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1288105010 con 0x7f1288101840 2024-04-12T06:20:27.278 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.276+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f127800fa60 con 0x7f1288101840 2024-04-12T06:20:27.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.277+0000 7f126ffff700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 0x7f127006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.277+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f127808d690 con 0x7f1288101840 2024-04-12T06:20:27.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.277+0000 7f1286ffd700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 0x7f127006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.277+0000 7f1286ffd700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 0x7f127006f1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f127c00ac50 tx=0x7f127c00a380 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:27.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.281+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f127805b210 con 0x7f1288101840 2024-04-12T06:20:27.317 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:27 smithi008 ceph-mon[146084]: pgmap v763: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 6 op/s 2024-04-12T06:20:27.317 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:27 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:27.317 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:27 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:27.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:27 smithi194 ceph-mon[106982]: pgmap v763: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 0 B/s wr, 6 op/s 2024-04-12T06:20:27.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:27 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:27.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:27 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.575+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1288199da0 con 0x7f1288101840 2024-04-12T06:20:27.575 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.575+0000 7f126ffff700 1 -- 172.21.15.8:0/1143949903 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f127805ada0 con 0x7f1288101840 2024-04-12T06:20:27.575 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:20:27.576 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:20:27.577 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:20:27.577 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:20:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.577+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 msgr2=0x7f127006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.577+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 0x7f127006f1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f127c00ac50 tx=0x7f127c00a380 comp rx=0 tx=0).stop 2024-04-12T06:20:27.577 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.577+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 msgr2=0x7f128819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.577+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f127800bda0 tx=0x7f127800bdd0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 shutdown_connections 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f127006cce0 0x7f127006f1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1288100e90 0x7f128819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 --2- 172.21.15.8:0/1143949903 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1288101840 0x7f128819ed50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 >> 172.21.15.8:0/1143949903 conn(0x7f12880fc740 msgr2=0x7f12880fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 shutdown_connections 2024-04-12T06:20:27.578 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.578+0000 7f128d2e7700 1 -- 172.21.15.8:0/1143949903 wait complete. 2024-04-12T06:20:27.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3455510192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 msgr2=0x7f7df4100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.684 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3455510192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df4100080 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7de8009910 tx=0x7f7de8009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3455510192 shutdown_connections 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3455510192 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df4100080 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3455510192 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df4105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3455510192 >> 172.21.15.8:0/3455510192 conn(0x7f7df4075670 msgr2=0x7f7df4075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3455510192 shutdown_connections 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.684+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3455510192 wait complete. 2024-04-12T06:20:27.685 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 Processor -- start 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 -- start start 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df419cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df419d150 con 0x7f7df40ffc20 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7dfa03a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df4196600 con 0x7f7df4105bd0 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7df37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7df37fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38676/0 (socket says 172.21.15.8:38676) 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7df37fe700 1 -- 172.21.15.8:0/3666968272 learned_addr learned my addr 172.21.15.8:0/3666968272 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.685+0000 7f7df2ffd700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df419cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df37fe700 1 -- 172.21.15.8:0/3666968272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 msgr2=0x7f7df419cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.687 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df37fe700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df419cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df37fe700 1 -- 172.21.15.8:0/3666968272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80095f0 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df2ffd700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df37fe700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d900 tx=0x7f7de400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de40041f0 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df41968e0 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7de4004cd0 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de40056d0 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.686+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df4196e30 con 0x7f7df40ffc20 2024-04-12T06:20:27.688 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.687+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df404f2e0 con 0x7f7df40ffc20 2024-04-12T06:20:27.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.688+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7de4004e40 con 0x7f7df40ffc20 2024-04-12T06:20:27.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.688+0000 7f7df0ff9700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 0x7f7ddc06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:27.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.688+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7de408cc00 con 0x7f7df40ffc20 2024-04-12T06:20:27.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.688+0000 7f7df2ffd700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 0x7f7ddc06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:27.689 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.689+0000 7f7df2ffd700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 0x7f7ddc06f140 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7df4075d70 tx=0x7f7de801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:27.692 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.692+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7de405a780 con 0x7f7df40ffc20 2024-04-12T06:20:27.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.951+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7df4197170 con 0x7f7df40ffc20 2024-04-12T06:20:27.951 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.951+0000 7f7df0ff9700 1 -- 172.21.15.8:0/3666968272 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 44 v44) v1 ==== 76+0+1870 (secure 0 0 0) 0x7f7de4016020 con 0x7f7df40ffc20 2024-04-12T06:20:27.951 INFO:teuthology.orchestra.run.smithi008.stdout:e44 2024-04-12T06:20:27.951 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 44 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:20:09.472436+0000 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:20:27.952 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=34330,1=34362} 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:20:27.953 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:34330} state up:active seq 245 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{0:44331} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{1:34362} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:44361} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:20:27.954 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 msgr2=0x7f7ddc06f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 0x7f7ddc06f140 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7df4075d70 tx=0x7f7de801a040 comp rx=0 tx=0).stop 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 msgr2=0x7f7df419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7de400d900 tx=0x7f7de400dcc0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 shutdown_connections 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ddc06cc80 0x7f7ddc06f140 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7df40ffc20 0x7f7df419c580 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.955 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 --2- 172.21.15.8:0/3666968272 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7df4105bd0 0x7f7df419cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:27.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 >> 172.21.15.8:0/3666968272 conn(0x7f7df4075670 msgr2=0x7f7df4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:27.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 shutdown_connections 2024-04-12T06:20:27.956 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:27.953+0000 7f7dfa03a700 1 -- 172.21.15.8:0/3666968272 wait complete. 2024-04-12T06:20:27.956 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 44 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 -- 172.21.15.8:0/1148818318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 msgr2=0x7f2490105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 --2- 172.21.15.8:0/1148818318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f2490105fb0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f248c009910 tx=0x7f248c009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 -- 172.21.15.8:0/1148818318 shutdown_connections 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 --2- 172.21.15.8:0/1148818318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 0x7f2490100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 --2- 172.21.15.8:0/1148818318 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f2490105fb0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 -- 172.21.15.8:0/1148818318 >> 172.21.15.8:0/1148818318 conn(0x7f2490075670 msgr2=0x7f2490075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 -- 172.21.15.8:0/1148818318 shutdown_connections 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.061+0000 7f24981e0700 1 -- 172.21.15.8:0/1148818318 wait complete. 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 Processor -- start 2024-04-12T06:20:28.063 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 -- start start 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 0x7f249019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249019d160 con 0x7f2490105bd0 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f24981e0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490196610 con 0x7f24900ffc20 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f249577b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f249577b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38694/0 (socket says 172.21.15.8:38694) 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f249577b700 1 -- 172.21.15.8:0/105506258 learned_addr learned my addr 172.21.15.8:0/105506258 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f2495f7c700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 0x7f249019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.062+0000 7f249577b700 1 -- 172.21.15.8:0/105506258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 msgr2=0x7f249019c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f249577b700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 0x7f249019c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f249577b700 1 -- 172.21.15.8:0/105506258 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0095f0 con 0x7f2490105bd0 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f249577b700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f248000c8a0 tx=0x7f248000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:28.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248000cea0 con 0x7f2490105bd0 2024-04-12T06:20:28.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2480004830 con 0x7f2490105bd0 2024-04-12T06:20:28.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24901968f0 con 0x7f2490105bd0 2024-04-12T06:20:28.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2480005500 con 0x7f2490105bd0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.063+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2490196e40 con 0x7f2490105bd0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.064+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249004f2e0 con 0x7f2490105bd0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.064+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f24800049a0 con 0x7f2490105bd0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.065+0000 7f2486ffd700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 0x7f247c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.065+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f248008c270 con 0x7f2490105bd0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.065+0000 7f2495f7c700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 0x7f247c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.066 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.065+0000 7f2495f7c700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 0x7f247c06f1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f248c006010 tx=0x7f248c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:28.069 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.069+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2480059d70 con 0x7f2490105bd0 2024-04-12T06:20:28.295 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.295+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2490197150 con 0x7f247c06cce0 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.296+0000 7f2486ffd700 1 -- 172.21.15.8:0/105506258 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f2490197150 con 0x7f247c06cce0 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:72f2afc3fa84ea067411fbf5c1562b80cbcb72ca6ea6f1e44b170e45174046f3", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": true, 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [ 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "osd", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "mds", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph-exporter", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "mon", 2024-04-12T06:20:28.296 INFO:teuthology.orchestra.run.smithi008.stdout: "crash" 2024-04-12T06:20:28.297 INFO:teuthology.orchestra.run.smithi008.stdout: ], 2024-04-12T06:20:28.297 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": "18/23 daemons upgraded", 2024-04-12T06:20:28.297 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-12T06:20:28.297 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:20:28.297 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 msgr2=0x7f247c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 0x7f247c06f1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f248c006010 tx=0x7f248c01a040 comp rx=0 tx=0).stop 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 msgr2=0x7f249019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f248000c8a0 tx=0x7f248000cc60 comp rx=0 tx=0).stop 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 shutdown_connections 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f247c06cce0 0x7f247c06f1a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24900ffc20 0x7f249019c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 --2- 172.21.15.8:0/105506258 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2490105bd0 0x7f249019cad0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 >> 172.21.15.8:0/105506258 conn(0x7f2490075670 msgr2=0x7f2490068700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:28.299 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 shutdown_connections 2024-04-12T06:20:28.300 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.298+0000 7f24981e0700 1 -- 172.21.15.8:0/105506258 wait complete. 2024-04-12T06:20:28.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:28.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:28.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1143949903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T06:20:28.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:28 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3666968272' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:20:28.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 -- 172.21.15.8:0/3792258130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 msgr2=0x7f043c105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.404 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 --2- 172.21.15.8:0/3792258130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c105fb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f042c009910 tx=0x7f042c009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 -- 172.21.15.8:0/3792258130 shutdown_connections 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 --2- 172.21.15.8:0/3792258130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 --2- 172.21.15.8:0/3792258130 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c105fb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 -- 172.21.15.8:0/3792258130 >> 172.21.15.8:0/3792258130 conn(0x7f043c075670 msgr2=0x7f043c075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 -- 172.21.15.8:0/3792258130 shutdown_connections 2024-04-12T06:20:28.405 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.404+0000 7f044202e700 1 -- 172.21.15.8:0/3792258130 wait complete. 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 Processor -- start 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 -- start start 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c19c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c19d160 con 0x7f043c105bd0 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.405+0000 7f044202e700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c196610 con 0x7f043c0ffc20 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:38708/0 (socket says 172.21.15.8:38708) 2024-04-12T06:20:28.407 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 -- 172.21.15.8:0/1880416530 learned_addr learned my addr 172.21.15.8:0/1880416530 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043b7fe700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c19c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 -- 172.21.15.8:0/1880416530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 msgr2=0x7f043c19c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c19c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 -- 172.21.15.8:0/1880416530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0095f0 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043b7fe700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f043affd700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f042400c8a0 tx=0x7f042400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f042400cea0 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c1968f0 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0424004830 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0424005500 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.406+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c196e40 con 0x7f043c105bd0 2024-04-12T06:20:28.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.407+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c04f2e0 con 0x7f043c105bd0 2024-04-12T06:20:28.409 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.408+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f04240049a0 con 0x7f043c105bd0 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1143949903' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:28.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-12T06:20:28.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:28 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3666968272' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:20:28.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.408+0000 7f0438ff9700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 0x7f042806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:28.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.408+0000 7f043b7fe700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 0x7f042806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:28.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.408+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f042408c2a0 con 0x7f043c105bd0 2024-04-12T06:20:28.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.409+0000 7f043b7fe700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 0x7f042806f1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f042c00b5c0 tx=0x7f042c005fd0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:28.422 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.413+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0424059d40 con 0x7f043c105bd0 2024-04-12T06:20:28.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.704+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f043c197120 con 0x7f043c105bd0 2024-04-12T06:20:28.704 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.705+0000 7f0438ff9700 1 -- 172.21.15.8:0/1880416530 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f04240598d0 con 0x7f043c105bd0 2024-04-12T06:20:28.705 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 msgr2=0x7f042806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 0x7f042806f1a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f042c00b5c0 tx=0x7f042c005fd0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 msgr2=0x7f043c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f042400c8a0 tx=0x7f042400cc60 comp rx=0 tx=0).stop 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 shutdown_connections 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f042806cce0 0x7f042806f1a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f043c0ffc20 0x7f043c19c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.707 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 --2- 172.21.15.8:0/1880416530 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f043c105bd0 0x7f043c19cad0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:28.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 >> 172.21.15.8:0/1880416530 conn(0x7f043c075670 msgr2=0x7f043c0686a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:28.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 shutdown_connections 2024-04-12T06:20:28.708 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:28.707+0000 7f044202e700 1 -- 172.21.15.8:0/1880416530 wait complete. 2024-04-12T06:20:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: pgmap v764: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 4 op/s 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: pgmap v765: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 3 op/s 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: Cluster is now healthy 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1880416530' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby-replay 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi008"}]: dispatch 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi008"}]': finished 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi194"}]: dispatch 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi194"}]': finished 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T06:20:29.411 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T06:20:29.412 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:29.413 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:29.414 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='client.35194 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: pgmap v764: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 4 op/s 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: pgmap v765: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 3 op/s 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: Upgrade: Setting filesystem cephfs Joinable 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: Cluster is now healthy 2024-04-12T06:20:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1880416530' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: fsmap cephfs:2 {0=cephfs.smithi008.rhdhsr=up:active,1=cephfs.smithi008.lmvepz=up:active} 2 up:standby-replay 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi008"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi008"}]': finished 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi194"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi194"}]': finished 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.576 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-12T06:20:29.577 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-12T06:20:29.578 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix": "config rm", "name": "mon_mds_skip_sanity", "who": "mon"}]': finished 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:29.579 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:29.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:20:29.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:20:29.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:20:29.580 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all iscsi 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all nfs 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Setting container_image for all nvmeof 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Finalizing container_image settings 2024-04-12T06:20:30.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:30 smithi008 ceph-mon[146084]: Upgrade: Complete! 2024-04-12T06:20:30.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:20:30.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all ceph-exporter 2024-04-12T06:20:30.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all iscsi 2024-04-12T06:20:30.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all nfs 2024-04-12T06:20:30.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Setting container_image for all nvmeof 2024-04-12T06:20:30.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Finalizing container_image settings 2024-04-12T06:20:30.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:30 smithi194 ceph-mon[106982]: Upgrade: Complete! 2024-04-12T06:20:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:31 smithi008 ceph-mon[146084]: pgmap v766: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 4 op/s 2024-04-12T06:20:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:31 smithi194 ceph-mon[106982]: pgmap v766: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 4 op/s 2024-04-12T06:20:33.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:33 smithi194 ceph-mon[106982]: pgmap v767: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 4 op/s 2024-04-12T06:20:33.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:33 smithi008 ceph-mon[146084]: pgmap v767: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 7.7 KiB/s rd, 4 op/s 2024-04-12T06:20:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:20:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:35 smithi008 ceph-mon[146084]: pgmap v768: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:35.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:35 smithi194 ceph-mon[106982]: pgmap v768: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:37 smithi008 ceph-mon[146084]: pgmap v769: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:37.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:37 smithi194 ceph-mon[106982]: pgmap v769: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:39 smithi008 ceph-mon[146084]: pgmap v770: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:39.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:39 smithi194 ceph-mon[106982]: pgmap v770: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:20:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:41 smithi008 ceph-mon[146084]: pgmap v771: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:20:41.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:41 smithi194 ceph-mon[106982]: pgmap v771: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:20:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:43 smithi008 ceph-mon[146084]: pgmap v772: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:20:43.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:43 smithi194 ceph-mon[106982]: pgmap v772: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:20:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:45 smithi008 ceph-mon[146084]: pgmap v773: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:45.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:45 smithi194 ceph-mon[106982]: pgmap v773: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:47 smithi008 ceph-mon[146084]: pgmap v774: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:47.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:47 smithi194 ceph-mon[106982]: pgmap v774: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:48 smithi008 ceph-mon[146084]: pgmap v775: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:48.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:20:48.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:48 smithi194 ceph-mon[106982]: pgmap v775: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:51.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:50 smithi008 ceph-mon[146084]: pgmap v776: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:20:51.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:50 smithi194 ceph-mon[106982]: pgmap v776: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:20:53.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:52 smithi008 ceph-mon[146084]: pgmap v777: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:20:53.216 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:52 smithi194 ceph-mon[106982]: pgmap v777: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:20:55.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:54 smithi008 ceph-mon[146084]: pgmap v778: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:54 smithi194 ceph-mon[106982]: pgmap v778: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:57.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:56 smithi008 ceph-mon[146084]: pgmap v779: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:57.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:56 smithi194 ceph-mon[106982]: pgmap v779: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:58.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3447174931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e0105bd0 msgr2=0x7fd5e0105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:58.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3447174931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0105fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009910 tx=0x7fd5d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:20:58.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3447174931 shutdown_connections 2024-04-12T06:20:58.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3447174931 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:58.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3447174931 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0105fb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:58.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3447174931 >> 172.21.15.8:0/3447174931 conn(0x7fd5e0075670 msgr2=0x7fd5e0075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:58.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3447174931 shutdown_connections 2024-04-12T06:20:58.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.828+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3447174931 wait complete. 2024-04-12T06:20:58.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.829+0000 7fd5e50ac700 1 Processor -- start 2024-04-12T06:20:58.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.829+0000 7fd5e50ac700 1 -- start start 2024-04-12T06:20:58.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5e50ac700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:58.830 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5e50ac700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5e50ac700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0196260 con 0x7fd5e00ffc20 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5e50ac700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0198f90 con 0x7fd5e0105bd0 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46612/0 (socket says 172.21.15.8:46612) 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 -- 172.21.15.8:0/3913621284 learned_addr learned my addr 172.21.15.8:0/3913621284 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5de59c700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 -- 172.21.15.8:0/3913621284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 msgr2=0x7fd5e0195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 -- 172.21.15.8:0/3913621284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00095f0 con 0x7fd5e00ffc20 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5de59c700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5ded9d700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006010 tx=0x7fd5d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d001d070 con 0x7fd5e00ffc20 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e0199210 con 0x7fd5e00ffc20 2024-04-12T06:20:58.831 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.830+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd5d000f460 con 0x7fd5e00ffc20 2024-04-12T06:20:58.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.831+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5d0021640 con 0x7fd5e00ffc20 2024-04-12T06:20:58.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.831+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e0199700 con 0x7fd5e00ffc20 2024-04-12T06:20:58.832 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.831+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e01080f0 con 0x7fd5e00ffc20 2024-04-12T06:20:58.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.832+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd5d000fa20 con 0x7fd5e00ffc20 2024-04-12T06:20:58.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.833+0000 7fd5d7fff700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:20:58.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.833+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd5d008d800 con 0x7fd5e00ffc20 2024-04-12T06:20:58.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.833+0000 7fd5de59c700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:20:58.833 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.833+0000 7fd5de59c700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005fd0 tx=0x7fd5c8005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:20:58.837 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:58.837+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd5d0057870 con 0x7fd5e00ffc20 2024-04-12T06:20:59.061 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.061+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e0199920 con 0x7fd5cc06cce0 2024-04-12T06:20:59.062 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.062+0000 7fd5d7fff700 1 -- 172.21.15.8:0/3913621284 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fd5e0199920 con 0x7fd5cc06cce0 2024-04-12T06:20:59.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 msgr2=0x7fd5cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:59.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8005fd0 tx=0x7fd5c8005ee0 comp rx=0 tx=0).stop 2024-04-12T06:20:59.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 msgr2=0x7fd5e0195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:20:59.064 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006010 tx=0x7fd5d000be20 comp rx=0 tx=0).stop 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 shutdown_connections 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd5cc06cce0 0x7fd5cc06f1a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd5e00ffc20 0x7fd5e0195690 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 --2- 172.21.15.8:0/3913621284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd5e0105bd0 0x7fd5e0195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.064+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 >> 172.21.15.8:0/3913621284 conn(0x7fd5e0075670 msgr2=0x7fd5e00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.065+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 shutdown_connections 2024-04-12T06:20:59.065 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:20:59.065+0000 7fd5e50ac700 1 -- 172.21.15.8:0/3913621284 wait complete. 2024-04-12T06:20:59.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:58 smithi008 ceph-mon[146084]: pgmap v780: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:58 smithi194 ceph-mon[106982]: pgmap v780: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:20:59.772 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch ps' 2024-04-12T06:21:00.057 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:00.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:20:59 smithi008 ceph-mon[146084]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:00.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:20:59 smithi194 ceph-mon[106982]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:01.127 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:00 smithi008 ceph-mon[146084]: pgmap v781: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:01.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:00 smithi194 ceph-mon[106982]: pgmap v781: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:02.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 -- 172.21.15.8:0/2045381846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 msgr2=0x7f8e1c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:02.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 --2- 172.21.15.8:0/2045381846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c1058a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009910 tx=0x7f8e10009c20 comp rx=0 tx=0).stop 2024-04-12T06:21:02.306 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 -- 172.21.15.8:0/2045381846 shutdown_connections 2024-04-12T06:21:02.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 --2- 172.21.15.8:0/2045381846 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c1058a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 --2- 172.21.15.8:0/2045381846 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 0x7f8e1c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 -- 172.21.15.8:0/2045381846 >> 172.21.15.8:0/2045381846 conn(0x7f8e1c0fc750 msgr2=0x7f8e1c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:02.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 -- 172.21.15.8:0/2045381846 shutdown_connections 2024-04-12T06:21:02.307 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.306+0000 7f8e20f93700 1 -- 172.21.15.8:0/2045381846 wait complete. 2024-04-12T06:21:02.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 Processor -- start 2024-04-12T06:21:02.308 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 -- start start 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 0x7f8e1c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c073b10 con 0x7f8e1c101850 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e20f93700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c073c50 con 0x7f8e1c100ea0 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e19d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e19d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:46638/0 (socket says 172.21.15.8:46638) 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e19d9b700 1 -- 172.21.15.8:0/483333933 learned_addr learned my addr 172.21.15.8:0/483333933 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.307+0000 7f8e1a59c700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 0x7f8e1c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e19d9b700 1 -- 172.21.15.8:0/483333933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 msgr2=0x7f8e1c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e19d9b700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 0x7f8e1c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e19d9b700 1 -- 172.21.15.8:0/483333933 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e100095f0 con 0x7f8e1c101850 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e19d9b700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000bd70 tx=0x7f8e1000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e1001d070 con 0x7f8e1c101850 2024-04-12T06:21:02.309 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e1000f460 con 0x7f8e1c101850 2024-04-12T06:21:02.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c073ed0 con 0x7f8e1c101850 2024-04-12T06:21:02.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e10021640 con 0x7f8e1c101850 2024-04-12T06:21:02.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.308+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c1a2d50 con 0x7f8e1c101850 2024-04-12T06:21:02.310 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.309+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c1092a0 con 0x7f8e1c101850 2024-04-12T06:21:02.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.310+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f8e1000f9e0 con 0x7f8e1c101850 2024-04-12T06:21:02.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.310+0000 7f8e0b7fe700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 0x7f8e0406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:02.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.310+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8e1008d6b0 con 0x7f8e1c101850 2024-04-12T06:21:02.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.310+0000 7f8e1a59c700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 0x7f8e0406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:02.311 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.311+0000 7f8e1a59c700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 0x7f8e0406f1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005950 tx=0x7f8e0c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:02.314 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.315+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f8e100577a0 con 0x7f8e1c101850 2024-04-12T06:21:02.529 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.529+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e1c0746d0 con 0x7f8e0406cce0 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.540+0000 7f8e0b7fe700 1 -- 172.21.15.8:0/483333933 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8e1c0746d0 con 0x7f8e0406cce0 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (22m) 39s ago 34m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (50s) 39s ago 34m 7075k - 18.2.2-1187-gd540ebac bb2c5eac26a4 dd25dcb3a496 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (45s) 40s ago 33m 6874k - 18.2.2-1187-gd540ebac bb2c5eac26a4 e155129f1ace 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (20m) 39s ago 34m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (20m) 40s ago 33m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:21:02.540 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (22m) 39s ago 34m 93.5M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (16m) 39s ago 28m 60.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (16m) 39s ago 28m 74.7M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (16m) 40s ago 28m 58.6M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (16m) 40s ago 28m 70.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (25m) 39s ago 35m 517M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (24m) 40s ago 33m 458M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (21m) 39s ago 36m 73.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:21:02.541 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (20m) 40s ago 33m 50.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (23m) 39s ago 34m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (23m) 40s ago 33m 18.7M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (20m) 39s ago 32m 111M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (19m) 39s ago 31m 121M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (18m) 39s ago 31m 104M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (18m) 40s ago 30m 122M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (17m) 40s ago 30m 119M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (17m) 40s ago 29m 97.0M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:21:02.542 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (23m) 39s ago 33m 82.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:21:02.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 msgr2=0x7f8e0406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 0x7f8e0406f1a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005950 tx=0x7f8e0c0058e0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 msgr2=0x7f8e1c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000bd70 tx=0x7f8e1000be50 comp rx=0 tx=0).stop 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 shutdown_connections 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f8e0406cce0 0x7f8e0406f1a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8e1c100ea0 0x7f8e1c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 --2- 172.21.15.8:0/483333933 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f8e1c101850 0x7f8e1c0735d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:02.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 >> 172.21.15.8:0/483333933 conn(0x7f8e1c0fc750 msgr2=0x7f8e1c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:02.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 shutdown_connections 2024-04-12T06:21:02.545 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:02.542+0000 7f8e20f93700 1 -- 172.21.15.8:0/483333933 wait complete. 2024-04-12T06:21:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:02 smithi008 ceph-mon[146084]: pgmap v782: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:03.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:03.272 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch upgrade status' 2024-04-12T06:21:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:02 smithi194 ceph-mon[106982]: pgmap v782: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:03.516 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:04.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:03 smithi008 ceph-mon[146084]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:04.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:03 smithi194 ceph-mon[106982]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:04.908 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:04 smithi008 ceph-mon[146084]: pgmap v783: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:05.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:04 smithi194 ceph-mon[106982]: pgmap v783: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:05.781 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 -- 172.21.15.8:0/3625675901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680068580 msgr2=0x7fc6800689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 --2- 172.21.15.8:0/3625675901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680068580 0x7fc6800689e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc674009910 tx=0x7fc674009c20 comp rx=0 tx=0).stop 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 -- 172.21.15.8:0/3625675901 shutdown_connections 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 --2- 172.21.15.8:0/3625675901 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680068580 0x7fc6800689e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 --2- 172.21.15.8:0/3625675901 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680106cd0 0x7fc6801070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.779+0000 7fc686ca8700 1 -- 172.21.15.8:0/3625675901 >> 172.21.15.8:0/3625675901 conn(0x7fc680075670 msgr2=0x7fc680075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 -- 172.21.15.8:0/3625675901 shutdown_connections 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 -- 172.21.15.8:0/3625675901 wait complete. 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 Processor -- start 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 -- start start 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680068580 0x7fc68019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019d130 con 0x7fc680106cd0 2024-04-12T06:21:05.782 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.780+0000 7fc686ca8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801965e0 con 0x7fc680068580 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60700/0 (socket says 172.21.15.8:60700) 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 -- 172.21.15.8:0/1151349715 learned_addr learned my addr 172.21.15.8:0/1151349715 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc684a44700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680068580 0x7fc68019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 -- 172.21.15.8:0/1151349715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680068580 msgr2=0x7fc68019c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680068580 0x7fc68019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 -- 172.21.15.8:0/1151349715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740095f0 con 0x7fc680106cd0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67ffff700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc6740092e0 tx=0x7fc67400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67401d070 con 0x7fc680106cd0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc674005e00 con 0x7fc680106cd0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc680196860 con 0x7fc680106cd0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.781+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc67400fe60 con 0x7fc680106cd0 2024-04-12T06:21:05.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.782+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc680196d50 con 0x7fc680106cd0 2024-04-12T06:21:05.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.783+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6801091d0 con 0x7fc680106cd0 2024-04-12T06:21:05.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.784+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc67400f6d0 con 0x7fc680106cd0 2024-04-12T06:21:05.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.784+0000 7fc67dffb700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 0x7fc66806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:05.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.784+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc67408db50 con 0x7fc680106cd0 2024-04-12T06:21:05.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.784+0000 7fc684a44700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 0x7fc66806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:05.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.785+0000 7fc684a44700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 0x7fc66806f1a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc670005950 tx=0x7fc6700058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:05.788 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:05.788+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc67408de60 con 0x7fc680106cd0 2024-04-12T06:21:06.009 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.009+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc680196fc0 con 0x7fc66806cce0 2024-04-12T06:21:06.010 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.010+0000 7fc67dffb700 1 -- 172.21.15.8:0/1151349715 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc680196fc0 con 0x7fc66806cce0 2024-04-12T06:21:06.010 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:21:06.010 INFO:teuthology.orchestra.run.smithi008.stdout: "target_image": null, 2024-04-12T06:21:06.010 INFO:teuthology.orchestra.run.smithi008.stdout: "in_progress": false, 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout: "which": "", 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout: "services_complete": [], 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout: "progress": null, 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout: "message": "", 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout: "is_paused": false 2024-04-12T06:21:06.011 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:21:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 msgr2=0x7fc66806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:06.012 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 0x7fc66806f1a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc670005950 tx=0x7fc6700058e0 comp rx=0 tx=0).stop 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 msgr2=0x7fc68019caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc6740092e0 tx=0x7fc67400be20 comp rx=0 tx=0).stop 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 shutdown_connections 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc66806cce0 0x7fc66806f1a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc680068580 0x7fc68019c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 --2- 172.21.15.8:0/1151349715 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc680106cd0 0x7fc68019caa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.012+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 >> 172.21.15.8:0/1151349715 conn(0x7fc680075670 msgr2=0x7fc6800fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.013+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 shutdown_connections 2024-04-12T06:21:06.013 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:06.013+0000 7fc686ca8700 1 -- 172.21.15.8:0/1151349715 wait complete. 2024-04-12T06:21:06.830 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph health detail' 2024-04-12T06:21:07.074 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:07.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:06 smithi008 ceph-mon[146084]: pgmap v784: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:07.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:06 smithi008 ceph-mon[146084]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:07.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:06 smithi194 ceph-mon[106982]: pgmap v784: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:07.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:06 smithi194 ceph-mon[106982]: from='client.35222 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 -- 172.21.15.8:0/2233002914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4101850 msgr2=0x7fedb41058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 --2- 172.21.15.8:0/2233002914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4101850 0x7fedb41058a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feda8009910 tx=0x7feda8009c20 comp rx=0 tx=0).stop 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 -- 172.21.15.8:0/2233002914 shutdown_connections 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 --2- 172.21.15.8:0/2233002914 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4101850 0x7fedb41058a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 --2- 172.21.15.8:0/2233002914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4100ea0 0x7fedb4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 -- 172.21.15.8:0/2233002914 >> 172.21.15.8:0/2233002914 conn(0x7fedb40fc750 msgr2=0x7fedb40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 -- 172.21.15.8:0/2233002914 shutdown_connections 2024-04-12T06:21:09.077 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.036+0000 7fedb8d53700 1 -- 172.21.15.8:0/2233002914 wait complete. 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 Processor -- start 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 -- start start 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 0x7fedb419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb419d110 con 0x7fedb4100ea0 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.037+0000 7fedb8d53700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41966a0 con 0x7fedb4101850 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60716/0 (socket says 172.21.15.8:60716) 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 -- 172.21.15.8:0/357260463 learned_addr learned my addr 172.21.15.8:0/357260463 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:21:09.078 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb1d9b700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 0x7fedb419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 -- 172.21.15.8:0/357260463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 msgr2=0x7fedb419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 0x7fedb419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 -- 172.21.15.8:0/357260463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda80095f0 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb1d9b700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 0x7fedb419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb259c700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feda400d8d0 tx=0x7feda400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda4009940 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feda4010460 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb4196980 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feda400f5f0 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.038+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb4196ed0 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.039+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb41092a0 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.040+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7feda40105d0 con 0x7fedb4100ea0 2024-04-12T06:21:09.079 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.040+0000 7feda37fe700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 0x7fed9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.041+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feda408c4c0 con 0x7fedb4100ea0 2024-04-12T06:21:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.041+0000 7fedb1d9b700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 0x7fed9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.041+0000 7fedb1d9b700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 0x7fed9c06f1a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feda80059d0 tx=0x7feda8005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:09.080 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.045+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7feda405a040 con 0x7fedb4100ea0 2024-04-12T06:21:09.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:08 smithi008 ceph-mon[146084]: pgmap v785: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:09.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:08 smithi194 ceph-mon[106982]: pgmap v785: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.331+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fedb404f2e0 con 0x7fedb4100ea0 2024-04-12T06:21:09.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.331+0000 7feda37fe700 1 -- 172.21.15.8:0/357260463 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fedb404f2e0 con 0x7fedb4100ea0 2024-04-12T06:21:09.331 INFO:teuthology.orchestra.run.smithi008.stdout:HEALTH_OK 2024-04-12T06:21:09.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.333+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 msgr2=0x7fed9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.333+0000 7fedb8d53700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 0x7fed9c06f1a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7feda80059d0 tx=0x7feda8005960 comp rx=0 tx=0).stop 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.333+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 msgr2=0x7fedb419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.333+0000 7fedb8d53700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feda400d8d0 tx=0x7feda400dc90 comp rx=0 tx=0).stop 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 shutdown_connections 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fed9c06cce0 0x7fed9c06f1a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fedb4100ea0 0x7fedb419c5b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 --2- 172.21.15.8:0/357260463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb4101850 0x7fedb419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 >> 172.21.15.8:0/357260463 conn(0x7fedb40fc750 msgr2=0x7fedb40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 shutdown_connections 2024-04-12T06:21:09.334 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:09.334+0000 7fedb8d53700 1 -- 172.21.15.8:0/357260463 wait complete. 2024-04-12T06:21:10.086 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:09 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/357260463' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:21:10.108 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions' 2024-04-12T06:21:10.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:09 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/357260463' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-12T06:21:10.346 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:11.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:10 smithi008 ceph-mon[146084]: pgmap v786: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:11.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:10 smithi194 ceph-mon[106982]: pgmap v786: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 -- 172.21.15.8:0/3555748822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10400ffb40 msgr2=0x7f10400fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 --2- 172.21.15.8:0/3555748822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10400ffb40 0x7f10400fffc0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1034009910 tx=0x7f1034009c20 comp rx=0 tx=0).stop 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 -- 172.21.15.8:0/3555748822 shutdown_connections 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 --2- 172.21.15.8:0/3555748822 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f10400ffb40 0x7f10400fffc0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 --2- 172.21.15.8:0/3555748822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1040105bf0 0x7f1040105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.233+0000 7f1048542700 1 -- 172.21.15.8:0/3555748822 >> 172.21.15.8:0/3555748822 conn(0x7f10400fb610 msgr2=0x7f10400fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:12.236 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 -- 172.21.15.8:0/3555748822 shutdown_connections 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 -- 172.21.15.8:0/3555748822 wait complete. 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 Processor -- start 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 -- start start 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 0x7f104019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104019d0c0 con 0x7f1040105bf0 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1048542700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040196570 con 0x7f10400ffb40 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1045add700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.234+0000 7f1045add700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:60732/0 (socket says 172.21.15.8:60732) 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1045add700 1 -- 172.21.15.8:0/3679595856 learned_addr learned my addr 172.21.15.8:0/3679595856 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f10462de700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 0x7f104019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1045add700 1 -- 172.21.15.8:0/3679595856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 msgr2=0x7f104019c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1045add700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 0x7f104019c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1045add700 1 -- 172.21.15.8:0/3679595856 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340095f0 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f10462de700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 0x7f104019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1045add700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1034006010 tx=0x7f103400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103401d070 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10401967f0 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f103400f460 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1034021620 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.235+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1040196ce0 con 0x7f1040105bf0 2024-04-12T06:21:12.238 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.236+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104004f2e0 con 0x7f1040105bf0 2024-04-12T06:21:12.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.236+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f103400fa40 con 0x7f1040105bf0 2024-04-12T06:21:12.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.237+0000 7f10337fe700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 0x7f102c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:21:12.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.237+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f103408d780 con 0x7f1040105bf0 2024-04-12T06:21:12.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.237+0000 7f10462de700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 0x7f102c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:21:12.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.237+0000 7f10462de700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 0x7f102c06f1a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c0058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:21:12.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.241+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f103405b280 con 0x7f1040105bf0 2024-04-12T06:21:12.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.527+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1040197a80 con 0x7f1040105bf0 2024-04-12T06:21:12.527 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.527+0000 7f10337fe700 1 -- 172.21.15.8:0/3679595856 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f103408be60 con 0x7f1040105bf0 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:21:12.528 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:21:12.529 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 msgr2=0x7f102c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 0x7f102c06f1a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f103c005950 tx=0x7f103c0058e0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 msgr2=0x7f104019ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1034006010 tx=0x7f103400be20 comp rx=0 tx=0).stop 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 shutdown_connections 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f102c06cce0 0x7f102c06f1a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f10400ffb40 0x7f104019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 --2- 172.21.15.8:0/3679595856 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1040105bf0 0x7f104019ca30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.529+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 >> 172.21.15.8:0/3679595856 conn(0x7f10400fb610 msgr2=0x7f1040068f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:21:12.530 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.530+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 shutdown_connections 2024-04-12T06:21:12.531 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:21:12.530+0000 7f1048542700 1 -- 172.21.15.8:0/3679595856 wait complete. 2024-04-12T06:21:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:12 smithi008 ceph-mon[146084]: pgmap v787: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:12 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3679595856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:21:13.263 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-12T06:21:13.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:12 smithi194 ceph-mon[106982]: pgmap v787: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:13.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:12 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3679595856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:21:13.502 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:15.141 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:14 smithi008 ceph-mon[146084]: pgmap v788: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:15.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:14 smithi194 ceph-mon[106982]: pgmap v788: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:15.439 INFO:teuthology.orchestra.run.smithi008.stdout:wait for servicemap items w/ changing names to refresh 2024-04-12T06:21:16.082 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'sleep 60' 2024-04-12T06:21:16.324 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:21:17.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:16 smithi008 ceph-mon[146084]: pgmap v789: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:17.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:16 smithi194 ceph-mon[106982]: pgmap v789: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:17.997 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:18.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:18 smithi008 ceph-mon[146084]: pgmap v790: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:19.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:18 smithi194 ceph-mon[106982]: pgmap v790: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:21.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:20 smithi008 ceph-mon[146084]: pgmap v791: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:21.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:20 smithi194 ceph-mon[106982]: pgmap v791: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:22 smithi008 ceph-mon[146084]: pgmap v792: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:23.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:22 smithi194 ceph-mon[106982]: pgmap v792: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:25.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:24 smithi008 ceph-mon[146084]: pgmap v793: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:25.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:24 smithi194 ceph-mon[106982]: pgmap v793: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:27.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:26 smithi008 ceph-mon[146084]: pgmap v794: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:27.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:26 smithi194 ceph-mon[106982]: pgmap v794: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:29.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:28 smithi008 ceph-mon[146084]: pgmap v795: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:29.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:28 smithi194 ceph-mon[106982]: pgmap v795: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:30.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:21:30.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:21:30.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:21:30.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:29 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:21:30.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:21:30.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:21:30.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:21:30.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:29 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:21:31.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:30 smithi008 ceph-mon[146084]: pgmap v796: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:31.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:30 smithi194 ceph-mon[106982]: pgmap v796: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:21:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:32 smithi008 ceph-mon[146084]: pgmap v797: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:33.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:32 smithi194 ceph-mon[106982]: pgmap v797: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:34 smithi194 ceph-mon[106982]: pgmap v798: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:34 smithi008 ceph-mon[146084]: pgmap v798: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:37.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:36 smithi194 ceph-mon[106982]: pgmap v799: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:37.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:36 smithi008 ceph-mon[146084]: pgmap v799: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:39.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:38 smithi194 ceph-mon[106982]: pgmap v800: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:38 smithi008 ceph-mon[146084]: pgmap v800: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:40 smithi194 ceph-mon[106982]: pgmap v801: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:21:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:40 smithi008 ceph-mon[146084]: pgmap v801: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:21:43.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:42 smithi194 ceph-mon[106982]: pgmap v802: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:21:43.327 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:42 smithi008 ceph-mon[146084]: pgmap v802: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:21:45.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:44 smithi194 ceph-mon[106982]: pgmap v803: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:21:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:44 smithi008 ceph-mon[146084]: pgmap v803: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-12T06:21:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:46 smithi194 ceph-mon[106982]: pgmap v804: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:21:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:46 smithi008 ceph-mon[146084]: pgmap v804: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-12T06:21:48.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:48.383 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:21:49.074 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:48 smithi194 ceph-mon[106982]: pgmap v805: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-12T06:21:49.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:48 smithi008 ceph-mon[146084]: pgmap v805: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-12T06:21:51.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:50 smithi194 ceph-mon[106982]: pgmap v806: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:21:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:50 smithi008 ceph-mon[146084]: pgmap v806: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-12T06:21:53.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:52 smithi194 ceph-mon[106982]: pgmap v807: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:52 smithi008 ceph-mon[146084]: pgmap v807: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:54 smithi194 ceph-mon[106982]: pgmap v808: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:55.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:54 smithi008 ceph-mon[146084]: pgmap v808: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:21:57.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:56 smithi194 ceph-mon[106982]: pgmap v809: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:57.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:56 smithi008 ceph-mon[146084]: pgmap v809: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:21:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:21:58 smithi194 ceph-mon[106982]: pgmap v810: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:21:59.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:21:58 smithi008 ceph-mon[146084]: pgmap v810: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:01.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:00 smithi194 ceph-mon[106982]: pgmap v811: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:01.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:00 smithi008 ceph-mon[146084]: pgmap v811: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:03.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:02 smithi194 ceph-mon[106982]: pgmap v812: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:03.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:02 smithi008 ceph-mon[146084]: pgmap v812: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:05.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:04 smithi194 ceph-mon[106982]: pgmap v813: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:05.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:04 smithi008 ceph-mon[146084]: pgmap v813: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:07.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:06 smithi194 ceph-mon[106982]: pgmap v814: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:07.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:06 smithi008 ceph-mon[146084]: pgmap v814: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:09.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:08 smithi194 ceph-mon[106982]: pgmap v815: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:09.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:08 smithi008 ceph-mon[146084]: pgmap v815: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:11.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:10 smithi194 ceph-mon[106982]: pgmap v816: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:11.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:10 smithi008 ceph-mon[146084]: pgmap v816: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:13.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:12 smithi194 ceph-mon[106982]: pgmap v817: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:13.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:12 smithi008 ceph-mon[146084]: pgmap v817: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:15.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:14 smithi194 ceph-mon[106982]: pgmap v818: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:15.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:14 smithi008 ceph-mon[146084]: pgmap v818: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:17.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:16 smithi194 ceph-mon[106982]: pgmap v819: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:17.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:16 smithi008 ceph-mon[146084]: pgmap v819: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:18.245 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:18.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:18.901 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph orch ps' 2024-04-12T06:22:19.145 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:22:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:18 smithi008 ceph-mon[146084]: pgmap v820: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:19.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:18 smithi194 ceph-mon[106982]: pgmap v820: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:21.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:20 smithi008 ceph-mon[146084]: pgmap v821: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:21.180 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 -- 172.21.15.8:0/1707699221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 msgr2=0x7f786c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1707699221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c1021c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7860009910 tx=0x7f7860009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 -- 172.21.15.8:0/1707699221 shutdown_connections 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1707699221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1707699221 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c1021c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 -- 172.21.15.8:0/1707699221 >> 172.21.15.8:0/1707699221 conn(0x7f786c0fb610 msgr2=0x7f786c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 -- 172.21.15.8:0/1707699221 shutdown_connections 2024-04-12T06:22:21.181 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.180+0000 7f7872ec8700 1 -- 172.21.15.8:0/1707699221 wait complete. 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 Processor -- start 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 -- start start 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c19d100 con 0x7f786c101de0 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7872ec8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c196640 con 0x7f786c102700 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7870c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7870c64700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:58318/0 (socket says 172.21.15.8:58318) 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f7870c64700 1 -- 172.21.15.8:0/1159338445 learned_addr learned my addr 172.21.15.8:0/1159338445 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.181+0000 7f786bfff700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7870c64700 1 -- 172.21.15.8:0/1159338445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 msgr2=0x7f786c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:21.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7870c64700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7870c64700 1 -- 172.21.15.8:0/1159338445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78600095f0 con 0x7f786c101de0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f786bfff700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7870c64700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7860006010 tx=0x7f786000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786001d070 con 0x7f786c101de0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f786000f460 con 0x7f786c101de0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c1968c0 con 0x7f786c101de0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7860021640 con 0x7f786c101de0 2024-04-12T06:22:21.184 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.182+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c196db0 con 0x7f786c101de0 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.183+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c1082f0 con 0x7f786c101de0 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.184+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f786000fa20 con 0x7f786c101de0 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.184+0000 7f7869ffb700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 0x7f785c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.184+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f786008d7a0 con 0x7f786c101de0 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.184+0000 7f786bfff700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 0x7f785c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:21.185 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.185+0000 7f786bfff700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 0x7f785c06f1a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f7858005e30 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:21.189 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.189+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f786005b320 con 0x7f786c101de0 2024-04-12T06:22:21.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:20 smithi194 ceph-mon[106982]: pgmap v821: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:21.401 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.401+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 --> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f786c197c30 con 0x7f785c06cce0 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.412+0000 7f7869ffb700 1 -- 172.21.15.8:0/1159338445 <== mgr.14660 v2:172.21.15.8:6800/2333971379 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f786c197c30 con 0x7f785c06cce0 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:alertmanager.smithi008 smithi008 *:9093,9094 running (24m) 118s ago 35m 28.0M - 0.25.0 c8568f914cd2 2afffd0480bb 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi008 smithi008 running (2m) 118s ago 35m 7075k - 18.2.2-1187-gd540ebac bb2c5eac26a4 dd25dcb3a496 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:ceph-exporter.smithi194 smithi194 running (2m) 119s ago 34m 6874k - 18.2.2-1187-gd540ebac bb2c5eac26a4 e155129f1ace 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi008 smithi008 running (21m) 118s ago 35m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 983499120c15 2024-04-12T06:22:21.412 INFO:teuthology.orchestra.run.smithi008.stdout:crash.smithi194 smithi194 running (21m) 119s ago 34m 6651k - 18.2.2-1187-gd540ebac bb2c5eac26a4 2ca71f633ecc 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:grafana.smithi008 smithi008 *:3000 running (23m) 118s ago 35m 93.5M - 9.4.7 954c08fa6188 48079030765e 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.lmvepz smithi008 running (17m) 118s ago 29m 60.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 8a9e864d7f81 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi008.rhdhsr smithi008 running (18m) 118s ago 29m 74.7M - 18.2.2-1187-gd540ebac bb2c5eac26a4 2803992e6e19 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.gbfdvc smithi194 running (17m) 119s ago 29m 58.6M - 18.2.2-1187-gd540ebac bb2c5eac26a4 0e00cc9ff480 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mds.cephfs.smithi194.uzfpyu smithi194 running (17m) 119s ago 29m 70.3M - 18.2.2-1187-gd540ebac bb2c5eac26a4 b9c383b4cdfd 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi008.ixmnlp smithi008 *:8443,9283,8765 running (26m) 118s ago 37m 517M - 18.2.2-1187-gd540ebac bb2c5eac26a4 6c9549be63f4 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mgr.smithi194.xbiyaj smithi194 *:8443,9283,8765 running (25m) 119s ago 34m 458M - 18.2.2-1187-gd540ebac bb2c5eac26a4 09fb7f45695d 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi008 smithi008 running (22m) 118s ago 37m 73.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 69cc2ae6e915 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:mon.smithi194 smithi194 running (22m) 119s ago 34m 50.0M 2048M 18.2.2-1187-gd540ebac bb2c5eac26a4 e7818c807a51 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi008 smithi008 *:9100 running (25m) 118s ago 35m 17.2M - 1.5.0 0da6a335fe13 3e0d6f0ba479 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:node-exporter.smithi194 smithi194 *:9100 running (24m) 119s ago 34m 18.7M - 1.5.0 0da6a335fe13 8b5e4f93a26c 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:osd.0 smithi008 running (21m) 118s ago 33m 111M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 eb3bf9318105 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:osd.1 smithi008 running (20m) 118s ago 33m 121M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 da7123940284 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:osd.2 smithi008 running (20m) 118s ago 32m 104M 1228M 18.2.2-1187-gd540ebac bb2c5eac26a4 566586cc138f 2024-04-12T06:22:21.413 INFO:teuthology.orchestra.run.smithi008.stdout:osd.3 smithi194 running (19m) 119s ago 32m 122M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 95a8d9917f81 2024-04-12T06:22:21.414 INFO:teuthology.orchestra.run.smithi008.stdout:osd.4 smithi194 running (19m) 119s ago 31m 119M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 8f6f5d666aeb 2024-04-12T06:22:21.414 INFO:teuthology.orchestra.run.smithi008.stdout:osd.5 smithi194 running (18m) 119s ago 31m 97.0M 2251M 18.2.2-1187-gd540ebac bb2c5eac26a4 860f425d081e 2024-04-12T06:22:21.414 INFO:teuthology.orchestra.run.smithi008.stdout:prometheus.smithi008 smithi008 *:9095 running (24m) 118s ago 35m 82.4M - 2.43.0 a07b618ecd1d 6aa289284771 2024-04-12T06:22:21.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 msgr2=0x7f785c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:21.414 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 0x7f785c06f1a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7858005fd0 tx=0x7f7858005e30 comp rx=0 tx=0).stop 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 msgr2=0x7f786c19c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7860006010 tx=0x7f786000be20 comp rx=0 tx=0).stop 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 shutdown_connections 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f785c06cce0 0x7f785c06f1a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f786c101de0 0x7f786c19c5c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 --2- 172.21.15.8:0/1159338445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f786c102700 0x7f786c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 >> 172.21.15.8:0/1159338445 conn(0x7f786c0fb610 msgr2=0x7f786c105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 shutdown_connections 2024-04-12T06:22:21.415 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:21.414+0000 7f7872ec8700 1 -- 172.21.15.8:0/1159338445 wait complete. 2024-04-12T06:22:22.096 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions' 2024-04-12T06:22:22.336 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:22:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:22 smithi008 ceph-mon[146084]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:22:23.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:22 smithi008 ceph-mon[146084]: pgmap v822: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:23.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:22 smithi194 ceph-mon[106982]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-12T06:22:23.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:22 smithi194 ceph-mon[106982]: pgmap v822: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.236+0000 7f3823e35700 1 -- 172.21.15.8:0/2238559975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c101850 msgr2=0x7f381c1058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.236+0000 7f3823e35700 1 --2- 172.21.15.8:0/2238559975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c101850 0x7f381c1058a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3810009910 tx=0x7f3810009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.236+0000 7f3823e35700 1 -- 172.21.15.8:0/2238559975 shutdown_connections 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.236+0000 7f3823e35700 1 --2- 172.21.15.8:0/2238559975 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c101850 0x7f381c1058a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 --2- 172.21.15.8:0/2238559975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c100ea0 0x7f381c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 -- 172.21.15.8:0/2238559975 >> 172.21.15.8:0/2238559975 conn(0x7f381c0fc750 msgr2=0x7f381c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 -- 172.21.15.8:0/2238559975 shutdown_connections 2024-04-12T06:22:24.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 -- 172.21.15.8:0/2238559975 wait complete. 2024-04-12T06:22:24.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 Processor -- start 2024-04-12T06:22:24.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.237+0000 7f3823e35700 1 -- start start 2024-04-12T06:22:24.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3823e35700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:24.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3823e35700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c101850 0x7f381c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:24.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3823e35700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c19d1f0 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3823e35700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c1966a0 con 0x7f381c101850 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36372/0 (socket says 172.21.15.8:36372) 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 -- 172.21.15.8:0/1399235668 learned_addr learned my addr 172.21.15.8:0/1399235668 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f38213d0700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c101850 0x7f381c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 -- 172.21.15.8:0/1399235668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c101850 msgr2=0x7f381c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c101850 0x7f381c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 -- 172.21.15.8:0/1399235668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100095f0 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.238+0000 7f3821bd1700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f381800e920 tx=0x7f381800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.239+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f381800c9d0 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.239+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3818010430 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.239+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c196980 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.239+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3818018650 con 0x7f381c100ea0 2024-04-12T06:22:24.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.239+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c196ea0 con 0x7f381c100ea0 2024-04-12T06:22:24.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.240+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c1092a0 con 0x7f381c100ea0 2024-04-12T06:22:24.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.240+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f38180105a0 con 0x7f381c100ea0 2024-04-12T06:22:24.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.240+0000 7f380effd700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 0x7f380806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:24.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.241+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3818014070 con 0x7f381c100ea0 2024-04-12T06:22:24.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.241+0000 7f38213d0700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 0x7f380806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:24.242 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.241+0000 7f38213d0700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 0x7f380806f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fd0 tx=0x7f3810005940 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:24.245 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.245+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3818052ab0 con 0x7f381c100ea0 2024-04-12T06:22:24.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.539+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f381c04f2e0 con 0x7f381c100ea0 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout:{ 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "mon": { 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "mgr": { 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 2 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "osd": { 2024-04-12T06:22:24.540 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 6 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: "mds": { 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 4 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: }, 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: "overall": { 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)": 14 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout: } 2024-04-12T06:22:24.541 INFO:teuthology.orchestra.run.smithi008.stdout:} 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.539+0000 7f380effd700 1 -- 172.21.15.8:0/1399235668 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f381805a360 con 0x7f381c100ea0 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 msgr2=0x7f380806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 0x7f380806f1a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3810009fd0 tx=0x7f3810005940 comp rx=0 tx=0).stop 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 msgr2=0x7f381c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f381800e920 tx=0x7f381800ec30 comp rx=0 tx=0).stop 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 shutdown_connections 2024-04-12T06:22:24.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f380806cce0 0x7f380806f1a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f381c100ea0 0x7f381c19c620 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 --2- 172.21.15.8:0/1399235668 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f381c101850 0x7f381c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 >> 172.21.15.8:0/1399235668 conn(0x7f381c0fc750 msgr2=0x7f381c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 shutdown_connections 2024-04-12T06:22:24.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:24.541+0000 7f3823e35700 1 -- 172.21.15.8:0/1399235668 wait complete. 2024-04-12T06:22:25.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:24 smithi008 ceph-mon[146084]: pgmap v823: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:25.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:24 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1399235668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:25.255 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-12T06:22:25.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:24 smithi194 ceph-mon[106982]: pgmap v823: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:25.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:24 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1399235668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:25.501 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:22:27.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:26 smithi008 ceph-mon[146084]: pgmap v824: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:27.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:26 smithi194 ceph-mon[106982]: pgmap v824: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/4207889162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 msgr2=0x7fa5c8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/4207889162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c8100080 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009910 tx=0x7fa5b0009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/4207889162 shutdown_connections 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/4207889162 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c8100080 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/4207889162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c8105fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/4207889162 >> 172.21.15.8:0/4207889162 conn(0x7fa5c8075670 msgr2=0x7fa5c8075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/4207889162 shutdown_connections 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.510+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/4207889162 wait complete. 2024-04-12T06:22:27.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 Processor -- start 2024-04-12T06:22:27.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 -- start start 2024-04-12T06:22:27.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:27.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:27.513 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c819d0c0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.511+0000 7fa5cd0ef700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8196600 con 0x7fa5c8105bd0 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36378/0 (socket says 172.21.15.8:36378) 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 -- 172.21.15.8:0/95088100 learned_addr learned my addr 172.21.15.8:0/95088100 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c659c700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 -- 172.21.15.8:0/95088100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 msgr2=0x7fa5c819cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c819cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 -- 172.21.15.8:0/95088100 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b00095f0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c659c700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5c6d9d700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c930 tx=0x7fa5b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b8007ad0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa5b8007c30 con 0x7fa5c80ffc20 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c81968e0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.514 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b8018630 con 0x7fa5c80ffc20 2024-04-12T06:22:27.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.512+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c8196e30 con 0x7fa5c80ffc20 2024-04-12T06:22:27.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.513+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c804f2e0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.514+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa5b8018790 con 0x7fa5c80ffc20 2024-04-12T06:22:27.515 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.514+0000 7fa5bffff700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 0x7fa5b406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:27.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.514+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5b808c750 con 0x7fa5c80ffc20 2024-04-12T06:22:27.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.514+0000 7fa5c659c700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 0x7fa5b406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:27.516 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.515+0000 7fa5c659c700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 0x7fa5b406f1a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8075de0 tx=0x7fa5b001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:27.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.519+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa5b805a410 con 0x7fa5c80ffc20 2024-04-12T06:22:27.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.806+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa5c81970e0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.806 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.806+0000 7fa5bffff700 1 -- 172.21.15.8:0/95088100 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fa5b8059fa0 con 0x7fa5c80ffc20 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 msgr2=0x7fa5b406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 0x7fa5b406f1a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8075de0 tx=0x7fa5b001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 msgr2=0x7fa5c819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c930 tx=0x7fa5b800ccf0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 shutdown_connections 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa5b406cce0 0x7fa5b406f1a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa5c80ffc20 0x7fa5c819c580 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 --2- 172.21.15.8:0/95088100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5c8105bd0 0x7fa5c819cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:27.809 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 >> 172.21.15.8:0/95088100 conn(0x7fa5c8075670 msgr2=0x7fa5c8068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:27.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 shutdown_connections 2024-04-12T06:22:27.810 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:27.809+0000 7fa5cd0ef700 1 -- 172.21.15.8:0/95088100 wait complete. 2024-04-12T06:22:27.820 INFO:teuthology.orchestra.run.smithi008.stdout:true 2024-04-12T06:22:28.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:27 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/95088100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:28.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:28 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/95088100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:28.609 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -e sha1=d540ebaca6b131a1dd560e7f69e024b133bbaa42 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-12T06:22:28.853 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:22:29.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:29 smithi194 ceph-mon[106982]: pgmap v825: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:29.395 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:29 smithi008 ceph-mon[146084]: pgmap v825: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:30.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:30 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:22:30.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:30 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:22:30.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 -- 172.21.15.8:0/3931817606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 msgr2=0x7f9d580689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:30.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 --2- 172.21.15.8:0/3931817606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580689e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009910 tx=0x7f9d48009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:30.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 -- 172.21.15.8:0/3931817606 shutdown_connections 2024-04-12T06:22:30.838 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 --2- 172.21.15.8:0/3931817606 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580689e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 --2- 172.21.15.8:0/3931817606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 0x7f9d581070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 -- 172.21.15.8:0/3931817606 >> 172.21.15.8:0/3931817606 conn(0x7f9d58075670 msgr2=0x7f9d58075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 -- 172.21.15.8:0/3931817606 shutdown_connections 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.823+0000 7f9d5d853700 1 -- 172.21.15.8:0/3931817606 wait complete. 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 Processor -- start 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 -- start start 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 0x7f9d58100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58100770 con 0x7f9d58068580 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d5d853700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581008b0 con 0x7f9d58106cd0 2024-04-12T06:22:30.839 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.824+0000 7f9d56ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:36394/0 (socket says 172.21.15.8:36394) 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 -- 172.21.15.8:0/1773993019 learned_addr learned my addr 172.21.15.8:0/1773993019 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d567fc700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 0x7f9d58100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 -- 172.21.15.8:0/1773993019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 msgr2=0x7f9d58100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 0x7f9d58100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 -- 172.21.15.8:0/1773993019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480095f0 con 0x7f9d58068580 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d56ffd700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00c8a0 tx=0x7f9d4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c0078e0 con 0x7f9d58068580 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9d4c00f450 con 0x7f9d58068580 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c00e5e0 con 0x7f9d58068580 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d581a2940 con 0x7f9d58068580 2024-04-12T06:22:30.840 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.825+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d581a2d50 con 0x7f9d58068580 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.826+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d581091d0 con 0x7f9d58068580 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.827+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9d4c00f5c0 con 0x7f9d58068580 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.827+0000 7f9d5c851700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 0x7f9d4006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.827+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9d4c08c190 con 0x7f9d58068580 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.827+0000 7f9d567fc700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 0x7f9d4006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.828+0000 7f9d567fc700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 0x7f9d4006f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9d581014e0 tx=0x7f9d4801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:30.841 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:30.832+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9d4c056200 con 0x7f9d58068580 2024-04-12T06:22:31.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.119+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d58108bb0 con 0x7f9d58068580 2024-04-12T06:22:31.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.119+0000 7f9d5c851700 1 -- 172.21.15.8:0/1773993019 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9d4c059820 con 0x7f9d58068580 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 msgr2=0x7f9d4006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 0x7f9d4006f1a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9d581014e0 tx=0x7f9d4801a040 comp rx=0 tx=0).stop 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 msgr2=0x7f9d580ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00c8a0 tx=0x7f9d4c00cbb0 comp rx=0 tx=0).stop 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 shutdown_connections 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9d4006cce0 0x7f9d4006f1a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:31.122 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9d58068580 0x7f9d580ffcf0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:31.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 --2- 172.21.15.8:0/1773993019 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9d58106cd0 0x7f9d58100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:31.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 >> 172.21.15.8:0/1773993019 conn(0x7f9d58075670 msgr2=0x7f9d580fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:31.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 shutdown_connections 2024-04-12T06:22:31.123 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:31.122+0000 7f9d5d853700 1 -- 172.21.15.8:0/1773993019 wait complete. 2024-04-12T06:22:31.132 INFO:teuthology.orchestra.run.smithi008.stdout: "ceph version 18.2.2-1187-gd540ebac (d540ebaca6b131a1dd560e7f69e024b133bbaa42) reef (stable)" 2024-04-12T06:22:31.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:31 smithi194 ceph-mon[106982]: pgmap v826: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:31.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:22:31.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:22:31.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:22:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:31 smithi008 ceph-mon[146084]: pgmap v826: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:22:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:22:31.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:22:31.800 DEBUG:teuthology.parallel:result is None 2024-04-12T06:22:31.801 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-12T06:22:31.819 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T06:22:31.819 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-12T06:22:32.060 INFO:teuthology.orchestra.run.smithi008.stderr:Inferring config /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/mon.smithi008/config 2024-04-12T06:22:32.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:32 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1773993019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:32.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:32 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1773993019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-12T06:22:33.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:33 smithi194 ceph-mon[106982]: pgmap v827: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:33.350 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:33 smithi008 ceph-mon[146084]: pgmap v827: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:33.350 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 -- 172.21.15.8:0/4067185416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 msgr2=0x7f0640105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:33.941 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 --2- 172.21.15.8:0/4067185416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640105fb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0630009910 tx=0x7f0630009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 -- 172.21.15.8:0/4067185416 shutdown_connections 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 --2- 172.21.15.8:0/4067185416 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 0x7f0640100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 --2- 172.21.15.8:0/4067185416 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640105fb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 -- 172.21.15.8:0/4067185416 >> 172.21.15.8:0/4067185416 conn(0x7f0640075670 msgr2=0x7f0640075a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 -- 172.21.15.8:0/4067185416 shutdown_connections 2024-04-12T06:22:33.942 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.941+0000 7f0644e3f700 1 -- 172.21.15.8:0/4067185416 wait complete. 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 Processor -- start 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 -- start start 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 0x7f06401975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06401980b0 con 0x7f0640105bd0 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.942+0000 7f0644e3f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0640198220 con 0x7f06400ffc20 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55618/0 (socket says 172.21.15.8:55618) 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 -- 172.21.15.8:0/3567397612 learned_addr learned my addr 172.21.15.8:0/3567397612 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063e59c700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 0x7f06401975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 -- 172.21.15.8:0/3567397612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 msgr2=0x7f06401975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:33.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 0x7f06401975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 -- 172.21.15.8:0/3567397612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300095f0 con 0x7f0640105bd0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f063dd9b700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f063400c930 tx=0x7f063400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0634007ad0 con 0x7f0640105bd0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06401a0650 con 0x7f0640105bd0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0634007c30 con 0x7f0640105bd0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0634018630 con 0x7f0640105bd0 2024-04-12T06:22:33.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.943+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06401a0ac0 con 0x7f0640105bd0 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.944+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06401080d0 con 0x7f0640105bd0 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.945+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f063400f450 con 0x7f0640105bd0 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.945+0000 7f062f7fe700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 0x7f062806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.945+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f063408c750 con 0x7f0640105bd0 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.945+0000 7f063e59c700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 0x7f062806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:33.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.946+0000 7f063e59c700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 0x7f062806f1a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0630009fd0 tx=0x7f063001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:33.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:33.950+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f06340567c0 con 0x7f0640105bd0 2024-04-12T06:22:34.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.197+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0640107ab0 con 0x7f0640105bd0 2024-04-12T06:22:34.197 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.197+0000 7f062f7fe700 1 -- 172.21.15.8:0/3567397612 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 45 v45) v1 ==== 76+0+1870 (secure 0 0 0) 0x7f0634059de0 con 0x7f0640105bd0 2024-04-12T06:22:34.197 INFO:teuthology.orchestra.run.smithi008.stdout:e45 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:legacy client fscid: 1 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:Filesystem 'cephfs' (1) 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:fs_name cephfs 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:epoch 45 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:created 2024-04-12T05:52:43.616162+0000 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:modified 2024-04-12T06:20:28.780381+0000 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:tableserver 0 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:root 0 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:session_timeout 60 2024-04-12T06:22:34.198 INFO:teuthology.orchestra.run.smithi008.stdout:session_autoclose 300 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:max_file_size 1099511627776 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:max_xattr_size 65536 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:required_client_features {} 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure 0 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:last_failure_osd_epoch 79 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:max_mds 2 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:in 0,1 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:up {0=34330,1=34362} 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:failed 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:damaged 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:stopped 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:data_pools [3] 2024-04-12T06:22:34.199 INFO:teuthology.orchestra.run.smithi008.stdout:metadata_pool 2 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:inline_data disabled 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:balancer 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:bal_rank_mask -1 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:standby_count_wanted 1 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.rhdhsr{0:34330} state up:active seq 245 join_fscid=1 addr [v2:172.21.15.8:6826/1645196626,v1:172.21.15.8:6827/1645196626] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.uzfpyu{0:44331} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.194:6824/996613107,v1:172.21.15.194:6825/996613107] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi008.lmvepz{1:34362} state up:active seq 239 join_fscid=1 addr [v2:172.21.15.8:6828/2252467909,v1:172.21.15.8:6829/2252467909] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout:[mds.cephfs.smithi194.gbfdvc{1:44361} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.194:6826/3564821977,v1:172.21.15.194:6827/3564821977] compat {c=[1],r=[1],i=[7ff]}] 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:34.200 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 msgr2=0x7f062806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 0x7f062806f1a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0630009fd0 tx=0x7f063001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 msgr2=0x7f0640197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f063400c930 tx=0x7f063400ccf0 comp rx=0 tx=0).stop 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 shutdown_connections 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f062806cce0 0x7f062806f1a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06400ffc20 0x7f06401975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:34.201 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 --2- 172.21.15.8:0/3567397612 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0640105bd0 0x7f0640197b20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 >> 172.21.15.8:0/3567397612 conn(0x7f0640075670 msgr2=0x7f06400fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 shutdown_connections 2024-04-12T06:22:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:34.199+0000 7f0644e3f700 1 -- 172.21.15.8:0/3567397612 wait complete. 2024-04-12T06:22:34.202 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 45 2024-04-12T06:22:35.118 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-12T06:22:35.130 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T06:22:35.270 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/1911300476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 msgr2=0x7fdf34106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/1911300476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34106e50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009910 tx=0x7fdf24009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/1911300476 shutdown_connections 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/1911300476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 0x7fdf34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/1911300476 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34106e50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/1911300476 >> 172.21.15.8:0/1911300476 conn(0x7fdf340fc510 msgr2=0x7fdf340fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/1911300476 shutdown_connections 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.268+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/1911300476 wait complete. 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 Processor -- start 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 -- start start 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 0x7fdf341965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.271 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf341971c0 con 0x7fdf34106a70 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf3c1f7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3419af40 con 0x7fdf34100a60 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55650/0 (socket says 172.21.15.8:55650) 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 -- 172.21.15.8:0/861196493 learned_addr learned my addr 172.21.15.8:0/861196493 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39f93700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 0x7fdf341965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 -- 172.21.15.8:0/861196493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 msgr2=0x7fdf341965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 0x7fdf341965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 -- 172.21.15.8:0/861196493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf240095f0 con 0x7fdf34106a70 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.269+0000 7fdf39792700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000d8d0 tx=0x7fdf3000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.270+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf300098a0 con 0x7fdf34106a70 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.270+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf30010460 con 0x7fdf34106a70 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.270+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3419b220 con 0x7fdf34106a70 2024-04-12T06:22:35.272 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.270+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf30009ea0 con 0x7fdf34106a70 2024-04-12T06:22:35.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.270+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3419b780 con 0x7fdf34106a70 2024-04-12T06:22:35.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.271+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf34109060 con 0x7fdf34106a70 2024-04-12T06:22:35.273 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.271+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fdf300105d0 con 0x7fdf34106a70 2024-04-12T06:22:35.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.272+0000 7fdf2affd700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 0x7fdf2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.272+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdf3008c440 con 0x7fdf34106a70 2024-04-12T06:22:35.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.272+0000 7fdf39f93700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 0x7fdf2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.274 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.272+0000 7fdf39f93700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 0x7fdf2006f1a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b5c0 tx=0x7fdf2401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.276 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.276+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fdf300564b0 con 0x7fdf34106a70 2024-04-12T06:22:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:35 smithi194 ceph-mon[106982]: pgmap v828: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:35 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3567397612' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:22:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:35 smithi008 ceph-mon[146084]: pgmap v828: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:35 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3567397612' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-12T06:22:35.486 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.486+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf3404ea90 con 0x7fdf34106a70 2024-04-12T06:22:35.487 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.487+0000 7fdf2affd700 1 -- 172.21.15.8:0/861196493 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16251 (secure 0 0 0) 0x7fdf30010880 con 0x7fdf34106a70 2024-04-12T06:22:35.488 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:35.488 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":83,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T06:04:51.939854+0000","last_up_change":"2024-04-12T06:03:54.642238+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T05:52:42.555620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T05:52:43.040716+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6803","nonce":1483673533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6805","nonce":1483673533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6809","nonce":1483673533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6807","nonce":1483673533}]},"public_addr":"172.21.15.8:6803/1483673533","cluster_addr":"172.21.15.8:6805/1483673533","heartbeat_back_addr":"172.21.15.8:6809/1483673533","heartbeat_front_addr":"172.21.15.8:6807/1483673533","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6811","nonce":202303324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6813","nonce":202303324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6817","nonce":202303324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6815","nonce":202303324}]},"public_addr":"172.21.15.8:6811/202303324","cluster_addr":"172.21.15.8:6813/202303324","heartbeat_back_addr":"172.21.15.8:6817/202303324","heartbeat_front_addr":"172.21.15.8:6815/202303324","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6819","nonce":2007171174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6821","nonce":2007171174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6825","nonce":2007171174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6823","nonce":2007171174}]},"public_addr":"172.21.15.8:6819/2007171174","cluster_addr":"172.21.15.8:6821/2007171174","heartbeat_back_addr":"172.21.15.8:6825/2007171174","heartbeat_front_addr":"172.21.15.8:6823/2007171174","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6801","nonce":1524342463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6803","nonce":1524342463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6807","nonce":1524342463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6805","nonce":1524342463}]},"public_addr":"172.21.15.194:6801/1524342463","cluster_addr":"172.21.15.194:6803/1524342463","heartbeat_back_addr":"172.21.15.194:6807/1524342463","heartbeat_front_addr":"172.21.15.194:6805/1524342463","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6809","nonce":3298239604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6811","nonce":3298239604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6815","nonce":3298239604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6813","nonce":3298239604}]},"public_addr":"172.21.15.194:6809/3298239604","cluster_addr":"172.21.15.194:6811/3298239604","heartbeat_back_addr":"172.21.15.194:6815/3298239604","heartbeat_front_addr":"172.21.15.194:6813/3298239604","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6817","nonce":2816217628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6819","nonce":2816217628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6823","nonce":2816217628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6821","nonce":2816217628}]},"public_addr":"172.21.15.194:6817/2816217628","cluster_addr":"172.21.15.194:6819/2816217628","heartbeat_back_addr":"172.21.15.194:6823/2816217628","heartbeat_front_addr":"172.21.15.194:6821/2816217628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6824/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:6829/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.194:6825/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.194:0/1351746800":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/867529145":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.194:0/3684029331":"2024-04-13T05:56:17.276168+0000","172.21.15.194:0/665606011":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/3326205956":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/2972485178":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/3187654197":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6826/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.194:0/3304199431":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6828/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.8:6827/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/3071600710":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1577264968":"2024-04-13T05:55:13.372184+0000","172.21.15.194:6828/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.194:6829/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T06:22:35.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 msgr2=0x7fdf2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.489 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 0x7fdf2006f1a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400b5c0 tx=0x7fdf2401a040 comp rx=0 tx=0).stop 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 msgr2=0x7fdf34196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000d8d0 tx=0x7fdf3000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 shutdown_connections 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fdf2006cce0 0x7fdf2006f1a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf34100a60 0x7fdf341965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 --2- 172.21.15.8:0/861196493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fdf34106a70 0x7fdf34196ae0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 >> 172.21.15.8:0/861196493 conn(0x7fdf340fc510 msgr2=0x7fdf340fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 shutdown_connections 2024-04-12T06:22:35.490 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.489+0000 7fdf3c1f7700 1 -- 172.21.15.8:0/861196493 wait complete. 2024-04-12T06:22:35.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T05:49:53.787041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T05:52:42.555620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T05:52:43.040716+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T06:22:35.503 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T06:22:35.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 -- 172.21.15.8:0/2064460386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 msgr2=0x7f24d4106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.634 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 --2- 172.21.15.8:0/2064460386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d4106f70 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009910 tx=0x7f24d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 -- 172.21.15.8:0/2064460386 shutdown_connections 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 --2- 172.21.15.8:0/2064460386 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 0x7f24d40ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 --2- 172.21.15.8:0/2064460386 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d4106f70 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 -- 172.21.15.8:0/2064460386 >> 172.21.15.8:0/2064460386 conn(0x7f24d4074c00 msgr2=0x7f24d4075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 -- 172.21.15.8:0/2064460386 shutdown_connections 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.632+0000 7f24db93b700 1 -- 172.21.15.8:0/2064460386 wait complete. 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 Processor -- start 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 -- start start 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 0x7f24d4100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4100ee0 con 0x7f24d4106b90 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24db93b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4101020 con 0x7f24d40ff280 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.635 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55662/0 (socket says 172.21.15.8:55662) 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 -- 172.21.15.8:0/1837379925 learned_addr learned my addr 172.21.15.8:0/1837379925 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d96d7700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 0x7f24d4100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 -- 172.21.15.8:0/1837379925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 msgr2=0x7f24d4100460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 0x7f24d4100460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.633+0000 7f24d8ed6700 1 -- 172.21.15.8:0/1837379925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d00095f0 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24d8ed6700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d8d0 tx=0x7f24c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c40098a0 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f24c4010460 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d41a2750 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24c4009ea0 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.634+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d41a2b70 con 0x7f24d4106b90 2024-04-12T06:22:35.636 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.635+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d4103e50 con 0x7f24d4106b90 2024-04-12T06:22:35.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.635+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f24c40105d0 con 0x7f24d4106b90 2024-04-12T06:22:35.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.636+0000 7f24ca7fc700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.636+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f24c408c570 con 0x7f24d4106b90 2024-04-12T06:22:35.637 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.636+0000 7f24d96d7700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.638 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.636+0000 7f24d96d7700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 0x7f24c006f1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f24d0006010 tx=0x7f24d001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.640 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.640+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f24c40564b0 con 0x7f24d4106b90 2024-04-12T06:22:35.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.845+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f24d402cda0 con 0x7f24d4106b90 2024-04-12T06:22:35.845 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.845+0000 7f24ca7fc700 1 -- 172.21.15.8:0/1837379925 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24c4010880 con 0x7f24d4106b90 2024-04-12T06:22:35.846 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 msgr2=0x7f24c006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 0x7f24c006f1a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f24d0006010 tx=0x7f24d001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 msgr2=0x7f24d41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f24c400d8d0 tx=0x7f24c400dbe0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 shutdown_connections 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f24c006cce0 0x7f24c006f1a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f24d40ff280 0x7f24d4100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.848 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 --2- 172.21.15.8:0/1837379925 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f24d4106b90 0x7f24d41009a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 >> 172.21.15.8:0/1837379925 conn(0x7f24d4074c00 msgr2=0x7f24d40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:35.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.847+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 shutdown_connections 2024-04-12T06:22:35.849 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.848+0000 7f24db93b700 1 -- 172.21.15.8:0/1837379925 wait complete. 2024-04-12T06:22:35.860 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 -- 172.21.15.8:0/2950207074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854102550 msgr2=0x7f385410aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 --2- 172.21.15.8:0/2950207074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854102550 0x7f385410aa30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3850009910 tx=0x7f3850009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 -- 172.21.15.8:0/2950207074 shutdown_connections 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 --2- 172.21.15.8:0/2950207074 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854102550 0x7f385410aa30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 --2- 172.21.15.8:0/2950207074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3854101c30 0x7f3854102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 -- 172.21.15.8:0/2950207074 >> 172.21.15.8:0/2950207074 conn(0x7f38540fb430 msgr2=0x7f38540fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 -- 172.21.15.8:0/2950207074 shutdown_connections 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 -- 172.21.15.8:0/2950207074 wait complete. 2024-04-12T06:22:35.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.989+0000 7f385b606700 1 Processor -- start 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 -- start start 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3854102550 0x7f385419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385419d310 con 0x7f3854101c30 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854196820 con 0x7f3854102550 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55676/0 (socket says 172.21.15.8:55676) 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 -- 172.21.15.8:0/1341258957 learned_addr learned my addr 172.21.15.8:0/1341258957 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 -- 172.21.15.8:0/1341258957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3854102550 msgr2=0x7f385419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3854102550 0x7f385419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 -- 172.21.15.8:0/1341258957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500095f0 con 0x7f3854101c30 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f38593a2700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f384400d8d0 tx=0x7f384400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3844009940 con 0x7f3854101c30 2024-04-12T06:22:35.992 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3844010460 con 0x7f3854101c30 2024-04-12T06:22:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3854196aa0 con 0x7f3854101c30 2024-04-12T06:22:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.990+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f384400f5f0 con 0x7f3854101c30 2024-04-12T06:22:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.991+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854197000 con 0x7f3854101c30 2024-04-12T06:22:35.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.991+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854108310 con 0x7f3854101c30 2024-04-12T06:22:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.992+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f38440105d0 con 0x7f3854101c30 2024-04-12T06:22:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.992+0000 7f384a7fc700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 0x7f384006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.993+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f384408c5d0 con 0x7f3854101c30 2024-04-12T06:22:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.993+0000 7f3858ba1700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 0x7f384006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:35.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.993+0000 7f3858ba1700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 0x7f384006f1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f385000b5c0 tx=0x7f3850005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:35.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:35.997+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f384405a0d0 con 0x7f3854101c30 2024-04-12T06:22:36.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:36 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/861196493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:36.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:36 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1837379925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:36.205 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.205+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f385404ea90 con 0x7f3854101c30 2024-04-12T06:22:36.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.206+0000 7f384a7fc700 1 -- 172.21.15.8:0/1341258957 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f3844010880 con 0x7f3854101c30 2024-04-12T06:22:36.206 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:36.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 msgr2=0x7f384006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.208 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 0x7f384006f1a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f385000b5c0 tx=0x7f3850005960 comp rx=0 tx=0).stop 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 msgr2=0x7f385419c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f384400d8d0 tx=0x7f384400dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 shutdown_connections 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f384006cce0 0x7f384006f1a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3854101c30 0x7f385419c7b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 --2- 172.21.15.8:0/1341258957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3854102550 0x7f385419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 >> 172.21.15.8:0/1341258957 conn(0x7f38540fb430 msgr2=0x7f3854105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 shutdown_connections 2024-04-12T06:22:36.209 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.208+0000 7f385b606700 1 -- 172.21.15.8:0/1341258957 wait complete. 2024-04-12T06:22:36.220 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T06:22:36.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:36 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/861196493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:36.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:36 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1837379925' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 -- 172.21.15.8:0/2097147675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 msgr2=0x7fb4ac102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/2097147675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac102090 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009940 tx=0x7fb49c009c50 comp rx=0 tx=0).stop 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 -- 172.21.15.8:0/2097147675 shutdown_connections 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/2097147675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 0x7fb4ac10aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/2097147675 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac102090 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 -- 172.21.15.8:0/2097147675 >> 172.21.15.8:0/2097147675 conn(0x7fb4ac0fb470 msgr2=0x7fb4ac0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 -- 172.21.15.8:0/2097147675 shutdown_connections 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.350+0000 7fb4b23e0700 1 -- 172.21.15.8:0/2097147675 wait complete. 2024-04-12T06:22:36.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 Processor -- start 2024-04-12T06:22:36.352 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 -- start start 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 0x7fb4ac196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac197280 con 0x7fb4ac101cb0 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4b23e0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19b000 con 0x7fb4ac1025d0 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4abfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.351+0000 7fb4abfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55698/0 (socket says 172.21.15.8:55698) 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4abfff700 1 -- 172.21.15.8:0/793930244 learned_addr learned my addr 172.21.15.8:0/793930244 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:36.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4ab7fe700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 0x7fb4ac196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4abfff700 1 -- 172.21.15.8:0/793930244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 msgr2=0x7fb4ac196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4abfff700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 0x7fb4ac196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4abfff700 1 -- 172.21.15.8:0/793930244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0095f0 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4abfff700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00b580 tx=0x7fb49c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c01d070 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac19b280 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb49c00bea0 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac19b780 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.352+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c00fe80 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.353+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac1081b0 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.354+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fb49c00f6f0 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.354+0000 7fb4a97fa700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 0x7fb49406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.354+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb49c08d6e0 con 0x7fb4ac101cb0 2024-04-12T06:22:36.355 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.354+0000 7fb4ab7fe700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 0x7fb49406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.356 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.354+0000 7fb4ab7fe700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 0x7fb49406f1a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005950 tx=0x7fb4a00058e0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:36.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.359+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fb49c057800 con 0x7fb4ac101cb0 2024-04-12T06:22:36.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.567+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb4ac04ea90 con 0x7fb4ac101cb0 2024-04-12T06:22:36.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.568+0000 7fb4a97fa700 1 -- 172.21.15.8:0/793930244 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb49c05ae20 con 0x7fb4ac101cb0 2024-04-12T06:22:36.568 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 msgr2=0x7fb49406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 0x7fb49406f1a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0005950 tx=0x7fb4a00058e0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 msgr2=0x7fb4ac196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00b580 tx=0x7fb49c0056a0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 shutdown_connections 2024-04-12T06:22:36.570 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fb49406cce0 0x7fb49406f1a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fb4ac101cb0 0x7fb4ac196660 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 --2- 172.21.15.8:0/793930244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb4ac1025d0 0x7fb4ac196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 >> 172.21.15.8:0/793930244 conn(0x7fb4ac0fb470 msgr2=0x7fb4ac1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:36.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 shutdown_connections 2024-04-12T06:22:36.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.570+0000 7fb4b23e0700 1 -- 172.21.15.8:0/793930244 wait complete. 2024-04-12T06:22:36.585 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.711+0000 7fcba0303700 1 -- 172.21.15.8:0/2931763545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 msgr2=0x7fcb98105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.711+0000 7fcba0303700 1 --2- 172.21.15.8:0/2931763545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98105d60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009910 tx=0x7fcb94009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- 172.21.15.8:0/2931763545 shutdown_connections 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 --2- 172.21.15.8:0/2931763545 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb980ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 --2- 172.21.15.8:0/2931763545 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98105d60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- 172.21.15.8:0/2931763545 >> 172.21.15.8:0/2931763545 conn(0x7fcb980fb420 msgr2=0x7fcb980fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- 172.21.15.8:0/2931763545 shutdown_connections 2024-04-12T06:22:36.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- 172.21.15.8:0/2931763545 wait complete. 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 Processor -- start 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- start start 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb98195520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98196140 con 0x7fcb98105980 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.712+0000 7fcba0303700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98199e70 con 0x7fcb980ff970 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55718/0 (socket says 172.21.15.8:55718) 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 -- 172.21.15.8:0/994981955 learned_addr learned my addr 172.21.15.8:0/994981955 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9e09f700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb98195520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 -- 172.21.15.8:0/994981955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 msgr2=0x7fcb98195520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb98195520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 -- 172.21.15.8:0/994981955 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb940095f0 con 0x7fcb98105980 2024-04-12T06:22:36.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9e09f700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb98195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb9d89e700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e920 tx=0x7fcb8800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb88009790 con 0x7fcb98105980 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9819a150 con 0x7fcb98105980 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcb8800cba0 con 0x7fcb98105980 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb88018650 con 0x7fcb98105980 2024-04-12T06:22:36.716 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.713+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9819a6b0 con 0x7fcb98105980 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.714+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb98107f70 con 0x7fcb98105980 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.715+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fcb8800cd10 con 0x7fcb98105980 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.715+0000 7fcb8f7fe700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 0x7fcb8406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.715+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcb88014070 con 0x7fcb98105980 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.715+0000 7fcb9e09f700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 0x7fcb8406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:36.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.715+0000 7fcb9e09f700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 0x7fcb8406f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400b6e0 tx=0x7fcb94009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:36.719 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.719+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fcb88052ab0 con 0x7fcb98105980 2024-04-12T06:22:36.968 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.968+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb9804ea90 con 0x7fcb98105980 2024-04-12T06:22:36.969 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.969+0000 7fcb8f7fe700 1 -- 172.21.15.8:0/994981955 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4858 (secure 0 0 0) 0x7fcb8805a440 con 0x7fcb98105980 2024-04-12T06:22:36.969 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:36.969 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:28.780381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:active","state_seq":245,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:active","state_seq":239,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44331":{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44361":{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 msgr2=0x7fcb8406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 0x7fcb8406f1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fcb9400b6e0 tx=0x7fcb94009f70 comp rx=0 tx=0).stop 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 msgr2=0x7fcb98195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800e920 tx=0x7fcb8800ece0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 shutdown_connections 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fcb8406cce0 0x7fcb8406f1a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb980ff970 0x7fcb98195520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.971 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 --2- 172.21.15.8:0/994981955 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fcb98105980 0x7fcb98195a60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 >> 172.21.15.8:0/994981955 conn(0x7fcb980fb420 msgr2=0x7fcb980fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 shutdown_connections 2024-04-12T06:22:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:36.971+0000 7fcba0303700 1 -- 172.21.15.8:0/994981955 wait complete. 2024-04-12T06:22:36.972 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 45 2024-04-12T06:22:36.983 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 21, 'max_mds': 2, 'flags': 50} 2024-04-12T06:22:36.983 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-12T06:22:37.021 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:37 smithi194 ceph-mon[106982]: pgmap v829: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:37.021 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:37 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1341258957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:37.022 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:37 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/793930244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:37.022 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:37 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/994981955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3615139652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 msgr2=0x7f3d98100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3615139652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98100ee0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009910 tx=0x7f3d80009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3615139652 shutdown_connections 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3615139652 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98100ee0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3615139652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 0x7f3d98106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3615139652 >> 172.21.15.8:0/3615139652 conn(0x7f3d980fc570 msgr2=0x7f3d980fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.110+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3615139652 shutdown_connections 2024-04-12T06:22:37.111 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3615139652 wait complete. 2024-04-12T06:22:37.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 Processor -- start 2024-04-12T06:22:37.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 -- start start 2024-04-12T06:22:37.113 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 0x7f3d980735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98073b10 con 0x7f3d98100a60 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.111+0000 7f3d9cfb5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98073c50 con 0x7f3d98106a70 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55740/0 (socket says 172.21.15.8:55740) 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 -- 172.21.15.8:0/3140128838 learned_addr learned my addr 172.21.15.8:0/3140128838 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d95d9b700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 0x7f3d980735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 -- 172.21.15.8:0/3140128838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 msgr2=0x7f3d980735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 0x7f3d980735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 -- 172.21.15.8:0/3140128838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800095f0 con 0x7f3d98100a60 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9659c700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c8a0 tx=0x7f3d8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8800cea0 con 0x7f3d98100a60 2024-04-12T06:22:37.114 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3d88004830 con 0x7f3d98100a60 2024-04-12T06:22:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d981a2750 con 0x7f3d98100a60 2024-04-12T06:22:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d88005500 con 0x7f3d98100a60 2024-04-12T06:22:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.112+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d981a2b70 con 0x7f3d98100a60 2024-04-12T06:22:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.113+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98108ff0 con 0x7f3d98100a60 2024-04-12T06:22:37.115 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.114+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3d880049a0 con 0x7f3d98100a60 2024-04-12T06:22:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.114+0000 7f3d8f7fe700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 0x7f3d8406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.114+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d8808c2c0 con 0x7f3d98100a60 2024-04-12T06:22:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.114+0000 7f3d95d9b700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 0x7f3d8406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.116 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.115+0000 7f3d95d9b700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 0x7f3d8406f1a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3d98074880 tx=0x7f3d8001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.119 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.119+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3d880562a0 con 0x7f3d98100a60 2024-04-12T06:22:37.375 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.375+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f3d9802cc70 con 0x7f3d98100a60 2024-04-12T06:22:37.376 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.376+0000 7f3d8f7fe700 1 -- 172.21.15.8:0/3140128838 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v45) v1 ==== 107+0+4854 (secure 0 0 0) 0x7f3d880598c0 con 0x7f3d98100a60 2024-04-12T06:22:37.376 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:37.376 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T05:54:49.406413+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.8:6827/534481461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":534481461},{"type":"v1","addr":"172.21.15.8:6827","nonce":534481461}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.194:6827/3772682938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3772682938},{"type":"v1","addr":"172.21.15.194:6827","nonce":3772682938}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 msgr2=0x7f3d8406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 0x7f3d8406f1a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3d98074880 tx=0x7f3d8001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 msgr2=0x7f3d98073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c8a0 tx=0x7f3d8800cc60 comp rx=0 tx=0).stop 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 shutdown_connections 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3d8406cce0 0x7f3d8406f1a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3d98100a60 0x7f3d98073090 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 --2- 172.21.15.8:0/3140128838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3d98106a70 0x7f3d980735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.378+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 >> 172.21.15.8:0/3140128838 conn(0x7f3d980fc570 msgr2=0x7f3d980fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:37.379 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.379+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 shutdown_connections 2024-04-12T06:22:37.380 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.379+0000 7f3d9cfb5700 1 -- 172.21.15.8:0/3140128838 wait complete. 2024-04-12T06:22:37.380 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 22 2024-04-12T06:22:37.390 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:37 smithi008 ceph-mon[146084]: pgmap v829: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:37.390 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:37 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1341258957' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:37.390 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:37 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/793930244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:37.390 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:37 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/994981955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-12T06:22:37.391 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-12T06:22:37.518 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 -- 172.21.15.8:0/2078216630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 msgr2=0x7f0a10105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 --2- 172.21.15.8:0/2078216630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10105d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009910 tx=0x7f0a04009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 -- 172.21.15.8:0/2078216630 shutdown_connections 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 --2- 172.21.15.8:0/2078216630 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 0x7f0a100ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 --2- 172.21.15.8:0/2078216630 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10105d60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 -- 172.21.15.8:0/2078216630 >> 172.21.15.8:0/2078216630 conn(0x7f0a100fb4c0 msgr2=0x7f0a100fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 -- 172.21.15.8:0/2078216630 shutdown_connections 2024-04-12T06:22:37.519 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.518+0000 7f0a17edc700 1 -- 172.21.15.8:0/2078216630 wait complete. 2024-04-12T06:22:37.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 Processor -- start 2024-04-12T06:22:37.521 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 -- start start 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 0x7f0a10197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10197f80 con 0x7f0a10105980 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a17edc700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101980f0 con 0x7f0a100ff970 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a15477700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a15477700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55762/0 (socket says 172.21.15.8:55762) 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.519+0000 7f0a15477700 1 -- 172.21.15.8:0/3529590744 learned_addr learned my addr 172.21.15.8:0/3529590744 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a15c78700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 0x7f0a10197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a15477700 1 -- 172.21.15.8:0/3529590744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 msgr2=0x7f0a10197420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a15477700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 0x7f0a10197420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a15477700 1 -- 172.21.15.8:0/3529590744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040095f0 con 0x7f0a10105980 2024-04-12T06:22:37.522 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a15477700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c8f0 tx=0x7f0a0c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0c004a00 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a0c005020 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0c007c40 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a101a0490 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.520+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a101a0990 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.521+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a10107f70 con 0x7f0a10105980 2024-04-12T06:22:37.523 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.522+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f0a0c005190 con 0x7f0a10105980 2024-04-12T06:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.522+0000 7f0a02ffd700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 0x7f09fc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.522+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a0c013070 con 0x7f0a10105980 2024-04-12T06:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.522+0000 7f0a15c78700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 0x7f09fc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.524 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.522+0000 7f0a15c78700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 0x7f09fc06f1a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.526 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.526+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0a0c056440 con 0x7f0a10105980 2024-04-12T06:22:37.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.780+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f0a1004ea90 con 0x7f0a10105980 2024-04-12T06:22:37.780 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.780+0000 7f0a02ffd700 1 -- 172.21.15.8:0/3529590744 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v45) v1 ==== 107+0+4855 (secure 0 0 0) 0x7f0a0c059a60 con 0x7f0a10105980 2024-04-12T06:22:37.781 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:37.781 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T05:55:45.534456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.8:6827/534481461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":534481461},{"type":"v1","addr":"172.21.15.8:6827","nonce":534481461}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.194:6827/3772682938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3772682938},{"type":"v1","addr":"172.21.15.194:6827","nonce":3772682938}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:37.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 msgr2=0x7f09fc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.783 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 0x7f09fc06f1a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0401a040 comp rx=0 tx=0).stop 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 msgr2=0x7f0a10197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00c8f0 tx=0x7f0a0c00cc00 comp rx=0 tx=0).stop 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 shutdown_connections 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f09fc06cce0 0x7f09fc06f1a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0a100ff970 0x7f0a10197420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 --2- 172.21.15.8:0/3529590744 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0a10105980 0x7f0a10197960 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 >> 172.21.15.8:0/3529590744 conn(0x7f0a100fb4c0 msgr2=0x7f0a100fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 shutdown_connections 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.783+0000 7f0a17edc700 1 -- 172.21.15.8:0/3529590744 wait complete. 2024-04-12T06:22:37.784 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 23 2024-04-12T06:22:37.795 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 -- 172.21.15.8:0/3551520350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59200ff150 msgr2=0x7f59200ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 --2- 172.21.15.8:0/3551520350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59200ff150 0x7f59200ff5d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5914009910 tx=0x7f5914009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 -- 172.21.15.8:0/3551520350 shutdown_connections 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 --2- 172.21.15.8:0/3551520350 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f59200ff150 0x7f59200ff5d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 --2- 172.21.15.8:0/3551520350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5920106c30 0x7f5920107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 -- 172.21.15.8:0/3551520350 >> 172.21.15.8:0/3551520350 conn(0x7f5920074c50 msgr2=0x7f5920075060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 -- 172.21.15.8:0/3551520350 shutdown_connections 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.922+0000 7f592544a700 1 -- 172.21.15.8:0/3551520350 wait complete. 2024-04-12T06:22:37.923 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 Processor -- start 2024-04-12T06:22:37.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 -- start start 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 0x7f592019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019d800 con 0x7f5920106c30 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.923+0000 7f592544a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592019d970 con 0x7f59200ff150 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55790/0 (socket says 172.21.15.8:55790) 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 -- 172.21.15.8:0/2216865918 learned_addr learned my addr 172.21.15.8:0/2216865918 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591effd700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 0x7f592019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 -- 172.21.15.8:0/2216865918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 msgr2=0x7f592019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 0x7f592019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 -- 172.21.15.8:0/2216865918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59140095f0 con 0x7f5920106c30 2024-04-12T06:22:37.926 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591effd700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 0x7f592019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f591e7fc700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f591400b5c0 tx=0x7f591400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591401d070 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f591400f460 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59201a06f0 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5914021640 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.924+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59201a0bf0 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.925+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5920108f90 con 0x7f5920106c30 2024-04-12T06:22:37.927 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.926+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f591400f9b0 con 0x7f5920106c30 2024-04-12T06:22:37.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.926+0000 7f5907fff700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 0x7f590806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:37.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.926+0000 7f591effd700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 0x7f590806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:37.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.926+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f591408d850 con 0x7f5920106c30 2024-04-12T06:22:37.928 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.927+0000 7f591effd700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 0x7f590806f1b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f5910005e20 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:37.931 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:37.931+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f59140578c0 con 0x7f5920106c30 2024-04-12T06:22:38.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:38 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3140128838' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-12T06:22:38.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:38 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3529590744' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-12T06:22:38.182 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.183+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f592019dfd0 con 0x7f5920106c30 2024-04-12T06:22:38.183 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.183+0000 7f5907fff700 1 -- 172.21.15.8:0/2216865918 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v45) v1 ==== 107+0+4854 (secure 0 0 0) 0x7f591405aee0 con 0x7f5920106c30 2024-04-12T06:22:38.183 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:38.183 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:00:00.958556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.8:6827/534481461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":534481461},{"type":"v1","addr":"172.21.15.8:6827","nonce":534481461}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.194:6827/3772682938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3772682938},{"type":"v1","addr":"172.21.15.194:6827","nonce":3772682938}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 msgr2=0x7f590806f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 0x7f590806f1b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5910005fd0 tx=0x7f5910005e20 comp rx=0 tx=0).stop 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 msgr2=0x7f592019d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f591400b5c0 tx=0x7f591400be20 comp rx=0 tx=0).stop 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 shutdown_connections 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f590806ccf0 0x7f590806f1b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f59200ff150 0x7f592019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 --2- 172.21.15.8:0/2216865918 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f5920106c30 0x7f592019d200 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.185+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 >> 172.21.15.8:0/2216865918 conn(0x7f5920074c50 msgr2=0x7f59200fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.186+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 shutdown_connections 2024-04-12T06:22:38.186 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.186+0000 7f592544a700 1 -- 172.21.15.8:0/2216865918 wait complete. 2024-04-12T06:22:38.187 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 24 2024-04-12T06:22:38.198 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-12T06:22:38.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:38 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3140128838' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-12T06:22:38.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:38 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3529590744' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 -- 172.21.15.8:0/3094396455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0067d90 msgr2=0x7f64e0068210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 --2- 172.21.15.8:0/3094396455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0067d90 0x7f64e0068210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009910 tx=0x7f64d0009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 -- 172.21.15.8:0/3094396455 shutdown_connections 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 --2- 172.21.15.8:0/3094396455 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0067d90 0x7f64e0068210 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 --2- 172.21.15.8:0/3094396455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0106b90 0x7f64e0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.327 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 -- 172.21.15.8:0/3094396455 >> 172.21.15.8:0/3094396455 conn(0x7f64e0075400 msgr2=0x7f64e0075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 -- 172.21.15.8:0/3094396455 shutdown_connections 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.325+0000 7f64e874a700 1 -- 172.21.15.8:0/3094396455 wait complete. 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 Processor -- start 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 -- start start 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0067d90 0x7f64e00ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0100610 con 0x7f64e0106b90 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0100750 con 0x7f64e0067d90 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55818/0 (socket says 172.21.15.8:55818) 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 -- 172.21.15.8:0/718550304 learned_addr learned my addr 172.21.15.8:0/718550304 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:38.328 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e64e6700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0067d90 0x7f64e00ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 -- 172.21.15.8:0/718550304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0067d90 msgr2=0x7f64e00ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0067d90 0x7f64e00ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 -- 172.21.15.8:0/718550304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d00095f0 con 0x7f64e0106b90 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e5ce5700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f64d0006010 tx=0x7f64d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d001d070 con 0x7f64e0106b90 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64d000f460 con 0x7f64e0106b90 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.326+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e01009d0 con 0x7f64e0106b90 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.327+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d0021640 con 0x7f64e0106b90 2024-04-12T06:22:38.329 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.327+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e01a2bf0 con 0x7f64e0106b90 2024-04-12T06:22:38.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.327+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e0109130 con 0x7f64e0106b90 2024-04-12T06:22:38.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.328+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f64d000fa20 con 0x7f64e0106b90 2024-04-12T06:22:38.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.328+0000 7f64d77fe700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 0x7f64cc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.329+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f64d008d7a0 con 0x7f64e0106b90 2024-04-12T06:22:38.330 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.329+0000 7f64e64e6700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 0x7f64cc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.331 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.329+0000 7f64e64e6700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 0x7f64cc06f1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f64dc005fd0 tx=0x7f64dc00a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:38.333 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.333+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f64d0057890 con 0x7f64e0106b90 2024-04-12T06:22:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.586+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f64e004ea90 con 0x7f64e0106b90 2024-04-12T06:22:38.586 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.587+0000 7f64d77fe700 1 -- 172.21.15.8:0/718550304 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v45) v1 ==== 107+0+4853 (secure 0 0 0) 0x7f64d005aeb0 con 0x7f64e0106b90 2024-04-12T06:22:38.587 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:38.587 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:00:14.502437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14488":{"gid":14488,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.8:6827/534481461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":534481461},{"type":"v1","addr":"172.21.15.8:6827","nonce":534481461}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.194:6827/3772682938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3772682938},{"type":"v1","addr":"172.21.15.194:6827","nonce":3772682938}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 msgr2=0x7f64cc06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 0x7f64cc06f1a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f64dc005fd0 tx=0x7f64dc00a560 comp rx=0 tx=0).stop 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 msgr2=0x7f64e01000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f64d0006010 tx=0x7f64d000be20 comp rx=0 tx=0).stop 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 shutdown_connections 2024-04-12T06:22:38.589 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f64cc06cce0 0x7f64cc06f1a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64e0067d90 0x7f64e00ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 --2- 172.21.15.8:0/718550304 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f64e0106b90 0x7f64e01000d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 >> 172.21.15.8:0/718550304 conn(0x7f64e0075400 msgr2=0x7f64e00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 shutdown_connections 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.589+0000 7f64e874a700 1 -- 172.21.15.8:0/718550304 wait complete. 2024-04-12T06:22:38.590 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 25 2024-04-12T06:22:38.601 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 -- 172.21.15.8:0/4168737738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 msgr2=0x7f3a28105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 --2- 172.21.15.8:0/4168737738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28105d70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3a18009910 tx=0x7f3a18009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 -- 172.21.15.8:0/4168737738 shutdown_connections 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 --2- 172.21.15.8:0/4168737738 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a280ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 --2- 172.21.15.8:0/4168737738 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28105d70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.728+0000 7f3a2d589700 1 -- 172.21.15.8:0/4168737738 >> 172.21.15.8:0/4168737738 conn(0x7f3a280fb4b0 msgr2=0x7f3a280fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.729+0000 7f3a2d589700 1 -- 172.21.15.8:0/4168737738 shutdown_connections 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.729+0000 7f3a2d589700 1 -- 172.21.15.8:0/4168737738 wait complete. 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.729+0000 7f3a2d589700 1 Processor -- start 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.729+0000 7f3a2d589700 1 -- start start 2024-04-12T06:22:38.731 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a281965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a281971d0 con 0x7f3a28105990 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819af50 con 0x7f3a280ff980 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55834/0 (socket says 172.21.15.8:55834) 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 -- 172.21.15.8:0/717268514 learned_addr learned my addr 172.21.15.8:0/717268514 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a26ffd700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a281965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 -- 172.21.15.8:0/717268514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 msgr2=0x7f3a281965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a281965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 -- 172.21.15.8:0/717268514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a180095f0 con 0x7f3a28105990 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a26ffd700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a281965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a267fc700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d8d0 tx=0x7f3a1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c009940 con 0x7f3a28105990 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3a1c010460 con 0x7f3a28105990 2024-04-12T06:22:38.732 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2819b230 con 0x7f3a28105990 2024-04-12T06:22:38.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c00f5f0 con 0x7f3a28105990 2024-04-12T06:22:38.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.730+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2819b790 con 0x7f3a28105990 2024-04-12T06:22:38.733 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.731+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28108f90 con 0x7f3a28105990 2024-04-12T06:22:38.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.732+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3a1c0105d0 con 0x7f3a28105990 2024-04-12T06:22:38.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.732+0000 7f3a0ffff700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 0x7f3a1006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:38.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.732+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3a1c08c670 con 0x7f3a28105990 2024-04-12T06:22:38.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.732+0000 7f3a26ffd700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 0x7f3a1006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:38.734 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.733+0000 7f3a26ffd700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 0x7f3a1006f1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006010 tx=0x7f3a1801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:38.736 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.737+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3a1c056640 con 0x7f3a28105990 2024-04-12T06:22:38.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.990+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f3a2819ba70 con 0x7f3a28105990 2024-04-12T06:22:38.991 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.991+0000 7f3a0ffff700 1 -- 172.21.15.8:0/717268514 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v45) v1 ==== 107+0+3284 (secure 0 0 0) 0x7f3a1c020cf0 con 0x7f3a28105990 2024-04-12T06:22:38.991 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:38.992 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.000122+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14476,"mds_1":24277},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14476":{"gid":14476,"name":"cephfs.smithi008.lmvepz","rank":0,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.8:6829/2135087803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2135087803},{"type":"v1","addr":"172.21.15.8:6829","nonce":2135087803}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24277":{"gid":24277,"name":"cephfs.smithi194.uzfpyu","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.194:6825/1464587758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":1464587758},{"type":"v1","addr":"172.21.15.194:6825","nonce":1464587758}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.993+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 msgr2=0x7f3a1006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.993+0000 7f3a2d589700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 0x7f3a1006f1a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006010 tx=0x7f3a1801a040 comp rx=0 tx=0).stop 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.993+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 msgr2=0x7f3a28196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.993+0000 7f3a2d589700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d8d0 tx=0x7f3a1c00dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 shutdown_connections 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f3a1006cce0 0x7f3a1006f1a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a280ff980 0x7f3a281965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 --2- 172.21.15.8:0/717268514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3a28105990 0x7f3a28196af0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 >> 172.21.15.8:0/717268514 conn(0x7f3a280fb4b0 msgr2=0x7f3a280690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 shutdown_connections 2024-04-12T06:22:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:38.994+0000 7f3a2d589700 1 -- 172.21.15.8:0/717268514 wait complete. 2024-04-12T06:22:38.995 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 26 2024-04-12T06:22:39.006 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-12T06:22:39.007 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-12T06:22:39.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 -- 172.21.15.8:0/3702672128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 msgr2=0x7f95400fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 --2- 172.21.15.8:0/3702672128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f95400fff20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9530009910 tx=0x7f9530009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:39.134 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 -- 172.21.15.8:0/3702672128 shutdown_connections 2024-04-12T06:22:39.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 --2- 172.21.15.8:0/3702672128 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f95400fff20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 --2- 172.21.15.8:0/3702672128 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 0x7f9540105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 -- 172.21.15.8:0/3702672128 >> 172.21.15.8:0/3702672128 conn(0x7f9540075400 msgr2=0x7f9540075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:39.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 -- 172.21.15.8:0/3702672128 shutdown_connections 2024-04-12T06:22:39.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.134+0000 7f9547f07700 1 -- 172.21.15.8:0/3702672128 wait complete. 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 Processor -- start 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 -- start start 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 0x7f9540197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540198080 con 0x7f95400ffac0 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9547f07700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401981f0 con 0x7f9540105aa0 2024-04-12T06:22:39.137 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55858/0 (socket says 172.21.15.8:55858) 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 -- 172.21.15.8:0/2657700054 learned_addr learned my addr 172.21.15.8:0/2657700054 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f95454a2700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 0x7f9540197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 -- 172.21.15.8:0/2657700054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 msgr2=0x7f9540197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 0x7f9540197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 -- 172.21.15.8:0/2657700054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300095f0 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.135+0000 7f9545ca3700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e920 tx=0x7f953c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.136+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c009790 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.136+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f953c00cba0 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.136+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c018650 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.136+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95401a0590 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.136+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95401a0a90 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.137+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540108020 con 0x7f95400ffac0 2024-04-12T06:22:39.138 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.137+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f953c00cd10 con 0x7f95400ffac0 2024-04-12T06:22:39.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.138+0000 7f9536ffd700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 0x7f952c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.138+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f953c014070 con 0x7f95400ffac0 2024-04-12T06:22:39.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.138+0000 7f95454a2700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 0x7f952c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.139 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.138+0000 7f95454a2700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 0x7f952c06f1a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9540068660 tx=0x7f9530009f70 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.142 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.142+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f953c052ab0 con 0x7f95400ffac0 2024-04-12T06:22:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:39 smithi194 ceph-mon[106982]: pgmap v830: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2216865918' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-12T06:22:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/718550304' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-12T06:22:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:39 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/717268514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-12T06:22:39.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.391+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f954004ea90 con 0x7f95400ffac0 2024-04-12T06:22:39.391 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.391+0000 7f9536ffd700 1 -- 172.21.15.8:0/2657700054 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v45) v1 ==== 107+0+3260 (secure 0 0 0) 0x7f953c05a3d0 con 0x7f95400ffac0 2024-04-12T06:22:39.392 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:39.392 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/885712375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":885712375},{"type":"v1","addr":"172.21.15.8:6827","nonce":885712375}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 msgr2=0x7f952c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 0x7f952c06f1a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9540068660 tx=0x7f9530009f70 comp rx=0 tx=0).stop 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 msgr2=0x7f9540197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f953c00e920 tx=0x7f953c00ece0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 shutdown_connections 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f952c06cce0 0x7f952c06f1a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f95400ffac0 0x7f9540197520 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.394 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 --2- 172.21.15.8:0/2657700054 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9540105aa0 0x7f9540197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 >> 172.21.15.8:0/2657700054 conn(0x7f9540075400 msgr2=0x7f95400fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:39.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 shutdown_connections 2024-04-12T06:22:39.395 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.394+0000 7f9547f07700 1 -- 172.21.15.8:0/2657700054 wait complete. 2024-04-12T06:22:39.395 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 27 2024-04-12T06:22:39.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:39 smithi008 ceph-mon[146084]: pgmap v830: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:39.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2216865918' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-12T06:22:39.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/718550304' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-12T06:22:39.405 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:39 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/717268514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-12T06:22:39.406 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:39.406 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-12T06:22:39.406 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 -- 172.21.15.8:0/582107459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80ff650 msgr2=0x7f15e80ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/582107459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80ff650 0x7f15e80ffa30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009910 tx=0x7f15e4009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 -- 172.21.15.8:0/582107459 shutdown_connections 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/582107459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80fff70 0x7f15e810ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/582107459 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80ff650 0x7f15e80ffa30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 -- 172.21.15.8:0/582107459 >> 172.21.15.8:0/582107459 conn(0x7f15e80fb250 msgr2=0x7f15e80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 -- 172.21.15.8:0/582107459 shutdown_connections 2024-04-12T06:22:39.539 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.538+0000 7f15ee7a2700 1 -- 172.21.15.8:0/582107459 wait complete. 2024-04-12T06:22:39.541 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.539+0000 7f15ee7a2700 1 Processor -- start 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.539+0000 7f15ee7a2700 1 -- start start 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 0x7f15e8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8199120 con 0x7f15e80fff70 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e819cea0 con 0x7f15e80ff650 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55866/0 (socket says 172.21.15.8:55866) 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 -- 172.21.15.8:0/310550575 learned_addr learned my addr 172.21.15.8:0/310550575 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ed7a0700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 0x7f15e8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 -- 172.21.15.8:0/310550575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 msgr2=0x7f15e8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 0x7f15e8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 -- 172.21.15.8:0/310550575 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40095f0 con 0x7f15e80fff70 2024-04-12T06:22:39.542 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ed7a0700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 0x7f15e8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ecf9f700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d8d0 tx=0x7f15dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc009940 con 0x7f15e80fff70 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f15dc010460 con 0x7f15e80fff70 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e819d180 con 0x7f15e80fff70 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15dc00f5f0 con 0x7f15e80fff70 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.540+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e819d6e0 con 0x7f15e80fff70 2024-04-12T06:22:39.543 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.541+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e810a430 con 0x7f15e80fff70 2024-04-12T06:22:39.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.542+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f15dc0105d0 con 0x7f15e80fff70 2024-04-12T06:22:39.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.542+0000 7f15da7fc700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 0x7f15d406f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.543+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f15dc08c520 con 0x7f15e80fff70 2024-04-12T06:22:39.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.543+0000 7f15ed7a0700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 0x7f15d406f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.544 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.543+0000 7f15ed7a0700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 0x7f15d406f260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15e400b5c0 tx=0x7f15e4005960 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.546 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.546+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f15dc0566a0 con 0x7f15e80fff70 2024-04-12T06:22:39.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.800+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f15e804f2e0 con 0x7f15e80fff70 2024-04-12T06:22:39.800 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.800+0000 7f15da7fc700 1 -- 172.21.15.8:0/310550575 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v45) v1 ==== 107+0+4818 (secure 0 0 0) 0x7f15dc010880 con 0x7f15e80fff70 2024-04-12T06:22:39.801 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:39.801 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34318,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/885712375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":885712375},{"type":"v1","addr":"172.21.15.8:6827","nonce":885712375}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":34326,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2959889136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2959889136},{"type":"v1","addr":"172.21.15.8:6829","nonce":2959889136}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44299,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/2084066297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2084066297},{"type":"v1","addr":"172.21.15.194:6825","nonce":2084066297}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.802+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 msgr2=0x7f15d406f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.802+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 0x7f15d406f260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f15e400b5c0 tx=0x7f15e4005960 comp rx=0 tx=0).stop 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 msgr2=0x7f15e8198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d8d0 tx=0x7f15dc00dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 shutdown_connections 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f15d406cda0 0x7f15d406f260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f15e80ff650 0x7f15e8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.803 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 --2- 172.21.15.8:0/310550575 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f15e80fff70 0x7f15e8198a40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 >> 172.21.15.8:0/310550575 conn(0x7f15e80fb250 msgr2=0x7f15e80fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:39.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 shutdown_connections 2024-04-12T06:22:39.804 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.803+0000 7f15ee7a2700 1 -- 172.21.15.8:0/310550575 wait complete. 2024-04-12T06:22:39.804 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 28 2024-04-12T06:22:39.815 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:39.815 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-12T06:22:39.816 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-12T06:22:39.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 -- 172.21.15.8:0/1374305555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 msgr2=0x7f52680ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.944 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 --2- 172.21.15.8:0/1374305555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f52680ff5e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5250009910 tx=0x7f5250009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 -- 172.21.15.8:0/1374305555 shutdown_connections 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 --2- 172.21.15.8:0/1374305555 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f52680ff5e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 --2- 172.21.15.8:0/1374305555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 0x7f5268106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 -- 172.21.15.8:0/1374305555 >> 172.21.15.8:0/1374305555 conn(0x7f5268074c60 msgr2=0x7f5268075070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 -- 172.21.15.8:0/1374305555 shutdown_connections 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.942+0000 7f526d2a8700 1 -- 172.21.15.8:0/1374305555 wait complete. 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 Processor -- start 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 -- start start 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 0x7f5268196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52681971a0 con 0x7f52680ff180 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f526d2a8700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526819af20 con 0x7f5268106ac0 2024-04-12T06:22:39.945 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f5266ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f5266ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55882/0 (socket says 172.21.15.8:55882) 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.943+0000 7f5266ffd700 1 -- 172.21.15.8:0/362899649 learned_addr learned my addr 172.21.15.8:0/362899649 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f52667fc700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 0x7f5268196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f5266ffd700 1 -- 172.21.15.8:0/362899649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 msgr2=0x7f5268196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f5266ffd700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 0x7f5268196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f5266ffd700 1 -- 172.21.15.8:0/362899649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500095f0 con 0x7f52680ff180 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f5266ffd700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f525800c8a0 tx=0x7f525800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f525800cea0 con 0x7f52680ff180 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526819b200 con 0x7f52680ff180 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5258004830 con 0x7f52680ff180 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5258005500 con 0x7f52680ff180 2024-04-12T06:22:39.946 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.944+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f526819b760 con 0x7f52680ff180 2024-04-12T06:22:39.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.945+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5268109040 con 0x7f52680ff180 2024-04-12T06:22:39.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.945+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f52580049a0 con 0x7f52680ff180 2024-04-12T06:22:39.947 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.946+0000 7f525ffff700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 0x7f525406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:39.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.946+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f525808c270 con 0x7f52680ff180 2024-04-12T06:22:39.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.946+0000 7f52667fc700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 0x7f525406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:39.948 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.946+0000 7f52667fc700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 0x7f525406f1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5268197b90 tx=0x7f525001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:39.950 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:39.950+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f52580562e0 con 0x7f52680ff180 2024-04-12T06:22:40.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:40 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2657700054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-12T06:22:40.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:40 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/310550575' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-12T06:22:40.203 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.203+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f526819baa0 con 0x7f52680ff180 2024-04-12T06:22:40.204 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.204+0000 7f525ffff700 1 -- 172.21.15.8:0/362899649 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v45) v1 ==== 107+0+4045 (secure 0 0 0) 0x7f5258059900 con 0x7f52680ff180 2024-04-12T06:22:40.204 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:40.204 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34326,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2959889136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2959889136},{"type":"v1","addr":"172.21.15.8:6829","nonce":2959889136}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44299,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/2084066297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2084066297},{"type":"v1","addr":"172.21.15.194:6825","nonce":2084066297}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:40.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 msgr2=0x7f525406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 0x7f525406f1a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5268197b90 tx=0x7f525001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:40.206 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 msgr2=0x7f5268196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f525800c8a0 tx=0x7f525800cc60 comp rx=0 tx=0).stop 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 shutdown_connections 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f525406cce0 0x7f525406f1a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f52680ff180 0x7f5268196580 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 --2- 172.21.15.8:0/362899649 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5268106ac0 0x7f5268196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 >> 172.21.15.8:0/362899649 conn(0x7f5268074c60 msgr2=0x7f52680fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 shutdown_connections 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.206+0000 7f526d2a8700 1 -- 172.21.15.8:0/362899649 wait complete. 2024-04-12T06:22:40.207 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 29 2024-04-12T06:22:40.219 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:40.219 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-12T06:22:40.219 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-12T06:22:40.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:40 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2657700054' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-12T06:22:40.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:40 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/310550575' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2580622865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 msgr2=0x7fae6410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2580622865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae6410aa50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fae50009910 tx=0x7fae50009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2580622865 shutdown_connections 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2580622865 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae6410aa50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2580622865 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae64102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2580622865 >> 172.21.15.8:0/2580622865 conn(0x7fae640fb470 msgr2=0x7fae640fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:40.348 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2580622865 shutdown_connections 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2580622865 wait complete. 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 Processor -- start 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.346+0000 7fae6b1f3700 1 -- start start 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae6b1f3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae641965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae6b1f3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae6b1f3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641971c0 con 0x7fae64102570 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae6b1f3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6419af40 con 0x7fae64101c50 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55906/0 (socket says 172.21.15.8:55906) 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 -- 172.21.15.8:0/2325633493 learned_addr learned my addr 172.21.15.8:0/2325633493 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae68f8f700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae641965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 -- 172.21.15.8:0/2325633493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 msgr2=0x7fae641965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae641965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.349 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 -- 172.21.15.8:0/2325633493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500095f0 con 0x7fae64102570 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae68f8f700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae641965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae63fff700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fae50006010 tx=0x7fae5000bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5001d070 con 0x7fae64102570 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fae50003cf0 con 0x7fae64102570 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6419b1c0 con 0x7fae64102570 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.347+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae50021410 con 0x7fae64102570 2024-04-12T06:22:40.350 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.348+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6419b6c0 con 0x7fae64102570 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.349+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae641080f0 con 0x7fae64102570 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.349+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fae50003850 con 0x7fae64102570 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.349+0000 7fae61ffb700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 0x7fae5406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.350+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae5008d740 con 0x7fae64102570 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.350+0000 7fae68f8f700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 0x7fae5406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.351 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.350+0000 7fae68f8f700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 0x7fae5406f1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae58005fd0 tx=0x7fae58005ee0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:40.354 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.354+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fae500577b0 con 0x7fae64102570 2024-04-12T06:22:40.611 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.611+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fae6402cd30 con 0x7fae64102570 2024-04-12T06:22:40.612 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.612+0000 7fae61ffb700 1 -- 172.21.15.8:0/2325633493 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v45) v1 ==== 107+0+4821 (secure 0 0 0) 0x7fae50003e60 con 0x7fae64102570 2024-04-12T06:22:40.612 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:40.612 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34326,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2959889136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2959889136},{"type":"v1","addr":"172.21.15.8:6829","nonce":2959889136}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44299,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/2084066297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2084066297},{"type":"v1","addr":"172.21.15.194:6825","nonce":2084066297}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 msgr2=0x7fae5406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 0x7fae5406f1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fae58005fd0 tx=0x7fae58005ee0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 msgr2=0x7fae64196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fae50006010 tx=0x7fae5000bbf0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 shutdown_connections 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fae5406cce0 0x7fae5406f1a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.614 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae64101c50 0x7fae641965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 --2- 172.21.15.8:0/2325633493 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fae64102570 0x7fae64196ae0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 >> 172.21.15.8:0/2325633493 conn(0x7fae640fb470 msgr2=0x7fae64105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:40.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 shutdown_connections 2024-04-12T06:22:40.615 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.614+0000 7fae6b1f3700 1 -- 172.21.15.8:0/2325633493 wait complete. 2024-04-12T06:22:40.615 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 30 2024-04-12T06:22:40.626 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:40.627 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-12T06:22:40.627 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-12T06:22:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 -- 172.21.15.8:0/271368007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 msgr2=0x7f1f30105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/271368007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f30105d40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009910 tx=0x7f1f18009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 -- 172.21.15.8:0/271368007 shutdown_connections 2024-04-12T06:22:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/271368007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 0x7f1f300ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.755 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/271368007 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f30105d40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 -- 172.21.15.8:0/271368007 >> 172.21.15.8:0/271368007 conn(0x7f1f300fb440 msgr2=0x7f1f300fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:40.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 -- 172.21.15.8:0/271368007 shutdown_connections 2024-04-12T06:22:40.756 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.755+0000 7f1f35ff3700 1 -- 172.21.15.8:0/271368007 wait complete. 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 Processor -- start 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 -- start start 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 0x7f1f30197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30197f20 con 0x7f1f30105960 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f35ff3700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f30198060 con 0x7f1f300ff950 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55930/0 (socket says 172.21.15.8:55930) 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 -- 172.21.15.8:0/3354822868 learned_addr learned my addr 172.21.15.8:0/3354822868 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2f7fe700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 0x7f1f30197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.758 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 -- 172.21.15.8:0/3354822868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 msgr2=0x7f1f30197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 0x7f1f30197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.756+0000 7f1f2effd700 1 -- 172.21.15.8:0/3354822868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f180095f0 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f2effd700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c930 tx=0x7f1f2000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f20007ad0 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f20007c30 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f20018630 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f301a0400 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.757+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f301a08b0 con 0x7f1f30105960 2024-04-12T06:22:40.759 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.758+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f30107f50 con 0x7f1f30105960 2024-04-12T06:22:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.758+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1f2000f450 con 0x7f1f30105960 2024-04-12T06:22:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.759+0000 7f1f2cff9700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 0x7f1f1c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.759+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1f2008c6d0 con 0x7f1f30105960 2024-04-12T06:22:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.759+0000 7f1f2f7fe700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 0x7f1f1c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:40.760 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.759+0000 7f1f2f7fe700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 0x7f1f1c06f1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f1801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:40.763 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:40.763+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1f20056870 con 0x7f1f30105960 2024-04-12T06:22:41.015 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.015+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f1f3002cc70 con 0x7f1f30105960 2024-04-12T06:22:41.016 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.016+0000 7f1f2cff9700 1 -- 172.21.15.8:0/3354822868 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v45) v1 ==== 107+0+4045 (secure 0 0 0) 0x7f1f20059e90 con 0x7f1f30105960 2024-04-12T06:22:41.016 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:41.016 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44299,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/2084066297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2084066297},{"type":"v1","addr":"172.21.15.194:6825","nonce":2084066297}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:41.018 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 msgr2=0x7f1f1c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 0x7f1f1c06f1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1f18006010 tx=0x7f1f1801a040 comp rx=0 tx=0).stop 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 msgr2=0x7f1f301978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c930 tx=0x7f1f2000ccf0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 shutdown_connections 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1f1c06cce0 0x7f1f1c06f1a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f300ff950 0x7f1f30197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 --2- 172.21.15.8:0/3354822868 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1f30105960 0x7f1f301978d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 >> 172.21.15.8:0/3354822868 conn(0x7f1f300fb440 msgr2=0x7f1f300fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 shutdown_connections 2024-04-12T06:22:41.019 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.018+0000 7f1f35ff3700 1 -- 172.21.15.8:0/3354822868 wait complete. 2024-04-12T06:22:41.020 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 31 2024-04-12T06:22:41.031 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:41.031 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-12T06:22:41.031 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-12T06:22:41.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.158+0000 7fe3a27b9700 1 -- 172.21.15.8:0/253968370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 msgr2=0x7fe39c105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.158+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/253968370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c105e80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009910 tx=0x7fe38c009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:41.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 -- 172.21.15.8:0/253968370 shutdown_connections 2024-04-12T06:22:41.159 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/253968370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 0x7fe39c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/253968370 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c105e80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 -- 172.21.15.8:0/253968370 >> 172.21.15.8:0/253968370 conn(0x7fe39c075400 msgr2=0x7fe39c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 -- 172.21.15.8:0/253968370 shutdown_connections 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 -- 172.21.15.8:0/253968370 wait complete. 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 Processor -- start 2024-04-12T06:22:41.160 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.159+0000 7fe3a27b9700 1 -- start start 2024-04-12T06:22:41.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 0x7fe39c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c19d070 con 0x7fe39c105aa0 2024-04-12T06:22:41.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c196520 con 0x7fe39c0ffac0 2024-04-12T06:22:41.162 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55946/0 (socket says 172.21.15.8:55946) 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 -- 172.21.15.8:0/1897222514 learned_addr learned my addr 172.21.15.8:0/1897222514 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39bfff700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 0x7fe39c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 -- 172.21.15.8:0/1897222514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 msgr2=0x7fe39c19c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 0x7fe39c19c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 -- 172.21.15.8:0/1897222514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0095f0 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe39b7fe700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d8d0 tx=0x7fe39000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3900098a0 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe390010460 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39c196800 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe390009ea0 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.160+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39c196d60 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.161+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39c04ea90 con 0x7fe39c105aa0 2024-04-12T06:22:41.163 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.162+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe3900105d0 con 0x7fe39c105aa0 2024-04-12T06:22:41.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.162+0000 7fe3997fa700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 0x7fe38406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.162+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe39008c440 con 0x7fe39c105aa0 2024-04-12T06:22:41.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.162+0000 7fe39bfff700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 0x7fe38406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.164 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.163+0000 7fe39bfff700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 0x7fe38406f1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.166 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.167+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe390059f40 con 0x7fe39c105aa0 2024-04-12T06:22:41.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:41 smithi194 ceph-mon[106982]: pgmap v831: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:41.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:41 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/362899649' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-12T06:22:41.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:41 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2325633493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-12T06:22:41.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:41 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3354822868' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-12T06:22:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:41 smithi008 ceph-mon[146084]: pgmap v831: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:41 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/362899649' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-12T06:22:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:41 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2325633493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-12T06:22:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:41 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3354822868' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-12T06:22:41.420 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.420+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fe39c196f80 con 0x7fe39c105aa0 2024-04-12T06:22:41.421 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.421+0000 7fe3997fa700 1 -- 172.21.15.8:0/1897222514 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v45) v1 ==== 107+0+4821 (secure 0 0 0) 0x7fe390010880 con 0x7fe39c105aa0 2024-04-12T06:22:41.421 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:41.421 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44299,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/2084066297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":2084066297},{"type":"v1","addr":"172.21.15.194:6825","nonce":2084066297}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:41.423 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 msgr2=0x7fe38406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 0x7fe38406f1a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe38c006010 tx=0x7fe38c01a040 comp rx=0 tx=0).stop 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 msgr2=0x7fe39c19c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d8d0 tx=0x7fe39000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 shutdown_connections 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe38406cce0 0x7fe38406f1a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe39c0ffac0 0x7fe39c19c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 --2- 172.21.15.8:0/1897222514 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe39c105aa0 0x7fe39c19c990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 >> 172.21.15.8:0/1897222514 conn(0x7fe39c075400 msgr2=0x7fe39c067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 shutdown_connections 2024-04-12T06:22:41.424 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.423+0000 7fe3a27b9700 1 -- 172.21.15.8:0/1897222514 wait complete. 2024-04-12T06:22:41.425 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 32 2024-04-12T06:22:41.436 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:41.436 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-12T06:22:41.436 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-12T06:22:41.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 -- 172.21.15.8:0/4259477155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc105990 msgr2=0x7fc9bc105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 --2- 172.21.15.8:0/4259477155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc105990 0x7fc9bc105d70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac009910 tx=0x7fc9ac009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:41.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 -- 172.21.15.8:0/4259477155 shutdown_connections 2024-04-12T06:22:41.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 --2- 172.21.15.8:0/4259477155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc0ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.565 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 --2- 172.21.15.8:0/4259477155 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc105990 0x7fc9bc105d70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 -- 172.21.15.8:0/4259477155 >> 172.21.15.8:0/4259477155 conn(0x7fc9bc0fb470 msgr2=0x7fc9bc0fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 -- 172.21.15.8:0/4259477155 shutdown_connections 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.563+0000 7fc9c236f700 1 -- 172.21.15.8:0/4259477155 wait complete. 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 Processor -- start 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 -- start start 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc105990 0x7fc9bc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc198020 con 0x7fc9bc0ff980 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9c236f700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc198190 con 0x7fc9bc105990 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55958/0 (socket says 172.21.15.8:55958) 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 -- 172.21.15.8:0/1615076614 learned_addr learned my addr 172.21.15.8:0/1615076614 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bb7fe700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc105990 0x7fc9bc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.566 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 -- 172.21.15.8:0/1615076614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc105990 msgr2=0x7fc9bc197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc105990 0x7fc9bc197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.564+0000 7fc9bbfff700 1 -- 172.21.15.8:0/1615076614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0095f0 con 0x7fc9bc0ff980 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9bbfff700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00b5c0 tx=0x7fc9ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9ac01d070 con 0x7fc9bc0ff980 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9ac005e00 con 0x7fc9bc0ff980 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc1a0470 con 0x7fc9bc0ff980 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9ac00fe60 con 0x7fc9bc0ff980 2024-04-12T06:22:41.567 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.565+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc1a0970 con 0x7fc9bc0ff980 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.566+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc107f80 con 0x7fc9bc0ff980 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.566+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fc9ac00f6d0 con 0x7fc9bc0ff980 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.567+0000 7fc9b97fa700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 0x7fc9a406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.567+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc9ac08d8f0 con 0x7fc9bc0ff980 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.567+0000 7fc9bb7fe700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 0x7fc9a406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.568 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.567+0000 7fc9bb7fe700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 0x7fc9a406f1a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005fd0 tx=0x7fc9b0005f60 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.571 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.571+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fc9ac0578c0 con 0x7fc9bc0ff980 2024-04-12T06:22:41.824 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.824+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fc9bc04ea90 con 0x7fc9bc0ff980 2024-04-12T06:22:41.825 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.825+0000 7fc9b97fa700 1 -- 172.21.15.8:0/1615076614 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v45) v1 ==== 107+0+4039 (secure 0 0 0) 0x7fc9ac05aee0 con 0x7fc9bc0ff980 2024-04-12T06:22:41.825 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:41.825 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:41.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 msgr2=0x7fc9a406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 0x7fc9a406f1a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005fd0 tx=0x7fc9b0005f60 comp rx=0 tx=0).stop 2024-04-12T06:22:41.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 msgr2=0x7fc9bc197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00b5c0 tx=0x7fc9ac00be20 comp rx=0 tx=0).stop 2024-04-12T06:22:41.828 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 shutdown_connections 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fc9a406cce0 0x7fc9a406f1a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fc9bc0ff980 0x7fc9bc197400 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 --2- 172.21.15.8:0/1615076614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9bc105990 0x7fc9bc197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 >> 172.21.15.8:0/1615076614 conn(0x7fc9bc0fb470 msgr2=0x7fc9bc0fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 shutdown_connections 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.827+0000 7fc9c236f700 1 -- 172.21.15.8:0/1615076614 wait complete. 2024-04-12T06:22:41.829 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 33 2024-04-12T06:22:41.840 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:41.840 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-04-12T06:22:41.840 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-12T06:22:41.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.969+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3282363420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 msgr2=0x7f9ff00fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.969+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3282363420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff00fff20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009910 tx=0x7f9fe4009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:41.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.969+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3282363420 shutdown_connections 2024-04-12T06:22:41.974 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.969+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3282363420 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff00fff20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3282363420 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 0x7f9ff0105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3282363420 >> 172.21.15.8:0/3282363420 conn(0x7f9ff0075400 msgr2=0x7f9ff0075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3282363420 shutdown_connections 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3282363420 wait complete. 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 Processor -- start 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- start start 2024-04-12T06:22:41.975 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 0x7f9ff0101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff01020e0 con 0x7f9ff00ffac0 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.970+0000 7f9ff49ca700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff0102220 con 0x7f9ff0105aa0 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:55982/0 (socket says 172.21.15.8:55982) 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 -- 172.21.15.8:0/3708251252 learned_addr learned my addr 172.21.15.8:0/3708251252 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fedd9b700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 0x7f9ff0101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 -- 172.21.15.8:0/3708251252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 msgr2=0x7f9ff0101ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 0x7f9ff0101ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:41.976 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 -- 172.21.15.8:0/3708251252 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40095f0 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fee59c700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d8d0 tx=0x7f9fe000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe0009940 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9fe0010460 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fe000f5f0 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff01a2860 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.971+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff01a2c80 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.972+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff0109030 con 0x7f9ff00ffac0 2024-04-12T06:22:41.977 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.973+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f9fe00105d0 con 0x7f9ff00ffac0 2024-04-12T06:22:41.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.973+0000 7f9fdf7fe700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 0x7f9fd806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:41.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.973+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9fe008c550 con 0x7f9ff00ffac0 2024-04-12T06:22:41.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.973+0000 7f9fedd9b700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 0x7f9fd806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:41.978 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.974+0000 7f9fedd9b700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 0x7f9fd806f1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0102e50 tx=0x7f9fe401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:41.979 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:41.977+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f9fe00565b0 con 0x7f9ff00ffac0 2024-04-12T06:22:42.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:42 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1897222514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-12T06:22:42.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:42 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1615076614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-12T06:22:42.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.237+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f9ff004ea90 con 0x7f9ff00ffac0 2024-04-12T06:22:42.237 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.237+0000 7f9fdf7fe700 1 -- 172.21.15.8:0/3708251252 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v45) v1 ==== 107+0+4818 (secure 0 0 0) 0x7f9fe0010880 con 0x7f9ff00ffac0 2024-04-12T06:22:42.238 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:42.238 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44291,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/1928513879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":1928513879},{"type":"v1","addr":"172.21.15.194:6827","nonce":1928513879}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:42.239 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 msgr2=0x7f9fd806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 0x7f9fd806f1a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0102e50 tx=0x7f9fe401a040 comp rx=0 tx=0).stop 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 msgr2=0x7f9ff0101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000d8d0 tx=0x7f9fe000dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 shutdown_connections 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f9fd806cce0 0x7f9fd806f1a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9ff00ffac0 0x7f9ff0101660 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 --2- 172.21.15.8:0/3708251252 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9ff0105aa0 0x7f9ff0101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 >> 172.21.15.8:0/3708251252 conn(0x7f9ff0075400 msgr2=0x7f9ff00680b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:42.240 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 shutdown_connections 2024-04-12T06:22:42.241 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.239+0000 7f9ff49ca700 1 -- 172.21.15.8:0/3708251252 wait complete. 2024-04-12T06:22:42.241 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 34 2024-04-12T06:22:42.252 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:42.252 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-04-12T06:22:42.252 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-12T06:22:42.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:42 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1897222514' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-12T06:22:42.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:42 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1615076614' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-12T06:22:42.381 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1583984836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb0105aa0 msgr2=0x7f6cb0105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1583984836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0105e80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009910 tx=0x7f6cac009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1583984836 shutdown_connections 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1583984836 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb00ffac0 0x7f6cb00fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1583984836 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0105e80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1583984836 >> 172.21.15.8:0/1583984836 conn(0x7f6cb0075400 msgr2=0x7f6cb0075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1583984836 shutdown_connections 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.379+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1583984836 wait complete. 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 Processor -- start 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 -- start start 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.382 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb01961f0 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb83a2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0199f70 con 0x7f6cb0105aa0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb613e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb613e700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56010/0 (socket says 172.21.15.8:56010) 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.380+0000 7f6cb613e700 1 -- 172.21.15.8:0/1818094502 learned_addr learned my addr 172.21.15.8:0/1818094502 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb593d700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb613e700 1 -- 172.21.15.8:0/1818094502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb0105aa0 msgr2=0x7f6cb0195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb613e700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb613e700 1 -- 172.21.15.8:0/1818094502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0095f0 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb613e700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00b5c0 tx=0x7f6cac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac01f070 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6cac019610 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb019a1f0 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cac00f690 con 0x7f6cb00ffac0 2024-04-12T06:22:42.383 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.381+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb019a6f0 con 0x7f6cb00ffac0 2024-04-12T06:22:42.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.382+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb004ea90 con 0x7f6cb00ffac0 2024-04-12T06:22:42.384 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.382+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f6cac003b20 con 0x7f6cb00ffac0 2024-04-12T06:22:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.383+0000 7f6ca77fe700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 0x7f6c9c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.383+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6cac08fb90 con 0x7f6cb00ffac0 2024-04-12T06:22:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.383+0000 7f6cb593d700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 0x7f6c9c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.385 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.383+0000 7f6cb593d700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 0x7f6c9c06f1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0005fd0 tx=0x7f6ca000a560 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:42.387 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.387+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f6cac059c00 con 0x7f6cb00ffac0 2024-04-12T06:22:42.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.642+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f6cb019a910 con 0x7f6cb00ffac0 2024-04-12T06:22:42.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.642+0000 7f6ca77fe700 1 -- 172.21.15.8:0/1818094502 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v45) v1 ==== 107+0+4036 (secure 0 0 0) 0x7f6cac019780 con 0x7f6cb00ffac0 2024-04-12T06:22:42.643 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:42.643 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 msgr2=0x7f6c9c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 0x7f6c9c06f1a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0005fd0 tx=0x7f6ca000a560 comp rx=0 tx=0).stop 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 msgr2=0x7f6cb01955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00b5c0 tx=0x7f6cac00be20 comp rx=0 tx=0).stop 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 shutdown_connections 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f6c9c06cce0 0x7f6c9c06f1a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f6cb00ffac0 0x7f6cb01955d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 --2- 172.21.15.8:0/1818094502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6cb0105aa0 0x7f6cb0195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.644+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 >> 172.21.15.8:0/1818094502 conn(0x7f6cb0075400 msgr2=0x7f6cb00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:42.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.645+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 shutdown_connections 2024-04-12T06:22:42.646 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.645+0000 7f6cb83a2700 1 -- 172.21.15.8:0/1818094502 wait complete. 2024-04-12T06:22:42.646 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 35 2024-04-12T06:22:42.657 DEBUG:tasks.fs:max_mds not reduced in epoch 35 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:42.657 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-04-12T06:22:42.657 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 -- 172.21.15.8:0/454987622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc105960 msgr2=0x7f65cc105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 --2- 172.21.15.8:0/454987622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc105960 0x7f65cc105d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009910 tx=0x7f65c8009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 -- 172.21.15.8:0/454987622 shutdown_connections 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 --2- 172.21.15.8:0/454987622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc0ff950 0x7f65cc0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 --2- 172.21.15.8:0/454987622 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc105960 0x7f65cc105d40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 -- 172.21.15.8:0/454987622 >> 172.21.15.8:0/454987622 conn(0x7f65cc0fb440 msgr2=0x7f65cc0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 -- 172.21.15.8:0/454987622 shutdown_connections 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.784+0000 7f65d3503700 1 -- 172.21.15.8:0/454987622 wait complete. 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 Processor -- start 2024-04-12T06:22:42.785 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 -- start start 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 0x7f65cc1959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1960b0 con 0x7f65cc0ff950 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d3503700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc199e30 con 0x7f65cc105960 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56026/0 (socket says 172.21.15.8:56026) 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 -- 172.21.15.8:0/4161899292 learned_addr learned my addr 172.21.15.8:0/4161899292 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d0a9e700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 0x7f65cc1959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.789 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 -- 172.21.15.8:0/4161899292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 msgr2=0x7f65cc1959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 0x7f65cc1959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d129f700 1 -- 172.21.15.8:0/4161899292 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80095f0 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.785+0000 7f65d0a9e700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 0x7f65cc1959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65d129f700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c801d070 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc19a0b0 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f65c80044c0 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c80039d0 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.786+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc19a550 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.787+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc107f50 con 0x7f65cc0ff950 2024-04-12T06:22:42.790 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.787+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f65c80048e0 con 0x7f65cc0ff950 2024-04-12T06:22:42.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.788+0000 7f65c27fc700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 0x7f65b806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:42.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.788+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f65c808dc00 con 0x7f65cc0ff950 2024-04-12T06:22:42.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.788+0000 7f65d0a9e700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 0x7f65b806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:42.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.788+0000 7f65d0a9e700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 0x7f65b806f1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f65cc196aa0 tx=0x7f65bc009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:42.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:42.792+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f65c8093020 con 0x7f65cc0ff950 2024-04-12T06:22:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.044+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f65cc1968e0 con 0x7f65cc0ff950 2024-04-12T06:22:43.044 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.044+0000 7f65c27fc700 1 -- 172.21.15.8:0/4161899292 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v45) v1 ==== 107+0+4818 (secure 0 0 0) 0x7f65cc1968e0 con 0x7f65cc0ff950 2024-04-12T06:22:43.044 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:43.045 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32},{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":27,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:03:56.166804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 msgr2=0x7f65b806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 0x7f65b806f1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f65cc196aa0 tx=0x7f65bc009260 comp rx=0 tx=0).stop 2024-04-12T06:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 msgr2=0x7f65cc195490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.047 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f65c8000c00 tx=0x7f65c800be90 comp rx=0 tx=0).stop 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 shutdown_connections 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f65b806cce0 0x7f65b806f1a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f65cc0ff950 0x7f65cc195490 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 --2- 172.21.15.8:0/4161899292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65cc105960 0x7f65cc1959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.046+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 >> 172.21.15.8:0/4161899292 conn(0x7f65cc0fb440 msgr2=0x7f65cc0fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.047+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 shutdown_connections 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.047+0000 7f65d3503700 1 -- 172.21.15.8:0/4161899292 wait complete. 2024-04-12T06:22:43.048 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 36 2024-04-12T06:22:43.060 DEBUG:tasks.fs:max_mds not reduced in epoch 36 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-12T06:22:43.060 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-04-12T06:22:43.060 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 -- 172.21.15.8:0/4059990642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380ff980 msgr2=0x7fd6380ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/4059990642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380ff980 0x7fd6380ffe00 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd62c009910 tx=0x7fd62c009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 -- 172.21.15.8:0/4059990642 shutdown_connections 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/4059990642 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd6380ff980 0x7fd6380ffe00 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/4059990642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd638105990 0x7fd638105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 -- 172.21.15.8:0/4059990642 >> 172.21.15.8:0/4059990642 conn(0x7fd6380fb470 msgr2=0x7fd6380fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 -- 172.21.15.8:0/4059990642 shutdown_connections 2024-04-12T06:22:43.191 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.187+0000 7fd63f3e5700 1 -- 172.21.15.8:0/4059990642 wait complete. 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 Processor -- start 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 -- start start 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 0x7fd6381974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381980a0 con 0x7fd638105990 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63f3e5700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638198210 con 0x7fd6380ff980 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:56036/0 (socket says 172.21.15.8:56036) 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 -- 172.21.15.8:0/2292534681 learned_addr learned my addr 172.21.15.8:0/2292534681 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63d181700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 0x7fd6381974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 -- 172.21.15.8:0/2292534681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 msgr2=0x7fd6381974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.192 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 0x7fd6381974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 -- 172.21.15.8:0/2292534681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0095f0 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63d181700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 0x7fd6381974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.188+0000 7fd63c980700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.189+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c01d070 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.189+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd62c00f460 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.189+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6381a0540 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.189+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c021640 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.189+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6381a09f0 con 0x7fd638105990 2024-04-12T06:22:43.193 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.190+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd638107fa0 con 0x7fd638105990 2024-04-12T06:22:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.190+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fd62c00fa60 con 0x7fd638105990 2024-04-12T06:22:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.190+0000 7fd62a7fc700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 0x7fd62406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.191+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd62c08d7c0 con 0x7fd638105990 2024-04-12T06:22:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.191+0000 7fd63d181700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 0x7fd62406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.194 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.191+0000 7fd63d181700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 0x7fd62406f1a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd634007900 tx=0x7fd634008040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:43.195 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.195+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fd62c0578b0 con 0x7fd638105990 2024-04-12T06:22:43.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:43 smithi194 ceph-mon[106982]: pgmap v832: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:43.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:43 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3708251252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-12T06:22:43.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:43 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1818094502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-12T06:22:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:43 smithi008 ceph-mon[146084]: pgmap v832: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:43 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3708251252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-12T06:22:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:43 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1818094502' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-12T06:22:43.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.450+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fd63804ea90 con 0x7fd638105990 2024-04-12T06:22:43.450 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.450+0000 7fd62a7fc700 1 -- 172.21.15.8:0/2292534681 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v45) v1 ==== 107+0+4840 (secure 0 0 0) 0x7fd62c05aed0 con 0x7fd638105990 2024-04-12T06:22:43.450 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:43.450 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:19:58.263577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 msgr2=0x7fd62406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 0x7fd62406f1a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd634007900 tx=0x7fd634008040 comp rx=0 tx=0).stop 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 msgr2=0x7fd638197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006010 tx=0x7fd62c00be20 comp rx=0 tx=0).stop 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 shutdown_connections 2024-04-12T06:22:43.453 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.452+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fd62406cce0 0x7fd62406f1a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.453+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd6380ff980 0x7fd6381974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.453+0000 7fd63f3e5700 1 --2- 172.21.15.8:0/2292534681 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fd638105990 0x7fd638197a10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.453+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 >> 172.21.15.8:0/2292534681 conn(0x7fd6380fb470 msgr2=0x7fd6380fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.453+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 shutdown_connections 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.453+0000 7fd63f3e5700 1 -- 172.21.15.8:0/2292534681 wait complete. 2024-04-12T06:22:43.454 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 37 2024-04-12T06:22:43.465 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-04-12T06:22:43.465 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-12T06:22:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.592+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/401840194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 msgr2=0x7f7ed410aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/401840194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed410aaa0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009940 tx=0x7f7ec4009c50 comp rx=0 tx=0).stop 2024-04-12T06:22:43.595 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/401840194 shutdown_connections 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/401840194 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed410aaa0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/401840194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 0x7f7ed4102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/401840194 >> 172.21.15.8:0/401840194 conn(0x7f7ed40fb420 msgr2=0x7f7ed40fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/401840194 shutdown_connections 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/401840194 wait complete. 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 Processor -- start 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.593+0000 7f7ed9ce0700 1 -- start start 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 0x7f7ed41965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed4197140 con 0x7f7ed41025c0 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed4197280 con 0x7f7ed4101ca0 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44904/0 (socket says 172.21.15.8:44904) 2024-04-12T06:22:43.596 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 -- 172.21.15.8:0/3655514179 learned_addr learned my addr 172.21.15.8:0/3655514179 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 -- 172.21.15.8:0/3655514179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 msgr2=0x7f7ed41965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed37fe700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 0x7f7ed41965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 0x7f7ed41965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 -- 172.21.15.8:0/3655514179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40095f0 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed2ffd700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006010 tx=0x7f7ec40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec401d070 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7ec400f460 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed419b090 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ec4021620 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.594+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed419b590 con 0x7f7ed41025c0 2024-04-12T06:22:43.597 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.595+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed4108140 con 0x7f7ed41025c0 2024-04-12T06:22:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.596+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f7ec4003a60 con 0x7f7ed41025c0 2024-04-12T06:22:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.596+0000 7f7ed0ff9700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 0x7f7ec006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.596+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7ec408d8f0 con 0x7f7ed41025c0 2024-04-12T06:22:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.596+0000 7f7ed37fe700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 0x7f7ec006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:43.598 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.597+0000 7f7ed37fe700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 0x7f7ec006f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00ba60 tx=0x7f7ebc005d50 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:43.601 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.601+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f7ec4057960 con 0x7f7ed41025c0 2024-04-12T06:22:43.855 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.855+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f7ed402cc70 con 0x7f7ed41025c0 2024-04-12T06:22:43.856 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.856+0000 7f7ed0ff9700 1 -- 172.21.15.8:0/3655514179 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v45) v1 ==== 107+0+4839 (secure 0 0 0) 0x7f7ec405af80 con 0x7f7ed41025c0 2024-04-12T06:22:43.856 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:43.856 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:19:59.265123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:43.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 msgr2=0x7f7ec006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 0x7f7ec006f1a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00ba60 tx=0x7f7ebc005d50 comp rx=0 tx=0).stop 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 msgr2=0x7f7ed4196af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006010 tx=0x7f7ec40056a0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 shutdown_connections 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f7ec006cce0 0x7f7ec006f1a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed4101ca0 0x7f7ed41965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 --2- 172.21.15.8:0/3655514179 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f7ed41025c0 0x7f7ed4196af0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 >> 172.21.15.8:0/3655514179 conn(0x7f7ed40fb420 msgr2=0x7f7ed4105280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 shutdown_connections 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.858+0000 7f7ed9ce0700 1 -- 172.21.15.8:0/3655514179 wait complete. 2024-04-12T06:22:43.859 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 38 2024-04-12T06:22:43.871 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- 172.21.15.8:0/981389853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 msgr2=0x7fef3c068210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 --2- 172.21.15.8:0/981389853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c068210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009910 tx=0x7fef2c009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- 172.21.15.8:0/981389853 shutdown_connections 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 --2- 172.21.15.8:0/981389853 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c068210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 --2- 172.21.15.8:0/981389853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.000 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- 172.21.15.8:0/981389853 >> 172.21.15.8:0/981389853 conn(0x7fef3c075400 msgr2=0x7fef3c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- 172.21.15.8:0/981389853 shutdown_connections 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- 172.21.15.8:0/981389853 wait complete. 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 Processor -- start 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.998+0000 7fef43b17700 1 -- start start 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef43b17700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef43b17700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef43b17700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1972c0 con 0x7fef3c067d90 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef43b17700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c199fe0 con 0x7fef3c106b90 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44936/0 (socket says 172.21.15.8:44936) 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 -- 172.21.15.8:0/541449592 learned_addr learned my addr 172.21.15.8:0/541449592 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef410b2700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 -- 172.21.15.8:0/541449592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 msgr2=0x7fef3c196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 -- 172.21.15.8:0/541449592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c0095f0 con 0x7fef3c067d90 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef410b2700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef418b3700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d8d0 tx=0x7fef3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef38009940 con 0x7fef3c067d90 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c19a2c0 con 0x7fef3c067d90 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fef38010460 con 0x7fef3c067d90 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:43.999+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef3800f5f0 con 0x7fef3c067d90 2024-04-12T06:22:44.002 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.000+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c19a820 con 0x7fef3c067d90 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.001+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c109130 con 0x7fef3c067d90 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.001+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fef380105d0 con 0x7fef3c067d90 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.001+0000 7fef32ffd700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 0x7fef2806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.002+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fef3808c5d0 con 0x7fef3c067d90 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.002+0000 7fef410b2700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 0x7fef2806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.003 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.002+0000 7fef410b2700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 0x7fef2806f1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef3c197d00 tx=0x7fef2c01a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.006 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.006+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fef38056640 con 0x7fef3c067d90 2024-04-12T06:22:44.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:44 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/4161899292' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-12T06:22:44.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:44 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2292534681' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-12T06:22:44.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:44 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3655514179' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-12T06:22:44.264 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.264+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fef3c04ea90 con 0x7fef3c067d90 2024-04-12T06:22:44.265 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.265+0000 7fef32ffd700 1 -- 172.21.15.8:0/541449592 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v45) v1 ==== 107+0+4842 (secure 0 0 0) 0x7fef38010880 con 0x7fef3c067d90 2024-04-12T06:22:44.265 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:44.265 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:05.447136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:replay","state_seq":1,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:resolve","state_seq":236,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 msgr2=0x7fef2806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 0x7fef2806f1a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fef3c197d00 tx=0x7fef2c01a040 comp rx=0 tx=0).stop 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 msgr2=0x7fef3c1966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d8d0 tx=0x7fef3800dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 shutdown_connections 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fef2806cce0 0x7fef2806f1a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fef3c067d90 0x7fef3c1966f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 --2- 172.21.15.8:0/541449592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef3c106b90 0x7fef3c196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 >> 172.21.15.8:0/541449592 conn(0x7fef3c075400 msgr2=0x7fef3c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.267+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 shutdown_connections 2024-04-12T06:22:44.268 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.268+0000 7fef43b17700 1 -- 172.21.15.8:0/541449592 wait complete. 2024-04-12T06:22:44.269 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 39 2024-04-12T06:22:44.280 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 40 2024-04-12T06:22:44.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:44 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/4161899292' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-12T06:22:44.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:44 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2292534681' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-12T06:22:44.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:44 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3655514179' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 -- 172.21.15.8:0/3138322023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 msgr2=0x7f9308105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 --2- 172.21.15.8:0/3138322023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f9308105f20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9304009910 tx=0x7f9304009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 -- 172.21.15.8:0/3138322023 shutdown_connections 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 --2- 172.21.15.8:0/3138322023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 0x7f93081006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 --2- 172.21.15.8:0/3138322023 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f9308105f20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 -- 172.21.15.8:0/3138322023 >> 172.21.15.8:0/3138322023 conn(0x7f9308074c50 msgr2=0x7f9308075060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 -- 172.21.15.8:0/3138322023 shutdown_connections 2024-04-12T06:22:44.408 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.407+0000 7f93103c2700 1 -- 172.21.15.8:0/3138322023 wait complete. 2024-04-12T06:22:44.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.408+0000 7f93103c2700 1 Processor -- start 2024-04-12T06:22:44.410 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.408+0000 7f93103c2700 1 -- start start 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.408+0000 7f93103c2700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 0x7f93081954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f93103c2700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f93103c2700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308196040 con 0x7f9308105b40 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f93103c2700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9308196180 con 0x7f9308100260 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44956/0 (socket says 172.21.15.8:44956) 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 -- 172.21.15.8:0/2209814224 learned_addr learned my addr 172.21.15.8:0/2209814224 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930e15e700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 0x7f93081954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 -- 172.21.15.8:0/2209814224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 msgr2=0x7f93081954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 0x7f93081954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 -- 172.21.15.8:0/2209814224 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040095f0 con 0x7f9308105b40 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f930d95d700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f92f800c930 tx=0x7f92f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f8007ad0 con 0x7f9308105b40 2024-04-12T06:22:44.411 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f92f800ce80 con 0x7f9308105b40 2024-04-12T06:22:44.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9308199ff0 con 0x7f9308105b40 2024-04-12T06:22:44.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f8018630 con 0x7f9308105b40 2024-04-12T06:22:44.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.409+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930819a550 con 0x7f9308105b40 2024-04-12T06:22:44.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.410+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9308102da0 con 0x7f9308105b40 2024-04-12T06:22:44.412 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.411+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f92f8007c30 con 0x7f9308105b40 2024-04-12T06:22:44.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.411+0000 7f92ff7fe700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 0x7f92f406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.411+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f92f808c540 con 0x7f9308105b40 2024-04-12T06:22:44.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.411+0000 7f930e15e700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 0x7f92f406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.413 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.412+0000 7f930e15e700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 0x7f92f406f1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9304006010 tx=0x7f930401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.416 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.416+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f92f8056590 con 0x7f9308105b40 2024-04-12T06:22:44.668 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.668+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f930804ea90 con 0x7f9308105b40 2024-04-12T06:22:44.669 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.669+0000 7f92ff7fe700 1 -- 172.21.15.8:0/2209814224 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v45) v1 ==== 107+0+4845 (secure 0 0 0) 0x7f92f8054e60 con 0x7f9308105b40 2024-04-12T06:22:44.669 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:44.669 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:06.449025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:resolve","state_seq":242,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:resolve","state_seq":236,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:44.671 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 msgr2=0x7f92f406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 0x7f92f406f1a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9304006010 tx=0x7f930401a040 comp rx=0 tx=0).stop 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 msgr2=0x7f93081959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f92f800c930 tx=0x7f92f800cc40 comp rx=0 tx=0).stop 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 shutdown_connections 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f92f406cce0 0x7f92f406f1a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9308100260 0x7f93081954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 --2- 172.21.15.8:0/2209814224 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f9308105b40 0x7f93081959f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 >> 172.21.15.8:0/2209814224 conn(0x7f9308074c50 msgr2=0x7f93080fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 shutdown_connections 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.671+0000 7f93103c2700 1 -- 172.21.15.8:0/2209814224 wait complete. 2024-04-12T06:22:44.672 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 40 2024-04-12T06:22:44.683 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 41 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3924322737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 msgr2=0x7f2c28105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3924322737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28105e80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009910 tx=0x7f2c1c009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3924322737 shutdown_connections 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3924322737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 0x7f2c280fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3924322737 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28105e80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3924322737 >> 172.21.15.8:0/3924322737 conn(0x7f2c28075400 msgr2=0x7f2c28075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:44.813 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3924322737 shutdown_connections 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.811+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3924322737 wait complete. 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 Processor -- start 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 -- start start 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 0x7f2c28101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c28102170 con 0x7f2c28105aa0 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2ef87700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c281022e0 con 0x7f2c280ffac0 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44972/0 (socket says 172.21.15.8:44972) 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 -- 172.21.15.8:0/3725099147 learned_addr learned my addr 172.21.15.8:0/3725099147 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c2cd23700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 0x7f2c28101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 -- 172.21.15.8:0/3725099147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 msgr2=0x7f2c28101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:44.814 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 0x7f2c28101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 -- 172.21.15.8:0/3725099147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0095f0 con 0x7f2c28105aa0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c27fff700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400e920 tx=0x7f2c1400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.812+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1400c9d0 con 0x7f2c28105aa0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.813+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2c14010430 con 0x7f2c28105aa0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.813+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c281a28c0 con 0x7f2c28105aa0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.813+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c14018650 con 0x7f2c28105aa0 2024-04-12T06:22:44.815 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.813+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c281a2d40 con 0x7f2c28105aa0 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.814+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2804ea90 con 0x7f2c28105aa0 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.814+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f2c140105a0 con 0x7f2c28105aa0 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.815+0000 7f2c25ffb700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 0x7f2c1806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.815+0000 7f2c2cd23700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 0x7f2c1806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.815+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2c14014070 con 0x7f2c28105aa0 2024-04-12T06:22:44.816 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.815+0000 7f2c2cd23700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 0x7f2c1806f1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b6e0 tx=0x7f2c1c005a90 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:44.819 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:44.819+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f2c14052ab0 con 0x7f2c28105aa0 2024-04-12T06:22:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.071+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7f2c281a2f60 con 0x7f2c28105aa0 2024-04-12T06:22:45.071 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.071+0000 7f2c25ffb700 1 -- 172.21.15.8:0/3725099147 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v45) v1 ==== 107+0+4849 (secure 0 0 0) 0x7f2c140568d0 con 0x7f2c28105aa0 2024-04-12T06:22:45.072 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:45.072 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:07.454444+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:reconnect","state_seq":243,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:reconnect","state_seq":237,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 msgr2=0x7f2c1806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 0x7f2c1806f1a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b6e0 tx=0x7f2c1c005a90 comp rx=0 tx=0).stop 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 msgr2=0x7f2c28101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400e920 tx=0x7f2c1400ec30 comp rx=0 tx=0).stop 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 shutdown_connections 2024-04-12T06:22:45.074 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f2c1806cce0 0x7f2c1806f1a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c280ffac0 0x7f2c28101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 --2- 172.21.15.8:0/3725099147 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2c28105aa0 0x7f2c28101ba0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 >> 172.21.15.8:0/3725099147 conn(0x7f2c28075400 msgr2=0x7f2c28067e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 shutdown_connections 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.074+0000 7f2c2ef87700 1 -- 172.21.15.8:0/3725099147 wait complete. 2024-04-12T06:22:45.075 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 41 2024-04-12T06:22:45.086 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 42 2024-04-12T06:22:45.215 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 -- 172.21.15.8:0/1095619839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3260105aa0 msgr2=0x7f3260105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 --2- 172.21.15.8:0/1095619839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3260105aa0 0x7f3260105e80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3250009910 tx=0x7f3250009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 -- 172.21.15.8:0/1095619839 shutdown_connections 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 --2- 172.21.15.8:0/1095619839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f32600ffac0 0x7f32600fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 --2- 172.21.15.8:0/1095619839 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f3260105aa0 0x7f3260105e80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 -- 172.21.15.8:0/1095619839 >> 172.21.15.8:0/1095619839 conn(0x7f3260075400 msgr2=0x7f3260075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 -- 172.21.15.8:0/1095619839 shutdown_connections 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.213+0000 7f3265a08700 1 -- 172.21.15.8:0/1095619839 wait complete. 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 Processor -- start 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 -- start start 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3260105aa0 0x7f3260195b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601961f0 con 0x7f32600ffac0 2024-04-12T06:22:45.216 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.214+0000 7f3265a08700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260199f70 con 0x7f3260105aa0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:44992/0 (socket says 172.21.15.8:44992) 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 -- 172.21.15.8:0/2446428167 learned_addr learned my addr 172.21.15.8:0/2446428167 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325e7fc700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3260105aa0 0x7f3260195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 -- 172.21.15.8:0/2446428167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3260105aa0 msgr2=0x7f3260195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3260105aa0 0x7f3260195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 -- 172.21.15.8:0/2446428167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500095f0 con 0x7f32600ffac0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f325effd700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f325000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f325001d070 con 0x7f32600ffac0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326019a1f0 con 0x7f32600ffac0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3250005dc0 con 0x7f32600ffac0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3250003940 con 0x7f32600ffac0 2024-04-12T06:22:45.217 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.215+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326019a690 con 0x7f32600ffac0 2024-04-12T06:22:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.216+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3260108040 con 0x7f32600ffac0 2024-04-12T06:22:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.217+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f3250004ae0 con 0x7f32600ffac0 2024-04-12T06:22:45.218 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.217+0000 7f3264a06700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 0x7f324806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.217+0000 7f325e7fc700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 0x7f324806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.217+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f325008db60 con 0x7f32600ffac0 2024-04-12T06:22:45.219 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.218+0000 7f325e7fc700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 0x7f324806f1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3260196be0 tx=0x7f3254009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:45.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.222+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f3250057bd0 con 0x7f32600ffac0 2024-04-12T06:22:45.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:45 smithi194 ceph-mon[106982]: pgmap v833: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:45.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:45 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/541449592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-12T06:22:45.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:45 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2209814224' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-12T06:22:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:45 smithi008 ceph-mon[146084]: pgmap v833: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:45 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/541449592' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-04-12T06:22:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:45 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2209814224' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-04-12T06:22:45.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.476+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 42, "format": "json"} v 0) v1 -- 0x7f32601969c0 con 0x7f32600ffac0 2024-04-12T06:22:45.476 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.476+0000 7f3264a06700 1 -- 172.21.15.8:0/2446428167 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 42, "format": "json"}]=0 dumped fsmap epoch 42 v45) v1 ==== 107+0+4843 (secure 0 0 0) 0x7f325005b1f0 con 0x7f32600ffac0 2024-04-12T06:22:45.477 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:45.477 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:08.462299+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:rejoin","state_seq":244,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:rejoin","state_seq":238,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:45.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 msgr2=0x7f324806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.479 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 0x7f324806f1a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3260196be0 tx=0x7f3254009260 comp rx=0 tx=0).stop 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 msgr2=0x7f32601955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f325000bde0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 shutdown_connections 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f324806cce0 0x7f324806f1a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f32600ffac0 0x7f32601955d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 --2- 172.21.15.8:0/2446428167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3260105aa0 0x7f3260195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 >> 172.21.15.8:0/2446428167 conn(0x7f3260075400 msgr2=0x7f32600febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 shutdown_connections 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.479+0000 7f3265a08700 1 -- 172.21.15.8:0/2446428167 wait complete. 2024-04-12T06:22:45.480 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 42 2024-04-12T06:22:45.492 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 43 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- 172.21.15.8:0/4073869340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 msgr2=0x7f2730106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 --2- 172.21.15.8:0/4073869340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730106e50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2720009910 tx=0x7f2720009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- 172.21.15.8:0/4073869340 shutdown_connections 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 --2- 172.21.15.8:0/4073869340 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 0x7f2730100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 --2- 172.21.15.8:0/4073869340 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730106e50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- 172.21.15.8:0/4073869340 >> 172.21.15.8:0/4073869340 conn(0x7f27300fc570 msgr2=0x7f27300fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- 172.21.15.8:0/4073869340 shutdown_connections 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- 172.21.15.8:0/4073869340 wait complete. 2024-04-12T06:22:45.621 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 Processor -- start 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.619+0000 7f2738570700 1 -- start start 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2738570700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 0x7f2730196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2738570700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2738570700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730197260 con 0x7f2730106a70 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2738570700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730199f80 con 0x7f2730100a60 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45002/0 (socket says 172.21.15.8:45002) 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 -- 172.21.15.8:0/3128304472 learned_addr learned my addr 172.21.15.8:0/3128304472 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f273630c700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 0x7f2730196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 -- 172.21.15.8:0/3128304472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 msgr2=0x7f2730196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 0x7f2730196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 -- 172.21.15.8:0/3128304472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27200095f0 con 0x7f2730106a70 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2735b0b700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f272c00b4b0 tx=0x7f272c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:45.622 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f272c011800 con 0x7f2730106a70 2024-04-12T06:22:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f272c011e00 con 0x7f2730106a70 2024-04-12T06:22:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273019a260 con 0x7f2730106a70 2024-04-12T06:22:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.620+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f272c010e60 con 0x7f2730106a70 2024-04-12T06:22:45.623 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.621+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273019a7c0 con 0x7f2730106a70 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.621+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2730109060 con 0x7f2730106a70 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.622+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f272c011960 con 0x7f2730106a70 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.622+0000 7f27277fe700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 0x7f271c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.622+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f272c08c140 con 0x7f2730106a70 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.622+0000 7f273630c700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 0x7f271c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:45.624 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.623+0000 7f273630c700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 0x7f271c06f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2720006010 tx=0x7f272001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:45.627 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.627+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f272c056110 con 0x7f2730106a70 2024-04-12T06:22:45.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.880+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 43, "format": "json"} v 0) v1 -- 0x7f273004ea90 con 0x7f2730106a70 2024-04-12T06:22:45.880 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.880+0000 7f27277fe700 1 -- 172.21.15.8:0/3128304472 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 43, "format": "json"}]=0 dumped fsmap epoch 43 v45) v1 ==== 107+0+4843 (secure 0 0 0) 0x7f272c059730 con 0x7f2730106a70 2024-04-12T06:22:45.881 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:45.881 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:09.466921+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:active","state_seq":245,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:active","state_seq":239,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:45.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 msgr2=0x7f271c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 0x7f271c06f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2720006010 tx=0x7f272001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:45.884 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 msgr2=0x7f2730196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f272c00b4b0 tx=0x7f272c00b7c0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 shutdown_connections 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f271c06cce0 0x7f271c06f1a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a60 0x7f2730196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 --2- 172.21.15.8:0/3128304472 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f2730106a70 0x7f2730196bd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:45.885 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 >> 172.21.15.8:0/3128304472 conn(0x7f27300fc570 msgr2=0x7f27300fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:45.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 shutdown_connections 2024-04-12T06:22:45.886 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:45.883+0000 7f2738570700 1 -- 172.21.15.8:0/3128304472 wait complete. 2024-04-12T06:22:45.886 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 43 2024-04-12T06:22:45.896 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 44 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/72346262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34106a70 msgr2=0x7f0b34106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/72346262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34106a70 0x7f0b34106e50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009910 tx=0x7f0b30009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/72346262 shutdown_connections 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/72346262 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34100a60 0x7f0b34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/72346262 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34106a70 0x7f0b34106e50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.026 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.023+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/72346262 >> 172.21.15.8:0/72346262 conn(0x7f0b340fc570 msgr2=0x7f0b340fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.024+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/72346262 shutdown_connections 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.024+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/72346262 wait complete. 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.024+0000 7f0b3bd2c700 1 Processor -- start 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.024+0000 7f0b3bd2c700 1 -- start start 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b3bd2c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b3bd2c700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34106a70 0x7f0b34196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b3bd2c700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b34197230 con 0x7f0b34100a60 2024-04-12T06:22:46.027 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b3bd2c700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3419afb0 con 0x7f0b34106a70 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45016/0 (socket says 172.21.15.8:45016) 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 -- 172.21.15.8:0/111804594 learned_addr learned my addr 172.21.15.8:0/111804594 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b392c7700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34106a70 0x7f0b34196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 -- 172.21.15.8:0/111804594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34106a70 msgr2=0x7f0b34196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34106a70 0x7f0b34196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 -- 172.21.15.8:0/111804594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300095f0 con 0x7f0b34100a60 2024-04-12T06:22:46.028 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b39ac8700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006010 tx=0x7f0b3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b3001d070 con 0x7f0b34100a60 2024-04-12T06:22:46.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3419b230 con 0x7f0b34100a60 2024-04-12T06:22:46.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0b3000f460 con 0x7f0b34100a60 2024-04-12T06:22:46.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.025+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b30021640 con 0x7f0b34100a60 2024-04-12T06:22:46.029 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.026+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3419b730 con 0x7f0b34100a60 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.026+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34109060 con 0x7f0b34100a60 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.027+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f0b3000fa60 con 0x7f0b34100a60 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.027+0000 7f0b2affd700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 0x7f0b2006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.027+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0b3008d820 con 0x7f0b34100a60 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.027+0000 7f0b392c7700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 0x7f0b2006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.031 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.028+0000 7f0b392c7700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 0x7f0b2006f1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0b24005fd0 tx=0x7f0b24005db0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.032 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.032+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f0b30057890 con 0x7f0b34100a60 2024-04-12T06:22:46.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:46 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3725099147' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-12T06:22:46.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:46 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2446428167' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-12T06:22:46.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:46 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3128304472' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-04-12T06:22:46.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.284+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 44, "format": "json"} v 0) v1 -- 0x7f0b3402cd30 con 0x7f0b34100a60 2024-04-12T06:22:46.284 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.284+0000 7f0b2affd700 1 -- 172.21.15.8:0/111804594 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 44, "format": "json"}]=0 dumped fsmap epoch 44 v45) v1 ==== 107+0+4858 (secure 0 0 0) 0x7f0b3005aeb0 con 0x7f0b34100a60 2024-04-12T06:22:46.285 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:46.285 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-12T05:52:43.616162+0000","modified":"2024-04-12T06:20:09.472436+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34330,"mds_1":34362},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34330":{"gid":34330,"name":"cephfs.smithi008.rhdhsr","rank":0,"incarnation":37,"state":"up:active","state_seq":245,"addr":"172.21.15.8:6827/1645196626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6826","nonce":1645196626},{"type":"v1","addr":"172.21.15.8:6827","nonce":1645196626}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_34362":{"gid":34362,"name":"cephfs.smithi008.lmvepz","rank":1,"incarnation":37,"state":"up:active","state_seq":239,"addr":"172.21.15.8:6829/2252467909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6828","nonce":2252467909},{"type":"v1","addr":"172.21.15.8:6829","nonce":2252467909}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44331":{"gid":44331,"name":"cephfs.smithi194.uzfpyu","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6825/996613107","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":996613107},{"type":"v1","addr":"172.21.15.194:6825","nonce":996613107}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_44361":{"gid":44361,"name":"cephfs.smithi194.gbfdvc","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6827/3564821977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3564821977},{"type":"v1","addr":"172.21.15.194:6827","nonce":3564821977}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 msgr2=0x7f0b2006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 0x7f0b2006f1a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0b24005fd0 tx=0x7f0b24005db0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 msgr2=0x7f0b34196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0b30006010 tx=0x7f0b3000be20 comp rx=0 tx=0).stop 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 shutdown_connections 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f0b2006cce0 0x7f0b2006f1a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0b34100a60 0x7f0b34196610 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 --2- 172.21.15.8:0/111804594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0b34106a70 0x7f0b34196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.287 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 >> 172.21.15.8:0/111804594 conn(0x7f0b340fc570 msgr2=0x7f0b340fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:46.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 shutdown_connections 2024-04-12T06:22:46.288 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.286+0000 7f0b3bd2c700 1 -- 172.21.15.8:0/111804594 wait complete. 2024-04-12T06:22:46.288 INFO:teuthology.orchestra.run.smithi008.stderr:dumped fsmap epoch 44 2024-04-12T06:22:46.299 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-12T06:22:46.310 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-12T06:22:46.311 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T06:22:46.311 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T06:22:46.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:46 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3725099147' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-04-12T06:22:46.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:46 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2446428167' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 42, "format": "json"}]: dispatch 2024-04-12T06:22:46.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:46 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3128304472' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 43, "format": "json"}]: dispatch 2024-04-12T06:22:46.331 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T06:22:46.331 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T06:22:46.391 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- 172.21.15.8:0/4059308174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 msgr2=0x7f104c105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 --2- 172.21.15.8:0/4059308174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c105e80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1040009910 tx=0x7f1040009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- 172.21.15.8:0/4059308174 shutdown_connections 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 --2- 172.21.15.8:0/4059308174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c0fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 --2- 172.21.15.8:0/4059308174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c105e80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- 172.21.15.8:0/4059308174 >> 172.21.15.8:0/4059308174 conn(0x7f104c075400 msgr2=0x7f104c075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- 172.21.15.8:0/4059308174 shutdown_connections 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- 172.21.15.8:0/4059308174 wait complete. 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 Processor -- start 2024-04-12T06:22:46.560 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.556+0000 7f1053227700 1 -- start start 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1053227700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1053227700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1053227700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1980d0 con 0x7f104c105aa0 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1053227700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c198210 con 0x7f104c0ffac0 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45036/0 (socket says 172.21.15.8:45036) 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 -- 172.21.15.8:0/794831314 learned_addr learned my addr 172.21.15.8:0/794831314 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1050fc3700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 -- 172.21.15.8:0/794831314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 msgr2=0x7f104c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.561 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 -- 172.21.15.8:0/794831314 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400095f0 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1050fc3700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f104bfff700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f103800d8d0 tx=0x7f103800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1038009940 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.557+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1038010460 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.558+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c1a0520 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.558+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103800f5f0 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.558+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c1a0a50 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.559+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c108040 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.559+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f10380105d0 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.559+0000 7f1049ffb700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 0x7f103c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.559+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f103808c670 con 0x7f104c105aa0 2024-04-12T06:22:46.562 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.559+0000 7f1050fc3700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 0x7f103c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.563 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.560+0000 7f1050fc3700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 0x7f103c06f1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f104000b5c0 tx=0x7f104001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.564 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.564+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1038056640 con 0x7f104c105aa0 2024-04-12T06:22:46.771 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.771+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f104c04ea90 con 0x7f104c105aa0 2024-04-12T06:22:46.772 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.772+0000 7f1049ffb700 1 -- 172.21.15.8:0/794831314 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16251 (secure 0 0 0) 0x7f1038010880 con 0x7f104c105aa0 2024-04-12T06:22:46.772 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:46.772 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":83,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T06:04:51.939854+0000","last_up_change":"2024-04-12T06:03:54.642238+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T05:52:42.555620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T05:52:43.040716+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6803","nonce":1483673533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6805","nonce":1483673533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6809","nonce":1483673533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6807","nonce":1483673533}]},"public_addr":"172.21.15.8:6803/1483673533","cluster_addr":"172.21.15.8:6805/1483673533","heartbeat_back_addr":"172.21.15.8:6809/1483673533","heartbeat_front_addr":"172.21.15.8:6807/1483673533","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6811","nonce":202303324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6813","nonce":202303324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6817","nonce":202303324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6815","nonce":202303324}]},"public_addr":"172.21.15.8:6811/202303324","cluster_addr":"172.21.15.8:6813/202303324","heartbeat_back_addr":"172.21.15.8:6817/202303324","heartbeat_front_addr":"172.21.15.8:6815/202303324","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6819","nonce":2007171174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6821","nonce":2007171174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6825","nonce":2007171174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6823","nonce":2007171174}]},"public_addr":"172.21.15.8:6819/2007171174","cluster_addr":"172.21.15.8:6821/2007171174","heartbeat_back_addr":"172.21.15.8:6825/2007171174","heartbeat_front_addr":"172.21.15.8:6823/2007171174","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6801","nonce":1524342463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6803","nonce":1524342463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6807","nonce":1524342463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6805","nonce":1524342463}]},"public_addr":"172.21.15.194:6801/1524342463","cluster_addr":"172.21.15.194:6803/1524342463","heartbeat_back_addr":"172.21.15.194:6807/1524342463","heartbeat_front_addr":"172.21.15.194:6805/1524342463","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6809","nonce":3298239604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6811","nonce":3298239604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6815","nonce":3298239604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6813","nonce":3298239604}]},"public_addr":"172.21.15.194:6809/3298239604","cluster_addr":"172.21.15.194:6811/3298239604","heartbeat_back_addr":"172.21.15.194:6815/3298239604","heartbeat_front_addr":"172.21.15.194:6813/3298239604","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6817","nonce":2816217628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6819","nonce":2816217628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6823","nonce":2816217628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6821","nonce":2816217628}]},"public_addr":"172.21.15.194:6817/2816217628","cluster_addr":"172.21.15.194:6819/2816217628","heartbeat_back_addr":"172.21.15.194:6823/2816217628","heartbeat_front_addr":"172.21.15.194:6821/2816217628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6824/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:6829/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.194:6825/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.194:0/1351746800":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/867529145":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.194:0/3684029331":"2024-04-13T05:56:17.276168+0000","172.21.15.194:0/665606011":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/3326205956":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/2972485178":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/3187654197":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6826/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.194:0/3304199431":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6828/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.8:6827/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/3071600710":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1577264968":"2024-04-13T05:55:13.372184+0000","172.21.15.194:6828/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.194:6829/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 msgr2=0x7f103c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 0x7f103c06f1a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f104000b5c0 tx=0x7f104001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 msgr2=0x7f104c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f103800d8d0 tx=0x7f103800dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 shutdown_connections 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f103c06cce0 0x7f103c06f1a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f104c0ffac0 0x7f104c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 --2- 172.21.15.8:0/794831314 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f104c105aa0 0x7f104c1979f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.775 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 >> 172.21.15.8:0/794831314 conn(0x7f104c075400 msgr2=0x7f104c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:46.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 shutdown_connections 2024-04-12T06:22:46.776 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.774+0000 7f1053227700 1 -- 172.21.15.8:0/794831314 wait complete. 2024-04-12T06:22:46.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T05:49:53.787041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T05:52:42.555620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T05:52:43.040716+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T06:22:46.787 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/3741111108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 msgr2=0x7fa1e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/3741111108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4106e60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009910 tx=0x7fa1d4009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/3741111108 shutdown_connections 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/3741111108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 0x7fa1e4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/3741111108 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4106e60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/3741111108 >> 172.21.15.8:0/3741111108 conn(0x7fa1e40fc580 msgr2=0x7fa1e40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/3741111108 shutdown_connections 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/3741111108 wait complete. 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 Processor -- start 2024-04-12T06:22:46.919 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.917+0000 7fa1e9fc9700 1 -- start start 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 0x7fa1e41965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e41971f0 con 0x7fa1e4106a80 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e419af70 con 0x7fa1e4100a70 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45052/0 (socket says 172.21.15.8:45052) 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 -- 172.21.15.8:0/2325542627 learned_addr learned my addr 172.21.15.8:0/2325542627 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e37fe700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 0x7fa1e41965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 -- 172.21.15.8:0/2325542627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 msgr2=0x7fa1e41965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 0x7fa1e41965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 -- 172.21.15.8:0/2325542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40095f0 con 0x7fa1e4106a80 2024-04-12T06:22:46.920 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e2ffd700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00d8d0 tx=0x7fa1cc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1cc0098a0 con 0x7fa1e4106a80 2024-04-12T06:22:46.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa1cc010460 con 0x7fa1e4106a80 2024-04-12T06:22:46.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e419b250 con 0x7fa1e4106a80 2024-04-12T06:22:46.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1cc009ea0 con 0x7fa1e4106a80 2024-04-12T06:22:46.921 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.918+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e419b7b0 con 0x7fa1e4106a80 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.919+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e4109070 con 0x7fa1e4106a80 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.920+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa1cc0105d0 con 0x7fa1e4106a80 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.920+0000 7fa1e0ff9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 0x7fa1d006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.921+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa1cc08c440 con 0x7fa1e4106a80 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.921+0000 7fa1e37fe700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 0x7fa1d006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:46.922 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.921+0000 7fa1e37fe700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 0x7fa1d006f1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4006010 tx=0x7fa1d401a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:46.925 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:46.925+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa1cc0564b0 con 0x7fa1e4106a80 2024-04-12T06:22:47.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.132+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1e419b9d0 con 0x7fa1e4106a80 2024-04-12T06:22:47.133 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.133+0000 7fa1e0ff9700 1 -- 172.21.15.8:0/2325542627 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1cc010880 con 0x7fa1e4106a80 2024-04-12T06:22:47.133 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-04-12T06:22:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 msgr2=0x7fa1d006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 0x7fa1d006f1a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4006010 tx=0x7fa1d401a040 comp rx=0 tx=0).stop 2024-04-12T06:22:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 msgr2=0x7fa1e4196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.135 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00d8d0 tx=0x7fa1cc00dbe0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 shutdown_connections 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa1d006cce0 0x7fa1d006f1a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1e4100a70 0x7fa1e41965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 --2- 172.21.15.8:0/2325542627 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa1e4106a80 0x7fa1e4196b10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 >> 172.21.15.8:0/2325542627 conn(0x7fa1e40fc580 msgr2=0x7fa1e40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 shutdown_connections 2024-04-12T06:22:47.136 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.135+0000 7fa1e9fc9700 1 -- 172.21.15.8:0/2325542627 wait complete. 2024-04-12T06:22:47.148 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.276+0000 7f1e28ca7700 1 -- 172.21.15.8:0/4137736714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e241059c0 msgr2=0x7f1e24105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.276+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/4137736714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e241059c0 0x7f1e24105da0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009910 tx=0x7f1e14009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 -- 172.21.15.8:0/4137736714 shutdown_connections 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/4137736714 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e240ffb90 0x7f1e24100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/4137736714 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e241059c0 0x7f1e24105da0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 -- 172.21.15.8:0/4137736714 >> 172.21.15.8:0/4137736714 conn(0x7f1e240753e0 msgr2=0x7f1e240757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 -- 172.21.15.8:0/4137736714 shutdown_connections 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 -- 172.21.15.8:0/4137736714 wait complete. 2024-04-12T06:22:47.279 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 Processor -- start 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 -- start start 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.277+0000 7f1e28ca7700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e241059c0 0x7f1e24196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e28ca7700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24197210 con 0x7f1e240ffb90 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e28ca7700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2419af90 con 0x7f1e241059c0 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45060/0 (socket says 172.21.15.8:45060) 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 -- 172.21.15.8:0/1970889131 learned_addr learned my addr 172.21.15.8:0/1970889131 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e21d9b700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e241059c0 0x7f1e24196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 -- 172.21.15.8:0/1970889131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e241059c0 msgr2=0x7f1e24196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e241059c0 0x7f1e24196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 -- 172.21.15.8:0/1970889131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140095f0 con 0x7f1e240ffb90 2024-04-12T06:22:47.280 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e2259c700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1e140092e0 tx=0x7f1e1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e1401d070 con 0x7f1e240ffb90 2024-04-12T06:22:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1e14005e00 con 0x7f1e240ffb90 2024-04-12T06:22:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2419b210 con 0x7f1e240ffb90 2024-04-12T06:22:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e14021410 con 0x7f1e240ffb90 2024-04-12T06:22:47.281 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.278+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2419b710 con 0x7f1e240ffb90 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.279+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e24108f00 con 0x7f1e240ffb90 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.280+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f1e1400f6d0 con 0x7f1e240ffb90 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.280+0000 7f1e137fe700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 0x7f1e0c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.280+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1e1408d8e0 con 0x7f1e240ffb90 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.280+0000 7f1e21d9b700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 0x7f1e0c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.282 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.281+0000 7f1e21d9b700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 0x7f1e0c06f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e18005fd0 tx=0x7f1e18005f60 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.285 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.285+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f1e14057950 con 0x7f1e240ffb90 2024-04-12T06:22:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:47 smithi194 ceph-mon[106982]: pgmap v834: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:47.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:47 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/111804594' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-04-12T06:22:47.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:47 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/794831314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:47 smithi008 ceph-mon[146084]: pgmap v834: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:47 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/111804594' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 44, "format": "json"}]: dispatch 2024-04-12T06:22:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:47 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/794831314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:47.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.495+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f1e2404ea90 con 0x7f1e240ffb90 2024-04-12T06:22:47.495 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.495+0000 7f1e137fe700 1 -- 172.21.15.8:0/1970889131 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1e1405af70 con 0x7f1e240ffb90 2024-04-12T06:22:47.495 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 msgr2=0x7f1e0c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 0x7f1e0c06f1a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1e18005fd0 tx=0x7f1e18005f60 comp rx=0 tx=0).stop 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 msgr2=0x7f1e241965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1e140092e0 tx=0x7f1e1400be20 comp rx=0 tx=0).stop 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 shutdown_connections 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f1e0c06cce0 0x7f1e0c06f1a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1e240ffb90 0x7f1e241965f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 --2- 172.21.15.8:0/1970889131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e241059c0 0x7f1e24196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 >> 172.21.15.8:0/1970889131 conn(0x7f1e240753e0 msgr2=0x7f1e24068070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 shutdown_connections 2024-04-12T06:22:47.498 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.497+0000 7f1e28ca7700 1 -- 172.21.15.8:0/1970889131 wait complete. 2024-04-12T06:22:47.510 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T06:22:47.641 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 -- 172.21.15.8:0/566376001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 msgr2=0x7fa2740ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 --2- 172.21.15.8:0/566376001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2740ffdf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa268009910 tx=0x7fa268009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 -- 172.21.15.8:0/566376001 shutdown_connections 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 --2- 172.21.15.8:0/566376001 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2740ffdf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 --2- 172.21.15.8:0/566376001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 0x7fa274105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 -- 172.21.15.8:0/566376001 >> 172.21.15.8:0/566376001 conn(0x7fa2740fb440 msgr2=0x7fa2740fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 -- 172.21.15.8:0/566376001 shutdown_connections 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.639+0000 7fa27baae700 1 -- 172.21.15.8:0/566376001 wait complete. 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 Processor -- start 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 -- start start 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.642 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 0x7fa274196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274197200 con 0x7fa2740ff970 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27baae700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27419af80 con 0x7fa274105980 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45082/0 (socket says 172.21.15.8:45082) 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 -- 172.21.15.8:0/3001781488 learned_addr learned my addr 172.21.15.8:0/3001781488 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa279049700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 0x7fa274196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 -- 172.21.15.8:0/3001781488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 msgr2=0x7fa274196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 0x7fa274196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.640+0000 7fa27984a700 1 -- 172.21.15.8:0/3001781488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2680095f0 con 0x7fa2740ff970 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa27984a700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d8d0 tx=0x7fa27000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2700098a0 con 0x7fa2740ff970 2024-04-12T06:22:47.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa270010460 con 0x7fa2740ff970 2024-04-12T06:22:47.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27419b260 con 0x7fa2740ff970 2024-04-12T06:22:47.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa270009ea0 con 0x7fa2740ff970 2024-04-12T06:22:47.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.641+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27419b7c0 con 0x7fa2740ff970 2024-04-12T06:22:47.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.642+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27404ea90 con 0x7fa2740ff970 2024-04-12T06:22:47.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.642+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fa2700105d0 con 0x7fa2740ff970 2024-04-12T06:22:47.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.643+0000 7fa266ffd700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 0x7fa26006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.643+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa27008c440 con 0x7fa2740ff970 2024-04-12T06:22:47.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.643+0000 7fa279049700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 0x7fa26006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.645 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.643+0000 7fa279049700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 0x7fa26006f1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa274197bf0 tx=0x7fa26801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.647+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fa2700564b0 con 0x7fa2740ff970 2024-04-12T06:22:47.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.850+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa274197930 con 0x7fa2740ff970 2024-04-12T06:22:47.850 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.850+0000 7fa266ffd700 1 -- 172.21.15.8:0/3001781488 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa270010880 con 0x7fa2740ff970 2024-04-12T06:22:47.850 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 msgr2=0x7fa26006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 0x7fa26006f1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa274197bf0 tx=0x7fa26801a040 comp rx=0 tx=0).stop 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 msgr2=0x7fa2741965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d8d0 tx=0x7fa27000dbe0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 shutdown_connections 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fa26006cce0 0x7fa26006f1a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fa2740ff970 0x7fa2741965e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 --2- 172.21.15.8:0/3001781488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa274105980 0x7fa274196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 >> 172.21.15.8:0/3001781488 conn(0x7fa2740fb440 msgr2=0x7fa274103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 shutdown_connections 2024-04-12T06:22:47.853 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.852+0000 7fa27baae700 1 -- 172.21.15.8:0/3001781488 wait complete. 2024-04-12T06:22:47.865 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T06:22:47.993 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 -- 172.21.15.8:0/735248474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100ed0 msgr2=0x7fad181045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 --2- 172.21.15.8:0/735248474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100ed0 0x7fad181045d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fad08009910 tx=0x7fad08009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 -- 172.21.15.8:0/735248474 shutdown_connections 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 --2- 172.21.15.8:0/735248474 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100ed0 0x7fad181045d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 --2- 172.21.15.8:0/735248474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad18100520 0x7fad18100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 -- 172.21.15.8:0/735248474 >> 172.21.15.8:0/735248474 conn(0x7fad18074bf0 msgr2=0x7fad18075000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 -- 172.21.15.8:0/735248474 shutdown_connections 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.991+0000 7fad1ffee700 1 -- 172.21.15.8:0/735248474 wait complete. 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 Processor -- start 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 -- start start 2024-04-12T06:22:47.994 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 0x7fad18198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1819f0d0 con 0x7fad18100520 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1ffee700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad18198b70 con 0x7fad1819eb00 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1dd8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1dd8a700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45108/0 (socket says 172.21.15.8:45108) 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1dd8a700 1 -- 172.21.15.8:0/2769671103 learned_addr learned my addr 172.21.15.8:0/2769671103 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1d589700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 0x7fad18198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1dd8a700 1 -- 172.21.15.8:0/2769671103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 msgr2=0x7fad18198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.992+0000 7fad1dd8a700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 0x7fad18198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad1dd8a700 1 -- 172.21.15.8:0/2769671103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080095f0 con 0x7fad18100520 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad1d589700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 0x7fad18198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad1dd8a700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d8d0 tx=0x7fad1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad14009940 con 0x7fad18100520 2024-04-12T06:22:47.995 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fad14010460 con 0x7fad18100520 2024-04-12T06:22:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad18198e50 con 0x7fad18100520 2024-04-12T06:22:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1400f5f0 con 0x7fad18100520 2024-04-12T06:22:47.996 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.993+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad181993b0 con 0x7fad18100520 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.994+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1804ea90 con 0x7fad18100520 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.994+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fad140105d0 con 0x7fad18100520 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.995+0000 7fad0effd700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 0x7fad0406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.995+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fad1408c4c0 con 0x7fad18100520 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.995+0000 7fad1d589700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 0x7fad0406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:47.997 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.995+0000 7fad1d589700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 0x7fad0406f1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad0801a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:47.999 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:47.999+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fad1405a040 con 0x7fad18100520 2024-04-12T06:22:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:48 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2325542627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:48 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/1970889131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:48 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3001781488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.222+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad18100f00 con 0x7fad18100520 2024-04-12T06:22:48.222 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.222+0000 7fad0effd700 1 -- 172.21.15.8:0/2769671103 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v83) v1 ==== 81+0+2059 (secure 0 0 0) 0x7fad14010880 con 0x7fad18100520 2024-04-12T06:22:48.222 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6824/1464587758 2024-04-13T06:03:56.166429+0000 2024-04-12T06:22:48.222 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6829/2135087803 2024-04-13T06:03:56.166416+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6825/1464587758 2024-04-13T06:03:56.166429+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/638708981 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/1351746800 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3166946531 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2313514201 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/1010894527 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/2294526141 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3166946531 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/1010894527 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/867529145 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3827652772 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3866476031 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3684029331 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.223 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/665606011 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/2294526141 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3326205956 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2871227956 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/2972485178 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3187654197 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6826/2911956822 2024-04-13T05:52:58.525944+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3304199431 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1444434986 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6828/2135087803 2024-04-13T06:03:56.166416+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6827/2911956822 2024-04-13T05:52:58.525944+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/4044975390 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3827652772 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:48.224 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2715380286 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3071600710 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1577264968 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6828/2335168333 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6829/2335168333 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/301277284 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:48.225 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3777789548 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:48.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 msgr2=0x7fad0406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 0x7fad0406f1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fad08006010 tx=0x7fad0801a040 comp rx=0 tx=0).stop 2024-04-12T06:22:48.226 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 msgr2=0x7fad1819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fad1400d8d0 tx=0x7fad1400dc90 comp rx=0 tx=0).stop 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 shutdown_connections 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fad0406cce0 0x7fad0406f1a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fad18100520 0x7fad1819e5c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 --2- 172.21.15.8:0/2769671103 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad1819eb00 0x7fad18198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.224+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 >> 172.21.15.8:0/2769671103 conn(0x7fad18074bf0 msgr2=0x7fad180fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.225+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 shutdown_connections 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.225+0000 7fad1ffee700 1 -- 172.21.15.8:0/2769671103 wait complete. 2024-04-12T06:22:48.227 INFO:teuthology.orchestra.run.smithi008.stderr:listed 35 entries 2024-04-12T06:22:48.237 DEBUG:teuthology.orchestra.run.smithi008:> set -ex 2024-04-12T06:22:48.237 DEBUG:teuthology.orchestra.run.smithi008:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-12T06:22:48.257 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-12T06:22:48.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:48 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2325542627' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:22:48.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:48 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/1970889131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:48 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3001781488' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:48.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- 172.21.15.8:0/2772733852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02800ffac0 msgr2=0x7f02800fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 --2- 172.21.15.8:0/2772733852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02800ffac0 0x7f02800fff20 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0268009910 tx=0x7f0268009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:48.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- 172.21.15.8:0/2772733852 shutdown_connections 2024-04-12T06:22:48.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 --2- 172.21.15.8:0/2772733852 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f02800ffac0 0x7f02800fff20 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.426 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 --2- 172.21.15.8:0/2772733852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0280105aa0 0x7f0280105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- 172.21.15.8:0/2772733852 >> 172.21.15.8:0/2772733852 conn(0x7f0280075400 msgr2=0x7f0280075810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- 172.21.15.8:0/2772733852 shutdown_connections 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- 172.21.15.8:0/2772733852 wait complete. 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 Processor -- start 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 -- start start 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 0x7f028019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.424+0000 7f0286629700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0286629700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028019f260 con 0x7f0280105aa0 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0286629700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0280198700 con 0x7f02800ffac0 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45120/0 (socket says 172.21.15.8:45120) 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 -- 172.21.15.8:0/193971322 learned_addr learned my addr 172.21.15.8:0/193971322 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:48.427 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f027ffff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 0x7f028019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 -- 172.21.15.8:0/193971322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 msgr2=0x7f028019e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 0x7f028019e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 -- 172.21.15.8:0/193971322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680095f0 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f027ffff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 0x7f028019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0277fff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f026800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f026801d070 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f02680044c0 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0280198980 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02680039d0 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.425+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280198e20 con 0x7f0280105aa0 2024-04-12T06:22:48.428 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.426+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028004ea90 con 0x7f0280105aa0 2024-04-12T06:22:48.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.427+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f02680048e0 con 0x7f0280105aa0 2024-04-12T06:22:48.429 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.427+0000 7f027dffb700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 0x7f026c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.427+0000 7f027ffff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 0x7f026c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:48.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.428+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f026808dbb0 con 0x7f0280105aa0 2024-04-12T06:22:48.430 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.428+0000 7f027ffff700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 0x7f026c06f1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0270009be0 tx=0x7f0270009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:48.432 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.432+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f026805b690 con 0x7f0280105aa0 2024-04-12T06:22:48.643 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.643+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02801990f0 con 0x7f0280105aa0 2024-04-12T06:22:48.644 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.644+0000 7f027dffb700 1 -- 172.21.15.8:0/193971322 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16251 (secure 0 0 0) 0x7f0268026090 con 0x7f0280105aa0 2024-04-12T06:22:48.644 INFO:teuthology.orchestra.run.smithi008.stdout: 2024-04-12T06:22:48.645 INFO:teuthology.orchestra.run.smithi008.stdout:{"epoch":83,"fsid":"b9cc93fe-f88f-11ee-bc8c-c7b262605968","created":"2024-04-12T05:45:00.023030+0000","modified":"2024-04-12T06:04:51.939854+0000","last_up_change":"2024-04-12T06:03:54.642238+0000","last_in_change":"2024-04-12T05:50:59.334352+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-12T05:49:53.787041+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-12T05:52:42.555620+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-12T05:52:43.040716+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"aa2f9673-7e1b-494b-89f2-f87fe4fbc40f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6802","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6803","nonce":1483673533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6804","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6805","nonce":1483673533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6808","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6809","nonce":1483673533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6806","nonce":1483673533},{"type":"v1","addr":"172.21.15.8:6807","nonce":1483673533}]},"public_addr":"172.21.15.8:6803/1483673533","cluster_addr":"172.21.15.8:6805/1483673533","heartbeat_back_addr":"172.21.15.8:6809/1483673533","heartbeat_front_addr":"172.21.15.8:6807/1483673533","state":["exists","up"]},{"osd":1,"uuid":"ad299755-103b-4710-8304-f0e5abed87d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6810","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6811","nonce":202303324}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6812","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6813","nonce":202303324}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6816","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6817","nonce":202303324}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6814","nonce":202303324},{"type":"v1","addr":"172.21.15.8:6815","nonce":202303324}]},"public_addr":"172.21.15.8:6811/202303324","cluster_addr":"172.21.15.8:6813/202303324","heartbeat_back_addr":"172.21.15.8:6817/202303324","heartbeat_front_addr":"172.21.15.8:6815/202303324","state":["exists","up"]},{"osd":2,"uuid":"ff12c02f-ab4b-4542-8b53-0c6112a339f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6818","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6819","nonce":2007171174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6820","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6821","nonce":2007171174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6824","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6825","nonce":2007171174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.8:6822","nonce":2007171174},{"type":"v1","addr":"172.21.15.8:6823","nonce":2007171174}]},"public_addr":"172.21.15.8:6819/2007171174","cluster_addr":"172.21.15.8:6821/2007171174","heartbeat_back_addr":"172.21.15.8:6825/2007171174","heartbeat_front_addr":"172.21.15.8:6823/2007171174","state":["exists","up"]},{"osd":3,"uuid":"6feb5af6-bf9e-48a4-bc8d-7cff6420c259","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6801","nonce":1524342463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6803","nonce":1524342463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6807","nonce":1524342463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":1524342463},{"type":"v1","addr":"172.21.15.194:6805","nonce":1524342463}]},"public_addr":"172.21.15.194:6801/1524342463","cluster_addr":"172.21.15.194:6803/1524342463","heartbeat_back_addr":"172.21.15.194:6807/1524342463","heartbeat_front_addr":"172.21.15.194:6805/1524342463","state":["exists","up"]},{"osd":4,"uuid":"79640d26-73e7-4d99-9933-f82e5e6bb487","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6809","nonce":3298239604}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6811","nonce":3298239604}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6815","nonce":3298239604}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":3298239604},{"type":"v1","addr":"172.21.15.194:6813","nonce":3298239604}]},"public_addr":"172.21.15.194:6809/3298239604","cluster_addr":"172.21.15.194:6811/3298239604","heartbeat_back_addr":"172.21.15.194:6815/3298239604","heartbeat_front_addr":"172.21.15.194:6813/3298239604","state":["exists","up"]},{"osd":5,"uuid":"b267a717-3b34-4518-ae46-420004107ef3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6817","nonce":2816217628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6819","nonce":2816217628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6823","nonce":2816217628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":2816217628},{"type":"v1","addr":"172.21.15.194:6821","nonce":2816217628}]},"public_addr":"172.21.15.194:6817/2816217628","cluster_addr":"172.21.15.194:6819/2816217628","heartbeat_back_addr":"172.21.15.194:6823/2816217628","heartbeat_front_addr":"172.21.15.194:6821/2816217628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:48:50.618084+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:20.236655+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:49:50.327906+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:18.092335+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:50:47.732692+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-12T05:51:18.011070+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.194:6824/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:6829/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.194:6825/1464587758":"2024-04-13T06:03:56.166429+0000","172.21.15.8:0/638708981":"2024-04-13T05:47:27.259061+0000","172.21.15.194:0/1351746800":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:0/2313514201":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6801/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3166946531":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/1010894527":"2024-04-13T05:46:10.032126+0000","172.21.15.8:0/867529145":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6801/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3866476031":"2024-04-13T05:46:10.032126+0000","172.21.15.194:0/3684029331":"2024-04-13T05:56:17.276168+0000","172.21.15.194:0/665606011":"2024-04-13T05:56:17.276168+0000","172.21.15.8:6800/2294526141":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/3326205956":"2024-04-13T05:55:13.372184+0000","172.21.15.8:0/2871227956":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/2972485178":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/3187654197":"2024-04-13T05:55:13.372184+0000","172.21.15.8:6826/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.194:0/3304199431":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1444434986":"2024-04-13T05:46:10.032126+0000","172.21.15.8:6828/2135087803":"2024-04-13T06:03:56.166416+0000","172.21.15.8:6827/2911956822":"2024-04-13T05:52:58.525944+0000","172.21.15.8:0/4044975390":"2024-04-13T05:45:34.567900+0000","172.21.15.8:6800/3827652772":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/2715380286":"2024-04-13T05:45:34.567900+0000","172.21.15.194:0/3071600710":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/1577264968":"2024-04-13T05:55:13.372184+0000","172.21.15.194:6828/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.194:6829/2335168333":"2024-04-13T05:56:17.276168+0000","172.21.15.8:0/301277284":"2024-04-13T05:47:27.259061+0000","172.21.15.8:0/3777789548":"2024-04-13T05:47:27.259061+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.646+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 msgr2=0x7f026c06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.646+0000 7f0286629700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 0x7f026c06f1a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0270009be0 tx=0x7f0270009260 comp rx=0 tx=0).stop 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.646+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 msgr2=0x7f028019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.646+0000 7f0286629700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0268000c00 tx=0x7f026800be90 comp rx=0 tx=0).stop 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 shutdown_connections 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f026c06cce0 0x7f026c06f1a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.647 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02800ffac0 0x7f028019e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 --2- 172.21.15.8:0/193971322 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f0280105aa0 0x7f028019ebd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 >> 172.21.15.8:0/193971322 conn(0x7f0280075400 msgr2=0x7f0280067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:48.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 shutdown_connections 2024-04-12T06:22:48.648 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.647+0000 7f0286629700 1 -- 172.21.15.8:0/193971322 wait complete. 2024-04-12T06:22:48.659 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-12T05:49:53.787041+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-12T05:52:42.555620+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-12T05:52:43.040716+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-12T06:22:48.660 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-12T06:22:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- 172.21.15.8:0/3145749174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28100a90 msgr2=0x7fbd28100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3145749174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28100a90 0x7fbd28100f10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fbd10009910 tx=0x7fbd10009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- 172.21.15.8:0/3145749174 shutdown_connections 2024-04-12T06:22:48.791 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3145749174 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28100a90 0x7fbd28100f10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3145749174 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28106aa0 0x7fbd28106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- 172.21.15.8:0/3145749174 >> 172.21.15.8:0/3145749174 conn(0x7fbd280fc580 msgr2=0x7fbd280fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- 172.21.15.8:0/3145749174 shutdown_connections 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- 172.21.15.8:0/3145749174 wait complete. 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 Processor -- start 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.789+0000 7fbd2e391700 1 -- start start 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28100a90 0x7fbd281965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd28197200 con 0x7fbd28106aa0 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2819af80 con 0x7fbd28100a90 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:48.792 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45134/0 (socket says 172.21.15.8:45134) 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 -- 172.21.15.8:0/3980084138 learned_addr learned my addr 172.21.15.8:0/3980084138 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 -- 172.21.15.8:0/3980084138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28100a90 msgr2=0x7fbd281965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28100a90 0x7fbd281965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 -- 172.21.15.8:0/3980084138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd100095f0 con 0x7fbd28106aa0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd277fe700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd1001d070 con 0x7fbd28106aa0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbd1000f460 con 0x7fbd28106aa0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2819b200 con 0x7fbd28106aa0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd10021640 con 0x7fbd28106aa0 2024-04-12T06:22:48.793 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.790+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2819b700 con 0x7fbd28106aa0 2024-04-12T06:22:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.791+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd28109020 con 0x7fbd28106aa0 2024-04-12T06:22:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.792+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fbd1000f9e0 con 0x7fbd28106aa0 2024-04-12T06:22:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.792+0000 7fbd257fa700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 0x7fbd1406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:48.794 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.792+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd1008d8c0 con 0x7fbd28106aa0 2024-04-12T06:22:48.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.792+0000 7fbd27fff700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 0x7fbd1406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:48.795 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.793+0000 7fbd27fff700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 0x7fbd1406f1a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800a990 tx=0x7fbd18005c80 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:48.797 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:48.797+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fbd10057930 con 0x7fbd28106aa0 2024-04-12T06:22:49.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.001+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd2804ea90 con 0x7fbd28106aa0 2024-04-12T06:22:49.001 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.001+0000 7fbd257fa700 1 -- 172.21.15.8:0/3980084138 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd1005af50 con 0x7fbd28106aa0 2024-04-12T06:22:49.001 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 1 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 msgr2=0x7fbd1406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 0x7fbd1406f1a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800a990 tx=0x7fbd18005c80 comp rx=0 tx=0).stop 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 msgr2=0x7fbd28196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fbd10006010 tx=0x7fbd1000be20 comp rx=0 tx=0).stop 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 shutdown_connections 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fbd1406cce0 0x7fbd1406f1a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbd28100a90 0x7fbd281965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 --2- 172.21.15.8:0/3980084138 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fbd28106aa0 0x7fbd28196b20 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 >> 172.21.15.8:0/3980084138 conn(0x7fbd280fc580 msgr2=0x7fbd280fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.003+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 shutdown_connections 2024-04-12T06:22:49.004 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.004+0000 7fbd2e391700 1 -- 172.21.15.8:0/3980084138 wait complete. 2024-04-12T06:22:49.016 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-12T06:22:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 -- 172.21.15.8:0/2036740309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 msgr2=0x7f1158106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 --2- 172.21.15.8:0/2036740309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f1158106e40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f114800b0a0 tx=0x7f114800b3b0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.149 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 -- 172.21.15.8:0/2036740309 shutdown_connections 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 --2- 172.21.15.8:0/2036740309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 --2- 172.21.15.8:0/2036740309 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f1158106e40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 -- 172.21.15.8:0/2036740309 >> 172.21.15.8:0/2036740309 conn(0x7f11580fc540 msgr2=0x7f11580fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 -- 172.21.15.8:0/2036740309 shutdown_connections 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.147+0000 7f115cb85700 1 -- 172.21.15.8:0/2036740309 wait complete. 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 Processor -- start 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 -- start start 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158073b10 con 0x7f1158106a60 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115cb85700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1158073c50 con 0x7f1158100a50 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f1155d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.150 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f1155d9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45158/0 (socket says 172.21.15.8:45158) 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f1155d9b700 1 -- 172.21.15.8:0/162084994 learned_addr learned my addr 172.21.15.8:0/162084994 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.148+0000 7f115659c700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f1155d9b700 1 -- 172.21.15.8:0/162084994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 msgr2=0x7f1158073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f1155d9b700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f1155d9b700 1 -- 172.21.15.8:0/162084994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148009d00 con 0x7f1158106a60 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f115659c700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f1155d9b700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f114000c8a0 tx=0x7f114000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114000cea0 con 0x7f1158106a60 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1140004830 con 0x7f1158106a60 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11581a2800 con 0x7f1158106a60 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1140005500 con 0x7f1158106a60 2024-04-12T06:22:49.151 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.149+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11581a2c20 con 0x7f1158106a60 2024-04-12T06:22:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.150+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1158109050 con 0x7f1158106a60 2024-04-12T06:22:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.150+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f11400049a0 con 0x7f1158106a60 2024-04-12T06:22:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.151+0000 7f114f7fe700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 0x7f114406f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.152 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.151+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f114008c2d0 con 0x7f1158106a60 2024-04-12T06:22:49.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.151+0000 7f115659c700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 0x7f114406f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.153 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.151+0000 7f115659c700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 0x7f114406f1a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1148000f80 tx=0x7f114800b7f0 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.155 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.156+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f11400562b0 con 0x7f1158106a60 2024-04-12T06:22:49.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:49 smithi194 ceph-mon[106982]: pgmap v835: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:49.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:49 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2769671103' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T06:22:49.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:49 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/193971322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:49.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:49 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/3980084138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:49.358 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.358+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f115804ea90 con 0x7f1158106a60 2024-04-12T06:22:49.359 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.359+0000 7f114f7fe700 1 -- 172.21.15.8:0/162084994 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f11400598d0 con 0x7f1158106a60 2024-04-12T06:22:49.359 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:49.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 msgr2=0x7f114406f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 0x7f114406f1a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1148000f80 tx=0x7f114800b7f0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.361 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 msgr2=0x7f11580735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f114000c8a0 tx=0x7f114000cc60 comp rx=0 tx=0).stop 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 shutdown_connections 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f114406cce0 0x7f114406f1a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1158100a50 0x7f1158073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 --2- 172.21.15.8:0/162084994 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f1158106a60 0x7f11580735d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 >> 172.21.15.8:0/162084994 conn(0x7f11580fc540 msgr2=0x7f11580fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 shutdown_connections 2024-04-12T06:22:49.362 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.361+0000 7f115cb85700 1 -- 172.21.15.8:0/162084994 wait complete. 2024-04-12T06:22:49.372 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:49 smithi008 ceph-mon[146084]: pgmap v835: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:49.372 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:49 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2769671103' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T06:22:49.372 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:49 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/193971322' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-12T06:22:49.372 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:49 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/3980084138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-12T06:22:49.374 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 -- 172.21.15.8:0/3784703731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 msgr2=0x7fe7b0105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 --2- 172.21.15.8:0/3784703731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0105d90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009910 tx=0x7fe7a0009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 -- 172.21.15.8:0/3784703731 shutdown_connections 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 --2- 172.21.15.8:0/3784703731 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 0x7fe7b00ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 --2- 172.21.15.8:0/3784703731 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0105d90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 -- 172.21.15.8:0/3784703731 >> 172.21.15.8:0/3784703731 conn(0x7fe7b00fb490 msgr2=0x7fe7b00fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.502+0000 7fe7b488b700 1 -- 172.21.15.8:0/3784703731 shutdown_connections 2024-04-12T06:22:49.503 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 -- 172.21.15.8:0/3784703731 wait complete. 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 Processor -- start 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 -- start start 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 0x7fe7b01965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b01971c0 con 0x7fe7b01059b0 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.503+0000 7fe7b488b700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b019af40 con 0x7fe7b00ff9a0 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45186/0 (socket says 172.21.15.8:45186) 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 -- 172.21.15.8:0/2212244857 learned_addr learned my addr 172.21.15.8:0/2212244857 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7ae59c700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 0x7fe7b01965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 -- 172.21.15.8:0/2212244857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 msgr2=0x7fe7b01965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.507 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 0x7fe7b01965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 -- 172.21.15.8:0/2212244857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00095f0 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7add9b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d900 tx=0x7fe7a400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a40041f0 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7a4004cd0 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b019b220 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7a4005660 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.504+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b019b780 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.505+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b0108fb0 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.506+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7fe7a4004e40 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.506+0000 7fe79f7fe700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 0x7fe79806f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.506+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe7a408c360 con 0x7fe7b01059b0 2024-04-12T06:22:49.508 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.506+0000 7fe7ae59c700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 0x7fe79806f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.509 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.507+0000 7fe7ae59c700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 0x7fe79806f1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a001a040 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.511 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.511+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7fe7a40563d0 con 0x7fe7b01059b0 2024-04-12T06:22:49.714 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.714+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fe7b004ea90 con 0x7fe7b01059b0 2024-04-12T06:22:49.715 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.715+0000 7fe79f7fe700 1 -- 172.21.15.8:0/2212244857 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe7a40599f0 con 0x7fe7b01059b0 2024-04-12T06:22:49.715 INFO:teuthology.orchestra.run.smithi008.stdout:pg_num: 32 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 msgr2=0x7fe79806f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 0x7fe79806f1a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a001a040 comp rx=0 tx=0).stop 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 msgr2=0x7fe7b0196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d900 tx=0x7fe7a400dc10 comp rx=0 tx=0).stop 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 shutdown_connections 2024-04-12T06:22:49.717 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7fe79806cce0 0x7fe79806f1a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe7b00ff9a0 0x7fe7b01965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 --2- 172.21.15.8:0/2212244857 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7fe7b01059b0 0x7fe7b0196ae0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 >> 172.21.15.8:0/2212244857 conn(0x7fe7b00fb490 msgr2=0x7fe7b0068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 shutdown_connections 2024-04-12T06:22:49.718 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.717+0000 7fe7b488b700 1 -- 172.21.15.8:0/2212244857 wait complete. 2024-04-12T06:22:49.729 DEBUG:teuthology.orchestra.run.smithi008:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-12T06:22:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.856+0000 7f4c44ff4700 1 -- 172.21.15.8:0/1126489756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c40105970 msgr2=0x7f4c40105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.856+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/1126489756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c40105970 0x7f4c40105d50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009910 tx=0x7f4c30009c20 comp rx=0 tx=0).stop 2024-04-12T06:22:49.857 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 -- 172.21.15.8:0/1126489756 shutdown_connections 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/1126489756 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c400ff960 0x7f4c400ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/1126489756 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c40105970 0x7f4c40105d50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 -- 172.21.15.8:0/1126489756 >> 172.21.15.8:0/1126489756 conn(0x7f4c400fb430 msgr2=0x7f4c400fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 -- 172.21.15.8:0/1126489756 shutdown_connections 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 -- 172.21.15.8:0/1126489756 wait complete. 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 Processor -- start 2024-04-12T06:22:49.858 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.857+0000 7f4c44ff4700 1 -- start start 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c40105970 0x7f4c4019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 -- --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4019f190 con 0x7f4c400ff960 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c40198630 con 0x7f4c40105970 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 --2- >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.8:3300/0 says I am v2:172.21.15.8:45212/0 (socket says 172.21.15.8:45212) 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 -- 172.21.15.8:0/2644274936 learned_addr learned my addr 172.21.15.8:0/2644274936 (peer_addr_for_me v2:172.21.15.8:0/0) 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3dd9b700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c40105970 0x7f4c4019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 -- 172.21.15.8:0/2644274936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c40105970 msgr2=0x7f4c4019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c40105970 0x7f4c4019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 -- 172.21.15.8:0/2644274936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c300095f0 con 0x7f4c400ff960 2024-04-12T06:22:49.861 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c3e59c700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4c30000c00 tx=0x7f4c3000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c3001d070 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c401988b0 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4c30005dc0 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40198d50 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.858+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c300038e0 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.860+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4004ea90 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.860+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 4 ==== mgrmap(e 32) v1 ==== 90848+0+0 (secure 0 0 0) 0x7f4c30004ae0 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.860+0000 7f4c2f7fe700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 0x7f4c2806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.860+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4c3008dbd0 con 0x7f4c400ff960 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.861+0000 7f4c3dd9b700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 0x7f4c2806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-12T06:22:49.862 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.861+0000 7f4c3dd9b700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 0x7f4c2806f1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4c400fde00 tx=0x7f4c34009260 comp rx=0 tx=0).ready entity=mgr.14660 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-12T06:22:49.865 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:49.865+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182936 (secure 0 0 0) 0x7f4c3005b630 con 0x7f4c400ff960 2024-04-12T06:22:50.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.089+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 --> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c40199b40 con 0x7f4c400ff960 2024-04-12T06:22:50.090 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.090+0000 7f4c2f7fe700 1 -- 172.21.15.8:0/2644274936 <== mon.0 v2:172.21.15.8:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v83) v1 ==== 81+0+2059 (secure 0 0 0) 0x7f4c3005b1c0 con 0x7f4c400ff960 2024-04-12T06:22:50.090 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6824/1464587758 2024-04-13T06:03:56.166429+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6829/2135087803 2024-04-13T06:03:56.166416+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6825/1464587758 2024-04-13T06:03:56.166429+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/638708981 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/1351746800 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3166946531 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2313514201 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/1010894527 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/2294526141 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3166946531 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:50.091 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/1010894527 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/867529145 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6801/3827652772 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3866476031 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3684029331 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/665606011 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/2294526141 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3326205956 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2871227956 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/2972485178 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3187654197 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6826/2911956822 2024-04-13T05:52:58.525944+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3304199431 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1444434986 2024-04-13T05:46:10.032126+0000 2024-04-12T06:22:50.092 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6828/2135087803 2024-04-13T06:03:56.166416+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6827/2911956822 2024-04-13T05:52:58.525944+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/4044975390 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:6800/3827652772 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/2715380286 2024-04-13T05:45:34.567900+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:0/3071600710 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/1577264968 2024-04-13T05:55:13.372184+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6828/2335168333 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.194:6829/2335168333 2024-04-13T05:56:17.276168+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/301277284 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:50.093 INFO:teuthology.orchestra.run.smithi008.stdout:172.21.15.8:0/3777789548 2024-04-13T05:47:27.259061+0000 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 msgr2=0x7f4c2806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 0x7f4c2806f1f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4c400fde00 tx=0x7f4c34009260 comp rx=0 tx=0).stop 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 msgr2=0x7f4c4019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4c30000c00 tx=0x7f4c3000bde0 comp rx=0 tx=0).stop 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 shutdown_connections 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:6800/2333971379,v1:172.21.15.8:6801/2333971379] conn(0x7f4c2806cd30 0x7f4c2806f1f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.8:3300/0,v1:172.21.15.8:6789/0] conn(0x7f4c400ff960 0x7f4c4019e5c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:50.095 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 --2- 172.21.15.8:0/2644274936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c40105970 0x7f4c4019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-12T06:22:50.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 >> 172.21.15.8:0/2644274936 conn(0x7f4c400fb430 msgr2=0x7f4c40068970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-12T06:22:50.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 shutdown_connections 2024-04-12T06:22:50.096 INFO:teuthology.orchestra.run.smithi008.stderr:2024-04-12T06:22:50.092+0000 7f4c44ff4700 1 -- 172.21.15.8:0/2644274936 wait complete. 2024-04-12T06:22:50.096 INFO:teuthology.orchestra.run.smithi008.stderr:listed 35 entries 2024-04-12T06:22:50.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi008.front.sepia.ceph.com... 2024-04-12T06:22:50.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-12T06:22:50.105 DEBUG:teuthology.orchestra.run.smithi008:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-12T06:22:50.141 INFO:teuthology.orchestra.run:waiting for 300 2024-04-12T06:22:50.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:50 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/162084994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:50.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:50 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2212244857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:50.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:50 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/162084994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-12T06:22:50.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:50 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2212244857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-12T06:22:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:51 smithi194 ceph-mon[106982]: pgmap v836: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:51 smithi194 ceph-mon[106982]: from='client.? 172.21.15.8:0/2644274936' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T06:22:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:51 smithi008 ceph-mon[146084]: pgmap v836: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:22:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:51 smithi008 ceph-mon[146084]: from='client.? 172.21.15.8:0/2644274936' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-12T06:22:53.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:53 smithi194 ceph-mon[106982]: pgmap v837: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:53 smithi008 ceph-mon[146084]: pgmap v837: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:55.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:55 smithi194 ceph-mon[106982]: pgmap v838: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:55.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:55 smithi008 ceph-mon[146084]: pgmap v838: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:22:57.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:57 smithi194 ceph-mon[106982]: pgmap v839: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:57.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:57 smithi008 ceph-mon[146084]: pgmap v839: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:22:59.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:22:59 smithi008 ceph-mon[146084]: pgmap v840: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:22:59.571 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:22:59 smithi194 ceph-mon[106982]: pgmap v840: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:01.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:01 smithi008 ceph-mon[146084]: pgmap v841: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:01.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:01 smithi194 ceph-mon[106982]: pgmap v841: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:03 smithi008 ceph-mon[146084]: pgmap v842: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:03 smithi194 ceph-mon[106982]: pgmap v842: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:05.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:05 smithi008 ceph-mon[146084]: pgmap v843: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:05.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:05 smithi194 ceph-mon[106982]: pgmap v843: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:07.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:07 smithi008 ceph-mon[146084]: pgmap v844: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:07.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:07 smithi194 ceph-mon[106982]: pgmap v844: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:09.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:09 smithi008 ceph-mon[146084]: pgmap v845: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:09.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:09 smithi194 ceph-mon[106982]: pgmap v845: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:11.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:11 smithi008 ceph-mon[146084]: pgmap v846: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:11.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:11 smithi194 ceph-mon[106982]: pgmap v846: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:13.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:13 smithi008 ceph-mon[146084]: pgmap v847: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:13.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:13 smithi194 ceph-mon[106982]: pgmap v847: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:15.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:15 smithi008 ceph-mon[146084]: pgmap v848: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:15.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:15 smithi194 ceph-mon[106982]: pgmap v848: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:17.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:17 smithi008 ceph-mon[146084]: pgmap v849: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:17.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:17 smithi194 ceph-mon[106982]: pgmap v849: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:18.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:18.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:19.382 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:19 smithi008 ceph-mon[146084]: pgmap v850: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:19.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:19 smithi194 ceph-mon[106982]: pgmap v850: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:21.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:21 smithi008 ceph-mon[146084]: pgmap v851: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:21.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:21 smithi194 ceph-mon[106982]: pgmap v851: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:23.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:23 smithi008 ceph-mon[146084]: pgmap v852: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:23.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:23 smithi194 ceph-mon[106982]: pgmap v852: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:25.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:25 smithi008 ceph-mon[146084]: pgmap v853: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:25.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:25 smithi194 ceph-mon[106982]: pgmap v853: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:27.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:27 smithi008 ceph-mon[146084]: pgmap v854: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:27.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:27 smithi194 ceph-mon[106982]: pgmap v854: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:29 smithi008 ceph-mon[146084]: pgmap v855: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:29.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:29 smithi194 ceph-mon[106982]: pgmap v855: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:31.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:31 smithi008 ceph-mon[146084]: pgmap v856: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:31.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:23:31.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:23:31.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:23:31.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:23:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:31 smithi194 ceph-mon[106982]: pgmap v856: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:23:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:23:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:23:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:23:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:33 smithi008 ceph-mon[146084]: pgmap v857: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:33.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:33 smithi194 ceph-mon[106982]: pgmap v857: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:33.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:35 smithi194 ceph-mon[106982]: pgmap v858: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:35 smithi008 ceph-mon[146084]: pgmap v858: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:37 smithi008 ceph-mon[146084]: pgmap v859: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:37.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:37 smithi194 ceph-mon[106982]: pgmap v859: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:39 smithi008 ceph-mon[146084]: pgmap v860: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:39.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:39 smithi194 ceph-mon[106982]: pgmap v860: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:41 smithi008 ceph-mon[146084]: pgmap v861: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:41.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:41 smithi194 ceph-mon[106982]: pgmap v861: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:43 smithi008 ceph-mon[146084]: pgmap v862: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:43.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:43 smithi194 ceph-mon[106982]: pgmap v862: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:45.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:45 smithi194 ceph-mon[106982]: pgmap v863: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:45 smithi008 ceph-mon[146084]: pgmap v863: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:47 smithi008 ceph-mon[146084]: pgmap v864: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:47.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:47 smithi194 ceph-mon[106982]: pgmap v864: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:48.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:23:49.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:49 smithi008 ceph-mon[146084]: pgmap v865: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:49.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:49 smithi194 ceph-mon[106982]: pgmap v865: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:51 smithi008 ceph-mon[146084]: pgmap v866: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:51.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:51 smithi194 ceph-mon[106982]: pgmap v866: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:23:53.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:53 smithi194 ceph-mon[106982]: pgmap v867: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:53.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:53 smithi008 ceph-mon[146084]: pgmap v867: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:55.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:55 smithi194 ceph-mon[106982]: pgmap v868: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:55.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:55 smithi008 ceph-mon[146084]: pgmap v868: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:23:57.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:57 smithi194 ceph-mon[106982]: pgmap v869: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:57.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:57 smithi008 ceph-mon[146084]: pgmap v869: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:23:59.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:23:59 smithi194 ceph-mon[106982]: pgmap v870: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:23:59.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:23:59 smithi008 ceph-mon[146084]: pgmap v870: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:01.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:01 smithi194 ceph-mon[106982]: pgmap v871: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:01.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:01 smithi008 ceph-mon[146084]: pgmap v871: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:03 smithi194 ceph-mon[106982]: pgmap v872: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:03.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:03 smithi008 ceph-mon[146084]: pgmap v872: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:03.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:05.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:05 smithi194 ceph-mon[106982]: pgmap v873: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:05.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:05 smithi008 ceph-mon[146084]: pgmap v873: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:07.574 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:07 smithi194 ceph-mon[106982]: pgmap v874: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:07.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:07 smithi008 ceph-mon[146084]: pgmap v874: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:08.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:08 smithi008 ceph-mon[146084]: pgmap v875: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:08.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:08 smithi194 ceph-mon[106982]: pgmap v875: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:11.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:10 smithi008 ceph-mon[146084]: pgmap v876: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:11.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:10 smithi194 ceph-mon[106982]: pgmap v876: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:13.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:12 smithi008 ceph-mon[146084]: pgmap v877: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:13.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:12 smithi194 ceph-mon[106982]: pgmap v877: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:15.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:14 smithi008 ceph-mon[146084]: pgmap v878: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:15.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:14 smithi194 ceph-mon[106982]: pgmap v878: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:17.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:16 smithi008 ceph-mon[146084]: pgmap v879: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:17.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:16 smithi194 ceph-mon[106982]: pgmap v879: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:18.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:18.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:19.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:18 smithi008 ceph-mon[146084]: pgmap v880: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:19.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:18 smithi194 ceph-mon[106982]: pgmap v880: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:21.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:20 smithi008 ceph-mon[146084]: pgmap v881: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:21.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:20 smithi194 ceph-mon[106982]: pgmap v881: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:23.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:22 smithi194 ceph-mon[106982]: pgmap v882: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:23.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:22 smithi008 ceph-mon[146084]: pgmap v882: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:25.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:24 smithi194 ceph-mon[106982]: pgmap v883: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:25.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:24 smithi008 ceph-mon[146084]: pgmap v883: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:27.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:26 smithi194 ceph-mon[106982]: pgmap v884: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:27.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:26 smithi008 ceph-mon[146084]: pgmap v884: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:29.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:28 smithi194 ceph-mon[106982]: pgmap v885: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:29.494 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:28 smithi008 ceph-mon[146084]: pgmap v885: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:31.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:30 smithi194 ceph-mon[106982]: pgmap v886: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:30 smithi008 ceph-mon[146084]: pgmap v886: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:32.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:24:32.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:24:32.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:24:32.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:24:32.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:24:32.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:24:32.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:24:32.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:24:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:32 smithi194 ceph-mon[106982]: pgmap v887: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:32 smithi008 ceph-mon[146084]: pgmap v887: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:35.214 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:34 smithi008 ceph-mon[146084]: pgmap v888: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:35.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:34 smithi194 ceph-mon[106982]: pgmap v888: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:37.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:36 smithi194 ceph-mon[106982]: pgmap v889: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:36 smithi008 ceph-mon[146084]: pgmap v889: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:39.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:38 smithi194 ceph-mon[106982]: pgmap v890: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:38 smithi008 ceph-mon[146084]: pgmap v890: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:41.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:40 smithi194 ceph-mon[106982]: pgmap v891: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:40 smithi008 ceph-mon[146084]: pgmap v891: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:43.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:42 smithi194 ceph-mon[106982]: pgmap v892: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:42 smithi008 ceph-mon[146084]: pgmap v892: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:44.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:44 smithi194 ceph-mon[106982]: pgmap v893: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:44.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:44 smithi008 ceph-mon[146084]: pgmap v893: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:47.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:46 smithi008 ceph-mon[146084]: pgmap v894: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:47.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:46 smithi194 ceph-mon[106982]: pgmap v894: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:48.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:48.326 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:24:49.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:48 smithi008 ceph-mon[146084]: pgmap v895: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:49.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:48 smithi194 ceph-mon[106982]: pgmap v895: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:51.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:50 smithi194 ceph-mon[106982]: pgmap v896: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:50 smithi008 ceph-mon[146084]: pgmap v896: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:24:53.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:52 smithi194 ceph-mon[106982]: pgmap v897: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:52 smithi008 ceph-mon[146084]: pgmap v897: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:55.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:54 smithi194 ceph-mon[106982]: pgmap v898: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:55.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:54 smithi008 ceph-mon[146084]: pgmap v898: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:24:57.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:56 smithi194 ceph-mon[106982]: pgmap v899: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:57.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:56 smithi008 ceph-mon[146084]: pgmap v899: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:24:59.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:24:58 smithi194 ceph-mon[106982]: pgmap v900: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:24:59.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:24:58 smithi008 ceph-mon[146084]: pgmap v900: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:00 smithi194 ceph-mon[106982]: pgmap v901: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:01.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:00 smithi008 ceph-mon[146084]: pgmap v901: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:03.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:02 smithi194 ceph-mon[106982]: pgmap v902: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:03.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:02 smithi008 ceph-mon[146084]: pgmap v902: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:04.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:04 smithi008 ceph-mon[146084]: pgmap v903: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:04.824 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:04 smithi194 ceph-mon[106982]: pgmap v903: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:07.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:06 smithi008 ceph-mon[146084]: pgmap v904: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:07.324 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:06 smithi194 ceph-mon[106982]: pgmap v904: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:09.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:08 smithi194 ceph-mon[106982]: pgmap v905: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:09.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:08 smithi008 ceph-mon[146084]: pgmap v905: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:11.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:10 smithi194 ceph-mon[106982]: pgmap v906: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:11.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:10 smithi008 ceph-mon[146084]: pgmap v906: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:13.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:12 smithi194 ceph-mon[106982]: pgmap v907: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:13.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:12 smithi008 ceph-mon[146084]: pgmap v907: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:15.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:14 smithi194 ceph-mon[106982]: pgmap v908: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:15.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:14 smithi008 ceph-mon[146084]: pgmap v908: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:17.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:16 smithi194 ceph-mon[106982]: pgmap v909: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:17.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:16 smithi008 ceph-mon[146084]: pgmap v909: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:18.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:17 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:18.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:17 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:19.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:18 smithi194 ceph-mon[106982]: pgmap v910: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:19.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:18 smithi008 ceph-mon[146084]: pgmap v910: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:21.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:20 smithi008 ceph-mon[146084]: pgmap v911: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:21.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:20 smithi194 ceph-mon[106982]: pgmap v911: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:23.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:22 smithi194 ceph-mon[106982]: pgmap v912: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:23.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:22 smithi008 ceph-mon[146084]: pgmap v912: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:25.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:24 smithi194 ceph-mon[106982]: pgmap v913: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:25.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:24 smithi008 ceph-mon[146084]: pgmap v913: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:27.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:26 smithi194 ceph-mon[106982]: pgmap v914: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:27.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:26 smithi008 ceph-mon[146084]: pgmap v914: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:29.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:28 smithi194 ceph-mon[106982]: pgmap v915: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:28 smithi008 ceph-mon[146084]: pgmap v915: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:31.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:30 smithi008 ceph-mon[146084]: pgmap v916: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:31.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:30 smithi194 ceph-mon[106982]: pgmap v916: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:32.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:31 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:25:32.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:31 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:25:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:32 smithi194 ceph-mon[106982]: pgmap v917: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:25:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:25:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:25:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:32 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:32 smithi008 ceph-mon[146084]: pgmap v917: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:25:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:25:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:25:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:32 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:35.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:34 smithi194 ceph-mon[106982]: pgmap v918: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:34 smithi008 ceph-mon[146084]: pgmap v918: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:37.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:36 smithi194 ceph-mon[106982]: pgmap v919: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:36 smithi008 ceph-mon[146084]: pgmap v919: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:38 smithi194 ceph-mon[106982]: pgmap v920: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:38 smithi008 ceph-mon[146084]: pgmap v920: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:41.302 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:40 smithi194 ceph-mon[106982]: pgmap v921: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:40 smithi008 ceph-mon[146084]: pgmap v921: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:43.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:42 smithi194 ceph-mon[106982]: pgmap v922: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:42 smithi008 ceph-mon[146084]: pgmap v922: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:45.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:44 smithi194 ceph-mon[106982]: pgmap v923: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:45.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:44 smithi008 ceph-mon[146084]: pgmap v923: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:47.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:46 smithi194 ceph-mon[106982]: pgmap v924: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:46 smithi008 ceph-mon[146084]: pgmap v924: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:48.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:47 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:47 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:25:49.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:48 smithi194 ceph-mon[106982]: pgmap v925: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:49.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:48 smithi008 ceph-mon[146084]: pgmap v925: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:50 smithi194 ceph-mon[106982]: pgmap v926: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:50 smithi008 ceph-mon[146084]: pgmap v926: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:25:53.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:52 smithi194 ceph-mon[106982]: pgmap v927: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:52 smithi008 ceph-mon[146084]: pgmap v927: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:55.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:54 smithi194 ceph-mon[106982]: pgmap v928: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:55.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:54 smithi008 ceph-mon[146084]: pgmap v928: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:25:57.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:56 smithi194 ceph-mon[106982]: pgmap v929: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:57.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:56 smithi008 ceph-mon[146084]: pgmap v929: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:25:59.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:25:58 smithi194 ceph-mon[106982]: pgmap v930: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:25:59.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:25:58 smithi008 ceph-mon[146084]: pgmap v930: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:01.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:00 smithi194 ceph-mon[106982]: pgmap v931: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:01.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:00 smithi008 ceph-mon[146084]: pgmap v931: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:03.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:02 smithi194 ceph-mon[106982]: pgmap v932: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:03.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:02 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:03.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:02 smithi008 ceph-mon[146084]: pgmap v932: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:03.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:02 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:05.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:04 smithi194 ceph-mon[106982]: pgmap v933: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:05.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:04 smithi008 ceph-mon[146084]: pgmap v933: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:07.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:07 smithi194 ceph-mon[106982]: pgmap v934: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:07.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:07 smithi008 ceph-mon[146084]: pgmap v934: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:09.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:09 smithi194 ceph-mon[106982]: pgmap v935: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:09.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:09 smithi008 ceph-mon[146084]: pgmap v935: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:11.267 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:11 smithi008 ceph-mon[146084]: pgmap v936: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:11.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:11 smithi194 ceph-mon[106982]: pgmap v936: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:13.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:13 smithi194 ceph-mon[106982]: pgmap v937: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:13.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:13 smithi008 ceph-mon[146084]: pgmap v937: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:15.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:15 smithi194 ceph-mon[106982]: pgmap v938: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:15.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:15 smithi008 ceph-mon[146084]: pgmap v938: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:17.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:17 smithi194 ceph-mon[106982]: pgmap v939: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:17.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:17 smithi008 ceph-mon[146084]: pgmap v939: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:18.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:18.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:19.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:19 smithi194 ceph-mon[106982]: pgmap v940: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:19.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:19 smithi008 ceph-mon[146084]: pgmap v940: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:21.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:21 smithi194 ceph-mon[106982]: pgmap v941: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:21.379 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:21 smithi008 ceph-mon[146084]: pgmap v941: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:23.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:23 smithi194 ceph-mon[106982]: pgmap v942: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:23.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:23 smithi008 ceph-mon[146084]: pgmap v942: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:25.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:25 smithi194 ceph-mon[106982]: pgmap v943: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:25.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:25 smithi008 ceph-mon[146084]: pgmap v943: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:27.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:27 smithi194 ceph-mon[106982]: pgmap v944: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:27.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:27 smithi008 ceph-mon[146084]: pgmap v944: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:29.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:29 smithi194 ceph-mon[106982]: pgmap v945: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:29 smithi008 ceph-mon[146084]: pgmap v945: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:31.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:31 smithi194 ceph-mon[106982]: pgmap v946: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:31.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:31 smithi008 ceph-mon[146084]: pgmap v946: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: pgmap v947: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:26:33.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:26:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: pgmap v947: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:26:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:26:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:26:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:26:35.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:35 smithi194 ceph-mon[106982]: pgmap v948: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:35.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:35 smithi008 ceph-mon[146084]: pgmap v948: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:37.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:37 smithi194 ceph-mon[106982]: pgmap v949: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:37 smithi008 ceph-mon[146084]: pgmap v949: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:39.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:39 smithi194 ceph-mon[106982]: pgmap v950: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:39 smithi008 ceph-mon[146084]: pgmap v950: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:41.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:41 smithi194 ceph-mon[106982]: pgmap v951: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:41.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:41 smithi008 ceph-mon[146084]: pgmap v951: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:43.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:43 smithi194 ceph-mon[106982]: pgmap v952: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:43.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:43 smithi008 ceph-mon[146084]: pgmap v952: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:45.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:45 smithi194 ceph-mon[106982]: pgmap v953: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:45.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:45 smithi008 ceph-mon[146084]: pgmap v953: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:47.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:47 smithi194 ceph-mon[106982]: pgmap v954: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:47.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:47 smithi008 ceph-mon[146084]: pgmap v954: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:48.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:26:49.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:49 smithi008 ceph-mon[146084]: pgmap v955: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:49.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:49 smithi194 ceph-mon[106982]: pgmap v955: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:51.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:51 smithi008 ceph-mon[146084]: pgmap v956: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:51.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:51 smithi194 ceph-mon[106982]: pgmap v956: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:26:53.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:53 smithi008 ceph-mon[146084]: pgmap v957: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:53.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:53 smithi194 ceph-mon[106982]: pgmap v957: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:55.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:55 smithi008 ceph-mon[146084]: pgmap v958: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:55.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:55 smithi194 ceph-mon[106982]: pgmap v958: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:26:57.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:57 smithi008 ceph-mon[146084]: pgmap v959: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:57.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:57 smithi194 ceph-mon[106982]: pgmap v959: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:26:59.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:26:59 smithi008 ceph-mon[146084]: pgmap v960: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:26:59.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:26:59 smithi194 ceph-mon[106982]: pgmap v960: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:01.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:01 smithi008 ceph-mon[146084]: pgmap v961: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:01.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:01 smithi194 ceph-mon[106982]: pgmap v961: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:03.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:03 smithi008 ceph-mon[146084]: pgmap v962: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:03.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:03 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:03 smithi194 ceph-mon[106982]: pgmap v962: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:03.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:03 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:05.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:05 smithi008 ceph-mon[146084]: pgmap v963: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:05.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:05 smithi194 ceph-mon[106982]: pgmap v963: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:07.158 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:07 smithi008 ceph-mon[146084]: pgmap v964: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:07.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:07 smithi194 ceph-mon[106982]: pgmap v964: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:09.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:09 smithi008 ceph-mon[146084]: pgmap v965: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:09.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:09 smithi194 ceph-mon[106982]: pgmap v965: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:11.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:11 smithi008 ceph-mon[146084]: pgmap v966: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:11.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:11 smithi194 ceph-mon[106982]: pgmap v966: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:13.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:13 smithi008 ceph-mon[146084]: pgmap v967: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:13.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:13 smithi194 ceph-mon[106982]: pgmap v967: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:15.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:15 smithi008 ceph-mon[146084]: pgmap v968: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:15.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:15 smithi194 ceph-mon[106982]: pgmap v968: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:17.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:17 smithi008 ceph-mon[146084]: pgmap v969: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:17.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:17 smithi194 ceph-mon[106982]: pgmap v969: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:18.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:18 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:18.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:18 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:19.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:19 smithi008 ceph-mon[146084]: pgmap v970: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:19.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:19 smithi194 ceph-mon[106982]: pgmap v970: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:21.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:21 smithi008 ceph-mon[146084]: pgmap v971: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:21.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:21 smithi194 ceph-mon[106982]: pgmap v971: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:23.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:23 smithi008 ceph-mon[146084]: pgmap v972: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:23.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:23 smithi194 ceph-mon[106982]: pgmap v972: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:25.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:25 smithi008 ceph-mon[146084]: pgmap v973: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:25.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:25 smithi194 ceph-mon[106982]: pgmap v973: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:27.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:27 smithi008 ceph-mon[146084]: pgmap v974: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:27.433 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:27 smithi194 ceph-mon[106982]: pgmap v974: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:29.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:29 smithi008 ceph-mon[146084]: pgmap v975: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:29.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:29 smithi194 ceph-mon[106982]: pgmap v975: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:31.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:31 smithi008 ceph-mon[146084]: pgmap v976: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:31.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:31 smithi194 ceph-mon[106982]: pgmap v976: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:33 smithi008 ceph-mon[146084]: pgmap v977: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:33.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:33.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:33 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:27:33.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:33 smithi194 ceph-mon[106982]: pgmap v977: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:33.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:33.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:33 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-12T06:27:34.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:34 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:27:34.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:34 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:27:34.409 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:34 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:27:34.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:34 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-12T06:27:34.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:34 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-12T06:27:34.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:34 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' 2024-04-12T06:27:35.410 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:35 smithi008 ceph-mon[146084]: pgmap v978: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:35.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:35 smithi194 ceph-mon[106982]: pgmap v978: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:37.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:37 smithi008 ceph-mon[146084]: pgmap v979: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:37.540 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:37 smithi194 ceph-mon[106982]: pgmap v979: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:39.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:39 smithi008 ceph-mon[146084]: pgmap v980: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:39.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:39 smithi194 ceph-mon[106982]: pgmap v980: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:41.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:41 smithi194 ceph-mon[106982]: pgmap v981: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:41.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:41 smithi008 ceph-mon[146084]: pgmap v981: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-12T06:27:43.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:43 smithi194 ceph-mon[106982]: pgmap v982: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:43.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:43 smithi008 ceph-mon[146084]: pgmap v982: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:45.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:45 smithi194 ceph-mon[106982]: pgmap v983: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:45.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:45 smithi008 ceph-mon[146084]: pgmap v983: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-12T06:27:47.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:47 smithi194 ceph-mon[106982]: pgmap v984: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:47.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:47 smithi008 ceph-mon[146084]: pgmap v984: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-12T06:27:48.408 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:48 smithi008 ceph-mon[146084]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:48.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:48 smithi194 ceph-mon[106982]: from='mgr.14660 172.21.15.8:0/2278942878' entity='mgr.smithi008.ixmnlp' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-12T06:27:49.210 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-12T06:27:49.210 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T06:27:49.212 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-12T06:27:49.225 INFO:tasks.cephadm:Teardown begin 2024-04-12T06:27:49.225 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/cephadm.py", line 1750, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T06:27:49.227 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T06:27:49.264 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T06:27:49.302 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-12T06:27:49.302 DEBUG:teuthology.orchestra.run.smithi008:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-12T06:27:49.326 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-12T06:27:49.363 INFO:tasks.cephadm:Stopping all daemons... 2024-04-12T06:27:49.363 INFO:tasks.cephadm.mon.smithi008:Stopping mon.smithi008... 2024-04-12T06:27:49.363 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008 2024-04-12T06:27:49.575 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:49 smithi194 ceph-mon[106982]: pgmap v985: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:49.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:49 smithi008 ceph-mon[146084]: pgmap v985: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-12T06:27:49.659 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:49 smithi008 systemd[1]: Stopping Ceph mon.smithi008 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:27:50.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:49 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:27:49.669+0000 7f39f7720700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi008 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:27:50.159 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:49 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008[146061]: 2024-04-12T06:27:49.669+0000 7f39f7720700 -1 mon.smithi008@0(leader) e2 *** Got Signal Terminated *** 2024-04-12T06:27:50.658 INFO:journalctl@ceph.mon.smithi008.smithi008.stdout:Apr 12 06:27:50 smithi008 bash[207366]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi008 2024-04-12T06:27:50.809 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi008.service' 2024-04-12T06:27:50.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:27:50.856 INFO:tasks.cephadm.mon.smithi008:Stopped mon.smithi008 2024-04-12T06:27:50.857 INFO:tasks.cephadm.mon.smithi194:Stopping mon.smithi194... 2024-04-12T06:27:50.857 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi194 2024-04-12T06:27:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:50 smithi194 systemd[1]: Stopping Ceph mon.smithi194 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:27:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:51 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194[106959]: 2024-04-12T06:27:51.134+0000 7f1dffd57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi194 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:27:51.325 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:51 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194[106959]: 2024-04-12T06:27:51.134+0000 7f1dffd57700 -1 mon.smithi194@1(peon) e2 *** Got Signal Terminated *** 2024-04-12T06:27:52.075 INFO:journalctl@ceph.mon.smithi194.smithi194.stdout:Apr 12 06:27:51 smithi194 bash[154173]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-mon-smithi194 2024-04-12T06:27:52.200 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@mon.smithi194.service' 2024-04-12T06:27:52.247 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:27:52.247 INFO:tasks.cephadm.mon.smithi194:Stopped mon.smithi194 2024-04-12T06:27:52.248 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-12T06:27:52.248 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.0 2024-04-12T06:27:52.529 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:27:52 smithi008 systemd[1]: Stopping Ceph osd.0 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:27:52.908 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:27:52 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:27:52.528+0000 7f689f064700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:27:52.908 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:27:52 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:27:52.528+0000 7f689f064700 -1 osd.0 83 *** Got signal Terminated *** 2024-04-12T06:27:52.908 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:27:52 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0[157980]: 2024-04-12T06:27:52.528+0000 7f689f064700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:27:58.658 INFO:journalctl@ceph.osd.0.smithi008.stdout:Apr 12 06:27:58 smithi008 bash[207530]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-0 2024-04-12T06:28:00.480 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.0.service' 2024-04-12T06:28:00.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:00.527 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-12T06:28:00.527 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-12T06:28:00.527 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.1 2024-04-12T06:28:00.908 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:28:00 smithi008 systemd[1]: Stopping Ceph osd.1 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:28:01.408 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:28:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:28:00.981+0000 7f0eab1db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:28:01.408 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:28:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:28:00.981+0000 7f0eab1db700 -1 osd.1 83 *** Got signal Terminated *** 2024-04-12T06:28:01.409 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:28:00 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1[164852]: 2024-04-12T06:28:00.981+0000 7f0eab1db700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:28:07.163 INFO:journalctl@ceph.osd.1.smithi008.stdout:Apr 12 06:28:07 smithi008 bash[207803]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-1 2024-04-12T06:28:09.256 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.1.service' 2024-04-12T06:28:09.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:09.305 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-12T06:28:09.306 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-12T06:28:09.306 DEBUG:teuthology.orchestra.run.smithi008:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.2 2024-04-12T06:28:09.601 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:28:09 smithi008 systemd[1]: Stopping Ceph osd.2 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:28:09.908 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:28:09 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:28:09.599+0000 7f3a2e42a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:28:09.908 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:28:09 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:28:09.599+0000 7f3a2e42a700 -1 osd.2 83 *** Got signal Terminated *** 2024-04-12T06:28:09.909 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:28:09 smithi008 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2[172358]: 2024-04-12T06:28:09.599+0000 7f3a2e42a700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:28:15.659 INFO:journalctl@ceph.osd.2.smithi008.stdout:Apr 12 06:28:15 smithi008 bash[208067]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-2 2024-04-12T06:28:17.528 DEBUG:teuthology.orchestra.run.smithi008:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.2.service' 2024-04-12T06:28:17.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:17.579 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-12T06:28:17.579 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-12T06:28:17.579 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.3 2024-04-12T06:28:17.870 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:17 smithi194 systemd[1]: Stopping Ceph osd.3 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:28:18.325 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:17 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:17.868+0000 7fb5bfeba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:28:18.325 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:17 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:17.868+0000 7fb5bfeba700 -1 osd.3 83 *** Got signal Terminated *** 2024-04-12T06:28:18.325 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:17 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:17.868+0000 7fb5bfeba700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:28:19.825 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:19 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:19.552+0000 7fb5bccb1700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:55.751334+0000 front 2024-04-12T06:27:55.751358+0000 (oldest deadline 2024-04-12T06:28:19.251257+0000) 2024-04-12T06:28:20.825 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:20 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:20.565+0000 7fb5bccb1700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:55.751334+0000 front 2024-04-12T06:27:55.751358+0000 (oldest deadline 2024-04-12T06:28:19.251257+0000) 2024-04-12T06:28:21.825 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:21 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:21.527+0000 7fb5bccb1700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:55.751334+0000 front 2024-04-12T06:27:55.751358+0000 (oldest deadline 2024-04-12T06:28:19.251257+0000) 2024-04-12T06:28:21.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:21 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:21.476+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:21.826 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:21 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:21.360+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:22.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:22 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:22.495+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:22.825 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:22 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3[118936]: 2024-04-12T06:28:22.492+0000 7fb5bccb1700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:55.751334+0000 front 2024-04-12T06:27:55.751358+0000 (oldest deadline 2024-04-12T06:28:19.251257+0000) 2024-04-12T06:28:22.826 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:22 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:22.330+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:23.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:23 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:23.490+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:23.575 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:23 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:23.309+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:24.075 INFO:journalctl@ceph.osd.3.smithi194.stdout:Apr 12 06:28:23 smithi194 bash[154329]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-3 2024-04-12T06:28:24.575 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:24 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:24.354+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:24.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:24 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:24.494+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:25.288 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.3.service' 2024-04-12T06:28:25.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:25.330 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-12T06:28:25.330 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-12T06:28:25.330 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.4 2024-04-12T06:28:25.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:25 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:25.525+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:25.575 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:25 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:25.329+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:25.575 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:25 smithi194 systemd[1]: Stopping Ceph osd.4 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:28:26.075 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:25 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:25.629+0000 7f2249ccb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:28:26.075 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:25 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:25.629+0000 7f2249ccb700 -1 osd.4 83 *** Got signal Terminated *** 2024-04-12T06:28:26.075 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:25 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:25.629+0000 7f2249ccb700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:28:26.825 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:26 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:26.344+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:26.826 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:26 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:26.482+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:27.825 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:27 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:27.368+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:27.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:27 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:27.512+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:27.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:27 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:27.512+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:28.825 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:28 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:28.397+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:28.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:28 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:28.529+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:28.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:28 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:28.529+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:29.825 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:29 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:29.353+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:29.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:29 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:29.530+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:29.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:29 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:29.530+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:30.825 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:30 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4[125269]: 2024-04-12T06:28:30.363+0000 7f2242aba700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.661090+0000 front 2024-04-12T06:27:56.661076+0000 (oldest deadline 2024-04-12T06:28:20.760932+0000) 2024-04-12T06:28:30.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:30 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:30.500+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:30.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:30 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:30.500+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:31.575 INFO:journalctl@ceph.osd.4.smithi194.stdout:Apr 12 06:28:31 smithi194 bash[154599]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-4 2024-04-12T06:28:31.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:31 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:31.470+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:31.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:31 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:31.470+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:32.765 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:32 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:32.427+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:32.765 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:32 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:32.427+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:33.023 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.4.service' 2024-04-12T06:28:33.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:33.063 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-12T06:28:33.063 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-12T06:28:33.064 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.5 2024-04-12T06:28:33.325 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 systemd[1]: Stopping Ceph osd.5 for b9cc93fe-f88f-11ee-bc8c-c7b262605968... 2024-04-12T06:28:33.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:33.388+0000 7fef3c4f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-12T06:28:33.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:33.388+0000 7fef3c4f7700 -1 osd.5 83 *** Got signal Terminated *** 2024-04-12T06:28:33.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:33.388+0000 7fef3c4f7700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-12T06:28:33.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:33.472+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:33.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:33 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:33.472+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:34.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:34 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:34.426+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:34.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:34 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:34.426+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:35.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:35.469+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:35.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:35.469+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:35.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:35 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:35.469+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6822 osd.2 since back 2024-04-12T06:28:11.158559+0000 front 2024-04-12T06:28:11.158572+0000 (oldest deadline 2024-04-12T06:28:35.258462+0000) 2024-04-12T06:28:36.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:36 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:36.467+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:36.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:36 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:36.467+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:36.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:36 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:36.467+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6822 osd.2 since back 2024-04-12T06:28:11.158559+0000 front 2024-04-12T06:28:11.158572+0000 (oldest deadline 2024-04-12T06:28:35.258462+0000) 2024-04-12T06:28:37.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:37 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:37.483+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6806 osd.0 since back 2024-04-12T06:27:56.755996+0000 front 2024-04-12T06:27:56.756080+0000 (oldest deadline 2024-04-12T06:28:20.856054+0000) 2024-04-12T06:28:37.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:37 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:37.483+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6814 osd.1 since back 2024-04-12T06:28:03.157032+0000 front 2024-04-12T06:28:03.157076+0000 (oldest deadline 2024-04-12T06:28:26.656801+0000) 2024-04-12T06:28:37.825 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:37 smithi194 ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5[131422]: 2024-04-12T06:28:37.483+0000 7fef392ee700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.8:6822 osd.2 since back 2024-04-12T06:28:11.158559+0000 front 2024-04-12T06:28:11.158572+0000 (oldest deadline 2024-04-12T06:28:35.258462+0000) 2024-04-12T06:28:39.575 INFO:journalctl@ceph.osd.5.smithi194.stdout:Apr 12 06:28:39 smithi194 bash[154867]: ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968-osd-5 2024-04-12T06:28:40.873 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b9cc93fe-f88f-11ee-bc8c-c7b262605968@osd.5.service' 2024-04-12T06:28:40.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-12T06:28:40.916 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-12T06:28:40.916 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 --force --keep-logs 2024-04-12T06:28:41.103 INFO:teuthology.orchestra.run.smithi008.stdout:Deleting cluster with fsid: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:28:45.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi008.stderr:ceph-fuse[101505]: fuse finished with error 0 and tester_r 0 2024-04-12T06:29:02.690 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 --force --keep-logs 2024-04-12T06:29:02.869 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:29:14.800 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T06:29:14.839 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-12T06:29:14.870 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-12T06:29:14.872 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi008/crash 2024-04-12T06:29:14.873 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash -- . 2024-04-12T06:29:14.911 INFO:teuthology.orchestra.run.smithi008.stderr:tar: /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash: Cannot open: No such file or directory 2024-04-12T06:29:14.911 INFO:teuthology.orchestra.run.smithi008.stderr:tar: Error is not recoverable: exiting now 2024-04-12T06:29:14.913 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi194/crash 2024-04-12T06:29:14.914 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash -- . 2024-04-12T06:29:14.940 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/crash: Cannot open: No such file or directory 2024-04-12T06:29:14.940 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-04-12T06:29:14.941 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-12T06:29:14.941 DEBUG:teuthology.orchestra.run.smithi008:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-04-12T06:29:14.987 INFO:teuthology.orchestra.run.smithi008.stderr:grep: /var/log/ceph/b9cc93fe-f88f-11ee-bc8c-c7b262605968/ceph.log: No such file or directory 2024-04-12T06:29:14.989 INFO:tasks.cephadm:Compressing logs... 2024-04-12T06:29:14.989 DEBUG:teuthology.orchestra.run.smithi008:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T06:29:15.032 DEBUG:teuthology.orchestra.run.smithi194:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T06:29:15.059 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-12T06:29:15.060 INFO:teuthology.orchestra.run.smithi008.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-12T06:29:33.047 INFO:teuthology.orchestra.run.smithi194.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-12T06:31:09.547 INFO:tasks.cephadm:Archiving logs... 2024-04-12T06:31:09.548 DEBUG:teuthology.misc:Transferring archived files from smithi008:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi008/log 2024-04-12T06:31:09.549 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T06:31:22.651 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi194/log 2024-04-12T06:31:22.652 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-12T06:31:34.626 INFO:tasks.cephadm:Removing cluster... 2024-04-12T06:31:34.627 DEBUG:teuthology.orchestra.run.smithi008:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 --force 2024-04-12T06:31:34.821 INFO:teuthology.orchestra.run.smithi008.stdout:Deleting cluster with fsid: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:31:35.163 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b9cc93fe-f88f-11ee-bc8c-c7b262605968 --force 2024-04-12T06:31:35.346 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: b9cc93fe-f88f-11ee-bc8c-c7b262605968 2024-04-12T06:31:35.646 INFO:tasks.cephadm:Removing cephadm ... 2024-04-12T06:31:35.646 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-12T06:31:35.666 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-12T06:31:35.685 INFO:tasks.cephadm:Teardown complete 2024-04-12T06:31:35.685 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-12T06:31:35.699 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T06:31:35.700 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-12T06:31:35.700 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T06:31:35.708 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-12T06:31:35.803 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-12T06:31:35.816 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-12T06:31:35.816 DEBUG:teuthology.orchestra.run.smithi008:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T06:31:35.819 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-12T06:31:35.835 INFO:teuthology.orchestra.run.smithi008.stderr:bash: ntpq: command not found 2024-04-12T06:31:35.835 INFO:teuthology.orchestra.run.smithi194.stderr:bash: ntpq: command not found 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:=============================================================================== 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +537us[ +541us] +/- 61ms 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 68 -808us[ -797us] +/- 65ms 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +451us[ +455us] +/- 64ms 2024-04-12T06:31:35.839 INFO:teuthology.orchestra.run.smithi008.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv01.front.sepia.ceph.com 3 7 377 6 +484us[ +491us] +/- 61ms 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 69 -927us[ -919us] +/- 65ms 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +379us[ +379us] +/- 64ms 2024-04-12T06:31:35.841 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-12T06:31:35.841 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-12T06:31:35.855 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-12T06:31:35.856 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-12T06:31:35.869 DEBUG:teuthology.orchestra.run.smithi008:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T06:31:35.959 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-12T06:31:36.030 DEBUG:teuthology.orchestra.run.smithi008:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-12T06:31:36.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T06:31:36.062 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-12T06:31:36.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T06:31:36.092 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-12T06:31:36.103 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-12T06:31:36.119 INFO:teuthology.task.internal:Duration was 3511.981410 seconds 2024-04-12T06:31:36.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-12T06:31:36.268 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-12T06:31:36.269 DEBUG:teuthology.orchestra.run.smithi008:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T06:31:36.271 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-12T06:31:36.317 INFO:teuthology.orchestra.run.smithi008.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T06:31:36.318 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-12T06:31:36.587 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-12T06:31:36.587 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi008.front.sepia.ceph.com 2024-04-12T06:31:36.588 DEBUG:teuthology.orchestra.run.smithi008:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T06:31:36.612 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-04-12T06:31:36.613 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-12T06:31:36.636 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-12T06:31:36.636 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T06:31:36.655 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-12T06:31:36.735 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-12T06:31:36.735 DEBUG:teuthology.orchestra.run.smithi008:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T06:31:36.737 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-12T06:31:36.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-12T06:31:36.886 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-12T06:31:36.886 DEBUG:teuthology.orchestra.run.smithi008:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T06:31:36.914 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-12T06:31:36.942 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-12T06:31:36.956 DEBUG:teuthology.orchestra.run.smithi008:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T06:31:36.958 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-12T06:31:36.983 INFO:teuthology.orchestra.run.smithi008.stdout:kernel.core_pattern = core 2024-04-12T06:31:37.008 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-04-12T06:31:37.062 DEBUG:teuthology.orchestra.run.smithi008:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T06:31:37.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T06:31:37.079 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-12T06:31:37.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T06:31:37.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-12T06:31:37.133 INFO:teuthology.task.internal:Transferring archived files... 2024-04-12T06:31:37.135 DEBUG:teuthology.misc:Transferring archived files from smithi008:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi008 2024-04-12T06:31:37.135 DEBUG:teuthology.orchestra.run.smithi008:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T06:31:37.182 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724/remote/smithi194 2024-04-12T06:31:37.182 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-12T06:31:37.224 INFO:teuthology.task.internal:Removing archive directory... 2024-04-12T06:31:37.224 DEBUG:teuthology.orchestra.run.smithi008:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T06:31:37.227 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-12T06:31:37.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-12T06:31:37.291 INFO:teuthology.task.internal:Not uploading archives. 2024-04-12T06:31:37.291 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-12T06:31:37.308 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-12T06:31:37.309 DEBUG:teuthology.orchestra.run.smithi008:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T06:31:37.311 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-12T06:31:37.327 INFO:teuthology.orchestra.run.smithi008.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 12 06:31 /home/ubuntu/cephtest 2024-04-12T06:31:37.327 INFO:teuthology.orchestra.run.smithi008.stdout: 527596 4 d--------- 2 ubuntu ubuntu 4096 Apr 12 05:53 /home/ubuntu/cephtest/mnt.0 2024-04-12T06:31:37.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-12T06:31:37.328 INFO:teuthology.orchestra.run.smithi008.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-12T06:31:37.328 INFO:teuthology.orchestra.run.smithi008.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-12T06:31:37.329 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_ceph_ceph-c_d540ebaca6b131a1dd560e7f69e024b133bbaa42/qa/tasks/cephfs/fuse_mount.py", line 400, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6c637841c215537a4502385240412f1966e0faab/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi008 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-12T06:31:37.329 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-12T06:31:37.345 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-12T06:31:37.360 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-12T06:31:37.360 INFO:teuthology.nuke:Checking targets against current locks 2024-04-12T06:31:37.387 DEBUG:teuthology.nuke:shortname: smithi008 2024-04-12T06:31:37.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T06:31:37.410 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi008.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 05:24:21.381449', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBHXrxyScZM767rrL0sGHsSXpHS0uRqNFIrwqtCyUM/tVsNzRVrGtWwPsCPEeSSUdpDpLpexQRk+YL4Hd9p5PA='} 2024-04-12T06:31:37.414 DEBUG:teuthology.nuke:shortname: smithi194 2024-04-12T06:31:37.414 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-12T06:31:37.438 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-11_20:35:13-orch-reef-release-distro-default-smithi/7652724', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-12 05:24:21.379489', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJLpm4x954BAugiC+FwA2zC+KhkrhLe0fOwwblJYsUgGIoXjcgpgaKzLpYnftQsmtTjghg2Ik4qXx70rbQDMt3Q='} 2024-04-12T06:31:37.455 INFO:teuthology.orchestra.console.smithi008:Power off 2024-04-12T06:31:37.456 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T06:31:37.479 DEBUG:teuthology.orchestra.console.smithi008:power off output: Chassis Power Control: Down/Off 2024-04-12T06:31:37.479 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:37.495 INFO:teuthology.orchestra.console.smithi194:Power off 2024-04-12T06:31:37.495 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-12T06:31:37.505 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-04-12T06:31:37.520 DEBUG:teuthology.orchestra.console.smithi194:power off output: Chassis Power Control: Down/Off 2024-04-12T06:31:37.520 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:37.544 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-12T06:31:41.507 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:41.546 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:41.617 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is on 2024-04-12T06:31:41.656 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is on 2024-04-12T06:31:45.622 DEBUG:teuthology.orchestra.console.smithi008:pexpect command: ipmitool -H smithi008.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:45.657 DEBUG:teuthology.orchestra.console.smithi194:pexpect command: ipmitool -H smithi194.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-12T06:31:45.733 DEBUG:teuthology.orchestra.console.smithi008:check power output: Chassis Power is off 2024-04-12T06:31:45.767 DEBUG:teuthology.orchestra.console.smithi194:check power output: Chassis Power is off 2024-04-12T06:31:45.834 INFO:teuthology.orchestra.console.smithi008:Power off completed 2024-04-12T06:31:45.869 INFO:teuthology.orchestra.console.smithi194:Power off completed 2024-04-12T06:31:45.973 INFO:teuthology.run:Summary data: description: orch/cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3511.9814100265503 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-04-12T06:31:45.974 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-12T06:31:46.070 INFO:teuthology.run:FAIL